You are not logged in.

Aggregation on the fly: Reducing traffic for big data in the cloud

Ke, Huan, Li, Peng, Guo, Song and Stojmenovic, Ivan 2015, Aggregation on the fly: Reducing traffic for big data in the cloud, IEEE Network, vol. 29, no. 5, pp. 17-23.

Attached Files
Name Description MIMEType Size Downloads

Title Aggregation on the fly: Reducing traffic for big data in the cloud
Author(s) Ke, Huan
Li, Peng
Guo, Song
Stojmenovic, Ivan
Journal name IEEE Network
Volume number 29
Issue number 5
Start page 17
End page 23
Total pages 7
Publisher IEEE
Place of publication New York, N.Y.
Publication date 2015-09-01
ISSN 0890-8044
Keyword(s) Science & Technology
Technology
Computer Science, Hardware & Architecture
Computer Science, Information Systems
Engineering, Electrical & Electronic
Telecommunications
Computer Science
Engineering
Summary As a leading framework for processing and analyzing big data, MapReduce is leveraged by many enterprises to parallelize their data processing on distributed computing systems. Unfortunately, the all-to-all data forwarding from map tasks to reduce tasks in the traditional MapReduce framework would generate a large amount of network traffic. The fact that the intermediate data generated by map tasks can be combined with significant traffic reduction in many applications motivates us to propose a data aggregation scheme for MapReduce jobs in cloud. Specifically, we design an aggregation architecture under the existing MapReduce framework with the objective of minimizing the data traffic during the shuffle phase, in which aggregators can reside anywhere in the cloud. Some experimental results also show that our proposal outperforms existing work by reducing the network traffic significantly.
Language eng
Field of Research 080109 Pattern Recognition and Data Mining
0906 Electrical And Electronic Engineering
0805 Distributed Computing
Socio Economic Objective 970108 Expanding Knowledge in the Information and Computing Sciences
HERDC Research category C1.1 Refereed article in a scholarly journal
ERA Research output type C Journal article
Persistent URL http://hdl.handle.net/10536/DRO/DU:30080773

Document type: Journal Article
Collection: School of Information Technology
Connect to link resolver
 
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 1 times in TR Web of Science
Scopus Citation Count Cited 3 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 21 Abstract Views, 2 File Downloads  -  Detailed Statistics
Created: Mon, 18 Jan 2016, 13:57:01 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.