abawajy-dataintensivetask-2020.pdf (366.35 kB)
Data-intensive task scheduling for heterogeneous big data analytics in IoT system
journal contribution
posted on 2020-09-01, 00:00 authored by Xin Li, Liangyuan Wang, Jemal AbawajyJemal Abawajy, Xiaolin Qin, Giovanni Pau, Ilsun YouEfficient big data analysis is critical to support applications or services in Internet of Things (IoT) system, especially for the time-intensive services. Hence, the data center may host heterogeneous big data analysis tasks for multiple IoT systems. It is a challenging problem since the data centers usually need to schedule a large number of periodic or online tasks in a short time. In this paper, we investigate the heterogeneous task scheduling problem to reduce the global task execution time, which is also an efficient method to reduce energy consumption for data centers. We establish the task execution for heterogeneous tasks respectively based on the data locality feature, which also indicate the relationship among the tasks, data blocks and servers. We propose a heterogeneous task scheduling algorithm with data migration. The core idea of the algorithm is to maximize the efficiency by comparing the cost between remote task execution and data migration, which could improve the data locality and reduce task execution time. We conduct extensive simulations and the experimental results show that our algorithm has better performance than the traditional methods, and data migration actually works to reduce th overall task execution time. The algorithm also shows acceptable fairness for the heterogeneous tasks.
History
Journal
EnergiesVolume
13Issue
17Article number
4508Pagination
1 - 14Publisher
Molecular Diversity Preservation InternationalLocation
Basel, SwitzerlandPublisher DOI
eISSN
1996-1073Language
engPublication classification
C1 Refereed article in a scholarly journalUsage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC