Scaleplus: towards fast scaling of distributed streaming dataflows

Zong, Zan, Wen, Lijie, Liu, Xiao, Lin, Li, Qian, Chen and Lin, Leilei 2020, Scaleplus: towards fast scaling of distributed streaming dataflows, in ISPA-BDCloud-SocialCom-SustainCom 2020 : Proceedings of the 2020 IEEE International Symposium on Parallel and Distributed Processing with Applications, 2020 IEEE International Conference on Big Data and Cloud Computing, 2020 IEEE International Symposium on Social Computing and Networking and 2020 IEEE International Conference on Sustainable Computing and Communications, Institute of Electrical and Electronics Engineers, Piscataway, N.J., pp. 260-268, doi: 10.1109/ISPA-BDCloud-SocialCom-SustainCom51426.2020.00058.

Attached Files
Name Description MIMEType Size Downloads

Title Scaleplus: towards fast scaling of distributed streaming dataflows
Author(s) Zong, Zan
Wen, Lijie
Liu, XiaoORCID iD for Liu, Xiao orcid.org/0000-0001-8400-5754
Lin, Li
Qian, Chen
Lin, Leilei
Conference name IEEE Computer Society. International Conference (2020 : Exeter, England)
Conference location Exeter, England
Conference dates 2020/12/17 - 2020/12/19
Title of proceedings ISPA-BDCloud-SocialCom-SustainCom 2020 : Proceedings of the 2020 IEEE International Symposium on Parallel and Distributed Processing with Applications, 2020 IEEE International Conference on Big Data and Cloud Computing, 2020 IEEE International Symposium on Social Computing and Networking and 2020 IEEE International Conference on Sustainable Computing and Communications
Editor(s) Hu, J
Min, G
Georgalas, N
Zhao, Z
Hao, F
Miao, W
Publication date 2020
Series IEEE Computer Society International Conference
Start page 260
End page 268
Total pages 9
Publisher Institute of Electrical and Electronics Engineers
Place of publication Piscataway, N.J.
Keyword(s) dataflow
auto-scaling
streaming
pluggable
Summary Streaming dataflows are usually deployed as longterm services and suffer from the fluctuation in arrival rates. Many scaling controllers provide the elasticity to scale up or down the dataflow to meet the target throughput. However, for most controllers with specific target Service Level Objective (SLO), it is difficult to use a single decision to accurately complete a scaling action. Reaching the target iteratively will increase the completion time of the scaling, which leads to the performance degeneration or the wasting of resources. In this paper, we present Scaleplus, which provides pluggable services for helping iterative scaling controllers make more accurate decisions to complete the scaling rapidly. Scaleplus builds decision models incrementally without the off-line sampling procedure, which can be used out of the box, then a recommendation strategy is leveraged to recommend accurate decisions. Besides, Scaleplus can be flexibly integrated with only 3 HTTP APIs. We evaluate Scaleplus with 3 different scaling controllers on Apache Flink and Apache Heron. For the simulated 9-day traces of Twitter, Scaleplus reduces the scaling duration by 36.5%, 51.1% and 54% respectively compared with DS2, mRB and Dhalion.
ISBN 9781665414852
Language eng
DOI 10.1109/ISPA-BDCloud-SocialCom-SustainCom51426.2020.00058
Indigenous content off
HERDC Research category E1 Full written paper - refereed
Persistent URL http://hdl.handle.net/10536/DRO/DU:30152894

Connect to link resolver
 
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in TR Web of Science
Scopus Citation Count Cited 0 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 12 Abstract Views, 5 File Downloads  -  Detailed Statistics
Created: Mon, 28 Jun 2021, 15:31:31 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.