Deakin University

File(s) under permanent embargo

Scaleplus: towards fast scaling of distributed streaming dataflows

conference contribution
posted on 2020-12-01, 00:00 authored by Z Zong, L Wen, Xiao LiuXiao Liu, L Lin, C Qian
Streaming dataflows are usually deployed as longterm services and suffer from the fluctuation in arrival rates. Many scaling controllers provide the elasticity to scale up or down the dataflow to meet the target throughput. However, for most controllers with specific target Service Level Objective (SLO), it is difficult to use a single decision to accurately complete a scaling action. Reaching the target iteratively will increase the completion time of the scaling, which leads to the performance degeneration or the wasting of resources. In this paper, we present Scaleplus, which provides pluggable services for helping iterative scaling controllers make more accurate decisions to complete the scaling rapidly. Scaleplus builds decision models incrementally without the off-line sampling procedure, which can be used out of the box, then a recommendation strategy is leveraged to recommend accurate decisions. Besides, Scaleplus can be flexibly integrated with only 3 HTTP APIs. We evaluate Scaleplus with 3 different scaling controllers on Apache Flink and Apache Heron. For the simulated 9-day traces of Twitter, Scaleplus reduces the scaling duration by 36.5%, 51.1% and 54% respectively compared with DS2, mRB and Dhalion.



IEEE Computer Society. International Conference (2020 : Exeter, England)


IEEE Computer Society International Conference


260 - 268


Institute of Electrical and Electronics Engineers


Exeter, England

Place of publication

Piscataway, N.J.

Start date


End date






Publication classification

E1 Full written paper - refereed


J Hu, G Min, N Georgalas, Z Zhao, F Hao, W Miao

Title of proceedings

ISPA-BDCloud-SocialCom-SustainCom 2020 : Proceedings of the 2020 IEEE International Symposium on Parallel and Distributed Processing with Applications, 2020 IEEE International Conference on Big Data and Cloud Computing, 2020 IEEE International Symposium on Social Computing and Networking and 2020 IEEE International Conference on Sustainable Computing and Communications