Optimizing microtask assignment on crowdsourcing platforms using Markov chain Monte Carlo

Moayedikia, Alireza, Ghaderi, Hadi and Yeoh, William 2020, Optimizing microtask assignment on crowdsourcing platforms using Markov chain Monte Carlo, Decision Support Systems, vol. 139, doi: 10.1016/j.dss.2020.113404.

Attached Files
Name Description MIMEType Size Downloads

Title Optimizing microtask assignment on crowdsourcing platforms using Markov chain Monte Carlo
Author(s) Moayedikia, Alireza
Ghaderi, Hadi
Yeoh, WilliamORCID iD for Yeoh, William orcid.org/0000-0002-2964-4518
Journal name Decision Support Systems
Volume number 139
Article ID 113404
Total pages 10
Publisher Elsevier
Place of publication Amsterdam, The Netherlands
Publication date 2020-12
ISSN 0167-9236
Keyword(s) crowdsourcing
task assignment
Markov Chain
crowd labeling
quality estimation
Summary Microtasking is a type of crowdsourcing, denoting the act of breaking a job into several tasks and allocating them to multiple workers to complete. The assignment of tasks to workers is a complex decision-making process, particularly when considering budget and quality constraints. While there is a growing body of knowledge on the development of task assignment algorithms, the current algorithms suffer from shortcomings including: after-worker quality estimation, meaning that workers need to complete all tasks after which point their quality can be estimated; and one-off quality estimation method which estimates workers' quality only at the start of microtasking using a set of pre-defined quality-control tasks. To address these shortcomings, we propose a Markov Chain Monte Carlo–based task assignment approach known as MCMC-TA which provides iterative estimations of workers' quality and dynamic task assignment. Specifically, we apply Gaussian mixture model (GMM) to estimate workers' quality and Markov Chain Monte Carlo to shortlist workers for task assignment. We use Google Fact Evaluation dataset to measure the performance of MCMC-TA and compare it against the state-of-the-art algorithms in terms of AUC and F-Score. The results show that the proposed MCMC-TA algorithm not only outperforms the rival algorithms, but also offers a spammer-resistant result that maximizes the learning of workers' quality with minimal budget.
Language eng
DOI 10.1016/j.dss.2020.113404
Indigenous content off
Field of Research 080605 Decision Support and Group Support Systems
01 Mathematical Sciences
08 Information and Computing Sciences
15 Commerce, Management, Tourism and Services
Socio Economic Objective 899999 Information and Communication Services not elsewhere classified
HERDC Research category C1 Refereed article in a scholarly journal
ERA Research output type C Journal article
Copyright notice ©2020, Elsevier
Persistent URL http://hdl.handle.net/10536/DRO/DU:30143240

Connect to link resolver
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in TR Web of Science
Scopus Citation Count Cited 0 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 78 Abstract Views, 1 File Downloads  -  Detailed Statistics
Created: Sat, 07 Nov 2020, 11:47:09 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact drosupport@deakin.edu.au.