Deakin University
Browse

File(s) under permanent embargo

Agree to disagree: on labelling helpful app reviews

conference contribution
posted on 2016-01-01, 00:00 authored by Andrew Simmons, Leonard HoonLeonard Hoon
Mobile apps designers seek to prioritise and refine app features so as to optimise user experience across the ensemble of possible situations and contexts in which the app is used. App reviews---some helpful, others irrelevant---can be analysed for feedback on this user experience. However, few studies have specifically examined the helpfulness of app reviews. In this paper, we surveyed users and developers to rate 167 reviews for helpfulness, obtaining a total of 2,558 helpfulness ratings captured on a 5 point Likert scale. We found only slight agreement (nominal Krippendorff's alpha = 0.039) between participants on the helpfulness of reviews. Differences between reviews become evident when we summarise all the helpfulness ratings per review. We conclude that the disagreement among users limits the potential of mobile app review recommender systems.

History

Event

Computer-Human Interaction Special Interest Group. Conference (28th : 2016 : Launceston, Tas.)

Series

Computer-Human Interaction Special Interest Group Conference

Pagination

416 - 420

Publisher

Association for Computing Machinery

Location

Launceston, Tas.

Place of publication

New York, N.Y.

Start date

2016-11-29

End date

2016-12-02

ISBN-13

978-1-4503-4618-4

Language

eng

Publication classification

E Conference publication; E1 Full written paper - refereed

Copyright notice

2016, ACM

Editor/Contributor(s)

C Parker

Title of proceedings

OzCHI 2016 : Connected futures : Proceedings of the 28th Australian Conference on Computer-Human Interaction