Does calibration reduce variability in the assessment of accounting learning outcomes?

O'Connell, Brendan, De Lange, Paul, Freeman, Mark, Hancock, Phil, Abraham, Anne, Howieson, Bryan and Watty, Kim 2016, Does calibration reduce variability in the assessment of accounting learning outcomes?, Assessment and evaluation in higher education, vol. 41, no. 3, pp. 331-349, doi: 10.1080/02602938.2015.1008398.

Attached Files
Name Description MIMEType Size Downloads

Title Does calibration reduce variability in the assessment of accounting learning outcomes?
Author(s) O'Connell, Brendan
De Lange, Paul
Freeman, Mark
Hancock, Phil
Abraham, Anne
Howieson, Bryan
Watty, KimORCID iD for Watty, Kim
Journal name Assessment and evaluation in higher education
Volume number 41
Issue number 3
Start page 331
End page 349
Total pages 19
Publisher Routledge
Place of publication Abingdon, Eng.
Publication date 2016
ISSN 0260-2938
Keyword(s) accounting education
Summary Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence.
Language eng
DOI 10.1080/02602938.2015.1008398
Field of Research 130203 Economics, Business and Management Curriculum and Pedagogy
Socio Economic Objective 930301 Assessment and Evaluation of Curriculum
HERDC Research category C1 Refereed article in a scholarly journal
ERA Research output type C Journal article
Copyright notice ©2015, Taylor & Francis
Persistent URL

Document type: Journal Article
Collections: Faculty of Business and Law
School of Accounting, Economics and Finance
Connect to link resolver
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 7 times in TR Web of Science
Scopus Citation Count Cited 9 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 522 Abstract Views, 4 File Downloads  -  Detailed Statistics
Created: Mon, 16 Mar 2015, 08:16:38 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact