Standard setting in specific-purpose language testing: what can a qualitative study add?

Manias, Elizabeth and McNamara, Tim 2016, Standard setting in specific-purpose language testing: what can a qualitative study add?, Language testing, vol. 33, no. 2, pp. 235-249, doi: 10.1177/0265532215608411.

Attached Files
Name Description MIMEType Size Downloads

Title Standard setting in specific-purpose language testing: what can a qualitative study add?
Author(s) Manias, ElizabethORCID iD for Manias, Elizabeth
McNamara, Tim
Journal name Language testing
Volume number 33
Issue number 2
Start page 235
End page 249
Total pages 15
Publisher Sage
Place of publication London, Eng.
Publication date 2016-04
ISSN 0265-5322
Keyword(s) healthcare communication
language for specific purpose testing
occupational English test
role play
standard setting
Social Sciences
Language & Linguistics
language for specific purposes testing
Summary This paper explores the views of nursing and medical domain experts in considering the standards for a specific-purpose English language screening test, the Occupational English Test (OET), for professional registration for immigrant health professionals. Since individuals who score performances in the test setting are often language experts rather than domain experts, there are possible tensions between what is being measured by a language test and what is deemed important by domain experts. Another concern is a lack of qualitative research on the process of the standard setting. To date, no published qualitative work has been identified about the contributions of domain experts in the standard setting for healthcare communication. In this study, a standard-setting exercise was conducted for the speaking component of the OET, using judgements of nursing and medical clinical educators and supervisors. In all, 13 medical and 18 nursing clinical educators and supervisors rated medical and nursing candidate performances respectively. These performances were audio-recorded OET role-plays that were selected across a range of proficiency levels. Domain experts were invited to comment on the basis of their decisions and the extent of alignment between these decisions and the criteria used to assess performance on the OET. Nursing and medical domain experts showed that they attended to all of the OET criteria in making their decisions about standards. However, clinical scenario simulation also invited judgements of clinical competence from participants, even where they knew that clinical competence should be excluded from their decision-making. Another concern related to the authenticity limitations of the role-play tasks as evidence of readiness to handle communication in the workplace. Overall, findings support the value of qualitative evidence from the standard setting in providing insight into the factors informing and impeding decision-making.
Language eng
DOI 10.1177/0265532215608411
Field of Research 111099 Nursing not elsewhere classified
Socio Economic Objective 920210 Nursing
HERDC Research category C1 Refereed article in a scholarly journal
ERA Research output type C Journal article
Copyright notice ©2015, The Authors
Persistent URL

Connect to link resolver
Unless expressly stated otherwise, the copyright for items in DRO is owned by the author, with all rights reserved.

Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 2 times in TR Web of Science
Scopus Citation Count Cited 3 times in Scopus
Google Scholar Search Google Scholar
Access Statistics: 264 Abstract Views, 2 File Downloads  -  Detailed Statistics
Created: Mon, 22 Aug 2016, 15:06:26 EST

Every reasonable effort has been made to ensure that permission has been obtained for items included in DRO. If you believe that your rights have been infringed by this repository, please contact