Deakin University
Browse

A collaborative comparison of objective structured clinical examination (OSCE) standard setting methods at Australian medical schools

Version 2 2024-06-03, 13:46
Version 1 2017-10-19, 14:44
journal contribution
posted on 2024-06-03, 13:46 authored by Bunmi Sherifat Malau-Aduli, Peta-Ann Teague, Karen D'SouzaKaren D'Souza, Clare Heal, Richard Turner, David L Garne, Cees van der Vleuten
BACKGROUND: A key issue underpinning the usefulness of the OSCE assessment to medical education is standard setting, but the majority of standard-setting methods remain challenging for performance assessment because they produce varying passing marks. Several studies have compared standard-setting methods; however, most of these studies are limited by their experimental scope, or use data on examinee performance at a single OSCE station or from a single medical school. This collaborative study between 10 Australian medical schools investigated the effect of standard-setting methods on OSCE cut scores and failure rates. METHODS: This research used 5256 examinee scores from seven shared OSCE stations to calculate cut scores and failure rates using two different compromise standard-setting methods, namely the Borderline Regression and Cohen's methods. RESULTS: The results of this study indicate that Cohen's method yields similar outcomes to the Borderline Regression method, particularly for large examinee cohort sizes. However, with lower examinee numbers on a station, the Borderline Regression method resulted in higher cut scores and larger difference margins in the failure rates. CONCLUSION: Cohen's method yields similar outcomes as the Borderline Regression method and its application for benchmarking purposes and in resource-limited settings is justifiable, particularly with large examinee numbers.

History

Journal

MEDICAL TEACHER

Volume

39

Pagination

1261-1267

Location

England

ISSN

0142-159X

eISSN

1466-187X

Language

English

Publication classification

C Journal article, C1 Refereed article in a scholarly journal

Copyright notice

2017, Informa UK [Taylor & Francis]

Issue

12

Publisher

TAYLOR & FRANCIS LTD