Influence of task properties and subjectivity on consistency of triage: a simulation study

Aim. This paper reports a study to determine nurses' levels of agreement using a standard 5-point triage scale and to explore the influence of task properties and subjectivity on decision-making consistency. Background. Triage scales are used to define time-to-treatment in hospital emergency departments. Studies of the inter-rater reliability of these scales using paper-based simulation methods report varying levels of consistency. Understanding how various components of the decision task and individual perceptions of the case influence agreement is critical to the development of strategies to improve consistency of triage. Method. Simulations were constructed from naturalistic observation, cue types and frequencies were classified. Data collection was conducted in 2002, and the final response rate was 41.3%. Participants were asked to allocate an urgency code for 12 scenarios using the Australasian Triage Scale, and provide estimates of case complexity, levels of certainty and available information. Data were analysed descriptively, agreement between raters was calculated using kappa. The influence of task properties and participants' subjective estimates of case complexity, levels of certainty and available information on agreement were explored using a general linear model. Findings. Agreement among raters varied from moderate to poor (K = 0.18-0.64). Participants' subjective estimates of levels of available information were found to influence consistency of triage by statistically significant amounts (F 5.68; ≤0.01). Conclusions. Strategies employed to optimize consistency of triage should focus on improving the quality of the simulations that are used. In particular, attention should be paid to the development of interactive simulations that will accommodate individual differences in information-seeking behaviour.