Recalibration and validation of a preoperative risk prediction model for mortality in major colorectal surgery

BACKGROUND: In 2009, Barwon Health designed a risk stratification model for mortality in major colorectal surgery with the use of only preoperative risk factors. The Barwon Health 2009 model was shown to predict mortality reliably, and it was comparable to other models, such as the original, POSSUM. However, the Barwon Health 2009 model was never validated with data other than those used to develop the model. OBJECTIVE: The aim of this study was to perform temporal and external validation of the Barwon Health 2009 model and to compare it with other published models. DESIGN: : The temporal validation was a prospective observational study, whereas the external validation was a retrospective observational study. The discrimination and calibration of the models were assessed by using the area under receiver operator characteristic and χ test of Hosmer-Lemeshow goodness-of-fi technique. SETTINGS: This is a multi-institutional study. Data were collected from 2008 to 2010. RESULTS: There were 474 major colorectal cases at Geelong Hospital (temporal validation) and 389 cases at Western Hospital (external validation). The overall mortality rate was 5.10% and 1.03%. In the comparison of the 2 demographics, Geelong Hospital had a higher proportion of patients who were older and had higher ASA scores and comorbidity counts, whereas Western Hospital surgeons were operating on a higher number of urgent cases. Despite the differences, the Barwon Health 2009 model was able to discriminate mortality reliably (area under receiver operator characteristic = 0.753) but had poor model calibration (p < 0.001) on temporal validation. Hence, the model was recalibrated to predict mortality accurately(area under receiver operator characteristic = 0.772; p = 0.83), and this was successfully validated at Western Hospital (area under receiver operator characteristic = 0.788; p = 0.24). CONCLUSIONS: We have developed a model that can accurately predict mortality after major colorectal surgery by using only data that are available preoperatively. After recalibration, the model was successfully validated in a second hospital.