Tags

Type your tag names separated by a space and hit enter

A comparison of the performance of a model based on administrative data and a model based on clinical data: effect of severity of illness on standardized mortality ratios of intensive care units.
Crit Care Med. 2012 Feb; 40(2):373-8.CC

Abstract

OBJECTIVES

It has been postulated that prognostic models based on administrative data can provide valid adjusted mortality rates in specific patient populations. In this study we compared the performance and robustness of a model based on administrative data (customized hospital standardized mortality ratio) and a model based on clinical data (customized Simplified Acute Physiology Score II) in the Dutch intensive care unit population.

DESIGN

Cohort study of intensive care unit records from a national intensive care unit quality registry linked to administrative records from the Dutch National Medical Registration. The hospital standardized mortality ratio and Simplified Acute Physiology Score II models were first-level customized on the intensive care unit population.

SETTING

Fifty-five Dutch intensive care units.

PATIENTS

A total of 66,564 intensive care unit patients admitted from 2005 to 2008.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Performance expressed by measures of discrimination, accuracy, and calibration (area under the receiver operating characteristic curve, Brier score, Hosmer-Lemeshow Ĉ-statistic, and calibration plots). Additionally, the robustness of the models was assessed by simulating changes in the population's severity of illness and analyzing the effect on the intensive care units' standardized mortality ratios.The area under the receiver operating characteristic curve and Brier score of the customized Simplified Acute Physiology Score II were significantly superior to that of the customized hospital standardized mortality ratio (0.85 and 0.11 vs. 0.77 and 0.13, respectively). Calibration plots showed good agreement between observed and predicted mortality for low-risk patients in both models, with more discrepancy in the high-risk patients when using the customized hospital standardized mortality ratio. Severity of illness had influence on the intensive care units' standardized mortality ratios in both models, but the customized Simplified Acute Physiology Score II showed more robustness.

CONCLUSIONS

The customized Simplified Acute Physiology Score II outperforms the customized hospital standardized mortality ratio in the Dutch intensive care unit population. Comparing institutions based on standardized mortality ratios can be unfavorable for those with a more severely ill intensive care unit population, especially when using the customized hospital standardized mortality ratio.

Authors+Show Affiliations

Department of Medical Informatics, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands. s.brinkman@amc.uva.nlNo affiliation info availableNo affiliation info availableNo affiliation info availableNo affiliation info available

Pub Type(s)

Comparative Study
Journal Article
Multicenter Study

Language

eng

PubMed ID

21983367

Citation

Brinkman, Sylvia, et al. "A Comparison of the Performance of a Model Based On Administrative Data and a Model Based On Clinical Data: Effect of Severity of Illness On Standardized Mortality Ratios of Intensive Care Units." Critical Care Medicine, vol. 40, no. 2, 2012, pp. 373-8.
Brinkman S, Abu-Hanna A, van der Veen A, et al. A comparison of the performance of a model based on administrative data and a model based on clinical data: effect of severity of illness on standardized mortality ratios of intensive care units. Crit Care Med. 2012;40(2):373-8.
Brinkman, S., Abu-Hanna, A., van der Veen, A., de Jonge, E., & de Keizer, N. F. (2012). A comparison of the performance of a model based on administrative data and a model based on clinical data: effect of severity of illness on standardized mortality ratios of intensive care units. Critical Care Medicine, 40(2), 373-8. https://doi.org/10.1097/CCM.0b013e318232d7b0
Brinkman S, et al. A Comparison of the Performance of a Model Based On Administrative Data and a Model Based On Clinical Data: Effect of Severity of Illness On Standardized Mortality Ratios of Intensive Care Units. Crit Care Med. 2012;40(2):373-8. PubMed PMID: 21983367.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - A comparison of the performance of a model based on administrative data and a model based on clinical data: effect of severity of illness on standardized mortality ratios of intensive care units. AU - Brinkman,Sylvia, AU - Abu-Hanna,Ameen, AU - van der Veen,André, AU - de Jonge,Evert, AU - de Keizer,Nicolette F, PY - 2011/10/11/entrez PY - 2011/10/11/pubmed PY - 2012/3/13/medline SP - 373 EP - 8 JF - Critical care medicine JO - Crit Care Med VL - 40 IS - 2 N2 - OBJECTIVES: It has been postulated that prognostic models based on administrative data can provide valid adjusted mortality rates in specific patient populations. In this study we compared the performance and robustness of a model based on administrative data (customized hospital standardized mortality ratio) and a model based on clinical data (customized Simplified Acute Physiology Score II) in the Dutch intensive care unit population. DESIGN: Cohort study of intensive care unit records from a national intensive care unit quality registry linked to administrative records from the Dutch National Medical Registration. The hospital standardized mortality ratio and Simplified Acute Physiology Score II models were first-level customized on the intensive care unit population. SETTING: Fifty-five Dutch intensive care units. PATIENTS: A total of 66,564 intensive care unit patients admitted from 2005 to 2008. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Performance expressed by measures of discrimination, accuracy, and calibration (area under the receiver operating characteristic curve, Brier score, Hosmer-Lemeshow Ĉ-statistic, and calibration plots). Additionally, the robustness of the models was assessed by simulating changes in the population's severity of illness and analyzing the effect on the intensive care units' standardized mortality ratios.The area under the receiver operating characteristic curve and Brier score of the customized Simplified Acute Physiology Score II were significantly superior to that of the customized hospital standardized mortality ratio (0.85 and 0.11 vs. 0.77 and 0.13, respectively). Calibration plots showed good agreement between observed and predicted mortality for low-risk patients in both models, with more discrepancy in the high-risk patients when using the customized hospital standardized mortality ratio. Severity of illness had influence on the intensive care units' standardized mortality ratios in both models, but the customized Simplified Acute Physiology Score II showed more robustness. CONCLUSIONS: The customized Simplified Acute Physiology Score II outperforms the customized hospital standardized mortality ratio in the Dutch intensive care unit population. Comparing institutions based on standardized mortality ratios can be unfavorable for those with a more severely ill intensive care unit population, especially when using the customized hospital standardized mortality ratio. SN - 1530-0293 UR - https://www.unboundmedicine.com/medline/citation/21983367/A_comparison_of_the_performance_of_a_model_based_on_administrative_data_and_a_model_based_on_clinical_data:_effect_of_severity_of_illness_on_standardized_mortality_ratios_of_intensive_care_units_ L2 - https://dx.doi.org/10.1097/CCM.0b013e318232d7b0 DB - PRIME DP - Unbound Medicine ER -