Minimal Impact of Implemented Early Warning Score and Best Practice Alert for Patient Deterioration

Objectives: Previous studies have looked at National Early Warning Score performance in predicting in-hospital deterioration and death, but data are lacking with respect to patient outcomes following implementation of National Early Warning Score. We sought to determine the effectiveness of National Early Warning Score implementation on predicting and preventing patient deterioration in a clinical setting. Design: Retrospective cohort study. Setting: Tertiary care academic facility and a community hospital. Patients: Patients 18 years old or older hospitalized from March 1, 2014, to February 28, 2015, during preimplementation of National Early Warning Score to August 1, 2015, to July 31, 2016, after National Early Warning Score was implemented. Interventions: Implementation of National Early Warning Score within the electronic health record and associated best practice alert. Measurements and Main Results: In this study of 85,322 patients (42,402 patients pre-National Early Warning Score and 42,920 patients post-National Early Warning Score implementation), the primary outcome of rate of ICU transfer or death did not change after National Early Warning Score implementation, with adjusted hazard ratio of 0.94 (0.84–1.05) and 0.90 (0.77–1.05) at our academic and community hospital, respectively. In total, 175,357 best practice advisories fired during the study period, with the best practice advisory performing better at the community hospital than the academic at predicting an event within 12 hours 7.4% versus 2.2% of the time, respectively. Retraining National Early Warning Score with newly generated hospital-specific coefficients improved model performance. Conclusions: At both our academic and community hospital, National Early Warning Score had poor performance characteristics and was generally ignored by frontline nursing staff. As a result, National Early Warning Score implementation had no appreciable impact on defined clinical outcomes. Refitting of the model using site-specific data improved performance and supports validating predictive models on local data. Drs. O'Brien and Goldstein are co-senior authors. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's website (https://ift.tt/29S62lw). Supported, in part, by grants from National Institute of Diabetes and Digestive and Kidney Diseases & Allergy and Infectious Diseases; Health and Human Services (HHS)|National Institutes of Health (NIH)|National Institute of Allergy and Infectious Diseases: T32-AI007392 (to Dr. Clement); HHS|NIH|National Institute of Diabetes and Digestive and Kidney Diseases: K25-DK097279 and Duke Center for Integrative Health (to Dr. Goldstein); and Duke Center for Integrative Health (to Drs. Steorts and O'Brien). The funding bodies had no role in the study's design, conduct, review, or reporting or the decision to submit the article for publication. Drs. Bedoya, Clement, and Goldstein received support for article research from the National Institutes of Health. Dr. Clement received funding from UpToDate (royalties). Dr. Steorts received funding from employment with U.S. Census Bureau; she holds an National Science Foundation (NSF) Career grant and NSF privacy grant (both not related to this study); and she received support for article research from a seed grant from Duke University. The remaining authors have disclosed that they do not have any potential conflicts of interest. For information regarding this article, E-mail: armando.bedoya@duke.edu Copyright © by 2018 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.

https://ift.tt/2I8SWNk