(COLUMBUS, Ohio) – In a report published in Pediatric Critical Care Medicine, a team from Nationwide Children's Hospital describes a machine learning tool for timely identification of hospitalized children at risk for deterioration – a worsening clinical condition with increased risk of morbidity and mortality. The tool, which is trained on disease-specific groups, promisingly outperformed the existing situational awareness program in identifying at-risk children.
“Predictive algorithms focused on improving clinical care have been increasingly developed over the years, but the vast majority are not operationalized. Transitioning the algorithm from the computer to the bedside can be a long process and requires engagement and collaboration from clinicians, data scientists and clinical informaticists,” says Laura Rust, MD, emergency medicine physician and physician informaticist at Nationwide Children’s and lead author of the paper. “This project has been a 5-plus year journey, and we are really proud of the successful integration within our safety culture and the impact on patient outcomes.”
The Deterioration Risk Index (DRI) was built off the foundation of the Watchstander situational awareness program, already in use at Nationwide Children’s. To promote adoption, the team utilized the same response mechanisms for alerts – patient assessment and huddle with the bedside care team within 30 minutes, risk mitigation and development of an escalation plan.
The three diagnostic groups – structural heart defect (cardiac), oncology (malignancy), general (neither cardiac nor malignancy) – were used to train three separate predictive models to develop the algorithms implemented.
“One of the design features that helped build trust with clinical teams is that we didn’t necessarily identify any new criteria. Our model simply identifies which existing situational awareness criteria are most important and weighs them accordingly,” says Tyler Gorham, data scientist in IT Research & Innovation at Nationwide Children’s and co-author of the publication.
According to Dr. Rust, there can be an overwhelming amount of clinical data within the electronic health record to process at any one time, especially after handoffs or transitions of care. The model helps to relieve this cognitive burden by automatically processing these risk criteria behind the scenes. Because it is integrated within the electronic medical record (EMR), it has the benefit of having all the data from all previous points of time – not just the current shift.
The DRI was 2.4 times as sensitive as the existing situational awareness program while also requiring 2.3 times fewer alarms per detected event. Notably, the team observed a four-fold sensitivity gain for the cardiac group and a three-fold gain for the malignancy group. The pilot study after implementation found that over the first 18 months, deterioration events decreased 77% compared to expected event rates in preceding years.
Perhaps the most important aspect of the model, according to the developers, is its transparency.
“This is not a black box. We show clinicians what goes in and how the algorithm evaluates the data to trigger alarms,” says Gorham. “The tool helps support clinical decision making because the clinical team is able to see why an alarm was triggered.”
The team also conducted road shows, visiting clinical units where the tool would be deployed, answering questions, doing simulations with the bedside care teams and incorporating feedback.
“At Nationwide Children’s, our teams are committed to a Zero Hero safety culture,” says Dr. Rust. “This provided the foundation and shared mission for our multidisciplinary team to see this across the finish line.”
More information, including details about the algorithm, is available in the publication.
“We shared our recipe in the publication,” says Gorham. “If others are interested, they could use their center’s data and retrain the model for their local populations. If we can share our success with others, we can hopefully support better outcomes for all children, even those who aren’t under our care.”
Citation: Rust LOH, Gorham TJ, Bambach S, Bode RS, Maa T, Hoffman JM, Rust SW. The deterioration risk index: Developing and piloting a machine learning algorithm to reduce pediatric inpatient deterioration. Pediatric Clinical Care Medicine. 2023;24(4).