Deploying AI on a national scale, especially in critical industries like healthcare, is no basic feat.
In HIMSS24 APAC session titled “Implementing AI-powered Triage for National Virtual and Digital Front Doors,” Dr. Nirvana Luckraj, Medical Director of Healthdirect, described how she selected, tested and implemented an AI-powered clinical decision support system to improve the service’s national hotline.
Healthdirect offers a range of free 24/7 helpline services, including nurse triage and GP telecare. It has dedicated lines for pregnant people, the elderly, and recently also people struggling with mental problems. In addition, Healthdirect provides information on medicines and curated health content on its website, as well as online booking and referral options that integrate directly with national care pathways, including My Health Record.
PROBLEM
The acquisition of CDSS is part of the actions taken by Healthdirect as it recently migrated its technology stack to its own operating platform.
The tool required many symptoms to be addressed at the outset. “We needed a CDSS that could tolerate and analyze multiple symptoms. We also wanted it to reflect a real-world consultation that took into account risk factors such as predisposing factors, chronic history and travel history.”
“We also needed something next-generation that would provide more comprehensive and accurate patient selection,” Dr. Luckraj added.
Previously, the service used a CDSS based on a decision tree algorithm that required “extended maintenance” and “could only tolerate a single symptom input.”
APPLICATION
The organization then went to the market to look for an AI-based CDSS provider. Initially, it selected five global suppliers whose offers were clinically tested.
To test the applicability of an AI system to Australia’s public health system, the Healthdirect team worked with AI experts from the Australian Institute of Health Innovation at Macquarie University. They adopted testing methodology from a recent study that used case vignettes to test the accuracy of the CDSS.
“We ran these vignettes through CDSS and found that the performance was consistent with a recently published study. We ran the same vignette through our old CDSS for comparison and found that the old and new CDSS produced similar results, but the new one was more accurate,” Dr. Luckraj noted.
Ultimately, Healthdirect chose AI CDSS from the Polish company Infermedica, which works based on a probabilistic Bayesian inference engine that takes into account symptoms, risk factors and diseases, an updated medical knowledge base and an API interface that connects to CRM.
MEETING THE CHALLENGE
Immediately after launching the up-to-date CDSS on the hotline in March 2023, the implementation team encountered resistance from staff. “We underestimated the change management required as healthcare workers began using new technologies to perform old tasks,” Dr. Luckraj admitted.
From entering one symptom into the CDSS, nurses now have to enter multiple patient symptoms. “Nurses were initially frustrated with this tool because it seemed to ask questions that were seemingly unrelated to the patient’s presentation. They also had to establish their past history to determine their individual risk factors, and the questions also seemed more advanced than the previous ones.”
Dealing with the nurses’ initial frustration, the team took their comments into account and helped them adapt to the up-to-date tool. According to Dr. Luckraj, one of the key areas for improvement was collaboration with the provider on a feedback loop mechanism in which nurses do not have to agree with every CDSS result.
“It’s a support system, not a diagnostic system. Nurses can therefore improve or lower triage scores by providing a reason for changing disposition. We also looked at flows where nurses disagreed with the CDSS results to continually improve the tool.”
RESULTS
One year after CDSS improvement, the implementation team observed that nurses changed triage results less, agreed more with CDSS results, and were more satisfied with the up-to-date user interface. Dr. Luckraj noted that the tool’s accuracy continues to improve as nurses also continue to provide feedback. On the consumer side, no grave clinical events attributable to CDSS were reported; are now more satisfied with the services, and their satisfaction level is the highest in three years.
Last December, the same AI-powered CDSS was rolled out to Healthdirect’s online symptom checker available to consumers, making it a “digital front door to a multitude of alternative care options, including direct access to urgent care services and virtual emergency departments “.
“In the future, we also want to provide consumers with information tailored to their symptoms to provide them with an even better experience.” Doctor Luckraj shared.
Since incorporating AI into its helpline, Healthdirect says half of emergency calls are being diverted to non-urgent services. From July 2023 to June 2024, it made 44,000 calls to the virtual emergency department. It has also booked almost 5,500 people online for GP-led urgent care services. Meanwhile, almost 350,000 people have been given advice on how to manage their symptoms safely at home. Overall, Healthdirect nurses have helped book over 370,000 GP appointments.