In the Children’s Hospital in Seattle, a enormous clinical obstacle revolved around the effective access and operate of the constantly developing universe of information based on evidence in an appropriate and contextual way. While the hospital has always prioritized the latest care, the volume and rapid evolution of medical knowledge were a significant challenge.
Challenge
“Our devoted pediatricians must navigate the landscape in which medical literature doubles every few months, and all this trying to critically assess the rigor and importance of every new study for the unique needs of their patients,” said Dr. Clara Lin, vice president and medical information director at Seattle Children’s Hospital. “The need to stay up to date from the latest discoveries, combined with the pressure of the growing pediatrician deficiency and more and more convoluted cases of patients, emphasized the urgency of improving the problem.
“Our involvement in the practice based on evidence led to the establishment of a clinical effectiveness program in 2010,” she continued. “This important initiative, currently conducted by Dr. Darrena Migita in our Patient Quality and Safety Center, includes a significant investment in time and specialist knowledge from many suppliers and scholars.”
Together, they meticulously review medical literature to develop and maintain clinical standard work paths or CSWS for over 70 diagnoses. These CSW proved to be invaluable in improving patients’ results and are accepted by healthcare workers around the world.
“However, even with these well-cut paths, the volume of information-thousands of text pages, images, block diagrams and tables representing the collective wisdom of our experts-can still be an intensive search process for the occupied clinicist looking for specific tips on a specific patient’s script,” she explained.
“The appearance of generative artificial intelligence was an exciting opportunity to examine the potential change of paradigm in the field of access and use of this key information,” she continued. “We have seen the potential to make models of large languages act as powerful tools, able to search our extensive CSW knowledge base with extraordinary speed and precision.”
The basic challenge was to determine whether this technology could reach the level of accuracy and reliability required by clinical health care. Can the AI agent effectively and effectively separate the precise, based on the evidence of observations, which the supplier needs at a key decision point, thus reducing the cognitive load and time on searching, and ultimately improve the provision of optimal patient care?
This fundamental question led to work for children in Seattle from Google to develop and strict testing of this technology.
APPLICATION
The proposal focused on using the AI agent based on LLM to act as an knowledgeable and competent navigator through the extensive repository of CSW paths. The staff predicted the agent as a animated interface that is able to understand the questions of the natural language from the healthcare providers, and then quickly indicate the most appropriate information in extensive path documentation.
“Instead of a clinicist who had to manually sieve through dozens of pages in one trail, AI would serve as a targeted search engine, capable of bringing specific answers based on the question of the question,” explained Lin.
“The key to alleviating the challenge of access to information concerned the agent’s ability to process and understand the complicated structure and content of our CSW paths, which includes text, images, tables and block diagrams,” she said. “Each path meticulously describes the normalized care of a specific condition, including each stage from the initial presentation and diagnostic work to the full treatment and observation plan.”
The staff offered that the AI agent would be refined to recognize various sections and decision points on these paths, enabling the contextualization of the user’s questions in the overall flow of clinical work.
“In addition, we expected that AI agent was involved in conversation dialogue with the user to ensure thorough search for information,” Lin noted. “If the initial query from the supplier lacked sufficient details to precisely locate the appropriate section on the path, the agent would proactively ask explanatory questions.
“This iterative process would help narrow the search and ensure that the information provided would be highly specific to the paths,” she continued. “By imitating the conversation with the” virtual consultant “, the agent intended to provide more intuitive and efficient experience compared to traditional key/text searches, ultimately saving valuable time and reducing the cognitive load of our healthcare providers when he was looking for guidelines based on evidence.”
Fulfillment of the challenge
To solve the challenge, Seattle Children cooperated with Google to create a path assistant using his Gemini models at Vertex AI Google Cloud. This agent is specially trained on the paths of CSW Children Seattle – thousands of pages of text, images and block diagrams.
“Suppliers can in a conversational way ask an assistant to questions about clinical scenarios,” Lin explained. “Then the agent quickly downloads the relevant information directly from the CSW paths. If the initial question is not specific enough, the agent task explains the questions to indicate the information you need.
“Over 50 children’s suppliers in Seattle took part in the assistant to the test path, submitting over 1,300 hints to confirm his accuracy,” she continued. “This strict tests have led to an accuracy exceeding 98% after many iterations. Now we are developing training materials for wider use and we plan to pilot a tool soon in some clinical areas.”
Currently, the Pathway assistant works independently and is not integrated with other applications or systems. This deliberate approach during the development and testing phase allowed the staff to focus only on the accuracy and reliability of its basic function, without complexity or the need to integrate data specific to the patient or other clinical applications.
“When preparing for pilot implementation in selected clinical areas, we focus on strengthening the position of suppliers with immediate access to collective knowledge embedded in our CSW, without the need for data specific to the patient,” she said.
RESULTS
Although the main endpoint of the health organization with AI’s assistant to the path is not to reduce the readmisja indicators, the wider CSW paths have documented achievements of improving the overall patient results. Concentration on this AI initiative concerns the enhance in availability and compliance with those based on evidence, normalized care paths.
“We will assess the shortening of the time needed to download critical information in CSW,” Lin informed. “By making it easier for clinicists and easier access to the most current guidelines at the care point, we anticipate the more consistent application of the best practices, which, as shown in general, CSW effectiveness, can contribute to improving patients’ trajectory.”
Tips for others
Children in Seattle have learned many lessons during their travel AI except for the development of this tool.
“My advice for every healthcare organization considering the use of AI agents technology is primarily to anchor their exploration in a clearly defined and burning problem,” said Lin. “Lean the temptation to accept a frosty technology for yourself. Instead, meticulously identify a specific bottleneck or ineffective, which directly affects patient care or work.
“In addition, be realistic and accurate in estimating a potential return on investments, both clinical and operational,” she continued. “A well -defined case of use with a clear path for measurable improvement will be crucial for the priorities of efforts and resources in this rapidly developing industry.”
Secondly, recognizing the operate of artificial intelligence, especially generative AI, in clinical contexts requires an extremely stringent approach to validation and testing.
“The inaccuracy of information generated by AI in clinical conditions can have significant consequences,” she said. “Therefore, organizations must commit to intensive and iterative tests to achieve a clearly high level of accuracy and reliability, and this often means investment in human knowledge and hours, often unexpected in initial planning of resources.
“In addition, it is necessary to establish a solid continuous monitoring system,” she continued. “This should include the active collection of feedback on AI results in real clinical scenarios to identify and solve potential drift or improvement areas. This commitment to continuous monitoring and validation is crucial for the responsible implementation of AI in healthcare.”
Finally, supporting trust and ensuring users’ acceptance are key factors for success for this type of technology, he offered lines.
“This goes beyond the explanation of technology; it is associated with comprehensive training and transparent communication on the possibilities and restrictions of the tool,” she said. “Healthcare employees will want to understand the intended goal of artificial intelligence, its data source and security to ensure accuracy.
“In addition, consider the user’s impression carefully,” she concluded. “The intuitive and smoothly integrated tool is much more likely that it will be adopted by busy clinicians. Investing in design focused on users and solid training programs will be necessary to maximize the benefits of this technology and ensure its successful integration with clinical work flows.”
Watch now: how the execution can become the director of AI and cooperate well with C-Suite