While AI can offer enormous potential to improve patient outcomes and streamline medical processes, system flaws can pose risks to data ethics, security and accuracy.
Dr. Gao Yujia, assistant technical director of the technology group of the National University Health System, urged caution in implementing artificial intelligence in health care during the program session “Exposing the Dark Side of Artificial Intelligence” HIMSS24 APAC.
“While artificial intelligence has undoubtedly improved patient care, it is extremely important to recognize potential pitfalls,” he emphasized.
The fundamental problem is the risk of systematic error in artificial intelligence algorithms. Dr. Gao cited an incident during the Covid-19 pandemic in which an artificial intelligence algorithm was trained on biased data people of color who had the same conditions as other patients were treated disproportionately.
Dr. Gao emphasized that it is extremely crucial that artificial intelligence systems are trained on diverse and representative datasets.
“Like a ‘garbage in, garbage out’ scenario, AI can only be as precise as the data that is fed to it. Some AI algorithms do not perform as expected because they have not been trained on the right data or have been over-fitted to specific datasets.”
In addition to data quality, Dr. Gao also expressed concerns about the ethical implications of AI in healthcare.
“We apply artificial intelligence to organize our thoughts, but the same purpose can be used maliciously. For example, AI-generated deepfakes are being used to impersonate healthcare workers for ransomware attacks. Fraudulent medical research is also possible because of how convincing artificial intelligence-generated content can be.
When asked if things are heading in the wrong direction for artificial intelligence in healthcare, Dr. Gao disagreed.
“A more immersive and immediate digital world is inevitable, but it protects us [from AI risks] strikes a balance between human agency and artificial intelligence utility.”
According to Dr. Gao, an ethical framework, transparency and human oversight are key factors in harnessing the power of AI while mitigating its potential harms.
He advised healthcare workers not to fear or fear artificial intelligence.
“We should take a step back to understand, appreciate, understand and plan for how to integrate artificial intelligence with healthcare in our society.”