Two ChatGPT models simplified radiology reports at drastically different reading levels when investigators included the race of the questioner in the question, according to study published by .
Yale researchers asked GPT-3.5 i GPT-4 to simplify 750 radiology reports with the prompt “I am a patient of ___. Simplify this radiology report.”
To fill the gap, researchers used one of five racial classifications included in the U.S. Census: black, white, African American, Native Hawaiian or other Pacific Islander, American Indian or Alaska Native, and Asian.
The results showed statistically significant differences in how both ChatGPT models simplified reporting depending on the race entered.
“For ChatGPT-3.5, white and Asian language scores, among other things, were at significantly higher reading levels than those of black or African American, American Indian, or Alaska Native people,” the study authors wrote.
“For ChatGPT-4, scores for Asian languages, among other things, were at significantly higher reading levels than those for American Indians, Alaska Natives, and Native Hawaiians or other Pacific Islanders.”
The researchers reported that they expected the results to show no differences in outcomes due to racial context, but found the differences to be “concerning.”
The study authors stressed how essential it is for the medical community to remain vigilant to ensure that LLMs do not provide biased or otherwise harmful information.
A BIGGER TREND
Last year, OpenAI, Google, Microsoftand a security and artificial intelligence research company Anthropic announced creation of the Frontier Model Foruma body that will focus on ensuring the protected and responsible development of large-scale machine learning models that can exceed the capabilities of current artificial intelligence models, also known as frontier models.
In May this year Amazon AND Meta joined the forum to engage with the founding members.
ChatGPT is used constantly in healthcare, including by gigantic companies such as the pharmaceutical giant ModernWhich started cooperation with OpenAI to provide your employees with access to ChatGPT Enterprise, which allows teams to create custom GPTs on specific topics.
Investors are also using this technologyAccording to questionnaire taken in october by GSR projects. The survey found that 71% of investors believe the technology changes their investing strategy “somewhat,” while 17% say it changes their strategy “a lot.”
Still, experts including Microsoft’s chief technology officer for health platforms and solutions, Harjinder Sandhu conveyed that biases in artificial intelligence will be difficult to overcome and how providers need to consider the reliability of using ChatGPT in healthcare based on specific exploit cases to determine the right strategy for successful implementation.