From building consensus on the definition of responsible AI to talking about semi-autonomous operate cases for AI in healthcare, industry and government appear to be moving toward shared values, even in these polarized times as we enter 2025, says Brian, director CEO of Anderson’s Coalition for Healthcare AI.
“We need our policymakers and regulatory officials to understand this kind of framework that is being developed in the private sector about what responsible AI looks like in health, and then develop a regulatory framework around that,” Anderson explained.
Compatibility to date
The technology industry and federal health care regulatory agencies have devoted significant attention to AI model cards, or “AI nutrition labels,” an accessible form of communication used to identify key aspects of AI model development for users.
On Thursday, as CHAI released the open-source version of its draft AI Model Charter, we spoke with Anderson about the coalition’s recent experience and his insights on what the near future may hold in developing a public-private framework for the unthreatening dissemination of AI in health care.
“It’s great to see alignment between where the private sector innovator community is heading and where the public sector regulatory community is heading,” he said.
While CHAI is seeking feedback this month on its draft open source model charter – which “originates” from the Office of the National Coordinator for Health Data, Technology and Interoperability on Health Technology: Certification Program Updates, Algorithm Transparency and Information Policies – With plans to roll out the update over the next six months, Anderson expressed hope that various regulators are and will continue to move in the same direction.
In particular, he reported the degree to which AI requirements are aligned with medical device regulators.
Earlier this week, the U.S. Food and Drug Administration included an example of a voluntary AI model charter in its draft recommendations for the total product lifecycle – design, development, maintenance and documentation – for AI-enabled devices.
“One of the exciting things when we look at the FDA sample model card, and then we certainly look at the ONC HTI-1 rule, is that the CHAI model card is very much consistent with both,” Anderson said.
According to the FDA, the AI-enabled medical device card model can address the communication challenges of presenting critical information to health care users – patients, physicians, regulators and researchers – and the public.
“Research has shown that the use of a card model can increase user trust and understanding,” the FDA said in its draft guidance. “They are a way to consistently summarize key aspects of AI-enabled devices and can be used to concisely describe their features, performance, and limitations.”
As an example of the regulation of artificial intelligence in health care, one points to where FDA regulators are moving in their work to build confidence in the operate of artificial intelligence.
“However, private and public sector stakeholder groups must work together to keep each other informed,” Anderson said.
When asked about this, he noted the incoming administration, “and every leader I’ve talked to in the Senate and the House is very interested in understanding how they can engage in public-private partnerships with organizations like CHAI.”
With the door open to pursuing more exacting AI in health care — such as government and industry alignment on the labs that provide AI control and how they operate — “there is still work to be done,” Anderson said.
“We need time to do this, and the appreciation from the new administration that they are willing to work with us and work with us – and hopefully provide us with that time – I think is very refreshing and exciting.”
Annual label updates and IRL operate
Anderson said the CHAI card model is intended to be a “living, breathing document as new opportunities emerge, particularly in the generative AI space.”
“It is very likely that the metrics and methodologies we use to assess emerging opportunities will need to change or need to be created,” he said.
Even before the FDA issued its draft guidance on the total lifecycle of medical devices, it finalized a review of its previously established change control plan for AI and machine learning submissions – without requiring modern marketing submissions.
“When we think about the different sections of the model card, there will be different metrics to consider – different assessment results, different metrics… different types of use cases,” etc., Anderson said.
“That kind of flexibility is going to be really important,” he added, noting that the system’s artificial intelligence model or nutrition label will need to be updated regularly, “certainly at least once a year.”
Providers must consider the high complexity when using AI-based clinical decision support tools to minimize errors or omissions.
“Imperfect transparency will be something we will struggle with and work on again,” he stressed.
Whether a model has been trained on a specific set of attributes that may apply to a specific patient may not be reflected in user-friendly model cards.
“You could put all the information under the sun on these model cards, but the retail community would be at risk of exposure[intellectual property]- he said “There is a balance between how to protect the supplier’s intellectual property and at the same time provide the customer – in this case the doctor – with the information necessary to make the right decision on whether he should use this model in the case of a patient he has before,” Anderson said[intellectualproperty”hesaid”Soit’sbalanceofhowdoyouprotecttheIPofthevendorbutgivethecustomer–thedoctorinthiscase–thenecessaryinformationtomaketherightdecisionaboutwhetherornottheyshouldusethatmodelwiththepatienttheyhaveinfrontofthem”Andersonsaid[własnościintelektualnej”–powiedział„Zatempozostajerównowagamiędzytymjakchronićwłasnośćintelektualnądostawcyajednocześniezapewnićklientowi–wtymprzypadkulekarzowi–informacjeniezbędnedopodjęciawłaściwejdecyzjiotymczypowinienstosowaćtenmodelwprzypadkupacjentaktóregomaprzednimi”–powiedziałAnderson[intellectualproperty”hesaid”Soit’sabalanceofhowdoyouprotecttheIPofthevendorbutgivethecustomer–thedoctorinthiscase–thenecessaryinformationtomaketherightdecisionaboutwhetherornottheyshouldusethatmodelwiththepatienttheyhaveinfrontofthem”Andersonsaid
“Causation has a really profound impact on how it might impact a particular outcome for the patient in front of you,” he admitted.
Inviting others to the AI evaluation table
While HTI-1’s 31 categorical areas — which include electronic health records and other certified health IT — “are a really great starting point,” they won’t be enough for various AI operate cases — especially in the direct-to-consumer space, he said. Anderson.
“The card models we are developing are intended to be used quite broadly across a variety of use cases, and in the consumer space, particularly in generative AI, there will be a whole host of new use cases emerging in the next year,” he explained.
However, over the next two to five years, evaluating AI models in healthcare will become even more convoluted, raising questions about how they define “human flourishing.”
Anderson said he thinks operate cases will be closely related to AI agents in healthcare, and developing a trust framework around them will require the support of “ethicists, philosophers, sociologists and spiritual leaders” who will aid advise technologists and AI experts to rethink framework for evaluating these tools.
“Developing an assessment framework for the future of agentic AI will be a real challenge,” he said. “It’s a very intimate personal space. How do we build this trust using these models? How do we evaluate these models?”
Anderson said that starting next year, CHAI will be leading a “very intentional effort to bring together community members and stakeholders that you wouldn’t necessarily think about first, what kind of stakeholders you would include in an effort like this.”
“We really need to make sure these models are aligned with our values, and we don’t have any rubrics for how to grade a model. I don’t know how to do it yet. I don’t think anyone knows that yet.”