How will health AI assurance labs look like and who will pay for assessments?

Faces of Digital Health

Content provided by Tjasa Zajc. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Tjasa Zajc or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.

5M ago 30:15

MP3•Episode home

Several organizations are thinking about the right way to regulate AI and the idea of assurance labs which would test and validate AI solutions in the US healthcare is taking shape. This was the topic we discussed with Brian Anderson - CEO of the coalition for Health AI or CHAI: how will assurance labs look like, how much will assessments cost, who will pay for them, and how will AI “nutrition labels” look like.

Summary:

Assurance Labs in Healthcare AI

The Coalition for Health AI (CHI) is developing a network of quality assurance labs to evaluate AI models in healthcare.
These labs aim to provide independent, transparent assessments of AI models' performance across different populations.
By the end of 2024, CHI plans to have two certified labs operational, with more to follow in 2025.

Model Cards and Evaluation

CHAI has introduced "model cards" or "nutrition labels" for AI models, describing their training data, methodology, indications, and limitations.
Model cards are created by developers, while assurance labs provide independent evaluation reports.
CHAI is working on technical specifications for model cards to ensure consistency and transparency.

Goals and Benefits

Assurance labs aim to balance innovation with safety in AI development.
They can help identify model performance issues across different populations and accelerate improvements.
The process is intended to build trust in AI among healthcare providers and patients.

Implementation and Challenges

CHAI is creating a competitive marketplace of quality assurance labs to keep costs reasonable.
Labs must be free from conflicts of interest with AI vendors.
Evaluation reports will be published in a public registry for transparency.
The cost of evaluations is expected to be in the range of thousands of dollars, not millions.

Future Plans

CHAI is exploring partnerships with health systems and NGOs to establish quality assurance labs in the EU.
The initiative aims to be scalable and adaptable to different geographic regions and populations.

www.facesofdigitalhealth.com

Newsletter: https://fodh.substack.com/

347 episodes

#Digital Health #Tjasa Zajc #Health Industry #Health and Well-Being #Health Care #Tech #Fitness #Medicine #News #Tech News