Dive Brief:
- Physicians, AI developers and healthcare organizations that deploy clinical AI tools all bear distinct responsibilities for ensuring the tools do not cause patient harm, the American Medical Association said in an issue brief released last week.
- While physicians remain responsible for exercising their clinical judgment, developers are accountable for designing safe AI tools, the physician advocacy group said. At the same time, deployers are responsible for selecting the right tool for clinical scenarios, providing physicians' training and monitoring the AI's performance.
- Currently, physicians are being asked to become a safety net to protect against clinical AI failures that could harm patients, but that means they may be held accountable for risks that they cannot control and, "in some cases, may not even be able to identify," Jennifer Goldsack, head of the AMA Center on Digital Health and AI, said in an interview.
Dive Insight:
Clinical AI tools are being widely adopted to support clinical decision-making and reduce administrative burdens that often result in clinician burnout. However, concerns about who is liable when AI tools make mistakes are top of mind for physicians. An AMA survey released earlier this year shows that a third of physicians cited “clear liability frameworks" as the primary driver of AI adoption.
This survey finding, along with the potential for patient harm if responsibility for AI is not clearly defined, led the AMA to create its latest framework, according to Goldsack.
“This [framework] is going to create the rubric by which every different stakeholder gets very clear on their responsibility to proactively implement best practices where they have the ability to mitigate risk,” she said.
Per the framework, physicians must exercise their clinical judgment when using AI, including interrogating AI outputs and, when necessary, dismissing AI recommendations. However, the AMA underscored that physicians cannot be held responsible for AI errors that they cannot control.
For instance, if a physician created a treatment plan based on the recommendation of an AI-driven clinical decision support tool, but the tool fails to take into account critical data to make that recommendation and the physician does not have visibility into this error, they should not be held responsible for the patient harm that may occur, Goldsack said.
It is up to developers to ensure that their tools offer safe and accurate recommendations, and that users, like physicians, can identify and respond to errors, according to the AMA framework. For this, users need access to information about intended use, performance and limitations, rather than just warnings and disclaimers.
According to Goldsack, disclaimers are not strong enough safeguards against the harms clinical AI tools could cause. Instead, the AMA framework suggests identifying where the risk of harm lies, which stakeholder can prevent it and what information they need to do so.
The third group responsible for clinical AI is the deployers, that is, healthcare organizations deploying the tools. These organizations, which include health systems, hospitals and physician practices, play a key role in selecting and implementing AI tools. According to the AMA's framework, deployers are responsible for evaluating the tools before asking physicians to use them and for providing the necessary training.
Currently, health systems appear to be lacking in the training department, with the 2026 AMA survey showing that a quarter of physicians received no AI training, and 92% were interested in more training.
In addition to training, deployers need to monitor the tool's performance to guard against situations where AI models become less accurate over time and to establish processes for responding to adverse events.
The framework not only aims to define each stakeholder's role in mitigating clinical AI risks, but also states that patients who have been harmed should have pathways to seek recourse when AI fails them, which, according to Goldsack, includes compensation.
“When we get this right and getting it right is key, not only are we protecting patients in cases in which they may be harmed so that they can seek compensation from the appropriate party, but much more importantly, it's going to make it much less likely that that happens, that a patient is harmed in the first place, because every stakeholder understands what they're responsible for,” she said.
As clinical AI advances — gaining the ability to act autonomously in some cases — the push-and-pull between AI companies and physicians on who bears the brunt of responsibility for errors intensifies.
In recent comments submitted to the Food and Drug Administration, provider groups, including the AMA, said that AI developers should be responsible for monitoring healthcare devices that use generative AI. They argued that asking providers to shoulder that burden would place undue strain on them, particularly small and rural provider organizations.
The comments, which were provided as feedback on the FDA's proposed approach to regulating generative AI-enabled medical devices, highlight providers' eagerness to limit their legal liability and malpractice lawsuit risk as the industry wrestles with how to govern the fast-evolving technology.
