Artificial intelligence

Health systems are hustling to implement artificial intelligence across their organizations, but how to measure returns on those investments isn’t always clear, experts said at the HLTH 2025 conference.

AI technology, which includes ambient documentation scribes, summarization tools and revenue cycle management products, costs money and time to put in place. And it’s not always obvious how AI produces hard-dollar ROI, which is important to justify expenses and measure success, experts say.

“The use of AI to simplify administration, the use of AI to assist with productivity is absolutely out there,” said Thom Bales, principal and health services advisory leader at PwC. “There still are questions around the ROI on it and hitting the bottom line.”

At the same time, health systems are also facing increased financial pressures. This summer, President Donald Trump signed a massive tax and policy bill that makes historic cuts to Medicaid, likely culling millions of people from the safety-net insurance program.

Meanwhile, more generous financial assistance for people who buy coverage on the Affordable Care Act exchanges — the issue at the center of a weeks-long government shutdown — is set to expire at the end of the year, also pushing more people off coverage.

More uninsured Americans means less revenue for providers and an increase in uncompensated care.

Still, the healthcare sector is hopeful that AI will move the needle on some of its biggest challenges, including workforce shortages, provider burnout and lengthy administrative tasks. Amid the challenging economic environment, healthcare companies are prioritizing AI investments most likely to improve profit margins and demonstrate clear returns, according to a survey published by Klas Research and Bain & Company.

But given those returns can be hard to show, hospitals may have to get creative in measuring the value of AI, including by looking at time savings for providers, stronger employee retention or improved patient satisfaction, experts say. Those softer measures could then be linked to financial improvement over time.

“I do think ultimately we can trace the majority of these things back to a bottom line, but you may have to connect more dots,” said Jennifer Goldsack, CEO of  industry association the Digital Medicine Society.

Piecing together ROI

Measuring financial returns is always tricky, given the complex interplay of factors that complicate attributing eventual savings to a specific product. This is also true when it comes to AI, where the direct financial impact of a product depends on the tool, the costs needed to adopt it and each provider’s specific priorities, experts said at HLTH.

“In healthcare, particularly in the healthcare delivery side, it’s really hard to apply line of sight to what we might think of as ROI,” Micky Tripathi, chief AI implementation officer at the Mayo Clinic, said during a Monday panel. “I don’t think that any of us should really expect you’re going to see a cost reduction that a CFO could look at and say, ‘Oh, I see it there.’”

Measuring financial returns is easier with certain AI products. For example, revenue cycle management tools might allow health systems to see metrics like time to collections improve, according to Sandra Johnson, senior vice president of client services at EHR vendor CliniComp.

But the impact of other AI tools is less obvious. There’s limited evidence that AI documentation assistants, which record physicians’ conversations with patients and create a clinical note, are affecting productivity and financial performance, according to a report published by the Peterson Health Technology Institute. 

However, the scribes are likely reducing clinician burnout, the report found — a significant concern that could be exacerbated by the amount of time and effort needed to document care and do other work in EHRs.

If AI scribes prevent a clinician from leaving the health system, that could be a major cost saver, experts say. The expense of replacing a physician can be two to three times the physician’s annual salary, studies suggest. 

And there are other potential financial and efficiency gains from ambient scribes, said Dr. Nele Jessel, the chief medical officer at health technology firm and EHR vendor Athenahealth.

For example, providers could complete notes more quickly and possibly see more patients. They also might be able to document services more accurately and bill for a higher level of services.

Patient satisfaction is another important metric for health systems. Patients tend to like ambient scribes in part because their doctors can be more present during appointments, instead of staring at a computer screen while they take notes. Documentation assistants might also help patients understand their care plans better, since physicians have to speak out loud during an exam to ensure the scribe picks up the information, Jessel said. 

“Providers are getting much better, for example, [at] vocalizing the physical exam, talking about the plan, right?” she said. “So it increases the transparency and also makes it a much more enjoyable experience for patients.”

How health systems are measuring AI’s impact

To implement AI successfully, health systems need to think critically about the biggest problems they want to solve and their main priorities, experts say. That plan should include how they’re going to evaluate the AI tool, which can help them determine if the project was a win — whether financially or by other metrics.

“To me, step zero is being very, very clear about, what is it that we’re doing? Why are we doing it? And what does success look like for that?” Mouneer Odeh, chief data and AI officer at Cedars-Sinai, said during a Sunday panel. 

Cleveland Clinic evaluates AI using two types of metrics, according to Sonja O’Malley, the general manager of business development and licensing at Cleveland Clinic Innovations, the system’s commercialization and innovation arm. 

Those include quantitative measures — like reducing no-show rates or cutting documentation time, compared to a baseline metric — as well as more qualitative criteria, like feedback on patient or clinician experience. 

Some tools will never be revenue generators. But they might do other important things, like allowing a clinician to spend time on tasks that require their full training and expertise, O’Malley said.

When an AI tool is brought before Ardent Health, metrics of success have to be clearly defined and measurable alongside a baseline, according to Anika Gardenhire, the health system’s chief digital and transformation officer. 

Financial return on investment is almost always important, but it might be correlative rather than direct causation, she said.

If a tool only breaks even, the system needs to consider other associated costs, such as implementation expenses or the time clinicians spent on the project when they could have been working elsewhere.

“If you get to a place where you’re just like, ‘Hey, this thing didn’t even break even. It doesn’t even pay for itself.’ Unless it’s doing something absolutely miraculous for patients, it’s out,” Gardenhire said.

Executives from some of the nation’s top health systems took to the stage at HLTH 2025 to share how their newly minted technology investments are improving operations ranging from capacity management to virtual specialty care. 

It’s a change from HLTH’s 2024 conference, when many major health systems’ more ambitious AI pilots and strategies were still in nascent stages and conference sessions on AI mostly featured technology companies.

In 2025, health systems including Risant Health, Advocate Health and Houston Methodist were eager to share data demonstrating that their forays into AI have been successful. 

Kelly Garrison, chief value officer at Atrium Health’s primary care network One Health, said Atrium had invested in AI solutions that cut expenses by identifying disease early, allowing the provider to implement lower cost solutions. For example, they might help a patient manage high blood pressure before they need medication. 

“We could all spend billions of dollars on new technologies, new tools that we all find interesting,” said Garrison. But such an investment strategy wouldn’t have the broad reach and cost savings of tools like the ones aimed at identifying early interventions, which can be used with 90% of patients.

Houston Methodist and Risant Health have leveraged AI to better manage capacity and coordinate patient care.

Capacity management has been a pain point for many overpacked health systems. Health systems are reluctant to build expensive new inpatient facilities, but they’re betting AI can help manage the churn of patients in the most appropriate and lowest cost setting. 

Houston Medical, for example, rolled out its “smart hospital” strategy this year. The initiative improves patient monitoring through the use of wearable technology and an AI-backed “care traffic control center” that helps direct and cue care.

The control center offers AI-powered “nudges” for clinicians when patients might need a higher level of care, helping move deteriorating patients more efficiently through care settings. Houston Methodist has seen a reduction in its average mortality rate since adopting the tools, according to President and CEO Dr. Mark Boom.

“Artificial intelligence, automation, remote monitoring and digital-first services are part of our future. They are the tools that will help make our care teams make faster, safer decisions,” said Boom during a panel.

The health system plans to apply the lessons learned from the inpatient setting to home health next, and expects to move 20% of its specialty care to virtual care settings over the next three years.

Meanwhile, Risant Health, the Kaiser Permanente-owned nonprofit health system launched in 2024, has integrated value-based care guides into its electronic health record to better direct care. The guides cover the most common conditions treated in primary care and include integrated orders a primary care doctor may want to order next based on evidence-based best practices. 

The primary goal of the guides is to standardize care across Kaiser Permanente and Risant facilities, which include Cone Health and Geisinger, Dr. Benjamin Hohmuth, chief medical informatics officer at Geisinger, said during a panel on Tuesday.

Risant’s parallel initiative, called Intelligent Triage, helps direct patients to the most appropriate care setting. The tool has resulted in about a 20% reduction in unnecessary emergency room encounters at Geisinger by prompting clinicians to consider lower acuity and more cost-effective options, according to Risant. Geisinger also used the tool to reduce unnecessary specialty referrals by 7% over the last year.

“It leads to faster resolution for patient concerns, lower cost sharing for patients and frees up specialty access,” said Hohmuth.

When specialists do need to be involved, Geisinger is pushing for virtual consults when possible, with Hohmuth noting “the specialist can do a lot more e-consults in an hour than they can do in person. It's a lot more convenient for the patient as well.”

Health systems also said they remain bullish on ambient AI and notetaking software to help reduce burnout.

Dr. Mark Collins, chief strategy officer at Atrium One Health, said the system’s investment in Microsoft's DAX Copilot, which automates notetaking, has helped give older providers “a new lease on caring for their patients.”

Meanwhile, Geisinger said physicians are saving approximately an hour per day through using ambient listening technology to assist in notetaking. 

Notetaking tools could be influential for recruiting talent too.

“If you don’t have this technology, people on the front lines may not want to come to your facility,” Amy Meister, senior vice president of UPMC and president of its community and ambulatory services division, said about documentation tools during a panel.

Sponsored content

By Cedar

If 2025 was the year everyone got an AI agent, 2026 is the year they have to justify it.

“It’s not going to be enough to say, ‘we have AI agents handling X percent of tasks,” says Dugan Winkie, Chief Strategy Officer at Cedar, a healthcare financial experience platform. “The question is going to be: how do we actually quantify the impact?”

For ApolloMD, a clinical practice management group that supports and operates Emergency Departments across the country, that question already has an answer.

Nine months after deploying Kora, Cedar's AI voice agent for patient billing support, ApolloMD has cut average call handle time by 27%, reduced outsourced call center staffing by 11% and routed more than 80,000 calls through the system.

Most AI pilots end as pilots. ApolloMD‘s didn't — and the reason is less about the technology than how they deployed it.

“We started small and tested each use case one at a time,” says Amy Katnik, Executive Advisor, Revenue Cycle and Strategy at ApolloMD. “While the technology enables you to scale rapidly, you need to be deliberate in how you do that — especially when it impacts patient experience."

A problem worth solving

AI is only as useful as the problem it's pointed at. For organizations like ApolloMD, patient billing has plenty.

"Patients are already upset — nobody wants to go to an ER," says Katnik. "They don't always understand that our bill is separate from the hospital's charges, creating a sticky situation."

But that's just the start. The patient may have presented insurance at the hospital that didn't transfer to the clinician's billing system. Or they may have had no coverage at all — a challenge that's only expected to become more prevalent as Medicaid changes in the One Big Beautiful Bill ripple through the system.

That context shaped how Cedar designed Kora.

Rather than optimizing for call containment, the system is built to meet patients where they are: authenticating their identity, pulling up their account and anticipating why they're calling. If it can resolve the issue on its own, it does. If it can't, it hands the call off to a human agent — along with everything it has already learned, so the conversation doesn't start from scratch.

There’s also a less obvious benefit. “Patients don’t really want to talk about their healthcare financial struggle,” says Katnik. “Kora enables them to do it in a way that feels less threatening.”

Embedded, not bolted on

No two billing operations are alike, and generic AI solutions tend to expose that quickly.

“The solution has to be very specific to each health system or physician group,” says Winkie. “The patients calling in have unique questions, and the AI has to be built to accommodate that.”

In emergency medicine, for example, insurance needs to be verified against the date of service, not when a patient calls. A patient seen three months ago may have had completely different coverage than they carry today, or none at all. In states like California, where Medicaid enrollment changes frequently, that can create even more complexity.

Uncovering that level of nuance required more than software. It required translation. That’s why Cedar embedded forward-deployed engineers (FDEs) directly with ApolloMD's team — a role that has surged more than 1,000% in tech job postings in 2025 as companies race to operationalize AI. FDEs analyzed real call recordings, mapped the edge cases of emergency medicine billing and built escalation logic one use case at a time.

The gains showed up quickly. Automating caller authentication alone, says Katnik, "dropped call time by about 45 seconds." Adding insurance capture saved another 40. "We reduced our calls by almost two minutes," she says.

It's a lesson that applies beyond any single deployment: the ROI of AI in the revenue cycle isn't found in the technology, but in the workflow.

What justification looks like

ApolloMD's roadmap extends well into 2026, with a focus on handling more complex billing inquiries. Full end-to-end call automation, where Kora resolves an inquiry without any human involvement, is expected to grow substantially as each new use case is added.

But for Katnik, the most meaningful measure of progress isn't on any dashboard.

"I just don't worry about it anymore," she says. "We used to get problem calls. Now I'm almost shocked when we get one."

For an industry asking whether AI can deliver, that might be the most honest answer out there.

North Country HealthCare is the “only game in town” for care delivery in some of the regions it serves, said Dr. Jennifer Cortes, quality and population health medical officer at the federally qualified health center. The provider operates 13 primary care clinics and two mobile units, caring for 55,000 people across rural Northern Arizona. 

Some communities are very remote — meaning patients may be forced to travel hours to reach specialty care — which makes recruiting providers a challenge, Cortes said. 

That’s one area artificial intelligence could help. Adopting an AI scribe, which typically records providers’ conversations with patients and drafts clinical documentation, could alleviate some of clinicians’ administrative work and decrease burnout, she said. 

“When ChatGPT first came out, I was like, ‘Oh my God, this might make things so much better for those of us working in this field,’” Cortes said. “I just want my job to not be so challenging all the time. It would be amazing if this works.”

But taking on an AI project isn’t easy for a safety-net provider. The technology can be labor intensive to implement, requiring technical expertise and oversight capabilities that many are unlikely to be able to easily access, experts say. 

And if health systems with the fewest resources — that often care for the most medically complex patients — aren’t able to realize the benefits of AI, they could fall even further behind larger or more affluent providers.

“If you look at the kinds of health systems that are actively deploying AI right now, those that can afford it are the ones that are more aggressively pursuing it,” said Brian Anderson, CEO of the Coalition for Health AI, an industry group developing guidelines for responsible AI use in healthcare. “Those that are in rural communities, for example, that don’t have the IT staff to deploy and configure different kinds of AI tools aren’t able to do that. That’s an example of the digital divide already being reinforced in the AI space.”

‘A ton of human effort’

Adopting AI products at health systems can require specialized human labor and technology resources to safely implement, creating significant barriers for cash-strapped safety-net providers, experts say. 

“People tend to talk about it or conceptualize it like you’re turning a light switch on,” said Paige Nong, assistant professor at the University of Minnesota School of Public Health. “It’s actually not that simple. These tools and these systems require a ton of human effort.”

Safety-net providers are likely to operate at slim margins, given their heavier reliance on Medicaid — a pressing challenge as the insurance program faces federal funding cuts — and higher uncompensated care demands. 

For example, the net margin at community health centers, which provide primary care to underserved populations, was just 1.6% in 2023, according to health policy research firm KFF. That fell from 4.5% in 2022, driven by inflation and the expiration of pandemic-era funding.

Many community health centers also face workforce concerns, with more than 70% reporting a primary care physician, nurse or mental health professional shortage last year, according to the Commonwealth Fund. Meanwhile, labor costs are a significant expense for many providers. 

And implementing AI will take plenty of work to manage. For example, health systems will need to set up AI governance structures that can evaluate products for safety and efficacy as well as maintain regulatory and legal compliance. Plus, providers should keep monitoring their AI tools, because the assumptions underlying the model, like patient characteristics, could change over time, potentially degrading its performance, experts say.

“It’s obvious when a scalpel gets rusty, you know you need to replace it or clean it,” CHAI’s Anderson said. “With a lot of these AI tools, we don’t necessarily know that yet. So how health systems can afford to do the kind of monitoring and management of these models over time is a real concern.”

Tech support

Additionally, providers will need IT staff with the technical expertise to handle the work needed to adopt AI tools, a particular challenge for cash-strapped and rural facilities that may struggle to attract the talent, experts say.

For example, many safety-net providers don’t have data scientists on staff, said Mark Sendak, population health and data science lead at the Duke Institute for Health Innovation. It likely doesn’t make sense for some of these care settings to employ them either, given the staff may command high salaries while not generating revenue from patient care, he added. 

Financial challenges can also make it hard to invest in the IT infrastructure needed to adopt AI tools, said Jennifer Stoll, chief external affairs officer at health IT provider and consultancy OCHIN.

“Compounding the challenges for community health organizations is that many have no choice but to rely on outdated, inefficient technology systems,” she said via email. “Not only are these outdated systems all they have access to, but some are not even capable of integrating AI tools—only exacerbating the technology divide.”

Meanwhile, AI might be at the bottom of an IT team’s to-do list. For example, at North Country, Wi-Fi at the clinics doesn’t always work well. And its legacy electronic health record won’t be supported by its vendor in a few years, so the provider will have to transition to a new one.

“We’re missing even the basics,” Cortes said. “Even if you’re not talking about AI, we’re behind.”

Missing out on AI

These constraints are likely already impacting how low-resource providers implement AI, experts say. 

For example, a study published early this year in Health Affairs found 61% of U.S. hospitals that used predictive AI models had evaluated them for accuracy using their own data, and just 44% locally evaluated their tools for bias — an important process to help health systems determine whether a tool will work well within their patient populations. 

Hospitals that developed their own predictive models, reported high operating margins and were part of a health system were more likely to locally evaluate their AI products. 

“You’ve got the resources, you’ve got an IT staff, you’ve got some data scientists who can design models or who can evaluate models from an EHR vendor,” said the University of Minnesota’s Nong, one of the study’s authors. “Resources were the critical necessary component for being able to really consistently conduct evaluation and also to design bespoke models.”

Without assistance, providers who lack the funds and technical capabilities to implement AI could miss out on the technology’s potential benefits or adopt AI without necessary safeguards. That could widen existing disparities between high- and low-resource providers and their patients, experts say.

For example, hiring and retaining staff is already a challenge for safety-net settings, Sendak said. When a new graduate from residency looks for a primary care job, would they rather take a position at a health system that could offer an AI documentation assistant — which could help alleviate provider burnout — or a clinic that’s unable to implement one?

Additionally, limiting AI uptake to the most well-resourced providers could reinforce biases inadvertently included in these tools, CHAI’s Anderson said. If data used to train algorithms continues to be only easily collected in urban, highly educated communities on the coasts, AI tools will miss out on information gathered from other groups. 

“I think our job as a society is to make sure that we are making that as easy as possible, so that we have AI that can serve the individual in rural Appalachia or in a farming community in Kansas just as well as it serves people in San Francisco or people in Boston,” he said.

Help wanted

Still, there are methods that could help small and low-resource providers adopt AI products — including mentorship and support models that have been used for other emerging technologies, experts say.

For example, the HITECH Act, enacted in 2009 to promote the use of EHRs, included funding for Regional Extension Centers, which provided on-the-ground technical assistance for small primary care practices, community health centers and critical access hospitals.

Similarly, the Health Resources and Services Administration funds Telehealth Resource Centers, a group of 12 regional and two national centers that offer education and resources to providers looking to implement virtual care. 

EHR vendors have a role to play as well, Nong said. Nearly 80% of the hospitals in her Health Affairs study used predictive models obtained through their EHR developer, so that could be an impactful point for helping providers safely deploy models, she added.

Larger health systems and academic medical centers could assist their small and resource-strapped counterparts too. The Health AI Partnership, a coalition that includes health systems like Duke Health and the Mayo Clinic, runs the Practice Network, which works with safety-net settings on adopting AI best practices with one-on-one support. 

North Country is one of the inaugural participants in the network. The program provides technical support to safety-net organizations, helping them work through AI procurement, evaluation and implementation, said Duke’s Sendak, who serves on HAIP’s leadership council. 

Still, the Practice Network currently works with five safety-net providers, while there are hundreds of federally qualified health centers across the country that may need support, he added. 

“There is a big difference between where all the conversation is about AI in healthcare and where a lot of the people who are delivering healthcare are in terms of ability to adopt and use it safely,” North Country’s Cortes said. “When they’re talking about all these huge investments going into AI, great, that’s exciting, but how are they going to bring us along too?”

The healthcare industry isn’t known for quick adaption to new technologies. But as the sector faces mounting challenges — like stressed providers, high costs and staffing shortages — executives are hopeful artificial intelligence could offer some relief.

Still, even as leaders look to adopt AI, the implementation process isn’t easy. And health systems will need to get their workforces on board to ensure success, experts said at the HIMSS AI Leadership Strategy Summit in Chicago. 

The technology tends to look like a black box to some healthcare professionals, said Dr. Everett Weiss, medical director for health informatics at Rochester Regional Health, during a panel discussion. Doctors might not understand how AI works or generates recommendations.

That could prove challenging for some providers, given concerns that AI tools could offer inaccurate, misleading or biased information, potentially putting patient care at risk.

Clinicians’ credibility is on the line — so ensuring they can trust the tools is key, he added.

“Being able to build up that trust is hugely important,” Weiss said. “It is very easy to tear down trust. It is very hard to build that back up.”

That means health systems need to make sure their AI tools are reliable, through plenty of testing before the AI is fully adopted and educating users on how the products work, experts said.

Keeping a “human in the loop” — ensuring a person is involved in an AI workflow to double check its work — can help too, said Susan Fenton, vice dean for education for the department of clinical and health informatics at the University of Texas Health Science Center at Houston.

“No AI is licensed to practice. None, zero,” she said. 

Where AI can succeed

One method for considering potentially successful use cases for AI is finding areas where the technology can complement human strengths, said Sagar Parikh, vice president of operations excellence and innovation at revenue cycle management company Ensemble Health Partners.

For example, an AI tool is probably much faster than a human worker at sifting through an entire electronic health record and pulling data points to support a response to a claim denial. But a nurse is likely better at providing the clinical context and choosing which facts are the most important, he said. 

“That’s where I see language models and AI generally to be helpful,” Parikh said. “Doing the things that a human is historically not great at, and pairing it with things that humans are historically great at.”

Alleviating some of clinicians’ heavy administrative burdens is another common use case for AI, experts said. Providers have said they spend long hours on documentation and other EHR tasks, taking time away from patient care and eating into after-work hours. 

AI documentation assistants are perhaps the “poster child” for improving clinician experience, Fenton said. But during a notetaking pilot in some clinics at UT Health Houston, the assistant only saved about nine minutes per day per provider — not a huge return on investment, she noted.

Still, the system decided to deploy the AI tool across the institution, because patient and provider satisfaction with the documentation assistant was high. That highlights the value of a pilot that measures metrics outside of finances, she added.

“It’s actually truly now improving that patient and provider experience,” Fenton said. “So I think that those are the things that we need to look for.”

Not all healthcare organizations need to be trailblazers when it comes to choosing an AI project either, Weiss said. If an AI project is doing well at other health systems, that could be a good place to start.

Additionally, organizations can look at places where AI has already been in use for a while, like using the technology in radiology to triage cases that need more immediate attention, he said. 

Still, health systems should consider that not all AI tools work for every clinician. For example, while an AI notetaking assistant was popular with 9 out of 10 providers at his system, some clinicians didn’t see efficiency gains, Weiss said. 

“We should not be forcing technology on our providers. These are tools to help them do their jobs even better,” he said. “We talk about the joy of medicine a lot in our organization, and so we want technology to be able to bring back that joy for all of our providers.” 

Artificial intelligence tools are improving rapidly, leaving regulators and lawmakers the challenging task of determining how — and when — to oversee the technology in healthcare, experts said at the 2026 HIMSS conference. 

Regulations for AI use in the sector are fragmented. The Trump administration has largely moved to limit rules that could slow the adoption of the technology, leaving healthcare organizations with less guidance on how to implement AI.

Meanwhile, some states are attempting to legislate how AI is used in healthcare, creating a convoluted regulatory environment for providers and payers who operate across state lines. 

“It remains a very complex environment, with few guardrails for the use of AI in healthcare, and still a lot of work to do to really create an environment that is going to produce safe, reliable artificial intelligence for clinical use,” Tina Joros, the vice president of policy and innovation at health IT company Veradigm, said in an interview. 

Still, the HHS has done a lot of work to understand what the sector needs from the federal government, Arman Sharma, the deputy chief AI officer and senior advisor in the office of the secretary at the HHS, said during a panel discussion. 

The department has moved to align AI work across its agencies and make sure they’re working in the same direction, Sharma said. Previously, agencies might start their own separate AI projects without clear visibility into what their colleagues were doing.

To help the massive department figure out how it could advance AI in clinical care across its divisions, the HHS released a request for information in December. 

The RFI, which received nearly 450 comments, asked the industry how regulations might need to change to include AI tools, how the department could simplify payment to encourage the use of the technology and what research and development investments could offer best practices for adoption.

“We recognize that this is a space that is very volatile. It’s moving very quickly,” Sharma said. “It’s very important that the industry gets clear signals from the agency about what we care about.”

Adapting to a fast-moving technology

Inside the HHS, the Food and Drug Administration is trying to set policy that works in 2026 but will also adapt as the technology evolves, Jared Seehafer, senior advisor in the office of commissioner at the agency, said during a panel. 

The agency has approved more than 1,300 AI medical devices since 1995, according to a tracker maintained by MedTech Dive. The number of submissions has spiked in recent years as investment in the technology has increased. 

But the advent of agentic AI, or AI systems that could act more autonomously and potentially improve themselves, adds new challenges to the FDA’s regulatory framework, given developers typically have to notify the agency about their update plans, Seehafer noted. 

AI tools also might need a more balanced approach between reviewing them before they hit the market and keeping an eye on the devices in operation, he added. The performance of AI models can degrade over time, so the tools could need continuous oversight to ensure they remain accurate.

“That is going to require of us a new regulatory framework, but it’s also going to require from the entire community — and all stakeholders of different types — data sharing in a way that heretofore does not really happen,” Seehafer said. 

Another challenge is an agentic system that relies fully on human oversight likely won’t be able to scale, given the amount of human resources needed to monitor all of the AI’s output, said Dr. Haider Warraich, a program manager at the Advanced Research Projects Agency for Health, or ARPA-H.

For example, ARPA-H, a relatively new agency that supports biomedical and health research, is working on a project that aims to develop clinical AI agents that can autonomously guide cardiovascular disease care, Warraich said.

The program also plans to build a supervisory agent that can oversee the clinical tools to ensure they continue to perform safety and effectively. 

“I think we’re still in the early innings. So a system that has no human oversight would be unacceptable, I think, not just to patients and health systems, but also to clinicians like myself,” Warraich said. “And really the goal of the supervisory agent is, how do you create a system that can optimize human oversight, especially in the absence of ground truth?”

A complex regulatory environment

Meanwhile, as AI technology advances, the current regulatory scheme remains complicated. For example, few states have one regulatory body responsible for AI, so rules could fall under a number of agencies, like state departments of insurance or attorneys general. 

Plus, state legislation can be slow-moving, so guardrails could become stale and out-of-date by the time the bill advances, Veradigm’s Joros said.

For now, the adoption of AI agents or tools for diagnosis and treatment might take time, Danny Sama, vice president of digital platform and strategy for information services at Northwestern Medicine, said in an interview. 

“Frankly, I think a lot of it’s going to depend on regulatory and legal,” he said. “At some point, some agent is going to make a mistake, and there will be a malpractice lawsuit, and the question is, who’s liable?”

State laws regulating artificial intelligence are on the rise, and industry leaders should step up to help lawmakers manage the technology, experts said at the HIMSS AI Leadership Strategy Summit. 

More than 1,000 bills on AI have been introduced at the state level this year, including about 280 that relate to health technology, said Tina Joros, vice president of policy and innovation at health IT firm Veradigm.

That compares to around 500 to 700 bills in 2024, depending on how the legislation is counted, she said during a panel discussion. 

“Definitely an increase over three to five years ago,” Joros said. “It’s just a flood.” 

The rise comes as federal oversight over many aspects of health AI is relatively nascent, and the Trump administration has signaled it wants to pursue a deregulatory path to spur development of the technology in the U.S.

Meanwhile, the healthcare sector has moved to implement AI for a variety of tasks, including assisting with clinical documentation, triaging imaging results or data analysis. 

Still, the technology comes with risks, like the potential for inaccuracies or biases that could lead to patient harm. Some states are interested in filling the regulation gap — and the healthcare sector should weigh in on how those laws are crafted as they work to roll out these tools, experts said. 

State laws around healthcare AI have so far been thoughtful and helpful, said Daniel Smith, chief medical information officer at Michigan health system Corewell Health. However, if the regulations become too broad, AI use at the organization could be at risk. 

“So what we would hope for is that what’s getting built out in our legal frameworks are taking a more of a risk-based assessment, so those lower risk opportunities can still flourish quickly and not be held back,” Smith said. 

Where states are looking to regulate health AI

A significant portion of the health AI legislation this year is related to prohibiting or monitoring the use of AI in insurers’ utilization management processes, said Sarah Jaromin, a policy specialist in the health program at the National Conference of State Legislatures. 

Some insurers have already faced scrutiny from Congress for using AI to deny claims or prior authorization requests, where a provider has to receive approval before offering a drug or service.

In 2024, a Senate report criticized Medicare Advantage insurers UnitedHealthcare, Humana and CVS for allegedly using predictive algorithms to inappropriately refuse coverage for post-acute care.  

States are looking to regulate the practice too. For example, Arizona passed a law this spring that requires payers to have a person verify claims denials or rejected prior authorization requests. Maryland also passed a law this year overseeing insurers’ use of the technology in claims management. 

Another significant area for legislation is regulating the use of mental health chatbots, Jaromin said. For example, a law passed in New York would require AI chatbots to put safeguards in place to detect signs of suicidal ideation or self-harm. 

Proponents say AI mental health tools could provide quick and accessible support, but other experts argue they might not be effective enough to replace a human therapist — and could be unsafe. 

AI and mental health has also received more public attention in recent months amid news reports on teens who died by suicide after developing deep relationships with chatbots.  

“Sometimes it’s like what you see in the news, and then you'll see some state regulation there,” Jaromin said. “And I think that’s what's happening with the payers and the mental health chatbots as well.”

Engaging with lawmakers

State lawmakers likely don’t have expertise in healthcare or AI, experts said. 

Additionally, many of the legislators who are making decisions on AI policy likely have limited knowledge on what they’re voting on, largely because no one in the community has come to speak to them about the impact, said State Sen. Bo Watson, R-Tenn.

Healthcare leaders should meet with legislators and offer themselves as a resource to policymakers, he said. And though it’s a challenging task in politics, they should approach lawmakers without bias or self-interest. 

“State government is the infrastructure of our political system,” he said. “It is critically important that you engage yourself in your state government, and you engage yourself with your state legislators, because ultimately they will be making 95% to 98% of the decisions that will have impact on your business model.”

Generative artificial intelligence has exploded in the healthcare sector in recent years, driven by hopes the technology could take on a variety of tasks — from clinical documentation to data analysis — and lessen the industry’s long-standing workforce challenges.

At the same time, healthcare organizations often struggle to manage cybersecurity, burdened by frequent cyberattack attempts as the sector adopts more internet-connected tools.

AI products could be another target for cybercriminals. Meanwhile, hackers can use their own AI to launch cyberattacks. That should create new work and security disciplines for healthcare cyber teams, said Taylor Lehmann, a director of the office of the chief information security officer at Google Cloud.

Lehmann sat down with Healthcare Dive to discuss how the advent of generative AI tools will impact healthcare cybersecurity and what organizations need to do to prepare for an increasingly AI-augmented security workforce. 

HEALTHCARE DIVE: Would you say these generative AI tools are more ripe for attack than any other piece of tech hospitals use? Or is that just another vendor they have to consider when they're thinking about cybersecurity?

TAYLOR LEHMANN: This is where I am concerned about the future. Today we’re talking with AI systems, tomorrow we’re watching AI systems do stuff — like the agentic push we’re still on the cusp of — so this answer might change a bit. But number one, it’s going to be very hard to detect when AI is wrong and whether that being wrong is due to it being manipulated by a nefarious actor. 

Detecting wrongness is already a challenge. We’ve been working on this for years. No one’s been able to completely eliminate hallucinations or inaccuracy. But I do think detecting more than just hallucinations, but functionality that has been introduced by a bad guy sitting in the middle to get a certain outcome — it’s going to be really hard to detect and increasingly difficult to detect. 

So organizations need to make sure that they have provenance for basically everything from the model being served to you, the technology that has been done with, all the way down to the provider of the model and the data they use to train it. That visibility is going to be critical. 

At Google we use model cards. We use practices like cryptographic binary signing, where we can tell exactly where the code came from that is running that model. We can tell you exactly where the data came from that trained that model. We can trace every record that went into that model’s training from birth to death. And organizations are going to need to be able to see that in order to manage those risks.

You mentioned the importance of being able to tell what models are trained on and what data is flowing in, and how that could help you figure out if the tool has been infiltrated. How should health systems be thinking about implementing workforces now to deal with those concerns?

There’s definitely new security disciplines coming. This is actually some of the work that we’re doing right now in the office of the CISO. Google is looking at some of these new roles and new capabilities, and then putting guidance together for folks on what to do.

The first thing I would say is, there are ways and methods that are needed to secure AI systems out of the gate. We feel very strongly that any reasonable approach to securing AI and AI systems needs to involve having strong identity controls in place and making sure that we know who’s using the model. We know what the model’s identity is, we know what the user’s identity is and we can differentiate between those two things. We also have radical transparency in everything the model does. Right or wrong, we can see it.

One of the new big areas is this concept of AI “red teaming.” Organizations deploy a team of people that do nothing but try to make these things break — basically try to get them to produce harmful content, take actions that were not intended — to test the limits of the safeguards, as well as evaluate how well the models are trained, whether they’re over- or under-fit for purpose.

I think you’re starting to see AI governance becoming hugely important. It’s always been important, but understanding the risks associated with AI — especially in regulated industries or in safety-critical use cases — requires a combination of technical skill as well as understanding of how AI works and is built, as well as the regulatory or the business context to determine what is an important risk.

Having really robust AI governance and people with these right skills is kind of a difficult place today. Because right now, you’ve got people who are great engineers, you’ve got people who are great doctors. You don't necessarily have both, with a risk hat on. So we’re going to see the rise of industry-specific, even practice-specific, governance professionals. 

Hackers are increasingly using AI to improve their intrusion attempts. How should health systems be thinking about the increased risks posed by cyber criminals now that they have access to AI?

Number one is, we have to wrangle how organizations provision and manage identities, like the identities of humans, the identities of machines, the identities of agents. 

I do not think your average hospital really thinks about identity more than how to identify the people walking in the door and what their user ID and password are. Identity is going to be the piece of digital evidence or digital artifact that is going to tie everything together. Based on identity is where we will apply controls with appropriate context.

The second is speed. AI systems move fast. They move faster than humans. Weaponized AI systems, especially those that have been trained to do very specific things, will move at lightning speed. We’re not going to say in the future, “Hey, you have less than an hour to fix a ransomware-positive infection of a piece of code.” You have seconds, if milliseconds, now. And if you can't operate with that same speed to detect and eliminate, you’re going to be in trouble. 

The strategy I have is, look at your architecture, look at the systems, look at your applications. Look at how quickly you can deploy new controls. Look how quickly you can rebuild the system from scratch, look how quickly you can deploy a patch. Look how quickly you can detect an event and take action on it, and drive down the things and eliminate the things that stand in your way from doing that — whether it be from hours to minutes or minutes to seconds. But prioritize those things that help you deploy corrections and detect and respond to issues extremely quickly. 

Adopting artificial intelligence tools could be a boon for safety-net health organizations, which frequently operate at low margins with fewer resources — and face more turmoil as significant Medicaid cuts loom, experts said at the 2026 HIMSS conference.

Implementing the technology is more challenging for the providers, which serve low-income populations and offer care regardless of patients’ ability to pay, given their workforce challenges and sometimes limited technical capabilities. 

Now, safety-net organizations will face new headwinds. Last summer, President Donald Trump signed sweeping tax and policy legislation into law that includes significant funding reductions to the insurance program Medicaid. 

The “Big Beautiful Bill” will likely cull millions of people from coverage and cut billions from state Medicaid programs.  

“[The law] presents an existential crisis to safety net providers,” Kara Carter, senior vice president of strategy and programs at the nonprofit California Health Care Foundation, said during a panel. “There’s no question about it. That is red states, blue states, purple states. It does not matter.”

However, the challenges brought on by the policies could be an opportunity for safety-net organizations to take advantage of AI to ameliorate some of the damage, experts said. 

In one of the law’s most significant impacts, many Medicaid beneficiaries will need to log 80 hours of work, school or volunteer hours per month to stay enrolled in the safety-net insurance program. 

That will likely create a lot of administrative work for enrollees and Medicaid programs. For example, before Arkansas’ work requirement policy was struck down during the first Trump administration, many beneficiaries didn’t know about the requirements and struggled to report their compliance. 

Additionally, the law passed last year requires states to check enrollees’ eligibility every six months instead of annually. 

AI could help with redeterminations and enrollment calls, Carter said. Some AI agents can handle outreach at the rate of 40 to 100 calls a second, she added. 

Many of the law’s provisions don’t go into effect immediately, giving states and providers some time to prepare, said David Ford, CEO of CMA Physician Services, the practice transformation subsidiary of the California Medical Association. 

“Before state legislatures start doing the really awful, terrible things we know they’re going to do — kicking people off the program, limiting benefits — the hope is that AI can generate some efficiencies and give us some savings and allow us to maintain some of that, even in a resource-strapped environment,” he said. 

Plus, the Rural Health Transformation Program — a $50 billion fund included in the law to improve care in rural communities — could give some states a lot of money to adopt AI and other technologies, said Adimika Meadows Arthur, executive director of the nonprofit Health Tech 4 Medicaid.

AI might be able to fill some gaps among providers too. If clinics need to cut back on staff, they’ll still need to make calls to patients — and AI could be one option to help, said Dr. Jeffrey Arroyo, associate director of clinical informatics at AltaMed Health Services, the nation’s largest federally qualified health center. 

“We’re already short staffed, but we’re probably going to lose more,” he said. “That doesn’t mean the work is going to go away. So it’s going to be put more on IT to take advantage of these AI solutions that are seen outside of healthcare and make magic happen.”

The CMS wants to deploy artificial intelligence tools to Medicare beneficiaries to help navigate their care, CMS officials said at the 2026 HIMSS conference. 

The agency is already using the technology to detect fraud. But the CMS also hopes to get the technology into patients’ hands, both to assist seniors and to hopefully bring down rising healthcare spending, which continues to outpace the rest of the economy. 

“The fundamental problem right now is that other sectors of the U.S. economy have advanced and been deflationary with their use of technology,” CMS Administrator Dr. Mehmet Oz said during a panel discussion. “Healthcare has remained inflationary.”

The federal government wants to introduce AI agents, or AI systems that can act more autonomously, to Medicare beneficiaries to help them find doctors or choose Medicare Advantage plans, Oz said. That tool should be available “by the time we’re done in this administration,” he said.

The biggest challenge is that Medicare beneficiaries don’t trust AI, Oz noted.

Only 31% of Medicare enrollees aged 65 and older trust AI “a great deal” or “a fair amount” to access medical records and provide personalized health advice, according to a survey published last year by health policy researcher KFF.

“No one has gotten to them with the use case of why it’ll transform their life for the better,” Oz said. “It seems like a tool that we use to market to them, or help hospitals deal with issues, but not necessarily their issues.”

One goal of the CMS’ Health Tech Ecosystem — an initiative launched last year that aims to ease data exchange through partnerships with the private sector and increase availability of digital health tools — is to expand access to conversational AI to help patients navigate care and discuss their health, said Amy Gleason, acting administrator of the U.S. DOGE Service and senior advisor to the CMS, during the panel.

Some companies that have signed onto the Health Tech Ecosystem pledge have begun to release dedicated AI chatbots geared toward health queries, she noted. For example, OpenAI launched its tool in January, and Microsoft announced its Copilot Health assistant this week. 

Still, misuse of AI chatbots could be a risk to patients, given the tools can provide false or misleading information, experts say. Research on OpenAI’s consumer health tool published last month found the chatbot often under-triaged cases, failing to send patients to the emergency room for serious health issues like impending respiratory failure.

Hallucinations are a concern, but doctors can also make mistakes, Gleason said. 

“The same will happen with AI, and they’re the dumbest today that they’ll ever be,” she said. “So I think we need to embrace this and help people understand how much this can help them.”

The Trump administration has also been using AI in healthcare to help the CMS detect fraud, said Kimberly Brandt, COO and deputy administrator at the agency. 

Fraud has become a significant priority for the Trump administration, which last month halted $259 million in Medicaid funds from Minnesota over what it called unsupported and potentially fraudulent claims. The agency is also imposing a six-month moratorium on Medicare enrollment for some suppliers of durable medical equipment, as part of its efforts to crack down on fraud. 

The CMS has been using an algorithm trained on previous Medicare providers that the agency has taken enforcement actions against, and tested new providers enrolling in the program against those bad actors, Brandt said. 

“We wanted to use AI to really help us be able to find out where the problems are,” she said.