top of page

FDA's GenAI Medical Device Paper Is a Starting Gun, Not a Warning Shot

Sep 14
5 min read

Updated: 7 days ago

The FDA just issued a discussion paper on regulating Generative AI as a medical device. Most healthcare executives will file it under "watch and wait." That is the wrong read. I have spent my career inside federal health IT, at NCQA, and now advising healthcare organizations through some of the most demanding compliance frameworks in the industry. I know exactly what happens between a federal discussion paper and an enforcement action. The gap is shorter than you think.



The Regulatory Architecture Is Already Being Built


Generative AI in clinical settings is not a future consideration. Health systems are deploying large language models right now to assist with radiology interpretation, clinical documentation, prior authorization, and diagnostic decision support. The FDA's discussion paper floats possible approaches to a distinct regulatory lane for GenAI; FDA has not decided whether new guidance or rules will follow that is materially different from the existing Software as a Medical Device (SaMD) framework that most compliance teams think they understand.


The existing SaMD pathway was built around deterministic software: a predictable input produces a predictable output. GenAI does not work that way. A foundation model fine-tuned on clinical data can produce outputs that vary with the same input, drift over time as the underlying model is updated, and generate plausible-sounding but clinically incorrect conclusions. The FDA knows this. That is exactly why the discussion paper exists.


Comment periods on FDA discussion papers are not formalities. They are the last opportunity for the regulated community to shape a framework before it hardens into enforceable requirements. I watched this dynamic play out with EHR certification criteria during my time at the Department of Defense and later at NCQA. The organizations that engaged during the comment window had their concerns reflected in the final rule. The ones that waited received a compliance mandate with a hard deadline and no margin to negotiate.



The Problem: CISOs Are Being Written Out of the Conversation


Here is what concerns me most about current AI governance across healthcare: the people making AI deployment decisions are not talking to their security leaders. I see this consistently. A clinical informatics team evaluates a GenAI vendor, legal reviews the BAA, IT provisions the infrastructure, and the CISO learns about the deployment when the vendor contract lands for signature review.


That sequence has always been problematic for cloud deployments. For GenAI in clinical decision support, it is genuinely dangerous.


The risk categories that GenAI introduces are security problems, not just clinical informatics problems. Data poisoning involves deliberate manipulation of training data to cause a model to produce biased or harmful outputs. In a clinical context, a poisoned model could systematically underweight symptoms in specific patient populations. That is both a patient safety issue and a cybersecurity incident. Hallucination in clinical pathways, where a model generates confident-sounding but factually incorrect guidance, requires the same kind of incident response planning that governs a ransomware event. De-identified data used to train or fine-tune models carries re-identification risk that the HIPAA Safe Harbor method was not designed to address at this level of data complexity.


These are not hypotheticals. The Change Healthcare cyberattack in early 2024 affected an estimated 192.7 million individuals and exposed the cascading operational risk that comes when a single vendor becomes a critical dependency across clinical workflows. GenAI vendors are business associates with even deeper access to clinical data. Their model training practices are your compliance exposure, and HHS has demonstrated consistently that covered entities bear accountability for vendor failures under the HIPAA Security Rule.




Healthcare information security

What This Means Strategically


Organizations that move now have a real advantage. Not because FDA enforcement is imminent, but because building AI governance infrastructure takes longer than executives expect. The organizations that wait until a final rule is published will face compressed timelines, expensive retrofits, and potential enforcement exposure that could have been avoided with 18 months of deliberate preparation.


This is the same dynamic I observed across information security framework (for example ISO 27001/27002 and SOC 2) adoption cycles. The early movers built governance structures methodically and absorbed new requirements efficiently as the framework matured. The late movers scrambled through point-in-time assessments under time pressure, producing compliance artifacts that looked sound on paper but were operationally fragile. When the next requirement arrived, they were back in catch-up mode.


FDA's GenAI framework will create a similar bifurcation. Health systems with AI governance structures already in place will absorb new requirements without disruption. Those without them will face a regulatory gap with the added complication that FDA's enforcement posture around patient safety failures is distinct from OCR's and carries consequences that go beyond financial penalties.



What Leaders Should Do Right Now


The comment period closed October 19, 2026; FDA is now reviewing feedback. That creates two simultaneous imperatives: engage externally to shape the framework, and build internally to prepare for it.


First, inventory every GenAI tool touching clinical workflows and assess it against SaMD classification criteria today. If your organization does not know what you have deployed, you cannot manage the regulatory exposure. This is the same inventory discipline that NIST CSF 2.0 describes under Identify, Asset Management (ID.AM), and it is consistently the capability that healthcare organizations underinvest in until a crisis forces the issue.


Second, give the CISO a formal seat in AI governance. Not as a late-stage contract reviewer, but as a voting member of whatever governance body evaluates AI vendor selection, deployment decisions, and model update cycles. HITRUST CSF supply chain risk management controls apply directly to AI vendors. Security has to be embedded in that process from the beginning, not appended at the end.


Third, strengthen GenAI vendor contracts with security-specific addenda covering model training data provenance, update notification requirements, incident response obligations, and audit rights. Generic BAA language written for a SaaS application is not sufficient for a vendor who has ingested your clinical data to train or fine-tune a foundation model. MTC Group is working through exactly this contract language with clients right now because standard vendor templates do not address it.


Fourth, add a GenAI-specific scenario to your incident response plan. That means a defined process for detecting model drift or anomalous output patterns, a clinical workflow fallback when a GenAI tool is taken offline, and a notification protocol that covers both HIPAA breach reporting and potential FDA adverse event reporting under whatever framework the final rule establishes. The HHS 405(d) Health Industry Cybersecurity Practices (HICP) framework is a reasonable starting structure that can be extended to cover these AI-specific scenarios without building from scratch.



The Window Is Open. For Now.


Federal regulatory frameworks take years to finalize and then arrive with compliance deadlines that assume organizations had time to prepare. The FDA's GenAI discussion paper is that preparation window opening. Healthcare organizations that engage with the comment process, build governance infrastructure, and position their security leaders as central participants in AI deployment decisions will find compliance manageable when the rules land.


The ones that wait will discover that "we were monitoring the regulatory environment" is not a defensible posture when FDA or OCR comes asking about your AI deployment practices. MTC Group works with healthcare organizations on exactly this kind of forward-looking compliance infrastructure, built before the mandate arrives. Reach out if you want to talk through where your organization stands.




Sources & Further Reading


Comments


bottom of page