FDA-Regulated AI in Life Sciences: SaMD, GxP & Clinical AI Compliance

AI in life sciences becomes FDA-regulated the moment it crosses one of three lines — analyzing patient-specific data to drive diagnosis or treatment (SaMD), operating inside a GxP manufacturing or quality process (validation required), or generating content that lands in regulated electronic records (21 CFR Part 11) — and knowing which side of each line a build sits on before the first commit is what separates fast iteration from a Form 483.

Pharma and medtech teams are deploying the same LLM tooling as every other industry — copilots, RAG assistants, document-drafting agents — but they operate under a regulator that most enterprise AI playbooks ignore. The result is a specific kind of paralysis: legal reflexively blocks anything that touches clinical or quality workflows, while shadow AI spreads through the workflows legal never sees. The way out is not less ambition; it is a precise map of what FDA actually regulates, and engineers who have shipped software inside validated environments before. That combination — regulatory literacy plus delivery speed — is the gap Gain America staffs for across healthcare and life sciences AI programs.

The regulatory map: three lines that turn productivity AI into regulated AI

Most AI in a pharma or medtech company is not FDA-regulated. A sales-territory assistant, an HR chatbot, a literature-summarization tool for early discovery — these carry ordinary enterprise risk, not regulatory risk. FDA jurisdiction attaches through three distinct doors:

1. Device jurisdiction. Software intended for a medical purpose — diagnosis, treatment, prevention, mitigation — is Software as a Medical Device (SaMD) and needs clearance or approval unless a statutory carve-out applies.
2. GxP jurisdiction. Software used in regulated manufacturing, laboratory, or clinical processes must be validated for intended use under the predicate rules (21 CFR Parts 210/211 for drugs, Part 820 for devices, GCP/GLP for trials and labs).
3. Records jurisdiction. Electronic records and signatures required by those predicate rules fall under 21 CFR Part 11, regardless of whether a human or a model produced the content.

A single LLM feature can cross all three lines at once, or none. The determination is a design-time exercise, and it should be documented the same way a security architecture review is — because inspectors and acquirers will ask for it.

When AI is a medical device: the SaMD framework and PCCPs

FDA has now authorized well over a thousand AI-enabled medical devices — the agency's public list stood at roughly 1,450 cumulative authorizations through the end of 2025, with radiology accounting for about three-quarters. Nearly all cleared through the 510(k) pathway against predicate devices; a smaller set used De Novo classification for novel-risk functions.

The structural problem SaMD creates for AI teams is that a cleared device is frozen at its cleared state. Retrain the model, and you have potentially modified the device — historically triggering a new submission per change, which is lethal to any team that wants to ship model improvements quarterly.

The Predetermined Change Control Plan (PCCP) is FDA's answer, finalized in guidance for AI-enabled device software functions in December 2024. A PCCP is authorized as part of the marketing submission and has three parts: a description of modifications (what you intend to change — retraining cadence, data sources, performance envelopes), a modification protocol (how each change will be developed, validated, and implemented, with acceptance criteria), and an impact assessment (how changes affect safety and effectiveness, including failure modes and rollback). Changes executed inside the authorized envelope do not require new submissions; changes outside it do.

A PCCP is essentially a regulator-approved MLOps contract: you negotiate your retraining and evaluation pipeline with FDA once, then operate it — which means the eval infrastructure has to exist and be documented before submission, not after clearance.

PCCP adoption is still early — appearing in roughly one in six AI-device summaries in 2024 — largely because most manufacturers lack the disciplined evaluation infrastructure a modification protocol presupposes. Teams that build eval harnesses, holdout datasets, and subgroup analysis into the development pipeline from day one get PCCPs; teams that treat evaluation as a pre-submission scramble do not.

Non-device CDS: four criteria, and how LLM features drift across the line

The 21st Century Cures Act (Section 520(o)(1)(E) of the FD&C Act) excludes certain clinical decision support software from the device definition. FDA's CDS guidance — significantly revised in January 2026 — requires all four criteria for non-device status. In plain terms, the software must:

1. not acquire, process, or analyze medical images or signals from an in vitro diagnostic or signal acquisition system;
2. display, analyze, or print medical information about a patient or general clinical information;
3. support or provide recommendations to a healthcare professional, rather than a specific directive or a time-critical output the clinician merely executes; and
4. enable the clinician to independently review the basis of the recommendation, so they need not rely primarily on the software's output.

The January 2026 revision loosened one historic pain point — software that surfaces a single recommendation can now qualify for enforcement discretion when only one option is clinically appropriate — while sharpening transparency expectations: the inputs, logic, and evidence behind a recommendation must be genuinely reviewable.

This is precisely where LLM features drift into device territory by accident. A RAG assistant that answers formulary questions from labeled sources is defensible non-device CDS. Add a feature that ingests a patient's chart and outputs "recommend holding metformin before contrast imaging," and criterion 4 is under strain: can the clinician independently review the basis, or is the model's synthesis effectively opaque? Add waveform or imaging analysis and criterion 1 is gone entirely. Because LLM products accrete capabilities sprint by sprint, the regulatory determination cannot be a one-time memo — it needs to be a gate in the release process, re-run whenever intended use, inputs, or output specificity change. Hospitals deploying vendor tools face the mirror-image problem, which is why HIPAA-compliant deployment programs increasingly include a device-status attestation in vendor review.

GxP validation of AI: from CSV theater to risk-based assurance

Inside manufacturing and quality systems, the question is not device status but validation. Any AI tool that participates in a GxP process — drafting deviation investigations, triaging complaints, reviewing batch records, flagging QC anomalies — is a computerized system that must be validated for intended use.

The good news is that FDA has spent the past several years dismantling the worst of legacy Computer System Validation. The Computer Software Assurance (CSA) final guidance, issued September 24, 2025, supersedes the script-everything era for production and quality system software: assurance effort should be proportionate to risk, unscripted and exploratory testing count as evidence, and vendor documentation can be leveraged rather than reproduced. GAMP 5 Second Edition points the same direction and explicitly contemplates machine-learning components.

For AI specifically, a defensible validation package looks like:

• Intended use definition narrow enough to test — "drafts deviation summaries for QA review" — plus explicit exclusions from autonomous GxP decisions;
• Acceptance criteria and challenge testing, including adversarial and edge-case inputs, with results against a fixed test set;
• Human-in-the-loop controls: every AI output that touches a quality decision passes through documented human review, and the reviewer's approval — not the model's draft — is the GxP act;
• Change control on the model: version pinning, revalidation triggers for model or prompt changes, and monitoring for drift in production.

Non-deterministic outputs do not break this framework; they change what "testing" means. You validate the system — model, prompts, retrieval corpus, guardrails, review workflow — against statistical acceptance criteria, the same way regulated industries have long validated processes with inherent variability.

21 CFR Part 11: AI-generated content in regulated records

Part 11 governs electronic records and electronic signatures wherever predicate rules require records — and AI creates two new pressures on it.

First, attributability. Part 11 requires secure, computer-generated, time-stamped audit trails that record who did what to a record and when. When an LLM drafts the first version of an adverse event narrative or a batch record comment, the audit trail must distinguish machine-generated content from human-authored and human-approved content. The clean pattern: the model writes under a system account, its draft is a versioned record event, and the human reviewer's edit-and-sign is a separate, attributable event. A UI that lets AI text flow silently into a signed record with no trace of its origin is an audit-trail failure waiting for an inspection.

Second, signature meaning. An electronic signature legally certifies that the signer reviewed and takes responsibility for the record. If reviewers rubber-stamp AI drafts at a pace that makes genuine review implausible, the signature's integrity — and the company's data-integrity posture — is compromised. Workflow design (queue pacing, mandatory diff views, sampling-based QC of reviews) is therefore a compliance control, not a UX nicety. The EU inspectorates read data integrity the same way, and teams operating globally should reconcile Part 11 design with Annex 11 and with EU AI Act obligations now landing on the same systems.

AI in clinical trials: protocol drafting, site selection, pharmacovigilance

Clinical development is where life sciences AI is scaling fastest, because most of it sits safely on the non-device side of the map while still demanding GxP discipline.

Protocol and document drafting. LLMs drafting protocols, informed-consent language, clinical study reports, and regulatory responses are productivity tools — but their outputs enter the GCP record stream, so versioning, review, and Part 11-compliant approval apply. The sponsor, not the model, owns every submitted word.

Site selection and enrollment forecasting. Models that rank investigative sites on historical enrollment, population data, and feasibility signals carry mostly business risk — unless their outputs feed claims to regulators, at which point FDA's January 2025 draft guidance on AI in regulatory decision-making applies. That guidance sets out a risk-based credibility assessment framework tied to context of use: define the question the model answers, assess model influence and decision consequence, and build validation evidence proportionate to that risk.

Pharmacovigilance case processing. This is the highest-volume prize — large pharma processes hundreds of thousands of individual case safety reports a year, and LLMs are strong at intake triage, narrative drafting, and MedDRA coding suggestions. It is also unforgiving: late or missed expedited reports are among the most common serious findings in PV inspections. Production programs keep humans on seriousness and causality assessment, measure model recall on the cases that matter most (serious, unexpected), and log every AI touch on a case for inspection.

In a validated environment, the eval report is not an engineering artifact — it is a regulatory record. If your model update evidence lives in a notebook on someone's laptop, you do not have evidence; you have a finding.

What regulators expect for model updates — and who builds it

Across all three jurisdictions the documentation pattern converges: pre-specified evaluation, versioned everything, human accountability. A model update in a regulated life sciences context should produce, as a byproduct of the pipeline: the model and prompt versions before and after; the frozen eval datasets and their provenance; performance against pre-set acceptance criteria, including subgroup and edge-case analysis; an impact assessment; the approval record; and rollback criteria with monitoring thresholds. That is a PCCP modification protocol, a CSA change-control package, and a credibility assessment — three regulatory dialects for the same engineering discipline.

Very few teams have people who speak both dialects fluently: engineers who can build LLM systems and who understand why an unsigned eval report is worthless in an inspection. That is the gap Gain America fills. We deploy forward-deployed engineers with validated-environment delivery experience directly into pharma and medtech teams — building the eval harnesses, audit-trail integrations, and change-control tooling inside the client's quality system, so the compliance evidence accrues as the system is built. The alternative — building fast outside the quality system and retrofitting validation later — is how programs end up choosing between a rewrite and a 483.