Hospitals need more than accuracy scores to validate medical AI

The International Medical Device Regulators Forum (IMDRF) finalized 10 good machine learning practice principles for artificial intelligence (AI) in medical devices on January 27, 2025. The principles place intended purpose, representative data, independent testing, human-AI interaction and post-deployment monitoring in the same product lifecycle.

For a hospital, validation is an evidence chain connecting a defined clinical use to test data, workflow behavior and action after deployment. APPI News's broader deployment framework separates purpose, evidence, interoperability, human control and monitoring; the narrower validation record determines whether those controls describe the same model in the same setting.

An accuracy score needs a defined clinical use

Accuracy counts correct outputs under a particular test design. It does not show which errors occurred, how often the system produced no usable result or what each error means for the decision that follows. Two systems with the same accuracy can have different false-positive and false-negative patterns.

An acceptance test therefore needs to name the intended user, patient population, input, output, care setting and decision that the output may influence. It also needs exclusions and a fallback for missing, poor-quality or unsupported inputs. A benchmark without those boundaries cannot establish readiness for a clinical workflow.

IMDRF connects intended purpose to expected benefits, patient risks and the context of use. Its principles also call for clinically meaningful performance goals derived from those benefits and risks, rather than one measure applied to every product.

Freeze the scope and version before testing

A validation record should identify the model and software version, the input specification, the intended population and the workflow being tested. It should also record the evaluation date, participating sites, equipment or record systems and any preprocessing applied before the model received the data.

Those details separate a repeatable test from a demonstration. If a vendor later changes the model, input mapping, decision threshold or interface, the hospital can identify which part of the evidence no longer describes the deployed system.

The scope must be narrower than a marketing phrase such as “supports diagnosis” or “assists clinicians.” A usable statement specifies whether the system ranks a worklist, flags a possible finding, drafts text or supplies information for another decision, and it identifies the person who can accept, reject or disregard the output.

Independent data must also represent the intended setting

IMDRF says clinical evaluation should use data representing the intended patient population, use environment and measurement inputs, while training and test datasets should remain appropriately independent. The document says potential dependence related to patients, sites and data acquisition should be addressed, with external validation proportionate to risk.

A random split does not settle independence when records from the same patient, site or acquisition process appear on both sides. Repeated images, copied notes or closely related encounters can make a test look separate while preserving information from development.

External data introduce a different question: whether the evidence covers the population and conditions in the proposed deployment. A test from another hospital may add useful variation, but it does not automatically represent different scanners, laboratory methods, coding practices, missing-data patterns or patient groups.

The evaluation file should state where each dataset came from, how records were separated, which groups were prespecified and where the sample is too small to support a conclusion. An aggregate result cannot fill an evidence gap for a subgroup that was not adequately represented.

The reference standard shapes the result

A model is tested against a chosen reference, not against an abstract form of truth. The record should explain who created image labels, which finalized note was used to judge a summary, how disagreements were resolved and how long an outcome was followed when time affects the answer.

IMDRF treats fit-for-purpose reference standards as a separate principle. It calls for documentation of why the reference was chosen, its limitations and its suitability for the intended environment.

That makes reference quality part of the result. A precise model score cannot correct inconsistent labels, a reference assembled after seeing the model output or an outcome definition that differs from the intended use.

Hospitals need to test the human-AI team

Clinical use adds time pressure, incomplete records, competing alerts and people with different levels of experience. IMDRF calls for assessment of the human-AI team in the intended environment, including user understanding, overreliance, autonomy and foreseeable misuse.

A workflow test can observe whether users notice missing inputs, understand the output and its limits, and retain a route to continue work during an outage. It can also measure whether the result arrives soon enough for the intended decision and whether the interface directs it to the correct record and user.

Human review is not established by placing a confirmation button on a screen. The validation record needs to show what the reviewer can inspect, change or reject, how disagreement is captured and who handles an unresolved or unsafe output.

Deployment turns validation into monitoring

A 2025 US Food and Drug Administration (FDA) request for public comment says retrospective testing and static benchmarks are not designed to predict behavior in dynamic clinical environments. It identifies changes in clinical practice, patient demographics, data inputs, infrastructure, user behavior and workflow integration as factors that can affect performance.

The US document seeks evidence and methods; it explicitly says it is not draft or final guidance and does not state regulatory expectations. Its questions cover monitoring metrics, data quality, clinical outcomes, user feedback, assessment triggers and responses to performance degradation.

A hospital monitoring plan can use those questions to define a local baseline before release. Candidate signals include missing-input rates, output distributions, unusable-result rates, performance in prespecified groups, clinician overrides, incident reports, latency and service interruptions.

A changed signal starts an investigation; it does not by itself prove that the model failed or identify the cause. The response may require checking data pipelines, coding changes, equipment, workflow, user behavior and the model version before deciding whether to continue, restrict or pause use.

Stop conditions need an owner and a recovery path

No cited international document supplies one numerical stop threshold for every medical AI system. The threshold has to match the intended use, baseline variation, consequence of error and time needed to detect a harmful change.

A monitoring record should state who reviews each signal, when an alert becomes an investigation and who can suspend the system. It should also identify the fallback process, the last accepted version and the evidence required before service resumes.

A pause is different from a model update. The first limits exposure while a problem is investigated; the second changes the system and requires evidence that the new version meets its acceptance criteria.

Model changes reopen the evidence file

Final US FDA guidance issued in August 2025 says a predetermined change control plan should describe planned modifications, methods for developing, validating and implementing them, and an assessment of their effects. The guidance applies to specified US 510(k), De Novo and premarket approval pathways, not to every country or every medical AI product.

A hospital still needs a release record connecting the product's authorization in the applicable country, deployed version, local interface and local acceptance test. APPI News's separate report on medical AI software updates explains the distinction between a regulator-authorized change plan in a named market and a hospital's decision to activate a version.

Retraining after drift should not overwrite the evidence for the prior model. Keeping both versions, their datasets, acceptance results and deployment dates allows investigators to reconstruct a change and restore the last accepted release when the system permits rollback.

FHIR can exchange data but cannot validate a model

Health Level Seven International (HL7) defines Fast Healthcare Interoperability Resources (FHIR) Release 5 as a standard for exchanging health information electronically. Its modular resources can structure information moving among a record system, device and AI service, but a successful exchange does not establish that a field is complete, correctly coded or suitable for the model.

HL7's security specification says nothing in FHIR requires a particular security implementation and that real-world systems must make their own authentication, authorization and access-control decisions. FHIR resources can support provenance and audit records, while the surrounding organization still decides who may retrieve data and how access is reviewed.

Interface validation should test patient identifiers, units, code systems, timestamps, missing values, source context and version identifiers. It also needs to show that each output reaches the intended record and user without losing the information needed to interpret it.

A validation file should connect six records

The following six-part file is an APPI News synthesis of the cited documents, not an IMDRF certification scheme or a legal test. Each record should identify the same intended use, population, inputs, outputs and deployed version.

• Scope: intended user, population, care setting, input, output, affected decision, exclusions and fallback.
• Data: provenance, collection sites, dates, equipment, preprocessing, subgroup coverage and separation from development data.
• Reference: label or outcome definition, adjudication method, follow-up period and known limitations.
• Results: prespecified measures, acceptance thresholds, false-positive and false-negative patterns, unusable inputs and subgroup findings.
• Workflow: interface behavior, human authority, audit trail, training, outage handling and unresolved errors.
• Monitoring: baseline, review schedule, alert and stop conditions, responsible roles, change control and recovery route.

The file exposes mismatches that one score can hide. A test population narrower than the procurement claim, an interface that changes a unit or a monitoring plan tied to another model version means the evidence does not describe the system being accepted.

Published guidance does not measure hospital adoption

The cited sources define international principles, technical structures and US regulatory guidance. They do not approve a specific product worldwide, assign liability after an error or establish that evidence from one hospital transfers to another.

APPI News could not find a published cross-country dataset showing how many hospitals use prespecified pause and rollback thresholds for deployed medical AI at the time of writing. Country-level policy surveys and individual case reports cannot establish hospital-level adoption of the full validation and monitoring process.

A hospital acceptance decision can be traced only when intended use, independent evidence, workflow testing, deployed version and monitoring response describe the same system. Without that chain, an accuracy score remains a result from a test, not evidence that the clinical deployment is controlled.