Most healthcare AI (artificial intelligence) bias is detected too late. By the time a team compares performance across demographic groups, many decisions that shape equity have already been made: what the model will predict, how the cohort will be defined, what data will count as evidence, and where the intervention threshold will be set.
Healthcare AI is increasingly used to identify risk earlier and support clinical decision-making at scale. But as these models move closer to clinical care, fairness cannot be treated as a final validation step, as key equity decisions often precede model training.
I see this firsthand in my work building models that scan years of patient history for early signs of arrhythmia, liver disease, and Type 1 diabetes. The data comes from tens of millions of patients and includes laboratory values, diagnosis codes, ECG findings, and medication histories. Electronic health records (EHRs) do not capture biology alone. They also reflect who had access to care, who was tested, whose symptoms were documented, and who returned for follow-up. Research in JAMA Internal Medicine has identified missingness, misclassification, measurement error, and unequal representation as sources of bias in EHR-based machine learning models.
That distinction matters because fairness and equity are not the same. Fairness refers to how a model’s performance and errors are distributed across groups, while equity asks whether the model and resulting interventions reduce or reinforce differences in access and outcomes.
Most discussions of AI bias start with the training data: Is it diverse enough? Is it representative? Those questions matter, but they assume a more consequential decision was made correctly: What exactly are we asking the model to predict?
A diagnosis code may capture who was diagnosed, not everyone who had the disease. Healthcare spending may capture who received care, not who needed it. Lab availability may capture who was tested, not who was sick.
Take an arrhythmia model. The first real decision is which signal should define a case: a diagnosis code, an ECG finding, or a medication that implies the condition. Each captures a different population, and relying on codes alone can skew the model toward patients with reliable access to cardiology.
Liver disease poses the mirror-image problem. A cirrhosis code often appears only after symptoms develop, while abnormal enzymes may be present much earlier. Defining a case by the diagnosis event can mean predicting who eventually got diagnosed rather than who actually had progressing disease.
In both cases, how a team defines a case shapes who the model will ever learn to flag.
Obermeyer and colleagues demonstrated a similar problem in a widely used population-health algorithm in Science. The system used healthcare cost as a proxy for health needs. Because less money was spent on Black patients than on equally ill white patients, the algorithm underestimated Black patients’ needs. Replacing cost with a more direct measure of illness substantially reduced the disparity.
The failure began not with model training, but with the choice of target.
Even with a defensible target and a well-calibrated score, people decide which score triggers outreach, additional screening, or lower priority. That threshold is where predictions become consequences.
The harms are not symmetric. A false negative can delay diagnosis and treatment; a false positive can create anxiety, additional testing, and cost. A Nature Medicine study found higher underdiagnosis rates among underserved groups in chest-radiograph AI models.
Fairness therefore needs to be evaluated at three levels: the score, the operating threshold, and the patient outcome. Teams should compare calibration and error rates across groups, but also ask who receives the resulting intervention and whether access to follow-up care differs. Equal-looking scores do not guarantee equitable outcomes.
Removing race, ethnicity, sex, age, or socioeconomic information from model inputs does not automatically make a model fair. Other variables may act as proxies, while deleting protected attributes can make disparate performance harder to detect.
My default approach is to preserve protected attributes for auditing even when they are excluded from model training. Their inclusion as predictive features should be a separate, documented decision supported by clinical rationale, subgroup validation, and appropriate legal and ethical review.
Auditing performance is not enough; someone must act on what the audit shows. Explainability can show which features influenced a prediction. It cannot determine whether the target was appropriate, whether the threshold was fair, or whether the inputs were shaped by unequal access to care.
The Coalition for Health AI’s Assurance Standards Guide treats fairness, safety, transparency, and accountability as related but distinct dimensions across the AI lifecycle. Model cards can make assumptions explicit, but only governance turns those observations into action.
Explainability must therefore be paired with accountability: a named owner, authority to pause or change the model, a process for responding to disparities, and ongoing monitoring after deployment.
Better models can identify missed disease, improve access, and allocate attention more consistently. The goal is not to wait for a perfect healthcare system before using AI, but to be precise about what improvement means and for whom.
For a new model, equity work should begin before training. Teams should document what the target measures and may proxy; examine who is missing from the cohort; define acceptable errors by use case and subgroup; and identify what happens on either side of the threshold.
Existing production models also deserve scrutiny. Auditing them is not a substitute for equity-first design, but is a necessary transitional step. Teams can compare what a target actually measures with its documentation and trace how scores become actions and outcomes across patient groups.
Healthcare AI does not become equitable because a checklist says so. Equity must be examined at score, threshold, and patient outcome levels. Without clarity across these three levels, a model may appear fair in evaluation while reinforcing inequity in practice.