How Medical AI Got Graded: The Evolution of Clinical LLM Benchmarks

The short answer. Medical AI evaluation wasn't designed — it accumulated, one layer at a time, each generation built because the previous one couldn't see something that mattered. Exam-style benchmarks such as MedQA became prominent, but high scores and contamination risks limit their interpretation. Realistic-conversation benchmarks followed. Then safety benchmarks, which found that accuracy doesn't predict harm. Then trials of physicians using the tools. Patient-outcome trials address a different question from all of these.

As of September 2026.

This is the history behind the five-layer evaluation framework. If you want the framework itself rather than how it came to exist, start there.

---

Era 1 (2019–2022): can it answer the question?

Early question-answering benchmarks grew out of natural language processing research and asked a relatively standardized question: can the model answer correctly?

PubMedQA [1] (2019) tested whether models could reason over biomedical abstracts to answer yes/no/maybe research questions.

MedQA [2] (2020 preprint; 2021 journal article) was built from professional licensing examinations including the USMLE. Its now-standard US test set contains 1,273 questions, and it became the field's default scoreboard.

MMLU [3] (2020) was a general reasoning benchmark that happened to include medical domains — anatomy, clinical knowledge, college medicine, medical genetics, professional medicine.

MedMCQA [4] (2022) added more than 194,000 questions from Indian AIIMS and NEET-PG examinations.

None of these were designed to determine whether an AI system was safe or useful at a bedside. They were tests of knowledge and question-answering, and they were adopted as proxies for clinical readiness because they were the tests that existed.

That inheritance shaped the next four years of discourse.

---

Era 2 (2023): the field notices exams aren't enough

The inflection came when Google consolidated MedQA, MedMCQA, PubMedQA, several medical MMLU subsets, and consumer health datasets into MultiMedQA, using it to evaluate Med-PaLM.

The headline numbers did what headline numbers do: Med-PaLM reached 67.6% on MedQA; Med-PaLM 2 soon reached 86.5%. These results came from different generations of the research, not a single 2023 experiment. [5] [6]

But the more important contribution was buried in the methods. The same authors added physician evaluation of long-form answers — scoring factuality, reasoning, alignment with scientific consensus, potential harm, usefulness, and bias.

They had recognized something the exam format couldn't capture: a model could produce the correct answer to a board question and still generate a clinically poor response.

That's the moment the field started moving from does it know medicine to does it behave like a useful medical system.

---

Era 3 (2024–2026): benchmarks built around what clinicians actually do

The next generation abandoned the single-answer exam format.

HealthBench [7] (2025) replaced exam questions with 5,000 realistic, often multi-turn health conversations, evaluated against 48,562 physician-written criteria developed with 262 physicians who had practiced across 60 countries. Its rubrics score accuracy, completeness, communication, context awareness, and appropriate escalation — not whether a final answer matches a key.

Others attacked the same problem from different angles:

• MedR-Bench [8] evaluated reasoning trajectories across 1,453 cases, not just final diagnoses
• DiagnosisArena [9] used difficult cases from clinical case reports to test professional diagnostic reasoning
• BRIDGE [10] assembled 87 tasks from 59 real-world clinical data sources across nine languages — triage, extraction, diagnosis, prognosis, coding
• MedHELM [11] attempted to organize the problem itself: a clinician-validated taxonomy of five categories, 22 subcategories, and 121 tasks encountered in actual medical work

Collectively these ask a different question: can the model perform the kinds of work medicine actually requires?

---

Era 4: safety becomes its own construct

The third shift was conceptual rather than technical. Researchers stopped assuming that accuracy predicts safety and started measuring safety directly.

NOHARM, cited here in its July 2026 preprint version, is one example. [12] Instead of asking whether a response was correct, specialists evaluated management recommendations by the benefit or harm that would result if acted upon.

Two findings reshaped how the field thinks:

Knowledge and safety require separate evaluation. The benchmark grades potential harms from recommendations and omissions, not patient outcomes.

Most severe errors were errors of omission. The model failed to recommend something necessary rather than recommending something obviously dangerous.

That second finding is the one worth carrying. A response can look reasonable, contain no falsehood, and still be unsafe because the clinically decisive action is missing.

CSEDB [13] (2026) formalized a parallel distinction — 32 specialists building 2,069 open-ended scenarios across 26 departments, scored on 30 safety and effectiveness measures including critical-illness recognition, guideline adherence, and medication safety. Performance again deteriorated substantially in high-risk scenarios.

The lesson is now hard to ignore: knowledge accuracy, clinical usefulness, and clinical safety are correlated but not interchangeable.

---

Era 5: stop evaluating the model alone

Beginning around 2023–2024, studies started asking what happens when a physician actually uses one of these systems.

A randomized trial gave 50 physicians conventional resources or conventional resources plus GPT-4. The model performed impressively on its own — and physician diagnostic reasoning didn't significantly improve. 76% versus 74%. [14]

Call it the model-to-user gap: capability in isolation doesn't guarantee benefit once inserted into human decision-making.

A subsequent trial of 92 practicing physicians found a different answer. For complex management cases, GPT-4 assistance improved performance by 6.5 percentage points, though physicians spent more time per case. [15]

Both results are valid. Whether AI helps depends on the task, the interface, and how the model enters the reasoning process — which is exactly why this layer can't be skipped.

A randomized study embedded within the NOHARM project pushed further: AI assistance improved physician performance, but physicians frequently failed to incorporate useful recommendations the AI had made. The limiting factor was no longer what the model knew. It was whether useful information survived the human–AI interaction. [12]

---

Era 6: does it help patients?

Patient outcomes must be measured directly when the intended claim is better care.

The most important example is a 2026 pragmatic cluster-randomized trial in Kenya — clinical officers across 16 primary-care facilities randomized to an EMR with or without an LLM-based clinical copilot, enrolling 9,691 patients. The primary outcome wasn't an exam score or a physician rating. It was treatment failure within 14 days.

Treatment failure was 2.2% with AI and 2.0% with usual care; the adjusted odds ratio was 0.77 (95% CI 0.55–1.08). The difference was not statistically significant. This does not establish equivalence or rule out a modest benefit. [16]

A US deployment trial, available as a preprint, randomized 284 outpatient clinicians across 42 specialties to a generative chart-summarization tool. It modestly reduced task load and some burnout measures, didn't meaningfully reduce charting time, and clinician engagement declined substantially over 90 days. Patient outcomes weren't the primary endpoint. [17]

These two studies illustrate why the endpoint matters: improved workload and improved patient health are different claims.

---

Why exam benchmarks need context

Exam scores remain useful for the task they measure. Four issues limit broader interpretations:

Saturation. Three frontier models scored 90.2–97.4% on a 500-question MedQA sample in one 2026 study. High scores leave less room to distinguish systems and say little about tasks that were not tested. [25]

Contamination. Researchers have documented overlap between public medical test questions and pretraining corpora, and memorization in medically adapted models. A temporal audit of diagnostic cases did not find lower accuracy on the newer cases it assessed. These are different tests; neither settles the training exposure of every model. [18] [19] [20]

Format inflation. Replace the correct option with "None of the other answers" and accuracy collapses across every model. That shows format sensitivity, not definitive proof that a model cannot reason. Deliver the same content conversationally rather than as a tidy vignette and diagnostic accuracy falls again. [21] [22]

The knowledge–practice gap. A systematic review of 39 benchmarks found exam-style accuracy of 84–90% against practice-based success of 45–69%, with safety performance at 40–50%. These heterogeneous benchmark results are not a pooled estimate of real-world clinical success. [23] On 2,400 real clinical cases, models performed significantly worse than clinicians, failed to follow guidelines, couldn't reliably interpret labs even with reference ranges, and were sensitive to prompt changes. [24]

---

What the sequence tells you

Read as a history, the benchmarks describe a field gradually discovering that the thing it could measure easily was not the thing that mattered.

Does it know the answer?
  → Can it reason and communicate realistically?
    → Could its recommendation harm someone?
      → Does a clinician decide better using it?
        → Does deploying it improve patient outcomes?

These approaches overlap in time; the sequence is a conceptual guide, not a claim that each stage took a fixed number of years.

When a vendor cites a benchmark at you, the useful question is which era it belongs to — and what the next era's methods would have caught that this one couldn't.

---

Frequently asked questions

What is MedQA? A benchmark built from professional medical licensing examinations including the USMLE; the standard US test set has 1,273 multiple-choice questions. It became the default scoreboard for medical AI but exam performance should not be treated as proof of safe clinical use.

What is HealthBench? A 2025 benchmark of 5,000 realistic, often multi-turn health conversations evaluated against 48,562 physician-written criteria, developed with 262 physicians across 60 countries. It scores accuracy, completeness, communication, context awareness, and escalation rather than matching an answer key.

Why are medical AI benchmark scores so high? Model capability, question format, and possible training overlap all matter. Scores from different benchmark designs cannot be combined into a single real-world success rate.

Do benchmark scores predict clinical safety? A knowledge test does not directly assess clinical harm. Safety-focused evaluations examine additional failure modes, including omitted actions.

Has any AI tool been shown to improve patient outcomes? The Kenya LLM trial discussed here found no significant difference in 14-day treatment failure. It does not settle whether other AI systems or implementations improve outcomes.

Related reading

• How to Evaluate a Clinical AI Tool: The Five Layers
• OpenEvidence vs ChatGPT: What the Evidence Actually Shows
• Google's AMIE: What the Evidence Actually Shows

Educational content for clinicians. Not medical advice. Figures as of September 2026.

References

Sources checked September 17, 2026. Preprints and conference versions are identified below.

1. Jin Q, Dhingra B, Liu Z, Cohen WW, Lu X. PubMedQA: A Dataset for Biomedical Research Question Answering. EMNLP-IJCNLP. 2019. doi:10.18653/v1/D19-1259.

2. Jin D, et al. What Disease Does This Patient Have? A Large-Scale Open Domain Question Answering Dataset from Medical Exams. Applied Sciences. 2021;11:6421. doi:10.3390/app11146421. Preprint first posted 2020.

3. Hendrycks D, et al. Measuring Massive Multitask Language Understanding. ICLR. 2021. Preprint first posted 2020. doi:10.48550/arXiv.2009.03300.

4. Pal A, Umapathi LK, Sankarasubbu M. MedMCQA: A Large-scale Multi-Subject Multi-Choice Dataset for Medical Domain Question Answering. PMLR. 2022;174:248–260. doi:10.48550/arXiv.2203.14371.

5. Singhal K, et al. Large language models encode clinical knowledge. Nature. 2023;620:172–180. doi:10.1038/s41586-023-06291-2.

6. Singhal K, et al. Toward expert-level medical question answering with large language models. Nature Medicine. 2025;31:943–950. doi:10.1038/s41591-024-03423-7.

7. Arora RK, et al. HealthBench: Evaluating Large Language Models Towards Improved Human Health. arXiv. 2025. Technical preprint. doi:10.48550/arXiv.2505.08775.

8. Qiu P, et al. Quantifying the reasoning abilities of LLMs on clinical cases. Nature Communications. 2025. doi:10.1038/s41467-025-64769-1.

9. Zhu Y, et al. DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models. arXiv:2505.14107v5. 2026. Accepted to Findings of ACL 2026. doi:10.48550/arXiv.2505.14107.

10. Wu J, et al. BRIDGE: benchmarking large language models for understanding real-world clinical practice texts. Nature Biomedical Engineering. 2026. doi:10.1038/s41551-026-01719-2.

11. Bedi S, et al. Holistic evaluation of large language models for medical tasks with MedHELM. Nature Medicine. 2026;32:943–951. doi:10.1038/s41591-025-04151-2.

12. Wu D, et al. First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations. arXiv:2512.01241v4. July 13, 2026. PREPRINT, not peer reviewed. doi:10.48550/arXiv.2512.01241.

13. Wang S, et al. A novel evaluation benchmark for medical LLMs illuminating safety and effectiveness in clinical domains. npj Digital Medicine. 2026;9:91. doi:10.1038/s41746-025-02277-8.

14. Goh E, et al. Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial. JAMA Network Open. 2024;7:e2440969. doi:10.1001/jamanetworkopen.2024.40969. PMID:39466245.

15. Goh E, et al. GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial. Nature Medicine. 2025;31:1233–1238. doi:10.1038/s41591-024-03456-y.

16. Agweyu A, et al. Generative AI-enabled clinical decision support system in primary care: a pragmatic, cluster-randomized trial. Nature Medicine. 2026;32:3032–3039. doi:10.1038/s41591-026-04503-6. PMID:42362867.

17. Chin AT, et al. A Pragmatic Randomized Trial of an EHR-Integrated Generative AI Chart Summarization Tool for Ambulatory Clinicians. medRxiv. 2026. PREPRINT, not peer reviewed. doi:10.64898/2026.08.26.26361496.

18. Gallifant J, et al. Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks. Findings of EMNLP. 2024. doi:10.18653/v1/2024.findings-emnlp.726.

19. Li A, et al. Memorization in large language models in medicine prevalence characteristics and implications. Nature Communications. 2026;17:7729. doi:10.1038/s41467-026-73779-6.

20. Sheppert AP, Adams B, Sheppert AD, Riley S. Reasoning or reciting? A temporal contamination audit of large language models in clinical medicine. JAMIA. 2026. doi:10.1093/jamia/ocag069. PMID:42281378.

21. Bedi S, et al. Fidelity of Medical Reasoning in Large Language Models. JAMA Network Open. 2025;8(8):e2526021. doi:10.1001/jamanetworkopen.2025.26021.

22. Johri S, et al. An evaluation framework for clinical use of large language models in patient interaction tasks. Nature Medicine. 2025. doi:10.1038/s41591-024-03328-5.

23. Gong EJ, Bang CS, Lee JJ, Baik GH. Knowledge-Practice Performance Gap in Clinical Large Language Models: Systematic Review of 39 Benchmarks. Journal of Medical Internet Research. 2025;27:e84120. doi:10.2196/84120.

24. Hager P, et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine. 2024;30:2613–2622. doi:10.1038/s41591-024-03097-1.

25. Vishwanath K, et al. General-purpose large language models outperform specialized clinical AI tools on medical benchmarks. Nature Medicine. 2026;32:2405–2409. doi:10.1038/s41591-026-04431-5.