Microsoft AI says it has a tool that can diagnose conditions "up to four times more accurately than human doctors", and this time the claim comes with a peer-reviewed paper in Science. The tool is the Diagnostic Orchestrator, a team of AI agents that behaves like a panel of specialists. The headline is strong. The evidence behind it is narrower: it was tested on past cases, not on live patients, and Microsoft's own AI chief says it "needs clinical validation."
This post lays out what is reported, what is confirmed, what is not, and how it fits with other clinical AI studies we have covered, such as Google's AMIE in a real clinic.
TL;DR: the claims and the checks
| Question | Answer |
|---|---|
| What is it? | A multi-agent system (Microsoft calls it the Diagnostic Orchestrator, earlier MAI-DxO) that asks questions, orders tests and reaches a diagnosis |
| Where was it published? | Science, on Thursday, October 22, 2026, per The Verge |
| Headline result? | About 83% on NEJM-derived cases vs roughly 36% (doctors with references) and 22% (without) |
| Emergency department result? | 80% vs an average of 47% for human physicians (Beth Israel Deaconess cases) |
| Which model? | OpenAI GPT-5 inside the orchestrator; paper does not test GPT-6 Astra, Gemini 4 Argon or Opus 5.5 |
| Tested on real patients? | No. Only past cases |
| Available to use? | No. No date or pricing from Microsoft |
| What is next? | Hospital testing with Beth Israel Deaconess, likely "in the next couple of months" |
What the Diagnostic Orchestrator actually is
The core idea is not a single bigger model. It is an orchestration layer. According to The Verge, the system "uses multiple AI agents to reason through symptoms and medical histories, decide what tests to order, and arrive at a diagnosis, mimicking the process doctors use to investigate cases." It is designed to work like a panel of medical experts, so one agent can propose a differential, another can challenge it, and another can weigh which test is worth its cost.
That structure matters because earlier medical AI benchmarks often handed the model a full case summary and asked for an answer. Real diagnosis is sequential. A doctor asks a question, gets an answer, orders a test, sees a result and revises. Microsoft's benchmark, SDBench, converts case reports into that step-by-step format. The Verge says Microsoft compiled the cases into SDBench and plans to open-source it.
If you build agent systems, the pattern will look familiar: a coordinator, specialist roles, a budget for tool calls (here, tests with a cost), and a stopping rule. We covered the general tradeoffs of this design in our posts on agent harnesses and the agent database-state benchmark.
The numbers, side by side
All figures below are as reported by The Verge from the Science paper. We have not independently reproduced them, and the paper's full tables are the place to check details.
| Test set | Diagnostic Orchestrator | Human physicians |
|---|---|---|
| NEJM case reports | about 83% | about 36% (with references), about 22% (without) |
| Beth Israel Deaconess emergency cases | 80% | average 47% |
| Cost to reach diagnosis | Lower estimated cost | Higher |
The "up to four times" line is the ratio at the favorable end: 83 divided by 22 is about 3.8, our own arithmetic. Against doctors who were allowed references, the gap is about 2.3 times, and on emergency cases about 1.7 times. All of those are large gaps, but "four times" is the best-case comparison, so read the lower rows too.
How this compares with the 2025 preprint
Microsoft first described MAI-DxO in mid-2025. Coverage at the time, including GeekWire and Fortune, reported about 85% accuracy on 304 New England Journal of Medicine cases against roughly 20% for 21 physicians from the UK and US, plus a diagnostic cost cut of about 20% versus physicians. The peer-reviewed numbers (83% vs 22% and 36%) are in the same range, which is a point in favor of the result being stable rather than a one-off. The new paper adds the emergency department data and the reference-assisted physician group.
What is claimed versus what is verified
| Claim | Status |
|---|---|
| Published in Science | Reported by The Verge; check the journal page for the paper itself |
| About 83% on NEJM cases | Reported figure from the paper; not independently reproduced |
| Beats frontier models used alone | Reported by the researchers; tested against models available when the study was run |
| Better than doctors in practice | Not verified. Only retrospective cases were tested |
| Cheaper diagnosis | Reported as lower estimated cost; this is a modeled number, not a billing result |
| Ready for deployment | No. Suleyman: "It needs clinical validation" |
Three caveats deserve plain language.
- Past cases are not patients. NEJM case reports are written up after the diagnosis is known. They are tidy narratives selected because they are instructive. The emergency cases come from a large database of admissions, which is closer to real data, but still retrospective.
- The human baseline is artificial. Doctors in these studies are asked to work through an unfamiliar case in a research setting. In the 2025 version, the physicians did not have colleagues, books or AI available, which made the comparison less like everyday practice. The new reference-assisted group helps, and it still scored far below the system.
- Models have moved on. The Verge notes the paper does not include newer models like GPT-6 Astra, Meta's Muse Wonder Full, Google's Gemini 4 Argon or Anthropic's Opus 5.5. The researchers say later testing suggests "improvements in model capability do not necessarily diminish the benefits" of using multiple agents. That is the authors' statement, not something the paper's main tables demonstrate for the newest models.
What Microsoft and Suleyman say
Microsoft AI CEO Mustafa Suleyman told The Verge that healthcare impact "is finally here" and predicted "something that is indistinguishable from human performance" on medical tasks within a couple of years. He also said: "It needs clinical validation. That is a leap, but this is a major milestone along that journey."
Dr. Dominic King, Microsoft's AI health lead and a coauthor, said testing in hospitals with Beth Israel Deaconess Medical Center will likely "get underway in the next couple of months," without a firm timeline. Microsoft also published a blog post and an article in a BMJ digital health journal laying out five more areas it believes AI must master: coordinating clinical care, making treatment decisions, and handling the human side of medicine.
The branding is worth noting. Microsoft calls the goal "medical superintelligence", a phrase it has used since before "superintelligence" became a catch-all term in policy circles. We looked at the company's wider framing in Suleyman's Humanist AI code of conduct. Whether you like the label or not, the test results should be judged on their own terms.
How this sits among other clinical AI results
Clinical AI news has moved fast this month, and the evidence quality varies a lot:
- Google AMIE was evaluated in a real clinic with patients, with a clear safety setup. See our breakdown of the Lancet study, which is a stronger form of evidence than retrospective cases.
- Nolla Health in Utah got permission for an AI to issue initial acne prescriptions, a narrow, supervised regulatory experiment: Nolla Health explained.
- Meta reported a research push using millions of records: Muse Health research.
- Mass General Brigham published a face-photo model for cancer survival in the Lancet family: FAHR.
- Radical Numerics reported an AI-designed phage given to a patient: Omnii-1.
The ladder of evidence runs roughly: benchmark on past cases, then prospective study in a clinic, then randomized outcome study, then post-deployment monitoring. Microsoft's paper sits on the first rung, with a strong peer-review stamp. The Beth Israel pilot would be the second.
What this means for builders
If you build agent products, even outside medicine, three lessons transfer.
- Orchestration can beat a bigger model. The authors argue the benefit of multiple cooperating agents persisted as the underlying model improved. That is a claim to test in your own domain before copying the design.
- Make the task sequential. SDBench's biggest idea is turning a static question into an interactive one with costs. Evaluations that include cost per step reward systems that know when to stop.
- Report the weakest baseline and the strongest one. The gap against doctors with references (36%) is the more honest comparison than the gap against doctors without (22%). Good eval writeups show both.
If you are a patient or clinician, the practical advice is simple: do not treat any chatbot, including this one, as a replacement for a clinician's judgment today, and watch for the prospective hospital results.
What to watch next
- The Science paper's full tables, especially the confidence intervals and case counts.
- The open-source release of SDBench, which would let others run the newest models against the same cases.
- The Beth Israel Deaconess hospital pilot and what endpoints it reports: accuracy, time to diagnosis, cost, and any harm events.
- Whether Microsoft says how it would commercialize the tool, given its existing healthcare footprint and the popularity of health questions in Copilot.
- Third-party re-runs using GPT-6 Astra, Gemini 4 Argon or Opus 5.5, which would show how much the orchestration layer still adds.
Related reading
- Google AMIE in a real clinic: the Lancet study
- Nolla Health and AI-issued acne prescriptions in Utah
- Meta Muse health research
- Mass General Brigham FAHR face model
- Omnii-1 and the first AI-designed phage given to a patient
- Suleyman's Humanist AI code of conduct
Figures are as reported on October 22, 2026, from The Verge's coverage of the Science paper and earlier 2025 reporting. Check the paper for exact tables.