Microsoft’s Diagnostic Orchestrator reached around 83 percent diagnostic accuracy on cases drawn from hundreds of reports in The New England Journal of Medicine, against roughly 36 percent for generalist physicians using references, in findings published in Science on 22 October, The Verge reported. Physicians working without references scored 22 percent. On emergency department cases drawn from admissions at Beth Israel Deaconess Medical Center in Boston, the system scored 80 percent, against an average of 47 percent for physicians.
Powered by OpenAI’s GPT-5, the system uses several AI agents to examine symptoms and medical histories, decide which tests to order and work towards a diagnosis. Microsoft assembled the cases into a benchmark it calls SDBench. The company says it plans to open-source it.
Getting a diagnosis could involve an AI system weighing possible explanations and choosing tests before reaching a conclusion, if its performance on past cases carries over to clinical care.
The system has been tested only on past cases, not validated in clinical settings. Microsoft AI chief Mustafa Suleyman told The Verge: “It needs clinical validation.”
