Microsoft unveiled the Maia 300, its third-generation AI accelerator, on 12 October and intends to use its own chips as the main hardware for running AI models in its services, CNBC reported. Production is due to begin in 2027. The plan would shift more of the work of answering incoming AI requests onto processors Microsoft designed for its Azure data centres, after years in which the company has relied heavily on Nvidia graphics processors.
Key points
- Maia 300 uses TSMC’s N2P process and eight HBM4 memory stacks.
- Microsoft expects 96 chips in each rack and production to begin in 2027.
- The company intends Maia to become its primary architecture for running AI models, while continuing to work with Nvidia and AMD on training.
- Microsoft’s Maia accelerators cannot currently be rented by the hour by other companies.
Maia 300 brings HBM4 close to the processor
The Maia 300 uses TSMC’s N2P manufacturing process and places eight stacks of HBM4 memory in the same package as its computing chip and supporting chiplets, Heise reported. Those memory stacks provide a combined data-transfer rate of around 18 TByte/s. The arrangement keeps memory beside the processor, rather than farther away in a data-centre machine: for work that repeatedly needs stored information, the distance between the two is part of the engineering problem.
Inference is the work a model does when it responds to a request. A token is a piece of text it produces or processes, so tokens per second measures output at a different point from the calculations a chip can perform internally. Microsoft says Maia 300 is tailored to the needs of its own AI offerings. Its predecessor, Maia 200, is already running productive workloads including Microsoft 365 Copilot, according to Microsoft’s account reported by Heise.
Drafting an email could take less time if the expected speed-up reaches the part of the service handling that request. A faster stream of generated words would shorten the wait for a reply, though the rack-level figures Microsoft has given concern how much output its hardware can produce, rather than the time taken by that particular task.
Microsoft announced Maia 100 in 2023 and followed it with Maia 200 in early 2026. The earlier chip uses TSMC’s N3P process and has 216 GByte of HBM3e memory. In June, Microsoft announced its first Maia 200 instances for selected pilot customers. Chief executive Satya Nadella told analysts in July that Maia 200 delivered 30% better performance per dollar than the latest-generation hardware in Microsoft’s fleet, a comparison against the company’s own equipment rather than every chip available elsewhere.
The 96-chip Maia 300 rack
Microsoft expects to fit 96 Maia 300 chips into a rack, compared with 48 Maia 200 chips in the preceding design. The company says the new rack can produce up to seven times as many tokens per second as a Maia 200 rack. That comparison combines a new chip with twice as many chips per rack, rather than isolating the improvement of one processor. Microsoft also puts a Maia 300 rack’s power use at up to 167 kW, according to Heise.
The two accounts give different comparisons for an approximately 40% output gain. CNBC reported Microsoft’s figure as 40% more tokens per megawatt than GPUs. Heise reported that a Maia 300 system is expected to produce around 40% more tokens than a Maia 200 system at the same power consumption. Tokens per megawatt matters in a data centre because electrical capacity limits how much computing equipment can run, but the named points of comparison differ: GPUs in one account, Microsoft’s previous Maia generation in the other.
Microsoft’s general manager Alistair Speirs said the company wants better use of the power available to it and linked data-centre power consumption directly to operating costs. Technical fellow Jim Kleewein said parts of Microsoft 365 Copilot should become faster with Maia 300. Those are expectations about a future deployment, alongside the company’s comparisons of hardware capacity.
Azure keeps Maia chips largely in-house
Amazon and Google have made successive generations of their own AI chips available to cloud customers, while Microsoft has primarily offered Azure instances using Nvidia accelerators. Unlike Google’s TPUs and Amazon’s Inferentia and Trainium, Maia accelerators cannot be rented by the hour by outside companies. Patrick Moorhead, chief executive of advisory firm Moor Insights and Strategy, told CNBC that nobody outside Microsoft had yet tested them.
Microsoft still works with Nvidia and AMD to train models, Speirs said. Evercore ISI analysts estimated that Maia would account for about 10% of Microsoft’s AI-chip spending by the first quarter of 2027. Microsoft is aiming for multi-gigawatt deployments across the Maia family, although Speirs declined to give CNBC a timetable for reaching that scale.
Microsoft plans to equip its Azure data centres with Maia 300 racks starting in 2027, with the planned installations drawing several gigawatts of electricity in total.
