Meta MTIA 400 Puts 9.4TB/s HBM3e Behind FP4 Inference

Custom AI silicon usually gets built for one job. Meta's newest accelerator is being built for two that pull in opposite directions, and the design choices it made to bridge them are the most interesting thing disclosed at Hot Chips 2026. On August 25 and 26, Meta detailed MTIA 400, the second of four planned generations of its Training and Inference Accelerator, currently in lab testing ahead of data center deployment.

• MTIA 400 carries eight HBM3e stacks delivering 9.4TB/s of memory bandwidth
• It adds hardware MXFP4 support, alongside MX4, MS8 and MS8S microscaling formats
• Up to 72 accelerators fit in a single scale-up domain, linked by an Ethernet fabric at 1.2TB/s
• Meta reports roughly 15x the FP16 compute and 46x the DRAM bandwidth of MTIA 200, with 5x more SRAM bandwidth

Why One Chip for Both Ad Ranking and LLM Work

Meta's two biggest compute bills look nothing alike. Deep learning recommender models, the systems that rank ads and feed content, are dominated by enormous embedding tables and are bound by memory capacity and random-access bandwidth. Large language model work is dominated by dense matrix math and is bound by arithmetic throughput and interconnect.

Building one accelerator that serves both is unusual, and MTIA 400's layout shows the compromise. The die places two compute chiplets in the centre, holding the core compute engines and the collective engines that handle cross-chip communication, with HBM stacks and networking arranged around them. An 8x6 grid of processing elements, including a redundancy row for yield, does the actual work. Meta also describes a host embedding cache specifically for managing the large embedding tables that recommender inference depends on — a feature that would be pure overhead on a pure LLM part.

The 46x jump in DRAM bandwidth over MTIA 200 is the number that gives that dual mandate away. Compute grew 15x; the memory path grew three times faster than that. This is a chip designed by people who know their bottleneck.

What Hardware FP4 Support Actually Buys

MTIA 400 introduces hardware support for MXFP4, the microscaling four-bit format, along with MX4, MS8 and MS8S. Microscaling works by attaching a shared exponent to a small block of low-precision values, which recovers much of the dynamic range that naive four-bit quantisation throws away.

The practical effect is more parameters per gigabyte of HBM and more useful math per watt, provided the model tolerates the precision drop. That trade-off is the same one running through the rest of this year's accelerator designs — we walked through the software side of it in our LLM quantization guide. MS8S, a 16x16 FP8 block configuration, is aimed at training workloads rather than inference, which fits the roadmap below.

How Big Is the Scale-Up Domain?

Seventy-two accelerators in one scale-up domain, connected by an Ethernet-based fabric running at 1.2TB/s. That number is not a coincidence: 72 has become the industry's convergence point for a single tightly-coupled domain, and it puts MTIA 400 in the same structural class as the rack-scale systems we covered in Nvidia's Vera Rubin NVL72 tokens-per-watt targets.

Choosing Ethernet for the scale-up fabric rather than a proprietary link is the notable call. It trades some peak efficiency for a supply chain and an operational model Meta already runs at enormous scale.

What Comes After MTIA 400

Meta laid out four generations rather than one. MTIA 300 was the DLRM training part that reached parity with GPUs on that workload. MTIA 400 broadens into general inference with FP4. MTIA 450 is aimed at improved generative AI inference performance, and MTIA 500 pushes scale-up domains beyond 72 accelerators.

Publishing a four-generation roadmap is itself a signal. It tells suppliers and internal teams that custom silicon is now a standing commitment rather than an experiment, which is the same message Intel sent with Diamond Rapids and Crescent Island at the same conference. More of our silicon coverage sits on the AI page.

Worth keeping in perspective: MTIA 400 is in lab testing, not deployment, and Meta's comparisons are against its own prior generation rather than against merchant silicon. The architecture disclosures are solid; the performance story will need production data behind it.

Sources: ServeTheHome — August 25, 2026; The Register — August 26, 2026; Tom's Hardware — August 2026.