Eigen RadarAI
Analysis

MLPerf adds agent post-training to AI system tests

MLCommons released MLPerf Training v6.1 results covering 153 systems from 24 organisations. The benchmark round adds a recommendation workload and a test of reinforcement learning after initial model training, including software-engineering tasks. Results span NVIDIA, AMD and Google hardware, with new visualisation tools for comparing systems and ranking configurations of similar scale rather than treating every submission as equivalent.

Artificial Intelligence··Morning
In a bright AI hardware test hall, differently designed accelerator system cabinets stand side by side under test while two engineers compare system runs at a shared console.

Training tests gain new workloads

MLCommons, an AI engineering consortium bringing together industry and academia, published MLPerf Training v6.1 results spanning complete hardware and software systems. The round adds Qwen3.5-397B, a workload measuring reinforcement learning after a model’s initial training. It covers software-engineering work, including tool calling. Another addition, DLRMv4-HSTU, replaces an earlier recommendation-system test and uses the Yambda dataset with an attention-based model architecture.[1]

Submissions span different hardware

The published round contains 250 results from 24 submitting organisations and 153 systems. Entries include NVIDIA, AMD and Google hardware. NVIDIA’s Rubin-generation graphics processors and Google’s Ironwood tensor processors appear for the first time. The results compare complete systems on defined training workloads, with cluster size and software affecting the reported training times.[1], [2]

Cloud-based configurations accounted for 57 systems, or 37 per cent, compared with 29 per cent in v6.0. A total of 12 submitted systems used at least 1,000 accelerators. The largest configuration contained 10,240. These counts describe the systems entered in this benchmark round.[1]

Tools compare systems of similar scale

New visualisation tools accompany the results, supporting comparisons across configurations and rankings within similar system scales. Full tables are available through the Training benchmark page, alongside supplemental explanations from submitters. Crusoe is a first-time participant in the Training round. MLCommons describes the expanded post-training test as a way of measuring the processes used to develop agent capabilities.[1]

References

  1. News sourceMLCommonsMLPerf Training results add agent post-training to system comparisons↩1↩2↩3↩4
  2. News sourceHardwareluxxMLPerf publishes new AI training benchmark results↩