MLPerf Storage v3.0 Expands Into AI Inference

MLCommons released results Tuesday for MLPerf Storage v3.0, the industry benchmark that measures how storage systems handle machine learning workloads. Version 3.0 adds two tests aimed at AI inference and opens the suite to S3 object storage for the first time, extending a benchmark that had measured only training and checkpointing.

The update pushes MLPerf Storage past training data delivery and into how storage supports AI systems already serving users. Nineteen organizations submitted 144 performance results this round, including 11 first-time entrants such as Azure, NVIDIA and Nebius. The results also reveal a wide spread in power efficiency among competing systems, evidence that AI storage architecture remains unsettled even as adoption grows.

New Tests Target AI Inference

Version 3.0 adds a KV cache test, which measures how storage systems handle the read/write operations behind LLM inference. KV caching lets a model reuse key-value vectors it already computed instead of recalculating them on every conversation turn, a common technique in transformer-based AI inference. The benchmark simulates multi-turn conversations that write a context once and read it repeatedly, with transfers ranging from 64 MiB to about 3 GiB and a median workload near 24,561 contexts totaling 13 TiB of data.

The suite also adds a vector database test, which measures performance for the indexing and query workloads behind RAG pipelines. The test uses the Milvus database with 1 million vectors at 1,536 dimensions and a DiskANN index, generating a stream of small, random read-only queries whose accuracy is checked against brute-force ground truth.

“These new additions to the benchmark suite round out the test collection, covering a larger range of AI inference workloads that drive storage needs,” said Brian Belgodere, MLPerf Storage working group co-chair.

He added: “Including tests that decompose monolithic AI systems and focus on specific storage uses and patterns, such as checkpointing, KV caching and vector databases, gives stakeholders a much clearer idea of how to engineer and provision AI systems to minimize storage performance bottlenecks.”

David Kanter, founder of MLCommons and the head of MLPerf, said the KV cache test differs from the suite's earlier training and checkpointing tests in one respect. Those benchmarks were derived from MLPerf's own industry-standard training workloads, while KV cache runs on an emulation the working group built itself, since no comparable industry-standard agentic workload existed in time for this release. He said the working group wants to align future KV cache rounds with MLPerf's own agentic inference benchmark once that work matures, replacing the purpose-built emulation with a workload derived from a documented standard.

Object Storage Joins the Comparison

Version 3.0 also adds support for S3 object storage as an access layer alongside the existing POSIX file system standard, letting submitters run training and checkpointing workloads against object storage instead of through a file system. About one-sixth of this round's submissions used the S3 layer.

“As the scale of AI contexts reaches into the trillions, we expect object-based storage systems to emerge as a viable, and possibly preferred, alternative to filesystem-based storage,” said Curtis Anderson, working group co-chair. “By enabling S3 support now, we are ensuring that stakeholders will have the performance information they need to make smart decisions.”

In a news briefing, Anderson put that scale in concrete terms. “If you look at the math for a KV cache environment, a billion iPhones or 5 million iPhones, every iPhone user has 1,000 contexts. Now you’re talking trillions of contexts that need to be stored. That’s an object problem, not a file system problem.”

Submitters including Nebius, NVIDIA and OpenLake used S3-compatible object storage for training and checkpointing workloads in this round.

Measuring Power Efficiency

The results also gave MLCommons its first broad look at power efficiency across submitted systems. On-premises submissions for the checkpointing write test posted a median of 14 GB/second per watt, with the top result reaching 201. The UNet3D read test showed a median of 34 GB/second per watt and a top result of 277.

“There is a wide range of power efficiencies represented in the results,” Anderson said. “It also shows that there is ample room for further improvement, and we encourage all suppliers to optimize for that metric.”

The working group also reframed how it wants benchmark customers to read the numbers. Rather than raw bandwidth, the training benchmark scores how many accelerators a system can keep above 90% utilization. Checkpointing scores duration. KV cache scores the number of conversations a system supports. Anderson said the comparisons that matter to data centers are performance per rack unit and performance per watt of provisioned power, since a working data center's space and power budgets are the constraints operators cannot expand on demand.

The TechArena Take

This round marks an expansion of what MLPerf Storage measures, from a single moment in a model's life to its full arc. Training and checkpointing belong in a model’s build phase. KV cache and vector database capture how it performs once deployed and serving real conversations and queries. That shift to covering a model's full life cycle tracks where AI investment is going as more organizations move models into production.

The economics are shifting to match. Anderson's accelerator-hours framing, storage judged by GPU time saved as well as dollars per terabyte, reflects a broader repricing of infrastructure happening across the AI stack, not just storage. Object storage's arrival alongside parallel file systems tells a similar story: As AI context volumes climb toward the trillions, storage architectures built for a smaller era are being tested by systems built for hyperscale conversational AI.

As AI infrastructure keeps changing shape, from training clusters to inference fleets to the agentic systems MLCommons is only beginning to benchmark, MLPerf Storage remains one of the few places buyers can compare vendors on equal footing.