Agentic AI Is an Open Systems Challenge...and Opportunity

OCP Global Summit week opened for me with brunch and a question that has percolated in my mind all year: What does it take to run AI agents in production? MiTAC Computing hosted its Executive Tech Forum on Monday morning , and I had the pleasure of moderating a panel with Intel, SK hynix, Solidigm and MiTAC under the title Powering the Agentic Era: Compute, Memory, Storage and Scale.

Each company took a few minutes to make the case for the layer it builds. Assembled end to end, the four presentations delivered a comprehensive view on how innovation is accelerating across the system and the data center. Only in working together can we solve this open systems challenge that is becoming more urgent. Agentic AI changes the demand on every part of a rack or pod level platform at once, and the companies that treat the challenge as a single design problem will deliver to true operator requirements. Here is more of what I took from the room.

From Training Once to Agents All the Time

SK hynix America’s Vis Valluri, senior director of industry intelligence, set the frame with a simple progression. Training happens once. Inference runs in real time. Agentic AI runs all the time. He made his claim citing data from OpenRouter and a16z showing that agents overtook humans as the main source of token traffic early this year, with token usage up 14 times since.

The mechanics of agentic workflows explain this surge precisely. A regular inference request makes one pass through prefill and decode and returns a response. Simple and linear! An agent works in a loop: prefill, decode, call a tool, observe the result, then fold that result back into the context for another round of prefill. The KV cache grows with every turn. Valluri put the consequence in one number. More than 85% of agent tokens come from cached prompts, which means agents spend most of their time rereading context they already hold.

That single behavior sets up everything the other speakers had to say and defined a challenge across compute, memory and storage working tightly in tandem.

Compute Sits at the Foundation

Gary Brown of Intel opened his segment with a message that compute acts as a foundation to memory, storage, broader infrastructure (don’t forget the network!). As the resident CPU builder, he made the case for compute with throughput data.

Brown showed Intel Xeon 6 against competing CPU instances on healthcare agent workloads covering reasoning, security and control operations, laying out performance benchmarks of the alternative platforms. And while Intel was seeking a performance leadership win, the bigger takeaway for me is that in the agentic era, core functions of workload management matter a lot... and these are the realm of CPUs.

Intel’s own recent announcements argue that as AI turns agentic, the bottleneck moves toward orchestration, concurrency and data movement. Brown’s portfolio slide spread agentic scale across what Intel calls the intelligence and execution layers, with GPUs handling prefill. An agent spends much of its life between model calls, running tools and checking results. Someone has to schedule all of that, and the processor is a natural orchestrator.

Memory Gets New Floors

Valluri took Intel’s view and built from it with a memory centric perspective on the same challenge. SK Hynix sees the hierarchy being rebuilt with new tiers that fill the gaps between SRAM, HBM, DRAM and SSD. He laid them out from fastest to largest: a G0.5 tier of 3D-stacked DRAM with near-SRAM speed, HBM at G1, a G1.5 tier of stacked flash called HBF offering roughly 1 TB at terabit-per-second scale, conventional DRAM, a G2.5 tier of CXL-pooled memory in the multi-terabyte range, and SSD beneath it all.

The demand driver he chose was personal agents, pointing to Meta’s personal agent running on Muse Spark. This agent remembers details you mentioned once, keeps working after you close the app and runs in a dedicated cloud VM for each user. Persistent context that grows per user means more memory at every tier. When I think about the scale of that arithmetic across a large user base, the new tiers look like necessities rather than options, and the importance of memory and storage innovation argue to take center stage as the existential challenge of agentic performance.

The Price of a Prompt

So where was storage in all of this? Enter Solidigm’s Ace Stryker. He made the bluntest case of the discussion, using simple math as his tool. He started with a 14-word question: what’s a cool bird I might see when I’m in San Jose next week? That prompt is 18 tokens. The model processes far more. System prompt and policies account for 10,000 tokens, retrieval for 15,000, tool schemas for 5,000 and conversation history for 12,000. The total reaches 42,018 tokens. My guess is, in reality, we could argue it being even higher.

And... that context has to live somewhere. For one user making one request, Stryker calculated about 13.1 GB of KV cache, scaled from NVIDIA’s figure of 40 GB at 128K tokens for Llama 3 70B at FP16. Multiply by concurrent users and sessions and the memory bill scales quickly.

Solidigm’s answer to drive efficiency into the equation is to transfer that model state on SSD and restore it rather than recompute it. In work with Metrum AI, retaining KV cache on a Solidigm D7-PS1010 cut time to first token from 31 seconds to 2.8 seconds in a single-user sports archive scenario. In a separate video analytics test, pairing KV cache with DiskANN retrieval lifted queries per second by 14%, from 2,071 to 2,371 at 1,024 query threads. Stryker flagged these as separate results rather than a combined speedup, and I will repeat that caution here. He closed on a line that deserves a spot on every infrastructure budget review: every token has a resource cost, and every resource has a dollar cost.

Rack-Scale Reality

MiTAC Computing ’s Raymond Huang took the conversation from cutting edge silicon innovation to system delivery at scale, under the banner, “Where AI Power Meets Rack-Level Scalability.” Silicon only matters once it ships as a working system, and his slides were about that last mile and connecting acute operator requirements in a couture fashion with bespoke systems leveraging industry standards.

MiTAC cited more than 30 MW of rack-scale manufacturing power in 2026, capacity for 5,000 racks per month, networking up to 800 Gb/s and power readiness from 480 VAC up to 800 VDC. The MiTAC California footprint anchors this: It's Newark site features an NPI direct liquid-cooling facility, while San Jose provides dedicated liquid-cooling manufacturing. A mega-scale, six-building campus in Fremont further strengthens the company’s ability to design, validate, and produce advanced computing solutions. From PCB and motherboard design to barebone, systems and complete AI clusters, MiTAC delivers vertically integrated, in-house engineering that accelerates time to market and ensures end-to-end quality. On the product side, Huang showed a 52U high-density liquid-cooled rack, including a direct liquid-cooled MI355X GPU configuration, and a 256-GPU cluster example with in-row coolant distribution and a separate aggregation switch rack.

MiTAC pairs the hardware with open firmware and management software called MiOBMC™, MiOPF™ and MiCoreView™, and with ecosystem partners including DDN, Rafay and Canonical Ubuntu. A partnership with Tonomia targets modular AI data centers. Its Diamond Cooling approach aims to keep token generation throttle-free at standard data center temperatures, which could trim power and cooling costs if it performs as described.

The delivery model closed the loop. US-based engineering and project management, local system and rack integration, and a build-to-inventory approach aim to shorten lead times for customers who cannot wait out a long supply chain.

The TechArena Take

From soup to nuts, one word runs through the discussion as a throughline. Agents carry context from step to step, and context is state. State has to live somewhere, and every element of infrastructure is now being asked to hold more of it, closer to the processor and for longer. SK Hynix described the tiers that hold it. Solidigm priced it in both gigs and dollars. Intel claimed the baton with the statement that the processor orchestrates it. MiTAC showed how to package it all into racks that arrive on schedule.

Run the memory math once and the stakes come into focus. A single request can carry 13.1 GB of cache, and a personal agent maintains its memory between sessions. The capacity question becomes a per-user number multiplied by a sea of agents and humans. Size up that investment, and the case for re-tiering and deeper performance tier SSD use comes into acute focus.

Capacity plans built around model size and GPU count will miss the largest cost driver in agentic deployments. Plan around the workflow and sustained memory: how large does it grow per session, how many sessions run at once, and where each piece of it should live. Operators who answer those questions across compute, memory, storage and rack design together leveraging all of the standards the OCP community provides will build agent infrastructure that scales at speed.