HPE Networking Goes Full Stack for the AI Era of Inference

ByJeni Barovian, Co-Founding Advisor, TechArena

The shift from AI training to inference and agentic applications is changing how data moves through infrastructure. Mixture-of-experts models, disaggregated inference, and distributed deployments are placing new demands on connectivity within the rack, across clusters, and between data centers. These developments are increasing the pressure on networking within AI infrastructure.

Network architecture increasingly affects GPU utilization, application performance, and the economics of AI deployments. As workloads become more distributed, operators need to consider not only compute capacity but also how efficiently data moves between accelerators, storage systems, and facilities.

At OCP26, I spoke with Amit Sanyal, who leads product marketing for data center networking at HPE. With Juniper Networks fully integrated into HPE, the company is bringing together switching, routing, software, and infrastructure management to support AI connectivity at multiple levels. Our conversation explored what that means for system builders, neoclouds, and enterprise operators, and how HPE is addressing these challenges through its expanded portfolio.

Inference changes what we're asking of the network

For years, AI networking discussions focused primarily on training. Unlike training, which typically involves coordinated processing across large GPU clusters, inference must also accommodate variable request patterns, latency-sensitive responses, and increasingly distributed workloads. As Amit explained, this introduces its own demands:

• Mixture-of-experts models route tokens between specialized experts, expanding communication across accelerators. When those experts reside on different GPUs, the network must support frequent data exchanges without introducing delays that affect response times.
• Disaggregated inference separates prefill and decode into distinct phases, requiring fast data transfers between them. This architecture allows operators to allocate resources independently to each phase, but also depends on efficient communication.
• Larger models span multiple GPUs, increasing the need for high-bandwidth connections. When model parameters are distributed across accelerators, network performance can affect how efficiently those GPUs work together.
• KV caches may be stored outside GPU memory and retrieved as needed, adding another layer of traffic. This approach can help manage limited GPU memory capacity, but retrieving cached data introduces additional bandwidth and latency considerations.

Agentic AI extends this complexity beyond individual facilities: as applications coordinate multiple models, tools, and services, traffic can move between data centers and across wide-area networks (WANs). These applications may also generate sustained activity as agents execute multiple steps, making network reliability and predictable performance important beyond individual inference requests.

For operators investing heavily in AI accelerators, network performance has a direct impact on utilization. HPE says its newest switches are designed to help GPUs spend more time processing and less time waiting on the network. The challenge is to support these different traffic patterns while maintaining high throughput and low latency.

Open always wins

As AI architectures diversify, open networking standards are gaining importance. Amit pointed to the growing adoption of Ethernet in scale-out networks, where InfiniBand has traditionally played a major role. Now, industry efforts are extending open interconnect options into scale-up environments within the rack to provide system designers more flexibility in how they connect accelerators.

The distinction matters because these environments have different performance requirements:

• Scale-up fabrics connect accelerators within tightly integrated systems, where extremely high bandwidth and low latency are critical.
• Scale-out networks link GPU clusters across racks, which require efficient traffic management to accommodate an increasing number of interconnected devices.

Open standards give infrastructure designers more options for meeting those needs without depending exclusively on proprietary interconnects, while allowing them to evaluate technologies based on specific performance requirements.

"Open networking has always won," Amit said.

HPE participates in several initiatives supporting this transition, including the Ultra Ethernet Consortium (UEC), Ultra Accelerator Link (UALink), and Ethernet for Scale-Up Networking (ESUN).

UEC focuses on enhancing Ethernet for demanding AI and high-performance computing workloads, including transport efficiency and congestion management. UALink defines an open, high-bandwidth interconnect for communication between accelerators within scale-up systems. ESUN explores Ethernet-based approaches to scale-up connectivity, extending the technology into environments traditionally served by specialized fabrics. Together, these efforts reflect an industry-wide push toward greater interoperability across AI infrastructure.

HPE is also incorporating emerging standards into its products. The company describes its QFX5250 switch as Ultra Ethernet Transport ready, an example of how these specifications are influencing commercial networking systems.

Why Juniper changed the equation

When HPE announced its plans to acquire Juniper Networks in 2024, CEO Antonio Neri emphasized the opportunity to deliver a modern AI-driven networking fabric for training and deploying AI. The acquisition expanded HPE's capabilities: Juniper brought an established portfolio of high-performance data center switches, routers, and network management software, extending the company's ability to support connectivity across different components of AI infrastructure.

Together, HPE's networking technologies enable communication across GPU clusters and provide routing between data centers, creating opportunities for customers to source more of their AI infrastructure from a single provider rather than integrating technologies from separate vendors.

"We came from Juniper, and now that we are part of HPE, we have the full stack," Amit said.

HPE can address AI environments at different scales, from individual GPU clusters to large, distributed facilities. Rather than treating connectivity as a separate infrastructure decision, operators can evaluate networking as part of the broader system architecture.

Silicon, systems, software, and the full stack

Amit described HPE's approach in three parts:

1. Silicon and systems. HPE uses merchant and custom silicon to support scale-up, scale-out, and scale-across networking, including the QFX5252 switch tray for Ethernet-based scale-up within AMD Helios racks and a fully liquid-cooled, 100-terabit-class scale-out switch. These designs address different levels of AI connectivity, from high-bandwidth communication within accelerator racks to the network fabrics that connect large GPU clusters. The liquid-cooled system also reflects the growing need to manage power and thermal constraints as switching capacity increases.
2. Software and management. Congestion management helps maintain throughput and reduce packet loss across AI clusters. HPE emphasizes rapid fabric deployment for neocloud operators, while agentic AIOps supports troubleshooting, issue resolution, and early detection of potential system and optics failures. For operators deploying thousands of accelerators, network congestion can reduce effective GPU utilization, while automated configuration and diagnostics help teams bring new infrastructure online and identify problems faster.
3. The full stack. With Juniper now part of HPE, the company can deliver networking alongside compute, storage, and management and orchestration software. This integrated approach supports customers from initial setup through ongoing operations. For system builders and neoclouds, tighter integration across components can simplify configuration. It also gives operators a more coordinated approach to monitoring performance and resolving issues across the AI environment.

Not hypothetical: customers are buying

HPE's expanded networking portfolio is already supporting major AI infrastructure projects. In September 2026, the company announced a $1.2 billion order from Vultr for AMD Helios AI Rack by HPE systems, which incorporate purpose-built networking hardware and software. Amit also highlighted HPE's collaboration with Oracle on multi-gigawatt data center deployments.

HPE expects that momentum to continue. At its Networking Investor Day, the company projected Data Center Networking revenue growth at a compound annual growth rate (CAGR) in the low-to-high 50% range through fiscal 2029.

For neocloud providers such as Vultr, bringing new GPU capacity online involves more than installing accelerators. Organizations must also deploy the networking, power, cooling, and management systems required to operate those resources efficiently. At multi-gigawatt scale, infrastructure decisions can have significant implications for deployment timelines, operational complexity, and long-term costs, particularly when operators must coordinate deployments across multiple facilities simultaneously.

By combining networking with compute, storage, and management capabilities, HPE is positioned to help operators address these challenges as they scale.

Final thoughts

My conversation with Amit highlighted just how central networking has become to the future of AI infrastructure. We spend so much time talking about GPUs, compute capacity, and model performance, but the ability to connect those resources efficiently is equally critical. As inference and agentic applications become prevalent, the relationships between compute, connectivity, and overall system performance are becoming more complex.

With Juniper fully integrated into the company, HPE has a full portfolio of solutions to address scale-up connectivity within the rack, scale-out communication across clusters, and scale-across networking between data centers. That coverage matters as customers build increasingly sophisticated AI environments, often with different technologies and operational needs at each level.

Perhaps the most interesting part of our discussion was how HPE is bringing these capabilities together. The combination of networking, compute, storage, and management software creates opportunities to simplify deployment and ongoing operations. For neoclouds and other operators racing to bring AI capacity online, faster infrastructure configuration and more efficient troubleshooting can have a meaningful impact.

As a more heterogeneous era of AI infrastructure emerges, open, standards-based approaches to networking are essential. Operators need to evaluate how their networking choices will accommodate evolving AI workloads, different accelerator configurations, distributed inference, and future capacity expansion. And solution providers need to be ready to deliver these capabilities with an open, interoperable approach. The goal is not simply to connect more hardware, but to build infrastructure that can adapt as operational requirements change.

Ultimately, networking is more central than ever within broader AI infrastructure planning. Efficient connectivity, configuration, and management and an open approach will be essential to maximizing compute utilization and the value of these investments.

‍