Welcome to
the Forum

PLVision’s Taras Chornyi chat’s SONIC at OCP Lisbon 2024.

TechArena host Allyson Klein chats with PLVision Director of Open Networking Solutions and Strategy, Taras Chornyi, about the progress of SONIC and open network infrastructure for the AI Era.

TechArena host Allyson Klein is joined by Solidigm’s Jeniece Wnorowski as they continue to explore rapid data innovation fueling today’s computing. In today’s episode, they chat with VAST Data’s Global VP of Engineering, Subramanian Kartik, as he describes how his team has delivered a breakthrough data platform for the AI Era.

OCP’s Lesya Dymyd provides insight into the Future Technologies Symposium and the European Data Center market from OCP Lisbon 2024.

TechArena host Allyson Klein chat’s with OCP’s Lesya Dymyd about her work steering the organization’s future technologies symposium as well as her deep collaboration with European technology providers and operators delivering to the region’s unique market requirements.

Qarnot CEO talks sustainable tech innovation at OCPLisbon24

TechArena host Allyson Klein chats with Qarnot CEO Paul Benoit about how VC backed startups are an essential element of vibrant industry innovation, and how the AI era has placed requirements on innovative approaches to sustainable IT.

Credo’s Don Barnetson chats optical innovation at OCP Lisbon 24

TechArena host Allyson Klein chats with Credo VP Don Barneston about how his company is delivering innovative connectivity solutions that address the AI era’s requirements for scalable data movement in the data center and beyond from OCP Lisbon 2024.

Open Compute Project’s Steve Helvie chats data center innovation from #OCPLisbon24

TechArena host Allyson Klein chats with Open Compute Project VP of Emerging Markets, Steve Helvie, about the proceedings in Lisbon this week and how OCP is helping to shape the cutting edge of infrastructure innovation.

ZeroPoint Technologies’ Nilesh Shah shares insights at #OCPLisbon24

TechArena host Allyson Klein chats with ZeroPoint Technologies’ VP of Business Development Nilesh Shah about the AI era demands for memory innovation, how advanced chiplet architectures will assist semiconductor teams in advancing memory access for balanced system delivery, and how ZeroPoint Technologies plans to play a strategic role in this major market transition.

HPE’s Jean-Marie Verdun discusses network delivered firmware via OpenBMC at #OCPLisbon24

TechArena host Allyson Klein chats with HPE’s Jean-Marie Verdun about his organization’s groundbreaking work to redefine firmware management using OpenBMC technology and how this breakthrough addresses data center customer demands.

Highlights from CircleB’s Matty Bakkeren at #OCPLisbon24

TechArena host Allyson Klein chats with CircleB’s Matty Bakkeren about how hisorganization is leveraging OCP specifications to deliver innovative and sustainable solutions to data center customers, how AI is re-shaping operator requirements, and how he sees the market shaping in 2024.

Portugal is an interesting place from a historic perspective. You could say that they peaked early, delivering the fastest ships in the Age of Discovery and carving a passage to India opening up the spice trade. This brought wealth and a colonial empire, and world influence to this small country, but as society advanced, other powers leapfrogged Portugal with maritime innovations of their own. In the 20th century, Portugal was a footnote on the history books becoming a military dictatorship until a democratic revolution freed the country 50 years ago this week. And its in this landscape that the Open Compute Project, an organization knowing a lot about the value of innovation, lands on Portuguese shores for its Regional Summit. How befitting of OCP to choose this location for arguably the most disruptive period in the history of data center computing, as we see AI place exponential performance demands on infrastructure and as infrastructure vendors feel the pressure to innovate or risk being disrupted to the footnotes of computing history.

The TechArena is delighted to once again be a media sponsor of OCP’s 2024 Summits, and we’ll be reporting this week on the latest innovations in open hardware configurations. I’m excited to engage with industry executives and leading operators about how OCP designs are influencing everything from balanced system performance across compute, memory and I/O, advances in cooling technologies to help operators deploy dense AI training clusters, and how a broader array of organizations beyond the largest cloud providers are leveraging OCP designs to advance their infrastructure. I’m also keen to see how European operators are navigating challenges of high energy costs and data regulatory restrictions and if we see any unique insights on the shaping of infrastructure and solution requirements flowing from European customers.

Watch this space as we report from the Summit, and please reach out if you’ve got specific questions you’d like answered from the experts assembled in Lisbon.

I recently attended NVIDIA GTC, called by some as the Woodstock moment of the AI Era, and I’m still unpacking what we learned there about industry innovation to fuel AI workloads. While the TechArena packed as many conversations possible with industry innovators at the event, one conversation that stood above the rest was our interview with CoreWeave’s Jacob Yundt. He leads infrastructure buildout for CoreWeave as they chart a trajectory for delivering unparalleled scale for AI training in the cloud.

How did they do it? As we have seen at many inflection points, CoreWeave took advantage of not being encumbered by legacy to deliver a cloud stack that was specially built for AI training clusters from initial provisioning to health checks, orchestration and scheduling. This enables the company to bring up a staggering amount of GPUs to a particular training task at warp speed while providing reliable compute throughout the training period. CoreWeave provides proactive oversight of its instances to ensure that precious training cycles are not disrupted based on potential hardware failures, I/O issues, or other maladies that confront data center infrastructure.

CoreWeave has developed a cult-like following amongst AI startups looking to train algorithms where speed to train often is the difference for market opportunity. Jacob clarified their market focus on any customer looking to do “ground-breaking work at incredible scale”, and this speaks to the type of underlying infrastructure requirements they have across compute, storage, and network. And the demand for this infrastructure is stark. CoreWeave has been on record stating that power demand alone from its training clusters may stress local power grids in the communities where it operates, and the demand for CoreWeave is also growing exponentially. Valued at $7B last December, the latest discussion of valuation of the company four months later has surged to $16B underscoring the growth potential for AI training.

So what infrastructure is CoreWeave tapping to deliver their AI service? It’s no secret that their training relies on NVIDIA GPUs, and CoreWeave will be integrating next generation Blackwell GPUs into clusters utilizing liquid cooling technologies. But Jacob stressed that there’s more than GPUs that goes into the groundbreaking scale they’ve been able to achieve. That scale starts with re-imagining the data pipeline, and CoreWeave has leaned into a strategic partnership with VAST Data to deliver innovative data management and control that scales with GPU performance needs. VAST Data’s platform has driven new capabilities for managing data sets to bring data more efficiently and quickly to the processing complex eliminating much of the overhead associated with traditional tiered storage solutions.

Jacob stated that the collaboration with VAST Data begins with his team’s love of QLC storage and the careful balance between performance, capacity and efficiency that QLC delivers. To say that Jacob is a fan of QLC is an understatement, and it’s no surprise given QLC’s advantages over TLC technology in delivering increased data density per cell. Jacob stated that his long-standing collaboration with Solidigm has ensured QLC deployment in his data centers with a partnership that extends beyond procurement to account and engineering support. When you consider the size of LLMs being trained at CoreWeave, it’s easy to guess that that’s a lot of QLC NAND being deployed.

So what’s next for CoreWeave? Watch this space to learn more about their continued infrastructure buildout as a harbinger of broader AI market adoption. I’m also interested to see if CoreWeave can make a dent in the cloud service provider landscape with their built for AI training stack. I’ll also be reporting on advances of the data pipeline infrastructure industry including in my Data Insights series with Solidigm.

TechArena host Allyson Klein chats with Eridan Communications CEO Doug Kirkpatrick about how a DARPA award innovation propelled an innovation in wireless radio technology that will help full broad proliferation of 5G.

TechArena host Allyson Klein chats with WEKA President Jonathan Martin about his vision for data pipeline disruption, what the WEKApod expands how the company is supporting customers in their delivery of innovative solutions, and why the company’s focus on sustainability is so critical in the AI era.

TechArena host Allyson Klein chats with Solidigm’s Jeniece Wronowski and Shirish Bhargava and Supermicro’s Paul McLeod about infrastructure demands for the AI Era, and how Supermicro is delivering new platform capabilities to address data and workload demands.

The world of technology is in a constant state of flux, and with the rise of generative AI, the demand for more powerful and efficient processing is skyrocketing. I recently had a chance to chat with Mohit Gupta, Senior Vice President and General Manager of Custom Silicon and IP at Alphawave Semi, about how his company is leveraging their IP and engineering prowess to address customer demand for custom silicon innovation. Our conversation sheds light on why companies are increasingly turning to custom solutions and how Alphawave Semi is leading the charge in this exciting field.

Why the Rise of Custom Silicon?

Gone are the days of one-size-fits-all solutions. The era of generative AI, with its diverse workloads and specialized requirements, demands a more tailored approach. As Mohit explained in our interview, "The need for custom silicon arises because companies are facing a vast array of workload challenges. They need solutions that can address these challenges head-on and extract the maximum efficiency from their systems." This requires a focus on optimizing for specific parameters where generic, off-the-shelf solutions might not always be able to deliver. Custom silicon allows companies to optimize for the specific parameters that are critical to their specific AI infrastructure requirements. Mohit highlighted some key areas that he’s hearing from customers: "Imagine a scenario where memory bandwidth is the bottleneck in your system. With custom silicon, you can design a chip that prioritizes increased bandwidth, ultimately leading to a significant performance boost. Similarly, network latency or even cost reduction can be targeted for optimization based on your specific needs."

Alphawave Semi's Approach to Custom Solutions

Alphawave Semi isn't just another player in the custom silicon game. They've developed a distinct approach that focuses on four key pillars:

1. Connectivity: High-performance interconnects are the backbone of any complex chip design. Alphawave Semi prioritizes building a robust connectivity foundation of IP and chiplets within custom silicon designs, ensuring seamless communication between different processing elements on the chip, and between chiplets in an advanced 2.5D/3D package.

2. Compute: The heart of AI processing lies in the compute engines. Here, Alphawave Semi uses cutting-edge design techniques to deliver the optimal balance between power and performance for specific workload requirements, and their modular design delivers compute scalability. "As a founding member of the Arm Total Design Partnership, Alphawave Semi provides an accelerated path for specialized SoC solutions based on Arm Neoverse Compute Subsystems (CSS).

3. Complex Silicon Building Blocks: Building a custom chip isn't just about slapping together existing components. Alphawave Semi leverages its expertise in designing complex silicon building blocks that cater to the unique needs of each project. Their modular design enables speed in delivery of chiplet-enabled custom solutions as well as dialed-in engineering that leverages known silicon building block configs vs. expensive one and done design.

4. Advanced Packaging: As chip designs become more intricate, advanced packaging techniques are crucial for efficient integration. Alphawave Semi offers state-of-the-art packaging solutions that ensure optimal functionality and performance of the chiplet-enable custom silicon designs.

The TechArena Take

Alphawave Semi's dedication to custom silicon solutions has seen significant market success. Mohit noted that their engineering and IP progress has unlocked customer traction: "We've had successful designs on both 5-nanometer and 3-nanometer processes, showcasing our ability to deliver cutting-edge solutions at the forefront of technological advancement." With the ever-evolving landscape of AI, this progress on leading-edge process node and advanced packaging, bodes well for continued success in market. As companies push boundaries and explore new frontiers in generative AI, the demand for tailored solutions will only increase. With its focus on specific needs, cutting-edge technologies, and successful track record, Alphawave Semi is well-positioned to be a leader in this exciting future. I, for one, will be following their journey closely as the world of chiplet-based custom design takes off.

TechArena host Allyson Klein chats with AlphaWave Semi’s Mohit Gupta about how his firm is assembling industry leading technology to deliver custom silicon to the market at scale.

TechArena host Allyson Klein chats with Muriel Medard from MIT about the trends in network innovation, how AI is infusing into telco, and the shaping of 6G.

TechArena hosts Allyson Klein and Jeniece Wnorowski chat with CoreWeave’s Jacob Yundt about how his organization is delivering a scalable data pipeline to AI customers utilizing breakthrough VAST Data solutions featuring Solidigm QLC SSDs.

TechArena’s host Allyson Klein reprises conversation at MWC24 with a talk with GlobalLogic’s SVP of Global Communication Services Business, Sameer Tikoo, about the future of AI in the network and how his firm is building solutions that meet swiftly evolving customer demands.

This week was a whirlwind at GTC 2024 in San Jose, and it was a conference where I felt myself absorbing new insights about the tech industry, AI and where we’re collectively headed at turbo speed. With so many messages and so many companies aligning themselves to the green lantern that is NVIDIA, what are the key takeaways from the event?

#1 NVIDIA IS DISTANCING THEMSELVES FROM THE FIELD

2023 taught us that NVIDIA had unquestioned leadership in foundational definition of the AI era infrastructure landscape. Last year’s H11 introduction, shortages of GPUs in market, and meteoric rise in LLM training cluster deployments underscored their importance to the industry. What we saw this week was a company operating on all cylinders to keep and extend their lead. First, we got the unveiling of Blackwell with performance deltas equivalent to greater that we saw from the A100 to H100. Next, we saw the announcement of NIM showcasing that NVIDIA is not satisfied with AI training, they want to own the inference landscape as well with powerful software tools to aid deployment. Jensen also unveiled sweeping collaborations with industry leaders led by a massive collaboration with Microsoft to bring GB200 Grace Blackwell computing into Microsoft Azure. Finally, and notably covered previously on the the TechArena, NVIDIA unveiled their strategy to extend their dominance to the network with a 6G strategy that centers squarely on AI.

#2 THE INDUSTRY IS ALIGNING THEMSELVES DESPERATELY TO NVIDIA’S STAR

The energy at the San Jose Convention Center was palpable including on the show floor where infrastructure vendors and service providers hawked NVIDIA centric gear to position themselves as part of this disruptive force. They worked to get selfies with Jensen and were keen to highlight the depth of collaboration they had with the company. I haven’t seen this kind of engagement since the earlier days of the Intel Developer Forum in terms of a conference that set the pace for the industry. Those who execute in alignment of this strategy are poised to benefit greatly, and they know it.

#3 THE DATA PIPELINE IS LEGITIMATELY CRITICAL AND BEING RE-DEFINED

One of the most interesting elements in AI infrastructure today is re-definition of the data pipeline as broad enterprise begin training LLMs and tapping their data. This data is located all over the map – in the cloud, on prem, and at the edge, and getting a handle on how to aggregate it for training is, well, really difficult. Disruption in this space is massive, and many companies, VAST Data and WEKA come to mind, have interesting solutions to aid companies in this realm. For the large scale of the large scale, Voltron Data just delivered some new insight about Theseus that needs unpacking as well. While we have covered these firms on the TechArena, we’ll be going even deeper in our new Data Insights series with Solidigm to learn more.

#4 THERE IS AN INTERCONNECT WAR BREWING

I sat in many GTC sessions describing architectural models for deployment of GPU clusters, and as important as the GPU performance is to these workloads, the ability to connect systems together with high bandwidth switching is critical. NVIDIA’s answer to this is InfiniBand, but there was open discussions from others in the industry that Ethernet was in play as well. We covered the Ultra Ethernet consortium last year at the OCP Global Summit, and it’s apparent that service providers and infrastructure leaders alike want Ethernet to compete here. Put a pin in this topic as we’ll be exploring it next month again at OCP Summit Lisbon.

#5 DPUs ARE FIGHTING FOR SUPREMACY

NVIDIA’s Bluefield Network solutions are a leading force in delivering DPU capability to network offload, but here they are not the only game in town. AMD’s Pensando technology has raised some eyebrows with pure capability, and this is a diffuse field with entrants from everyone including network leaders Broadcom and Marvell to cloud service providers like Microsoft and AWS. What’s interesting to me is that this arena seems ripe for fierce competition, and I expect to hear a lot more about DPU innovation in the coming months.

So are we ready to declare a GPU victory and place CPUs as legacy gear? Do we want to throw the towel in as well for the AI startup silicon arena? The answer is…no. AI is moving at a pace that requires incredible amounts of silicon, and this opens the door for a heterogeneous array of viable solutions. AI is not one workload – it’s a broad array of training and inference across LLMs, image and voice recognition, recommendation engines and more – each with their unique computing requirements. It’s also a force that will be delivered across the data center, into the edge, and at the device, each requiring its own platform optimizations and performance characteristics. Finally, it’s a power-hungry monster, and we will meet a moment in the not so distant future where pure efficiency will become as critical as liquid cooling tradeoffs. We are still in the infancy of the AI era, and the room is open for broad innovation starting with silicon. I can’t wait to see what happens next.

TechArena kicks off a Data Insights Series in collaboration with Solidigm, and TechArena host welcomes co-host Jeniece Wronowski and Solidigm data center marketing director Ace Stryker to the program to talk about data in the AI era, the series objectives, and how SSD innovation sits at the foundation of a new data pipeline.

TechArena host Allyson Klein chats with EY’s Global Innovation AI Officer, Rodrigo Madanes, about what he’s seeing from clients in their advancement with AI and what this means for the industry requirements for innovation.

TechArena host Allyson Klein chats with Intel’s Lisa Spelman about how compute requirements are changing for the AI era, where we are with broad enterprise adoption of AI, and how software, tools and standards are required to help implement solutions at scale.

With new industry, collaborations VAST Data accelerates their ambition as the AI data platform.

VAST Data has been on a tear. If you haven't been following this company, you've missed a great story about transformation from storage platform to insightful data control for the AI era. Today vast unveiled multiple industry collaborations that fill in details of their strategy and showcase VAST’s bold direction to re-define data storage and oversight from cloud to edge.

Let’s unpack today’s news. We’ll start with an announcement on the new VAST Data AI platform featuring NVIDIA DPU's. This is a fundamentally different take on data and redefines platform architecture into a powerful GPU platform complemented with Bluefield DPUs each running a containerized version of the VAST OS embedding storage and database processing directly into the GPU platform.

What you’ll notice is missing is any mention of x86 architecture eliminating the concept of the CPU head node. VAST points to the increased efficiency of this design claiming a power utilization and footprint reduction of up to 70% and a “net energy consumption” savings of up to 5% vs previous platforms featuring VAST distributed data services and NVIDIA silicon.

VAST is also leaning into the core capabilities delivered with this platform with increased QoS through direct reads and writes to shared name spaces across the cluster, enhanced zero-trust security through use of industry standard NFS, SMB, S3 and Apache Arrow service delivery, and the addition of native block storage to compliment historic file and object storage.

So where do you get this platform? Enter collaboration announcement number two, a comprehensive collaboration with Supermicro to fuel system delivery for the AI factory. Here, the two companies point to delivering platforms that scale to exabyte level clusters. VAST and Supermicro promise support for the entire data pipeline from data prep to data training with VAST’s DataBase and DataStore solutions. Supermicro is known for swiftly getting innovative platforms to market, so I’m excited to see what they actually deliver to support this bold move for both companies.

What’s TechArena’s take? The AI training game is moving from GPU centric to GPU native with new architectural frameworks fueling these massive clusters. VAST Data’s history in high performance computing and their DASE (disaggregated share everything) architecture places them as a central player for organizations looking to integrate distributed data into AI training. AI is the killer app of the era forcing re-definition of the fundamental constructs that have fueled computing for the last thirty years, and this magnitude of disruption will reshape the infrastructure industry. Based on recent history and today’s announcements, VAST Data is positioning itself well as a disruptive force. I can’t wait to see more from the company in 2024.

TechArena host Allyson Klein interviews Netflix’s Tejas Chopra about how Netflix’s recommendation engines require memory innovation across performance and efficiency in advance of his keynote at MemCon 2024 later this month.