TechArena’s 5 Top Takeaways of GTC 2024
This week was a whirlwind at GTC 2024 in San Jose, and it was a conference where I felt myself absorbing new insights about the tech industry, AI and where we’re collectively headed at turbo speed. With so many messages and so many companies aligning themselves to the green lantern that is NVIDIA, what are the key takeaways from the event?
#1 NVIDIA IS DISTANCING THEMSELVES FROM THE FIELD
2023 taught us that NVIDIA had unquestioned leadership in foundational definition of the AI era infrastructure landscape. Last year’s H11 introduction, shortages of GPUs in market, and meteoric rise in LLM training cluster deployments underscored their importance to the industry. What we saw this week was a company operating on all cylinders to keep and extend their lead. First, we got the unveiling of Blackwell with performance deltas equivalent to greater that we saw from the A100 to H100. Next, we saw the announcement of NIM showcasing that NVIDIA is not satisfied with AI training, they want to own the inference landscape as well with powerful software tools to aid deployment. Jensen also unveiled sweeping collaborations with industry leaders led by a massive collaboration with Microsoft to bring GB200 Grace Blackwell computing into Microsoft Azure. Finally, and notably covered previously on the the TechArena, NVIDIA unveiled their strategy to extend their dominance to the network with a 6G strategy that centers squarely on AI.
#2 THE INDUSTRY IS ALIGNING THEMSELVES DESPERATELY TO NVIDIA’S STAR
The energy at the San Jose Convention Center was palpable including on the show floor where infrastructure vendors and service providers hawked NVIDIA centric gear to position themselves as part of this disruptive force. They worked to get selfies with Jensen and were keen to highlight the depth of collaboration they had with the company. I haven’t seen this kind of engagement since the earlier days of the Intel Developer Forum in terms of a conference that set the pace for the industry. Those who execute in alignment of this strategy are poised to benefit greatly, and they know it.
#3 THE DATA PIPELINE IS LEGITIMATELY CRITICAL AND BEING RE-DEFINED
One of the most interesting elements in AI infrastructure today is re-definition of the data pipeline as broad enterprise begin training LLMs and tapping their data. This data is located all over the map – in the cloud, on prem, and at the edge, and getting a handle on how to aggregate it for training is, well, really difficult. Disruption in this space is massive, and many companies, VAST Data and WEKA come to mind, have interesting solutions to aid companies in this realm. For the large scale of the large scale, Voltron Data just delivered some new insight about Theseus that needs unpacking as well. While we have covered these firms on the TechArena, we’ll be going even deeper in our new Data Insights series with Solidigm to learn more.
#4 THERE IS AN INTERCONNECT WAR BREWING
I sat in many GTC sessions describing architectural models for deployment of GPU clusters, and as important as the GPU performance is to these workloads, the ability to connect systems together with high bandwidth switching is critical. NVIDIA’s answer to this is InfiniBand, but there was open discussions from others in the industry that Ethernet was in play as well. We covered the Ultra Ethernet consortium last year at the OCP Global Summit, and it’s apparent that service providers and infrastructure leaders alike want Ethernet to compete here. Put a pin in this topic as we’ll be exploring it next month again at OCP Summit Lisbon.
#5 DPUs ARE FIGHTING FOR SUPREMACY
NVIDIA’s Bluefield Network solutions are a leading force in delivering DPU capability to network offload, but here they are not the only game in town. AMD’s Pensando technology has raised some eyebrows with pure capability, and this is a diffuse field with entrants from everyone including network leaders Broadcom and Marvell to cloud service providers like Microsoft and AWS. What’s interesting to me is that this arena seems ripe for fierce competition, and I expect to hear a lot more about DPU innovation in the coming months.
So are we ready to declare a GPU victory and place CPUs as legacy gear? Do we want to throw the towel in as well for the AI startup silicon arena? The answer is…no. AI is moving at a pace that requires incredible amounts of silicon, and this opens the door for a heterogeneous array of viable solutions. AI is not one workload – it’s a broad array of training and inference across LLMs, image and voice recognition, recommendation engines and more – each with their unique computing requirements. It’s also a force that will be delivered across the data center, into the edge, and at the device, each requiring its own platform optimizations and performance characteristics. Finally, it’s a power-hungry monster, and we will meet a moment in the not so distant future where pure efficiency will become as critical as liquid cooling tradeoffs. We are still in the infancy of the AI era, and the room is open for broad innovation starting with silicon. I can’t wait to see what happens next.
This week was a whirlwind at GTC 2024 in San Jose, and it was a conference where I felt myself absorbing new insights about the tech industry, AI and where we’re collectively headed at turbo speed. With so many messages and so many companies aligning themselves to the green lantern that is NVIDIA, what are the key takeaways from the event?
#1 NVIDIA IS DISTANCING THEMSELVES FROM THE FIELD
2023 taught us that NVIDIA had unquestioned leadership in foundational definition of the AI era infrastructure landscape. Last year’s H11 introduction, shortages of GPUs in market, and meteoric rise in LLM training cluster deployments underscored their importance to the industry. What we saw this week was a company operating on all cylinders to keep and extend their lead. First, we got the unveiling of Blackwell with performance deltas equivalent to greater that we saw from the A100 to H100. Next, we saw the announcement of NIM showcasing that NVIDIA is not satisfied with AI training, they want to own the inference landscape as well with powerful software tools to aid deployment. Jensen also unveiled sweeping collaborations with industry leaders led by a massive collaboration with Microsoft to bring GB200 Grace Blackwell computing into Microsoft Azure. Finally, and notably covered previously on the the TechArena, NVIDIA unveiled their strategy to extend their dominance to the network with a 6G strategy that centers squarely on AI.
#2 THE INDUSTRY IS ALIGNING THEMSELVES DESPERATELY TO NVIDIA’S STAR
The energy at the San Jose Convention Center was palpable including on the show floor where infrastructure vendors and service providers hawked NVIDIA centric gear to position themselves as part of this disruptive force. They worked to get selfies with Jensen and were keen to highlight the depth of collaboration they had with the company. I haven’t seen this kind of engagement since the earlier days of the Intel Developer Forum in terms of a conference that set the pace for the industry. Those who execute in alignment of this strategy are poised to benefit greatly, and they know it.
#3 THE DATA PIPELINE IS LEGITIMATELY CRITICAL AND BEING RE-DEFINED
One of the most interesting elements in AI infrastructure today is re-definition of the data pipeline as broad enterprise begin training LLMs and tapping their data. This data is located all over the map – in the cloud, on prem, and at the edge, and getting a handle on how to aggregate it for training is, well, really difficult. Disruption in this space is massive, and many companies, VAST Data and WEKA come to mind, have interesting solutions to aid companies in this realm. For the large scale of the large scale, Voltron Data just delivered some new insight about Theseus that needs unpacking as well. While we have covered these firms on the TechArena, we’ll be going even deeper in our new Data Insights series with Solidigm to learn more.
#4 THERE IS AN INTERCONNECT WAR BREWING
I sat in many GTC sessions describing architectural models for deployment of GPU clusters, and as important as the GPU performance is to these workloads, the ability to connect systems together with high bandwidth switching is critical. NVIDIA’s answer to this is InfiniBand, but there was open discussions from others in the industry that Ethernet was in play as well. We covered the Ultra Ethernet consortium last year at the OCP Global Summit, and it’s apparent that service providers and infrastructure leaders alike want Ethernet to compete here. Put a pin in this topic as we’ll be exploring it next month again at OCP Summit Lisbon.
#5 DPUs ARE FIGHTING FOR SUPREMACY
NVIDIA’s Bluefield Network solutions are a leading force in delivering DPU capability to network offload, but here they are not the only game in town. AMD’s Pensando technology has raised some eyebrows with pure capability, and this is a diffuse field with entrants from everyone including network leaders Broadcom and Marvell to cloud service providers like Microsoft and AWS. What’s interesting to me is that this arena seems ripe for fierce competition, and I expect to hear a lot more about DPU innovation in the coming months.
So are we ready to declare a GPU victory and place CPUs as legacy gear? Do we want to throw the towel in as well for the AI startup silicon arena? The answer is…no. AI is moving at a pace that requires incredible amounts of silicon, and this opens the door for a heterogeneous array of viable solutions. AI is not one workload – it’s a broad array of training and inference across LLMs, image and voice recognition, recommendation engines and more – each with their unique computing requirements. It’s also a force that will be delivered across the data center, into the edge, and at the device, each requiring its own platform optimizations and performance characteristics. Finally, it’s a power-hungry monster, and we will meet a moment in the not so distant future where pure efficiency will become as critical as liquid cooling tradeoffs. We are still in the infancy of the AI era, and the room is open for broad innovation starting with silicon. I can’t wait to see what happens next.
