Welcome to
the Forum

TechArena host Allyson Klein chats with Eric Dahlen from Intel and Alex Rakow from Schneider Electric about their roles in the Compute Sustainability group of the Open Compute Project. They discuss AI's impact on data center sustainability, power and cooling innovations, and the upcoming OCP Summit.

The pace of innovation in AI is moving at light speed, so it was only a small surprise recently when Antti Karjalainen shared his company’s vision beyond large language models (LLMs). 

“So what is an agent? It's a piece of software that, contrasting to traditional software that we've been used to, does not just make you more efficient at your work. Agents will actually complete the work for you. So think about them as sort of a knowledge worker or software that can reason, collaborate and act with humans.”

Antti went on to explain that what we’ve seen with generative AI is just the beginning, and it’s in the application of these models, in his view with agents, that the real transformation of our industries and society will start taking shape.

What Sema4 is delivering is fascinating in terms of its broad reach across industries and the disruptive productivity it places in the hands of human collaborators. In Sema4’s vision, each individual can be unleashed with unlimited AI workers that can research, reason, and deliver task work, freeing the human for more value-added work. At first blush, this vision looks similar to Microsoft co-Pilot. But upon further analysis, it’s more optimized for unique industries or business processes. Antti explained the difference, being that “the AI agent is something that you can actually interact with to get a full work product done.” 

“Instead of just assisting on the side, an AI agent is kind of the main thing. An AI agent is going to be a new way to interact with enterprise applications, whereas a co-pilot is more of an additive layer on top,” he said.

When you consider the autonomy in this statement, you can start understanding the power that agents represent to autonomous control of digital functions holistically and start imagining the complexity of delivering this level of control to the enterprise. The team certainly has the technology chops and enterprise awareness to deliver the goods with far-ranging backgrounds from tech leaders including AWS, Cloudera, Docker, HortonWorks and more. Today, the Sema4 team is working with a range of business functions including customer support and finance operations to deploy initial agents into enterprise environments. Tomorrow, Antti described additional functions including HR and software development as part of a near-term vision for deployment.

So what about guardrails? Sema4 is tackling security as well as data integrity flowing into training agents as keys to success. Layer on top of that the organizational compliance and enterprise-class resilience of model adoption, and you can understand that the human trust in bringing AI agents into the enterprise is likely the main time investment in this powerful technology transition. Antti expects to see a wealth of use case deployment examples emerge in the second half of 2024 as the first signal of broad agent adoption.

What’s the TechArena take? We aren’t surprised anymore by the speed of innovation being driven by AI. IT organizations are being pushed to their limits to be agents of change, no pun intended, within the enterprise, and we expect business functions to directly adopt agents with or without internal IT support – much as they did with the initial activation of cloud services to fuel business agility. 2024 and 2025 will be rife with stories of success in this arena, and we can’t wait to hear them. As to worker response to agent deployment…we’ll leave that topic for an upcoming blog as it’s a lot to unpack and urgently needs to be discussed in order to get full value out of these powerful tools.

Advanced Driver Assistance Systems (ADAS) are all the rage. Increasingly, consumer car purchasing decisions are based on ADAS features as compared to vehicle style, engine size, or branding, which historically has been the case.

A quick internet search will provide one with an obligatory understanding of the capabilities associated with each ADAS level. A high-level summary of the different ADAS levels is as follows:

Level 0: No Driving Automation

Level 1 - Driver Assistance

Level 2 - Partial Driving Automation

Level 3 - Conditional Driving Automation

Level 4 - High Driving Automation

Level 5 - Full Driving Automation

What is profound, at least in my mind, is that there is no consistency regarding what specific ADAS capabilities are associated with a given ADAS level. Furthermore, the industry has introduced yet a new level called L3+. Not quite High Driving Automation but approaching autonomy. Apparently, good enough isn’t good enough.

The general goal of ADAS is to improve the overall safety of the driving experience. To keep this tone upbeat, I won’t focus on the mortality rate associated with driving, however, one key point to note is that it is estimated that 97% of all auto accidents are due to driver negligence. The key takeaway is that most accidents can be easily avoided through the use of technology, which is what ADAS brings to the table.

At the lower ADAS levels, the goal is to provide the driver with safety assistance features so the driver is more aware of their surroundings and can respond to the environment appropriately. Features like blind spot detection or backup cameras, typically considered Level 1/ Level 2 don’t have any physical impact on the operation of the vehicle itself but provide valuable insights to the driver to avoid what may have otherwise been a significant incident.

The progression from Level 2 to Level 3 and beyond is significant in terms of the level of electronic content that is employed to actively prevent accidents leading to the point where, at levels 4 and 5, the vehicle is driving autonomously without some level or any level of human intervention.

The impact in the reduction of accidents associated with each progressive ADAS level along with the benefits of freedom that comes from emancipating the need for a driver are profound. These benefits range from enabling a “greener” relationship between our cars and the environment to providing mobility to those who otherwise might be immobile.

The technology that is required to pull off these increasing levels of ADAS is pushing the state-of-the-art in every category. The overused phrase “data center on wheels” is an understatement regarding the level of complexity and technology that is under the hood of the modern car. Achieving level 3 ADAS typically requires 18 or more high-resolution cameras in addition to a dozen radar sensors and typically a LIDAR (light detection and range) sensor. The underlying computing engine that processes these massive amounts of sensor data typically employs AI computing that delivers 100s of Trillion - Tera Operations Per Second (TOPs).

To date, few passenger vehicles have been certified to be Level 3 compliant. And yet, Level 3 still requires a driver to be present and ready to take over control of the vehicle in the case where the capabilities of the electronics are exceeded in their ability to control the vehicle – a far cry from the vision of the self-driving car. Achieving Level 3 ADAS and beyond is pushing semiconductor process technology limits – TSMC has announced an automotive-qualified 3 nm process technology expressly to address the demands of this market as the number of transistors and their associated thermal footprint have become meaningful. One of the hottest technologies – chiplets – is also being readily embraced by the automotive market to most effectively address the mismatch in technologies required to achieve the vision of the self-driving car.

The emerging industry standard UCIe™ (Universal Chiplet Interconnect Express™) announced a 1.1 version of the specification expressly to address the automotive application space focusing on areas such as data integrity and reliability. More about chiplets and the UCIe specification will be discussed in future blogs. In short, outside of quantum computing, it seems like the automotive market is taking a leading role in defining the future of many of the emerging technologies and is a good reason to follow the exciting innovations that the automotive market is now “driving.”

Cornelis Networks shot a huge salvo today into the fabric community with the announcement of Intel veteran Lisa Spelman assuming the CEO seat for the startup. Rumblings started on the Internet yesterday that a major Intel executive was leaving the firm for a new home. Spelman was an Intel veteran, having served long-standing leadership roles in the data center and AI group as well as leading the company’s Xeon processor strategy. She’s been widely hailed as a face to watch for higher levels of corporate leadership, making this departure yet another strategic talent exit for the chip giant.

What does this mean for Cornelis? The company has a history of leadership in the HPC arena, having delivered Omni-Path fabrics to data centers and having delivered a robust portfolio of high speed fabric solutions. For those who may remember, Omni-Path was originally designed within the walls of Intel before being re-born as an independent company four years ago. And while HPC fabrics are interesting, the strategic value of Cornelis, and likely the reason why Spelman was interested in driving this tech further into the market, is AI’s insatiable fabric demand. We’ve written about the need for a fabric alternative to NVIDIA’s InfiniBand solutions on the TechArena at length before, and the Ultra Ethernet consortium is certainly building a groundswell of momentum for alternative solutions for AI clusters.

If you’d asked me yesterday who was going to lead new fabric innovation for AI, I likely wouldn’t have Cornelis on the top of my list. But the addition of Spelman, and her deep knowledge of the AI landscape, incredible relationships with ecosystem and customers built from her time driving Xeon processors, and frankly business savvy in terms of what it takes to create categories and grow tech leadership with customers, Cornelis has placed themselves squarely in the mix. I’m interested to learn more about how the Omni-Path based IP mixes with AI customer requirements, how Lisa will place a different focus on delivery of tech from previous leadership, and how strategic collaborations will be brought to bear to scale Cornelis’s ascent in the marketplace. Watch this space for more information in the weeks ahead as Cornelis begins to unveil it’s expected evolution.

TechArena host Allyson Klein chats with Sema4.ai co-founder Antti Karjalainen about his vision for AI agents and how he sees these powerful tools surpassing even what current AI models deliver today.

In our latest TechArena Data Insights interview, Jeniece Wnorowski and I had the pleasure of chatting with Doug Emby, Vice President of Cheetah RAID. We delved into the fascinating world of cutting-edge storage solutions tailored for edge environments, which are crucial for industries such as entertainment, defense, and autonomous vehicles.

Doug shared insights about the remarkable Cheetah RAID Raptor and Prowler servers. These servers are designed to handle the rigorous demands of media and entertainment as well as military applications. The Raptor 2U server, in particular, is a powerhouse with its high storage capacity, robust performance, and rugged design. It boasts up to 737.28TB (using Solidigm D5-P5336 61.44 SSD) of storage capacity, hot-swappable NVMe canisters, and support for PCIe Gen4, ensuring rapid data transfer and reliable performance even in extreme conditions. You can imagine how these hot-swappable canisters could be used to quickly capture data in rugged environments and transport it swiftly at the end of a shoot day or in a migrating environment. Given the reliability of SSD technology, this use case works effortlessly without significant risk of data loss or drive degradation.

How much data can these solutions handle? A key part of our discussion was about the importance of scalable storage solutions for managing vast amounts of data at the edge. The Solidigm D5-P5336 SSD, which is integrated into Cheetah RAID’s systems, stood out for its high capacity and performance. This SSD is optimized for data-intensive workloads, including AI-driven data lakes, big data analytics, and scale-out NAS, providing efficient and rapid storage and retrieval of extensive datasets. Jeniece shared some insights on how Solidigm’s SSD technology is integral to these advancements. She highlighted features such as enhanced power loss data protection, hardware encryption, and temperature monitoring, all of which are essential for maintaining data integrity and performance in various edge applications.

One of the most compelling parts of our conversation was understanding the synergistic innovation driven between the two companies. Their collaboration has resulted in powerful and efficient storage solutions that are being relied on by IT leaders across industries. These servers ensure that critical data is stored securely and accessed quickly when needed, making a significant impact in real-world scenarios. Cheetah RAID’s high-performance server/storage, featuring hot-swap drive canisters and Solidigm’s 61.44TB drives, makes a powerful statement unmatched by others in the market.

For those interested in the technical details and practical applications of these innovations, the full interview provides a wealth of information on the future of scalable storage solutions at the edge. To dive deeper into our discussion, you can visit the TechArena interview here and learn more about Cheetah RAID's innovative products on their official page here.

The Road to the AI Era is Paved in Semiconductor Manufacturing Innovation

This report provides insight to the force and speed of innovation required to propel artificial intelligence (AI), new requirements from across the computing landscape, and why foundational principles of semiconductor manufacturing are requiring re-invention to deliver the performance and scale of this new age. We cover the impact of generative AI and large language models (LLMs) across industries, current challenges in delivering performance to meet LLM requirements, how High Bandwidth Memory (HBM) has emerged as a foundational element of AI compute platforms, and a foreshadowing use case of upcoming chiplet-based processing solutions. We also look at how advanced 2.5D and 3D packaging, delivered in collaboration with market leader Lam Research, ensures the future of AI and continued semiconductor innovation.

Introduction

We’re seeing our world transform at warp speed, and the opportunities AI will unleash are just beginning to surface. What once seemed science fiction is actually closer than we may realize. OpenAI blew the doors off this arena with its recent demonstration of its Figure One robot performing complicated tasks and demonstrating complex decision processes. NVIDIA CEO Jensen Huang underscored this innovation, stating that robots will integrate across industries providing support for manual tasks and more.1 While robotics captures incredible inspiration for many of us, it is the tip of the iceberg for potential societal advancement in the years ahead. One of the most compelling areas for near-term benefit is pharmaceutical discovery. Here we’ve seen major tech players collaborating with traditional pharma and biotech startups to fuel a new generation of drug research that’s estimated to improve profitability in the sector by up to 25% according to McKinsey.2 What’s driving this investment? Bob Rogers, co-founder and Chief Scientific Advisor at leading healthcare AI startup BeeKeeperAI, explains,

“For each application area in drug development, the speedups reported by AI vendors are 3x to 10x. These accelerations by themselves are significant, but the real magic lies in the fact that every step of the drug development process is currently built from interlocking, inefficient human tasks. Replacement with AI tooling will result in wholesale reductions in the time it takes to propose, test, and report on new drugs in the market.”

Pharma transformation is being echoed across industries, and while each industry carries different near and far-term potential and will move at different speeds, it’s easy to conclude that entire industries will be transformed and our definition of work reshaped. In fact, we’re seeing the rapid evolution of a symbiotic relationship between humans and machines where AI can take on routine or tedious tasks, freeing humans to focus on innovation. This symbiosis even extends into the most complex invention undertaken by humans — the continued advancement of semiconductor manufacturing.

I’m excited to kick off a blog series on All Things Automotive. This topic is close to my heart. It is pretty much falling off a log to garner broad interest from a broad audience as we are now entering an era where the decades of fascination around cars that could drive themselves are now becoming a reality. I have been fortunate enough to have held many different roles throughout my career, both technical and business, that have been centered around the automotive market such that I have had a front-row seat to witness the transformation of this industry.

Cars are cool.

Why do I say that? Well, everyone who has ever owned a car has their own story about their car. It’s personal. For us boomers in the crowd, we knew exactly how to feather the accelerator when trying to start the car. It was somewhat of an art. If you didn’t understand what it took to get the car started, you would end up flooding the engine or running down the battery. Ultimately, over time you developed a personal relationship with your car and “understood” it.

Cars are personal.

When you ask someone about the first car they owned, you typically will hear a story that reflects some kind of unique relationship between them and their car. Many people have given their cars names, and everyone seems to have a mix of good and bad memories to share. If you’re ever looking for a good icebreaker– ask them about their first car. You probably won’t be disappointed.

My first car was a ’71 Ford LTD. Canary yellow with an olive-green interior. – What a color combo. This car was enormous. The trunk was big enough to allow me to pack all my possessions to go back and forth to college. With gasoline at 60 cents a gallon, fuel economy wasn’t even a thought. Complete with rotor and distributor cap, on a rainy day, if water got into the distributor cap the power steering, power brakes, power everything would fail. The first time this happened to me, it was quite a traumatic experience. In fact, every time it happened it was a traumatic experience. Today you probably need to go to a museum to see a distributor cap and most certainly, if you want to buy a fleer gauge to adjust the points. This has been long since replaced by electronic ignition and such.

For the first 150-plus years since the car was first invented, the adoption of electronics and technology in general in the car was slow. Seat belts were first introduced in 1885 and weren’t mandatory until somewhere around 1968. In California, it wasn’t until 1986 that you would receive a ticket if you weren’t wearing a seat belt. Growing up in the 60’s I used to enjoy watching Batman on TV. It was a light-hearted tongue-in-cheek adaptation of the comic book series. Not only was it one of the early shows to be filmed in color, something that was lost on me because we didn’t own a color TV, but every time Batman and Robin got into the Batmobile, they paused to show Batman and Robin both putting on their seatbelts. Kind of a public safety message at the time.

Fast forward to 2024, the advances in automobiles through the adoption of the most advanced semiconductor technologies, are, in my mind, even more profound than the introduction of the automobile itself. Self-driving cars, which have been the stuff of science fiction, today are a reality, albeit with some restrictions. These advances, which have only happened in the past 10 – 15 years, have led to significant improvements in safety over the seat belt, anti-lock brakes, or airbags, the mainstay for several decades.

Historically, the electronics in the vehicle were based upon mature, low-technology semiconductors at the level of 8-bit microcontrollers fabricated on semiconductor nodes that were at least a decade old. For many of the major automotive OEMs, electronics was considered context, not core to their business. This led to the introduction of Tier 1’s including Visteon and Delphi which were the spin-out of the electronics groups within Ford and GM respectively. Ford’s spin-out of their electronics group, named Visteon happened in 2000.

Today, auto OEMs have come to realize that the use of technology to enable a safer and more enjoyable driving experience is driving consumer purchasing decisions, not brand loyalty. This is a dramatic change from the past when brand, styling, engine, and transmission were the traditional factors that drove consumer purchase decisions. The technologies adopted in today’s vehicles are not for the faint of heart; they represent some of the most leading-edge technologies across multiple disciplines including, semiconductor technologies, packaging, artificial intelligence, and computing architectures, with many still on the drawing table as they are still being defined by some of the best minds across many different industries.

Automotive OEMs grappling with the fact that electronics has quickly moved from being context to core are now starting to “spin in” electronics organizations – a term, I have been told is referred to as “the double helix”. In 2017 – just 17 years after Ford spun out Visteon, they hired 400 engineers from Blackberry to accelerate the development of vehicle electronics. This is just one very small example of the expansive disruption that has been and continues to occur across the entire automotive value chain. The disruptions are profound and make for rich stories for a blog series. I will most definitely talk about these in future blogs.

As an introductory blog, I thought that this would serve as a good backdrop to understand the motivations for the adoption and development of leading technologies across multiple disciplines. Future blogs will be more technical, covering topics including chiplets, cybersecurity, functional safety, artificial intelligence, sensors, the evolution of automotive architectures, and the very popular software-defined vehicle, amongst many others.

TechArena host Allyson Klein and Solidigm’s Jeniece Wnorowski chat with Cheetah RAID VP Doug Emby about the innovative solutions his company is delivering to edge environments across a wide swath of applications from the entertainment industry to defense, and how innovative SSD designs from Solidigm help provide a foundation for storage performance and efficiency.

TechArena host Allyson Klein and Solidigm’s Jeniece Wnorowski chat with Taboola Vice President of Information Technology and Cyber, Ariel Pisetzky, about how his company is reshaping the marketing landscape with AI infused customer engagement tools.

TechArena spoke to over a dozen industry experts from  Circle B, Credo, London South Bank University, the Open Compute Project, Palo Alto Electron, PLVision, Qarnot, the Research Institutes of Sweden, and ZeroPoint Technologies,  and  to publish this comprehensive report on the state of open compute infrastructure innovation and how organizations should align data center planning and oversight with sustainability and performance objectives. If you manage an IT organization or oversee data center infrastructure, software, or sustainability initiatives, this report offers practical value for your organization.

I sat in on Andrew Dieckmann and Nidhi Chappell’s session at MS Build today to learn more about how Microsoft is delivering new AI capability leveraging the MI300X accelerator. Andrew leads Instinct accelerator development at AMD, and Nidhi oversees Azure AI and HPC infrastructure at Microsoft. While today’s MI300X instance delivery is a tremendous milestone for the companies, this has been a multi-year journey in the making starting in 2020 with the MI50 accelerator instance which was focused on a small scale cluster implementation.

Andrew called out that generative AI is the most demanding data center workload requiring incredible performance and capability from infrastructure and specifically silicon. The MI300X has been designed to integrate AMD technologies and manufacturing prowess to deliver a compelling choice of solutions to the marketplace. Nidhi furthered this concept, stating that until today’s launch, Microsoft did not have a choice of solutions to offer customers being limited to NVIDIA instances. For those customers who are seeking higher memory capacity and better price performance, the AMD based instances provide notable value.

She extended this thought by stating that this is not just about a silicon optimization but a holistic view across data center, AI accelerator, CPU, IO and network optimization, delivering the infrastructure environment that allows Microsoft to keep pace with broader corporate objectives on scaling both performance capability and energy efficiency within the Azure environment. Nidhi’s team is leveraging MI300X for Microsoft’s own Azure AI, a stunning group of workloads that collectively delivers 7.5 trillion characters translated per month, 54 million meeting hours transcribed in Teams per month, and 100 million monthly active users of AI text predictions per month. While she didn’t go into detail on how much of this work is delivered using MI300X today, we recommend watching this space for growth of platform usage given the value the platform represents to generative AI.

One notable observation from the discussion was the centrality of low-latency access to data for generative AI. Nidhi and Andrew both discussed the capabilities offered with the MI300X platform in HBM memory support as well as platform memory capacity scale. Another attribute of note was the central focus on Hugging Face and their use of MI300X services giving callouts to the software optimizations as well as core platform capability as differentiating factors for their use.

Congrats to Microsoft and AMD for this great milestone. We at the TechArena can’t wait to see more collaborative innovation.

MS Build has always been a fantastic conference for developer innovation within the Microsoft environment. In 2024 it has transformed into a must attend event to track AI innovation. Today, Satya Nadella and team did not disappoint as they delivered a maelstrom on new announcements to Azure AI, Co-Pilot, and more. The speed of announcements in the keynote was reflective of the speed of Microsoft innovation, and it starts with the foundational innovation of Azure infrastructure.

Satya shared a massive buildout of Azure data centers across the world from Thailand and Malaysia to Spain and Wisconsin. Microsoft announced the world’s largest supercomputing cluster in the world last fall, and Satya shared that they’ve grown this supercomputing capability by 30X in the last six months, an incredible pace of deployment reflective of the customer demand for Azure AI services.

Silicon Collaborations Fuel Azure AI Growth

They’re delivering this with tight partnerships with industry leaders along with home grown innovation starting with Microsoft silicon. This starts with their deep partnership with NVIDIA. The collaboration was discussed earlier this year at GTC and covered on the TechArena. Microsoft’s Nidhi Chappell discussed the nature of this collaboration as true co-invention in her interview on TechArena last week, and this was reflective of plans for delivery of H200 based instances later this year and expectations for Blackwell platforms among the first available cloud instances on Azure. These will be available to fuel MS365 and Co-Pilot Acceleration.

NVIDIA, however, is not the only game in town for Azure, and Satya stressed a commitment to the broadest choice of acceleration. Today, Satya announced expansion of the strategic collaboration with AMD with delivery of the industry’s first NDMI300 instances for customers. This is an enormous milestone for the two companies offering the best price performance instances for GPT 4o instances. I expect to hear more about this collaboration at the conference reflective of AI providers desire to support competitors to NVIDIA’s dominance of the AI acceleration arena.

Microsoft extends their investment in this space with their own silicon, and Satya did give a shout out to Microsoft Maia acceleration. However, more attention for home grown silicon was given to Microsoft Cobalt processors. Satya announced the public preview of Colbalt based VMs for cloud native computing. These ARM based solutions are being delivered to customers including Elastic, MongoDB, Snowflake and more and put the silicon industry on notice that while Microsoft was comparatively late to indigenous silicon development, they are not slowly exploring this space but integrating rapidly into customer services.

With this rapid development of compute capacity and capability, we need to consider Microsoft’s utility bill to power this infrastructure. Satya gave an update on his team’s goals in energy efficiency stating that Microsoft is on track to meet 100% renewable energy use across global Azure data centers by next year. He pointed to specific innovations in advanced power and cooling technologies helping Azure to meet these commitments. While this is a fantastic achievement especially given the challenge of renewable energy availability across the diverse geographical landscape that Microsoft is operating, I'd like to learn more about advancements on embedded carbon investment and true circularity given the speed of innovation investment.

Infrastructure Innovation Fuels AI Integration and Societal Transformation

So what does this buildout and innovation deliver? Satya spoke to the performance and efficiency advancements that Microsoft is delivering to customers giving an example of Chat GPT4 achieving 12X cost savings and 3X performance improvements since its launch in Q4 2022. That’s 1.5X performance gains vs. Moore’s Law in case you’re tracking.

But Chat GPT 4.0 is not the only LLM being delivered in Azure AI. Satya spoke to broad model support being tapped by over 50K organizations around the world, all grounded on the foundational partnership with OpenAI. GPT-4o the industry’s top performing model announced just last week, has already been integrated with MS Co-Pilot and in Azure AI.

Microsoft has also delivered Model as a Service (MaaS) capabilities with a handful of partners including NTT Data and expanded their ongoing opensource collaboration with Hugging Face with new capabilities for developers. Satya also claimed leadership on small language models including expansion of Phi-3. Microsoft is delivering Phi-3 vision as well as Phi-3 small, medium, and mini models– all with sizes to fit developer needs from ~ 3 billion to 12 billion parameters.

All of this capability fuels opportunity for integration across industries, and Satya briefly covered examples of customers taking advantage of this technology. A notable example of society changing integration of AI into our world is a new collaboration with the Khan Academy propelling AI’s power directly into US classrooms. Khanmigo, a Khan Academy AI tool will help support US educators to offload some of the crushing operational work for managing the classroom freeing time  to for educator engagement with students. And while the capability of AI will transform industries, deliver new revenue streams and create eye opening efficiency to work, this example provides a glimpse of how transformational a time we live in. We’re excited to see more and are thrilled to see what Microsoft is delivering to help usher in this new AI Era.

TechArena host Allyson Klein chats with Microsoft’s Vice President of Azure AI and HPC Infrastructure, Nidhi Chappell, in advance of Microsoft Build 2024. Nidhi shares how her organization is accelerating deployments of critical technology to fuel the insatiable demand for AI around the world and how Microsoft’s AI tools including co-pilot, Open AI and more have been met with overwhelming engagement from developers. She also talks about Microsoft’s silicon plans and strategic collaborations with NVIDIA and AMD.

Supermicro has been a player in the tech industry for over 30 years, focusing on building breakthrough solutions for data center compute requirements. Their history as a nimble infrastructure supplier has driven them ahead as a leader in AI era compute delivery. This is why I was so excited to invite Supermicro’s Paul McLeod to the TechArena Data Insights podcast sponsored by Solidigm. My co-host Jeniece Wnorowski and I put Paul through his paces to discuss Supermicro’s perspective on AI era computing, what customers are demanding of infrastructure, and how the data pipeline is a central innovation focus for today’s deployment targets.

The changing landscape of data management in AI workloads

Paul started by discussing the history of data management across data center environments stating that traditionally, IT has involved infrastructure silos for specific storage needs, with limited data accessibility across storage solutions. Paul added that with AI this is changing. AI demands all these data types and pipeline workloads to function simultaneously. Supermicro utilizes this evolving requirement to deliver value to customers. Paul pointed out that Supermicro’s heritage includes early use of NVMe technology, giving them valuable experience in storage solutions for AI.

This has shaped by a flattening of the traditional tiered storage model. Previously, cold tiers existed for data that rarely needed to be accessed. However, with AI, fast access to almost all data has become critical meaning that the cold tier is heating up into warm tier storage where flash alternatives shine. For this transition, Supermicro's solutions have featured Solidigm’s D5P5430 SSDs. These SSDs were designed to solve the unique challenges of data center enviroments, including delivery of high-density storage high performance storage drives needed for AI training. The P5430, their premier QLC-based offering, is available in various form factors to accommodate different server designs and thermal requirements and delivering impressive capacity, reaching up to 30 terabytes. Paul noted that the technology was dialed in for Supermicro’s requirements highlighting that bottlenecks have shifted from storage to compute and network. This is made even better with key collaborations with storage partners taking advantage of underlying infrastructure to fuel even the most grueling customer requirements.

Looking ahead: The future of data storage for AI workloads

Where does the market take platform innovation next? Paul pointed out the need for continued innovation of the data pipeline to reach additional scale in performance, compute density and efficiency. As large language models scale and customers demand more compute to train algorithms, keeping the data pipeline in balance and fed with rely on continued industry collaborations with partners like VAST Data and Solidigm. Be sure to visit Supermicro and Solidigm’s websites for more information about storage and compute solutions for the AI era, and continue following the TechArena as we explore data insights.

TechArena host Allyson Klein chats with Research Institute of Sweden’s Jon Summers about the latest research his team has conducted on efficient infrastructure and data center buildout in the wake of massive data center growth for the AI era.

TechArena host Allyson Klein chats with Palo Alto Electron CEO Jawad Nasrullah about his vision for an open chiplet economy, the semiconductor manufacturing hurdles standing in the way of broad chiplet market delivery, and how he plans to play a role in shaping this next evolution of the semiconductor landscape.

TechArena host Allyson Klein chats with OCP’s Raul Alvarez on his new charter accelerating growth of the data center market in Europe as well as his ongoing work in immersion cooling technologies from OCP Lisbon 2024.

TechArena host Allyson Klein and Solidigm’s Jeniece Wnorowski chat with Weka’s Joel Kaufman, as he tours the Weka data platform and how the company’s innovation provides sustainable data management that scales for the AI era.

Deborah Andrews talks data center sustainability at OCP Lisbon 2024.

TechArena host Allyson Klein chats with Deborah Andrews, Professor of Design for Sustainability and Circularity, at London South Bank University, about her research into delivering true circularity in data center production and operation and how the future is shaping to get closer to this vision.

PLVision’s Taras Chornyi chat’s SONIC at OCP Lisbon 2024.

TechArena host Allyson Klein chats with PLVision Director of Open Networking Solutions and Strategy, Taras Chornyi, about the progress of SONIC and open network infrastructure for the AI Era.

TechArena host Allyson Klein is joined by Solidigm’s Jeniece Wnorowski as they continue to explore rapid data innovation fueling today’s computing. In today’s episode, they chat with VAST Data’s Global VP of Engineering, Subramanian Kartik, as he describes how his team has delivered a breakthrough data platform for the AI Era.

OCP’s Lesya Dymyd provides insight into the Future Technologies Symposium and the European Data Center market from OCP Lisbon 2024.

TechArena host Allyson Klein chat’s with OCP’s Lesya Dymyd about her work steering the organization’s future technologies symposium as well as her deep collaboration with European technology providers and operators delivering to the region’s unique market requirements.

Qarnot CEO talks sustainable tech innovation at OCPLisbon24

TechArena host Allyson Klein chats with Qarnot CEO Paul Benoit about how VC backed startups are an essential element of vibrant industry innovation, and how the AI era has placed requirements on innovative approaches to sustainable IT.

Credo’s Don Barnetson chats optical innovation at OCP Lisbon 24

TechArena host Allyson Klein chats with Credo VP Don Barneston about how his company is delivering innovative connectivity solutions that address the AI era’s requirements for scalable data movement in the data center and beyond from OCP Lisbon 2024.