Welcome to
the Forum

Check out our in-depth look at key takeaways from AI Hardware and Edge AI Summit, highlighting how advancements in AI infrastructure, acceleration, and connectivity are transforming the future of computing.

I love talking to Letizia Guiliano. Letizia drives product marketing and management at Alphawave, but her passion for silicon design and standards innovation is infectious. In our latest chat on my In the Arena podcast, I spoke with her about the game-changing innovations required for AI advancement across connectivity and broader silicon foundations.

Letizia shared a fascinating view on how Alphawave is pushing the boundaries of high-speed interconnects to support the ever-growing demands of AI infrastructure, and how new competition is coming to the connectivity arena in this growing market.

Letizia and I have been discussing chiplets since last year, and last week’s chat highlighted how the designs are revolutionizing semiconductor implementation by improving both performance and efficiency. Unlike traditional monolithic chips, chiplets allow designers to break up complex components into smaller, interconnected pieces, optimizing for power, performance, and cost.

I challenged Letizia on progress towards an open chiplet industry, and she explained that while Alphawave is focusing on open standards to ensure interoperability between different chiplets and systems, current designs focus on bespoke customer solutions. There is still challenge in the market for standards-setting as well as tension that some players benefit more from an open industry than others.

One area that is advancing rapidly with standards delivery is scale-up fabric technology. With AI workloads growing at an unprecedented rate, moving massive amounts of data quickly and efficiently between processing units is a key challenge. Letizia was particularly excited about the industry’s progress in this space, highlighting that the new Ultra Acceleration Consortium is advancing standards that will enable interoperability across multiple company’s accelerator technology. Today, this area of technology is largely controlled by NVIDIA GPUs and NVlink serving as a barrier to entry to other accelerator alternatives.

We also touched on the importance of scalability. As AI models like GPT-4 and LLaMA continue to grow, the ability to scale up infrastructure to support these models is becoming increasingly critical. Alphawave is aiming to provide scalable solutions that not only meet today’s needs but are also future-proof, ensuring that data centers can easily expand to handle tomorrow’s AI workloads.

Letizia shared that the future of AI isn’t just about more powerful models, but about smarter infrastructure. And with Alphawave’s commitment to driving forward innovation in both connectivity and standards embracing design, the company is on track to be an important player in the future shaping of AI silicon.

For those who want to dive deeper into the details of Alphawave Semi’s solutions in this space, I highly recommend tuning into the full interview and visiting their website.

For those of you who follow the TechArena – or the technology landscape – you’re likely familiar with the name Lisa Spelman. Former head of all things Xeon at Intel, Lisa has driven tens of billions in business creation in her career as well as gaining real-world experience in the IT realm at the company.

Her knowledge of both the data center arena and of the large hyperscalers is unquestioned, and we at TechArena were excited by her recent move to take on leadership as CEO of Cornelis Networks as the company seeks to drive their technology into the AI landscape. Cornelis is the provider of Omni-Path fabric technology, a competitive fabric to InfiniBand that heretofore has had great success in HPC clusters across the globe.

I finally got to catch up with Lisa on her vision for the company at the AI Hardware and Edge Summit about how she is going to harness Cornelis technology to compete squarely with NVIDIA’s InfiniBand technology as a fabric foundation for AI training clusters. Lisa did not disappoint. We dove into the future of AI scale-out and how Cornelis is leading the charge with their Omni-Path solutions with an approach to maximize performance, scalability, and interoperability for data centers, ensuring that AI models can scale effortlessly across different environments.

Lisa highlighted how Omni-Path provides the foundation for building out next-gen AI infrastructure, making it easier for businesses to scale their AI workloads without running into bottlenecks of other technologies like Ethernet. She shared that Cornelis has already seen substantial traction with customers who are looking to optimize their data center operations for AI. One of the key takeaways for me from our conversation was how AI scale-out is moving beyond just a cloud discussion. Companies are increasingly interested in hybrid models that balance the power of cloud computing with the control and security of on-prem solutions.

What’s the TechArena take? If you’re keeping an eye on AI infrastructure, you’ll want to closely watch what Cornelis Networks is doing. Cornelis is helping companies deliver to the hybrid vision, and their goal of future-proofing AI infrastructure and providing an alternative technology to an all-NVIDIA solution is something that businesses can't afford to ignore. Cornelis’ focus on true interoperability with Ethernet and Ultra Ethernet extends Omni-Path into, from my view, a very viable and competitive solution in the market addressing both technology and business needs for new AI cluster deployments.

Their innovative approach to fabric delivery, and frankly their mere existence as a competitor to NVIDIA, will be a game changer for companies looking to optimize their near-term AI operations while maintaining the flexibility to grow in a constantly evolving tech landscape. Cornelis is indeed in a perfect position to disrupt a market, and with Lisa’s extensive knowledge of the data center landscape, key players, and business savvy, I expect a bright future for Omni-Path.

In my recent chat with Sascha Buehrle from Uptime Industries, we dove deep into the exciting world of AI at the edge, powered by their innovative platform Lemony. What stood out to me is how Lemony is designed to bring scalable and secure generative AI right to your desk. Instead of relying on massive, cloud-based infrastructure, Lemony offers an on-premise solution that is not only easy to deploy, but also incredibly accessible for small and mid-sized businesses.

One of the key highlights from Sascha was Lemony’s model support. It comes preconfigured to support several generative AI models with an emphasis on Meta’s Llama, the predominant open source tool de jour, which means you can hit the ground running with everything from natural language processing to image generation tasks and take advantage of a variety of tools to fit the job. It's a seamless integration for businesses that want to start using AI but don't have the resources to maintain large AI clusters or hire a specialized data science team.

Storage is another critical aspect when considering AI training at the deskside, and Lemony comes with options for up to 10TB of storage, making it perfect for businesses that need to manage substantial amounts of data securely. Sascha mentioned that Lemony’s edge AI solution can handle complex datasets while maintaining data sovereignty — a huge selling point for companies that prioritize privacy and data control. And since it's all on-premise, there’s no need to worry about the potential risks of sending sensitive information to the cloud.

One thing I’d called out in my pre-AIHW Summit show blog was a question on customer traction, and Sascha delivered insight into the early success the team has had with Lemony. It’s impressive to see the traction the company is gaining with customers. In just under a year, Uptime Industries has secured over 100 early adopters across a variety of industries — from finance to healthcare — who are using Lemony to streamline their operations with AI. This is a massive vote of confidence in the platform’s ability to scale and deliver real business value. Businesses are particularly drawn to the security features and the fact that Lemony is an all-in-one box solution, which means you can deploy it right in your office and start seeing the benefits immediately.

What’s the TechArena take? I don’t think Lemony represents the alpha and omega of the future of generative AI, but it does bring a fantastic product to the market that could tap a sweet spot of early adoption. Lemony is about bringing AI to where it’s most needed today: closer to your data, closer to your operations, and in a way that fits many companies’ specific usage model. I can’t wait to see where Uptime Industries goes from here, but with the momentum they’re building, I have no doubt they’ll play a key role in making AI practical and accessible for the rest of us. Watch this space as Lemony continues to evolve, and don’t miss out on the opportunities AI at the edge can bring to your business.

During our latest Data Insights podcast, sponsored by Solidigm, Ian McClarty of PhoenixNAP shares how AI is shaping data centers, discusses the rise of Bare Metal Cloud solutions, and more.

Sean Lie of Cerebras Systems shares insights on cutting-edge AI hardware, including their game-changing wafer-scale chips, Llama model performance, and innovations in inference and efficiency.

Letizia Giuliano of Alphawave Semi discusses advancements in AI connectivity, chiplet designs, and the path toward open standards at the AI Hardware Summit with host Allyson Klein.

Lisa Spelman, CEO of Cornelis Networks, discusses the future of AI scale-out, Omni-Path architecture, and how their innovative solutions drive performance, scalability, and interoperability in data centers.

Kelley Osburn gets storage. As an industry veteran and leader at Graid Technology, Kelley recently shared his insights on how the storage arena is rapidly transforming to fuel AI workloads and how his company’s SupremeRAID™ solution – a revolutionary approach to tackling modern data storage challenges – is hitting a sweet spot in the market.

So why is traditional RAID no longer sufficient? Kelley explained how these configurations struggle with high-performance computing demands, especially in data-intensive environments. He emphasized the need for innovation in data storage as the exponential growth in data continues to challenge existing systems, explaining that RAID's original purpose was to provide redundancy and protection against disk failures. While this redundancy is still valued, it lacks the performance desired by many customers.

As data stores grow and speed-of-delivery of data becomes more urgent, innovation to the approach helps extend RAID solution viability while meeting customer demand. Graid's SupremeRAID™ solution, for example, optimizes storage performance by offloading RAID tasks to a dedicated hardware device, enhancing speed and efficiency without compromising data integrity. This makes it an ideal solution for customers managing massive amounts of data for applications like AI, machine learning, and big data analytics.

Kelley detailed the core value of Graid’s solution, describing how SupremeRAID™ addresses critical bottlenecks in traditional storage systems by offering unprecedented performance gains while reducing the computational load on CPU and system resources. The innovation lies in its architecture, which integrates both hardware and software in a way that eliminates RAID-specific processing burdens from the host server, thus allowing server resources to focus on other tasks. The result is a solution that dramatically improves throughput and reduces latency, creating a more balanced and efficient data environment.

In addition to AI and ML, SupremeRAID™ also proves to be a valuable tool in applications in media and entertainment, where high-resolution content creation, editing, and rendering demand significant data processing power. Its ability to handle these workloads without compromising performance makes it a game-changer for companies managing large data sets.

Industry Implications and Future Outlook 

So what are the broader implications of Graid's innovations for the storage industry? Kelley explained that as companies continue to generate vast quantities of data, the demand for more efficient and performant storage solutions will only grow. Graid’s SupremeRAID™ is positioned to address these challenges head-on, providing enterprises with the tools they need to manage, protect, and access their data faster and more reliably than ever before. This will rely on underlying storage media delivering the performance and density required for these tasks, and Kelley pointed to Graid’s strategic collaboration with Solidigm as an example of how high performance QLC memory delivers unique value to customers.

Looking to the future, Graid plans to continue evolving its technology to meet the ever-increasing demands of the data economy. As data volumes grow, so too will the need for innovative storage solutions that can handle not just the size, but the speed and complexity of modern workloads. SupremeRAID™ represents a critical step in that direction, offering a glimpse into the future of RAID technology and its role in addressing the data challenges of tomorrow.

Want to learn more? Check the episode here.

‍

Join Sascha Buehrle of Uptime Industries as he reveals how Lemony AI offers scalable, secure, on-premise solutions, speeding adoption of genAI.

In the latest episode of The Tech Arena, I had a fantastic conversation with Neeraj Kumar, the Chief Data Scientist at Pacific Northwest National Laboratory (PNNL), about how AI is shaping the future, especially around energy efficiency and large-scale data processing.

Neeraj is a dynamic thinker who’s deeply invested in how AI can revolutionize everything from scientific discovery to sustainable technology. Together, we explored some of the key innovations and challenges in the AI space, with a special focus on energy consumption and the balance between advancing technology and environmental responsibility.

Neeraj kicked off by discussing one of the most pressing topics in AI today — how to manage the skyrocketing demand for computing power without exacerbating the climate crisis. AI is, after all, a double-edged sword. While it brings incredible breakthroughs in fields like medicine, climate modeling, and autonomous systems, it also consumes a huge amount of energy. Neeraj highlighted this tension between innovation and sustainability and stressed that AI's future hinges on solving this dilemma.

He shared insights into his work at PNNL, where AI is driving cutting-edge research. One area he focused on was the collaboration between PNNL and Micron to improve memory systems for AI and high-performance computing (HPC). As AI systems evolve, they require ever-greater amounts of data to be processed in real-time. That’s where memory plays a critical role. Neeraj pointed out that memory bottlenecks can cripple even the most advanced AI models, and optimizing these systems is crucial for enabling more energy-efficient AI. He believes that advancements in memory technology will be a game-changer in achieving the balance between computational power and energy efficiency.

I loved how Neeraj dove into the technical aspects of how PNNL and Micron are innovating in this space. They're working to enhance memory architectures to meet the needs of next-generation AI systems. The aim is to reduce the energy footprint of AI while improving its capacity to handle massive datasets. This kind of optimization is vital for industries like healthcare, where AI can sift through oceans of data to discover new treatment methods or predict patient outcomes, but only if the computational infrastructure can keep up.

We also discussed how intertwined the worlds of AI and compute efficiency have become. It’s no longer enough to simply innovate; those innovations must also be energy-conscious. Neeraj made a great point about how the energy demands of AI systems are growing exponentially, and we need to rethink how we design hardware and software. He highlighted how PNNL’s research is pushing the boundaries on what's possible, from creating AI models that are more efficient in their data use to developing entirely new computing architectures that reduce energy consumption without compromising performance.

Neeraj shared his vision that AI itself can be a powerful tool for combating climate change on a broader scale. For instance, AI can help optimize energy grids, improve the efficiency of wind and solar farms, and even model climate patterns more accurately. But to truly leverage AI in the fight against climate change, we must first ensure that the technology is sustainable. Neeraj was candid about the fact that we’re not there yet, but with ongoing research and innovation, we’re heading in the right direction.

Climate change is not the only area where AI is contributing to advancement in the scientific community. Neeraj shared some compelling examples of how AI is transforming research at PNNL. One standout example was how AI is being used to accelerate scientific discovery by identifying patterns in data that humans might miss. This is particularly important in fields like materials science and chemistry, where AI can analyze vast datasets from experiments and simulations to uncover new materials with potential applications in everything from energy storage to quantum computing.

The role of AI in scientific discovery is something Neeraj is particularly passionate about, and it was evident in the way he spoke about its potential. He emphasized that AI doesn’t replace scientists; rather, it amplifies their ability to make new discoveries. In many ways, AI acts as a partner to researchers, sifting through enormous amounts of data and generating insights that can lead to breakthroughs much faster than traditional methods. This is where AI’s promise really lies — in its ability to accelerate progress in some of the most complex and pressing challenges of our time. 

Towards the end of our conversation, we touched on the future of AI and what it might look like in the next decade. Neeraj is optimistic about where the field is headed but also realistic about the challenges that lie ahead. The next big leap, according to him, will come from integrating AI with other emerging technologies like quantum computing. This, he believes, will open up new frontiers in both AI capabilities and energy efficiency.

We cover a lot on the TechArena about enterprise applications of AI, and the podcast with Neeraj was a welcome reminder of how AI is contributing positively to society at large, advancing science in ways we wouldn’t otherwise accomplish. While there are significant challenges ahead to tap the full potential of AI to these use cases, especially when it comes to energy consumption, with leaders like PNNL and other US labs driving innovation, I’m confident that we’re on the path to a more sustainable, AI-driven future.

“I.N.T.E.L.L.I.G.E.N.C.E. is down! I repeat, we have no I.N.T.E.L.L.I.G.E.N.C.E.!”

-  Lisa, Team America: World Police*

Numbers and dates sometimes seem to take on an oversized significance. One could posit that the random patterns AI sometimes finds in a stream are the equivalent of the random patterns burned into the human noggins by repetition, creating reputation. This could lead to a topic for another time: does human bias lead to implicit programmatic bias by accident? Only the ones and zeroes know, and those got buried in the compiler eons ago in computer years. 

Anyway, some date combinations evoke visceral reactions among the subsets of populaces. Drop a 12/25 into a Western Culture discussion, and the twinkly lights and a sudden introduction of forestry into the household takes mental hold. In China, mention of 8/8 sends the wedding planner industry into a tizzy. Some dates are less about happiness. In former days, a mention of 12/7 (though they called it December 7th back then) brought out the somber reactions of those who remember when our nation was lured into war by the attack on Pearl Harbor. Though fading, that pattern still evokes a response.

These days, the date of today’s publication seems to resonate in the minds of many, 9/11. It’s fresh in the cultural mind, and the global implications of the day a bit more than two decades ago keep washing ashore like some geopolitical flotsam. Among we, the tech weenies, a lot of the response drifted to the feeling that technology needed to create change to protect the innocent. We’re only accidental technocrats, after all. Two subjects drift to mind, proving that patterns evoke patterns.

One that we can address later in detail was the realization that redundancy and continuity involved more than putting the backup tapes in the basement of the same building. The needed push of number-crunching close to the trading floor (someone wrote about distance and latency recently, can’t remember where…) was fighting the pull of real-estate values and commute times. One firm’s bright techie idea of building some redundancy across the Hudson in New Jersey led to copycats that did the same. That migration alone likely shortened the closure of the largest markets to six calendar days from the months it would have otherwise taken.

On another vector, though, the fairly new power the computer industry was feeling resonated off the political firewalls: “We can use technology to make us all safer against global threats.” The bright engineering minds educated in the 30-year trough of relative international peace saw the opportunity for sea change. Even as we said it, the creep of horror at what technology as a tool might unleash sowed the seeds of its own inertia.

Yer Humble Author (YHA) remembers not long after the infamous date visiting one of the largest database vendors in the world with a shiny new concept server. Said system would throw all of its 32 threads across eight processors at a then-incredible 8TB of DRAM. It was 20 years ago, you stand on the shoulders of ants, friends. Rhetorically, YHA asked the team of think-biggers what Homeland Security could do with an 8TB in-memory database. The conversation that followed was interesting, and a little scary, though the implications seemed to point to the positive…if you squinted.

Now that a quick trip to the open-source Gits and the virtual card swipe at Amazon provides everyone similar capability, how has technology made the globe safer against the shadowy baddies? Well, many of the apparent changes in safety and security seem to be fractional, in the diddly over squat range. Technology purchasing in the hopes of a more secure globe certainly absorbs the tax dollars of many, but our tools don’t appear to have created that vital change that moves us into the realm of 60’s Star Trek unity. Some of us want the transporter, airports really suck post-9/11.

And therein lies the conundrum. Technology is a tool. The redundant tools that saved the backsides of the finance industry aren’t all that different from those analytics tools that get tossed at global metadata for security. And yet, the conflicting thought processes – political of all stripes, apolitical liberty, the need for consensus of committee, trying to explain all this to leadership who stopped at algebra – grind some tools to a nub of inefficiency. The tool is not to blame, it’s the users. Accidental technocrats, untie (sic). 

Hope should never be lost where technology is available. The leaps of innovation repeatedly outperform the bloat of human decision making. And maybe the human bias projecting into machine bias is a re-reflection of the same human bias. We’d probably just feel better on some digitally-significant days if we didn’t always stand in our own way.

*(Seriously, that might have been the only quote from the movie that could be used in a family-friendly space. But now that you’re at the end, you know that’s the only movie we could have used.)

Mark Wade, CEO of Ayar Labs, explains how optical I/O technology is enhancing AI infrastructure, improving data movement, reducing bottlenecks, and driving efficiency in large-scale AI systems.

Gayathri Radhakrishnan knows how to spot differentiated value and understands well the difference between innovative technology and well-designed PowerPoint.

As a partner at Hitachi Ventures with a deep pedigree in venture capital, she’s a savvy expert in valuation of what comes next. This is why I was so excited to catch up with her at this week’s AIHW & Edge AI Summit in the valley. The AIHW conference is many things – a landscape of silicon innovation, a trend source for AI model development, and also a hub for venture deals in-the-making as startups showcase their new tech and silicon valley investors evaluate respective offerings. I wanted to check in with Gayathri on the state of the AI market, what areas of innovation will be garnering venture investment in the future, and how she sees the landscape solidifying as we enter the second half of the decade.

Hitachi Ventures has made a few moves of late in the AI space, most recently announcing investment in Archetype AI, Ema, Strikeready and Trustwise. Their portfolio shows an acute focus on mindful investment in use cases that will drive real enterprise value, fitting with Hitachi’s overarching stable approach to the tech arena. Gayathri shared that evaluation for real innovation is nothing new, harkening back to the cloud era rife with startups that were delivering virtualization solutions and branding them cloud. AI is no different. There may be fantastic analytics solutions under development, but ferreting out what is truly using machine learning, what is truly automated, is the key. That metric, Gayathri explains, is based on what layer of the stack is in play and what unique value is required for that layer of innovation to disrupt in the market.

Gayathri and her team are also surveying the infrastructure landscape for innovation in efficient delivery of computing to fuel AI. With US data centers forecast to consume 9% of the world’s energy by 2030 alone, it’s no surprise that this focus on efficiency is a priority. She underscored the push and pull in the market between customer demand for more efficient IT and industry engineering investment to deliver. This starts at the boardroom and C-level and trickles down to IT vendor conversations. influencing both disruptive tech evaluation as well as potential M&A moves from large infrastructure and cloud players with startup infrastructure solutions.s

All of this innovation and investment is ultimately focused on delivering value to the enterprise. Gayathri stressed Hitachi Ventures’ broader purview across industrial markets and the need for enterprises to adopt this new powerful technology responsibly, with focus on safety and reliability required for the solutions enabled by its use in the marketplace. While she sees continued advancement of large AI models and breakthrough technologies like AI agents, smaller model integration into reliable use cases will be a feature for 2025.

If you’re at AIHW Summit this week, connect with Gayathri to learn more about her perspectives and be sure to check out her panel, Industry Landscape Overview: Funding Trends & Emerging Business Models in Generative AI Start-Ups, & Enterprise Adoption Patterns, featuring insights from Applied Ventures, Silicon Catalyst Ventures, and NGP Capital.

It’s Oracle CloudWorld week, and I was excited to hear from industry legend, Larry Ellison, on his view of the future of computing and the AI era. Oracle holds a distinct view of the enterprise and adoption of technology, so there are likely very few people who can project enterprise adoption curves better than Larry.

The keynote did not disappoint. Larry emphasized Oracle’s vision of AI for the enterprise, where practical AI solutions are designed to address real business needs. He highlighted the integration of generative AI across Oracle’s cloud applications, with tools supporting functions like finance, HR, marketing, and customer service.

This was lack of splash of the Open.AI crowd and more meat and potatoes for grizzled IT veterans. Larry emphasized that Oracle’s AI focus is on transforming business processes, not just hype-driven tools, and delivering AI-driven automation at scale.

Larry also spotlighted Oracle Cloud Infrastructure (OCI) as a trusted platform, built to handle sovereign cloud needs and ensure secure, region-specific data processing. This commitment to AI readiness has positioned Oracle as a trusted provider for industries like healthcare, finance, and government. Oracle’s approach underscores its reliability and adaptability in helping enterprises deploy AI efficiently, combining performance with the security enterprises demand.

One of the most compelling aspects of Larry’s speech was the focus on vector search, a method Oracle uses to process large volumes of unstructured data, seamlessly embedding AI into database environments like Oracle Database 23ai. This ensures that enterprises don’t need to move their data to leverage AI — AI can operate directly within the database environment, optimizing workflows for speed and scalability.

Oracle’s generative AI technology is geared to simplify complex tasks, automate workflows, and enhance decision-making. For example, Oracle Fusion Applications integrate AI for tasks like project proposal generation, strategy development, and content summarization, tailored to support business functions directly.

One thing I thought was interesting was Oracle’s ambition to follow a similar path with AI as it did with the cloud — building a trusted ecosystem that delivers real value for enterprise applications. Larry noted that while Oracle’s focus isn't on flashy AI tools for consumers, its enterprise-first strategy offers the reliability and scale required for mission-critical applications, very similar to its path with delivery of cloud services that met specific enterprise requirements.

What’s TechArena’s take? I think of a lot of companies before Oracle in terms of pure AI innovation. Those born on the cloud who lean forward into the tip of innovation will likely deliver the most impactful transformative tech in the near term. But…for those meat-and-potatoes workload enhancements with AI automation and agent control, I think Oracle will make an outsized impact on adoption. Time will tell if Oracle can once again re-imagine itself for this latest era of computing and prove to continue its centrality on enterprise application and service delivery.

Neeraj Kumar, Chief Data Scientist at PNNL, discusses AI's role in scientific discovery, energy-efficient computing, and collaboration with Micron to advance memory systems for AI and high-performance computing.

I believe that AI at the edge will start dominating AI chatter in 2025 as enterprises large and small seek to integrate generative AI tools into operational applications.

This is what we think of at the TechArena as the 2nd wave of gen AI, the first being the training of behemoth models by brute force of massive AI clusters in the largest clouds on the planet. While the hyperscalers continue on their quest for AGI supremacy, the rest of us are looking for real world integration into business operations to ramp the efficiency and insight at our fingertips with this powerful technology. Of course, whenever we discuss enterprise adoption, we need to consider words like reliability, trust, security and more. Hallucinations and other risks of too careless of AI adoption can represent brand reputation loss or worse.

The team at Uptime Industries has designed a solution to enable smaller businesses and work teams to harness the power of generative AI for unique workloads with that trust and reliability in mind.

Meet Lemony, a secure, on-premise generative AI platform that delivers an all-in-a-box solution for on-prem deployment at the edge. A single Lemony box will run you $499 for up to five users, and boxes can scale to address larger team requirements. What’s best, your data, combined with generative AI model power, can work in tandem in a transparent and secure fashion, to capture the application advancements you need to fuel your business.

After reading about Lemony, I’m intrigued to learn more! I love innovative solutions that take the heady and complicated of new tech innovation into a consumable tool that even companies the size of TechArena can use. I love the focus on data security and on- prem deployment for organizations that don’t want to compromise on data location or control. I love the speed that this company has been moving as well, going from formation in March of 2023 to seed round in January of 2024 complete with a core team of experts from across the companies you’d expect to help make them a success.

Frankly, I have a lot of questions for the Uptime Industries team, which I’ll get answered at next week’s AIHW and Edge Summit. The Uptime Industries team will be showcasing their wares at the event. While most of the companies assembled will be discussing the latest innovations to pave the path to AGI, they will be discussing how those who aren’t running the world’s largest data centers can tap AI’s power to transform our businesses. Watch this space for more next week.

‍

Arun Nandi, VP of Global Data and Analytics at Unilever, knows his way around data and how to apply it for positive business outcomes.

In my recent conversation with Arun in advance of this week’s AIHW and Edge AI Summit, Arun shared how his company is leveraging AI and data analytics to enhance business operations, drive sustainability, foster innovation and more. 

Arun’s insights centered on the transformative potential of AI in improving corporate decision-making, optimizing supply chains, and refining product offerings. What’s better, Arun believes all of this can be delivered while maintaining a focus on environmental responsibility.

AI and Analytics Sit at the Core of Innovation

Unilever is one of the world's largest consumer goods companies. They handle vast amounts of data daily from across their business portfolio, ranging from information on supply chain logistics to culled insights on consumer preferences. Arun sees AI and advanced analytics as crucial in handling this data to extract meaningful insights that inform decisions at all levels of the organization, and that this extraction is critical for competitive advantage in today’s market.

From product development to marketing strategies, AI is being used to refine and predict consumer needs, ensuring that Unilever can keep ahead in an ever-changing market.

Arun shared some details about customer behavior modeling taken on by businesses like his. By analyzing consumer behavior data, organizations can personalize marketing and product recommendations, providing a more tailored and relevant experience for customers. This not only improves customer satisfaction, but also increases loyalty and retention, and this is widely deployed today utilizing existing analytics and AI capabilities. With new generative AI models, the ability to forecast consumer behavior will only grow more accurate and impactful.

Sustainability Through Data

Our interview also covered the challenge with efficient IT to fuel AI, and Arun highlighted that this is a core value at Unilever. In fact, AI is playing a pivotal role in helping the company achieve its environmental goals. Arun shared that Unilever is committed to reducing its carbon footprint and creating more eco-friendly products. Through data analytics, the company can identify areas where it can reduce waste, optimize resource use, and create sustainable products without compromising on quality or performance.

For example, AI helps Unilever track the environmental impact of its supply chain, ensuring that the company can source raw materials more responsibly and efficiently. AI also allows the company to forecast demand more accurately, reducing the chances of overproduction and excess waste. These efforts contribute to Unilever’s broader mission of achieving net-zero emissions and promoting a circular economy. And, they are part of a larger trend in corporate use of the powerful technology to enhance energy use and implement sustainable business practices.

The Future of Enterprise Data Architecture 

Arun also provided his vision for the future of enterprise data architecture as one where AI and machine learning will become even more integral where continued investing in AI advancement is required to stay ahead of the curve. This includes building robust data infrastructures that can handle the increasing complexity and volume of data Unilever manages while ensuring privacy and security for consumers.

One of the challenges that Arun touched upon is the need for a cultural shift within organizations to embrace AI and data-driven decision-making fully. He stressed the importance of upskilling employees and creating a data-driven mindset across all departments to ensure the successful integration of AI technologies. This requires new avenues of collaboration, not just across Unilever business groups, but across the industry. 

The solution for this collaboration is new partnerships between businesses, academia, and governments to drive innovation and tackle global challenges such as climate change and resource scarcity. By sharing data and working together, companies can amplify their positive impact on the world and achieve shared sustainability goals.

What’s the TechArena take? Our conversation with Arun was a fantastic reminder that AI has been implemented in IT organizations for years, automating critical functions in relation to advanced analytics. These powerful tools are opening new business opportunity and driving efficiency to corporate processes. While much of the chatter on AI is focused on advancement of large language models, the enterprise is happily deploying core AI-enabled applications across business functions with an eye to improve these functions with new model capability. As we look ahead to 2025 and expected deployments of gen AI in the enterprise, they likely will be built atop what’s already been done in many organizations.

Our second take? We see an important trendline on AI’s positive impact to overall corporate sustainability efforts and mindful use of energy and resources. Expect more stories from corporations in the months ahead on this theme as companies look to counter set the energy consumption utilized for deploying these models from data center to edge.

If you’re at AIHW Summit this week, be sure to check out Emerging Architectures for Applications Using LLMs, the Transition to LLM Agents, featuring Arun alongside experts from Stanford, Union.ai and Nava Ventures as well as his talk, Revolutionizing Language Models: Innovative Designs in Database Layers for Retrieval Augmented Generation, both within Tuesday’s lineup.

One of the most compelling factors driving autonomous vehicles is the reduction of fatalities and accidents. More than 90% of all car accidents are caused by human failures. Self-driving cars play a crucial role in realizing the industry vision of “zero accidents.” Beyond addressing the high power consumption typically associated with the high-performance AI computing required to achieve “just” level 3 ADAS, (the power goes up considerably more for L4 & L5), the fully self-driving car must address some additional really hard challenges: It needs to be able to learn, see, think and then act. And it must do so in a wide range of weather conditions, road conditions, lighting conditions, and also do so flawlessly in a wide range of traffic scenarios.

For the vehicle to “see” (perception) and ultimately understand its surroundings, the vehicle employs a mix of different sensors which includes cameras, radar, and LIDAR (light detection and ranging). While controversy exists in the industry regarding the need for LIDAR, reduced price points and learnings from many millions of miles of trials are leading to broader industry adoption. The reason different types of sensors are employed is to offset the limitations inherent to a given sensor. - i.e. cameras don’t see well in the dark, so by creating a composite image by combining the LIDAR sensor (which works well at night) with the camera sensor, the vehicle can see at night. Unlike cameras and LIDAR, radar is not affected by fog. When radar is combined with the other sensors, the vehicle can now “see” in a wider range of driving conditions. The combining of data from the different sensors is known as sensor fusion.

The AI processing performance required for sensor fusion is relatively modest when compared to the AI performance required for perception. The Convolutional Neural Network (CNN), which is typically employed in sensor fusion, is a form of a deep learning neural network commonly used in computer vision. While the AI processing required to address aspects of sensor fusion may be lightweight, addressing other aspects of sensor fusion are quite complex.

To illustrate that point, today there are vehicles on the road that employ 11 cameras, 1 long-range radar, and 1 LIDAR, in addition to 12 ultrasonic sensors (radar). It’s important to note this collection of sensors and sensor types is employed just to achieve Level 2+ ADAS. Successfully creating a composite image through the fusion of this large number of disparate sensors is a very challenging task. Not only does the aggregated data rate of all these different sensors present significant dataflow and data processing challenges, but the cameras typically have different resolutions, frame rates, and operate asynchronously. These challenges are further compounded when fusing the LIDAR and radar images with the camera images.

4 approaches to sensor fusion commonly employed to yield a composite image include:

• Early fusion
• Mid-level fusion
• Sequential fusion
• Late fusion

Each approach describes the point when the fusion of the different sensors occurs within the fusion signal processing chain. Early fusion typically delivers higher precision at the expense of completeness of scene coverage whereas late fusion offers more coverage of the environment at the expense of precision. Mid-level and sequential fusion offers more balanced trade-offs between coverage and precision. Once fusion has been completed, the composite image is sent to the Perception engine which is responsible for making sense of the fused image.

‍

As shown, the autonomy workload consists of 4 major tasks: perception, prediction, planning, and control. Level 4 autonomy, where a vehicle can operate in self-driving mode with minimal human interaction, requires autonomous driving tasks (e.g. scene understanding, motion detection, image inference tasks, etc.) to be executed continuously with significant consideration for functional safety. While the above signal chain shows a linear signal flow from left to right, functional safety considerations drive the need for redundant signal paths, leading to even further complexity both to the overall signal chain architecture and the underlying computations.

Except for the motion control block, deep-learning neural networks are employed across the entire ADAS signal chain. Each block has different demands regarding the required AI computing performance and the neural nets that are employed. There is an ever-growing population of neural networks that tend to differ in structure, network connectivity, and their target end application while offering different trade-offs in complexity, accuracy, and performance. What they all have in common is that the networks mirror the human brain, (hence the name neural networks), in their approach to solving a very different class of problems than traditional CPUs have for many generations. This field is rapidly evolving and hit its stride when research led to the development of neural networks that could more accurately perform object detection and recognition than a human. Hence why the application of AI in autonomous systems is very logical.

The Convolutional Neural Network CNN, which was widely employed in ADAS perception for object detection, was quickly displaced by the advent of region-based CNNs (RCNNs). RCNNs marked a significant improvement in perception performance, by focusing primarily on just a region of interest within the image vs the entire image.  However, this drove up a corresponding increase in AI computational demands. Since then, newer models including Single Shot Multibox Detector (SSD) and "You Only Look Once" (YOLO), which deliver even better efficiency and improved accuracy for object detection have gained industry momentum.

Most recently, vision transformers, which borrow from neural network concepts used in natural language processing (NLP) – think Siri – are now beginning to emerge as a leading neural network considered for image classification. Vision transformers surpass the performance of Recurrent Neural Networks (RNNs), in accuracy however, this is at the expense of extended training times – which we haven’t discussed yet. But just like learning how to ride a bike, there is a training phase required before you can ride any distance without training wheels. Networks need to be trained to be able to accurately detect and recognize objects in the vehicle’s surroundings.

As another benefit, vision transformers are also less sensitive to visual occlusion. Here again, with this more advanced and accurate neural network, there is an increased cost associated with the AI computing performance required to achieve similar frames-per-second performance when compared to earlier networks.

Suffice it to say, it’s awe-inspiring to witness the almost breakneck pace at which the automotive industry is embracing state-of-the-art technologies, including neural networks, that, in some cases are not quite off the drawing room table yet, but are being embraced ultimately to improve the safety of automotive transportation.

As someone who has spent a quarter of a century immersed in the technology sector, I’ve witnessed seismic shifts in how this disruptive force has changed how we work, communicate, and innovate. But never have I seen a moment quite like this. 

The convergence of AI, cloud and edge computing and advanced network technology is reshaping the world around us at a pace that’s sometimes difficult to comprehend, let alone keep up with. We are at a critical juncture in human history, where technology doesn’t just support daily life — it defines it. IT leaders are being called to quickly harness this technology to fuel massive organization transformation or be left behind. And industry technologists are being challenged to innovate every element of computing to deliver to this north star of opportunity.  

We are all challenged to keep up with the pace of innovation and change. We need connection with those at the center of the storm and guidance about how to harness new opportunities ahead. Innovators have stories to tell about their unique delivery of technology that will advance AI’s influence on our industries and society, will make hyperscalers’ quest for AGI more attainable, and will tip the industry landscape to deliver new market introduction.  

 That’s why I’m thrilled to announce the launch of TechArena 2.0. — an enhanced media platform designed to fill today’s tech news and information void and connect the architects of technology deployment with the engineering gurus driving its creation. This isn’t just another tech blog or news site; it’s an unparalleled hub of innovator connection dedicated to bringing you in-depth coverage and expert perspectives on the topics that matter most: AI, data centers, edge computing, networking, and compute efficiency. Our foundation is based on the premise that direct access to those driving industry innovation is your best bet to charting a successful course in next-generation technology adoption. Our content features direct access to the brightest minds in the tech landscape with an arena filled with tech titans representing >$9T in market capitalization and scrappy startup CEOs and founders sharing their disruptive visions for the future. 

Why Now? Because Timing is Everything. 

 The truth is, tech storytelling has not evolved fast enough to keep pace with the innovations reshaping our world. Traditional media often falls into the trap of either overwhelming audiences with jargon or caring more about winners, losers and earnings, and less about sound guidance on deployment strategies. In this landscape, TechArena is stepping up to do what no other platform is doing: combining tech domain expertise with a clear-eyed, authentic approach that connects innovators directly with the audiences they care about. Our new platform features exclusive interviews, technology deep dives, and expert analysis — delivered in a way that cuts through the noise and speaks to the heart of what IT organizations really need to track. 

Unmatched Expertise, Unfiltered Voices 

We’re bringing together an editorial team that isn’t just reporting on the industry — they are the industry. From seasoned technologists to data center architects, our contributors bring unparalleled expertise and a deep understanding of the solutions that are redefining our world. This isn’t content created from the sidelines pontificating about what may happen; it’s first-person perspectives from those who have lived in the arena itself and know the stakes of tech decision making. 

I started TechArena in 2022 with a vision for a hub for tech innovation in mind. Today, we move closer to that vision’s realization, and I couldn’t be prouder of the entire TechArena team with its delivery. In order to tell this story authentically, I’ve brought in tech content expert Rachel Horton – a longtime tech editor and journalist – to lead our editorial, and Chad Koontz, a digital swami with years of tech sector expertise, to head our digital practice – because I believe our job is not done until content published is delivered to its intended audience every time. These seasoned leaders will help ensure that the TechArena platform hums with valuable stories that educate our audience and influence technology deployments. 

Covering the Critical Issues in Compute Efficiency 

In a time when energy use is as much a technological challenge as it is an ethical imperative, our platform will also focus heavily on how innovations in AI, data center, edge, and networking can drive more efficient solutions. We know that the future of technology cannot exist in isolation from the future of our planet, and we actually believe the tech sector will play a leading role in helping mitigate our climate crisis ahead. TechArena will continue to explore the intersection of tech and energy utilization with the urgency and depth that the topic demands, and we welcome stories from innovators focused on efficient and sustainable technology across the cloud to edge landscape. 

Building a Network for You 

Our goal is not just to inform, but to engage. We’re building a community of like-minded professionals, engineers, and business leaders who are passionate about tech and pushing the boundaries of what’s possible. Through live online discussions, community forums, and face-to-face network engagements at industry events, we’ll offer you more than just articles — we’ll provide engagements for meaningful connection and collaboration that can help reshape your business opportunity.  

Join Us on This Journey 

TechArena’s vision will be fully realized with your collaboration, whether that be engaging in consumption of our content, contributing your voice to our platform, or connecting with us to drive content creation. We want to hear from you. Thank you for being a part of the TechArena community, and I look forward to having you join us on this journey as we continue to shine a light on the voices of innovation.

Allyson Klein
Principal and Founder, TechArena

‍

Guest Gayathri “G” Radhakrishnan, Partner at Hitachi Ventures​, joins host Allyson Klein on the eve of the AIHW and Edge Summit to discuss innovation in the AI space, future adoption of AI, and more.

Join Allyson Klein and Jeniece Wnorowski as they chat with Rita Kozlov from Cloudflare about their innovative cloud solutions, AI integration, and commitment to privacy and sustainability.

To say that the TechArena team is excited for next week's AIHW and Edge Summit is an understatement. With 2024 being completely dedicated to all things AI, this event is a fantastic opportunity to check in on disruptive innovation to fuel the next wave of AI buildouts from hyperscale to the enterprise.

And with industry conversations focused on supply chain constraints of critical technology like GPUs and HBM, it's clear to see that innovation is required to keep up with customer demand and introduce more choice of solutions into the market. We will be coming at you with a deluge of content featuring bright stars in the AI hardware development arena which we recently kicked off with our conversation with Arun Nandi from Unilever. Here's what we will be tracking at the event next week:

Will NVIDIA GPUs Finally Face Real AI Competition in 2025?

The year has been filled with stories of NVIDIA AI Factories packed with Blackwell platforms and broad deployments of GPU clusters scaling to 100,000 nodes. While the extreme shortages of GPUs of the first half of the year have eased, the cost and power requirements to fuel these performance engines are eye-opening. AMD, the event's primary sponsor, has delivered the MI300X accelerator to the market, a viable performance alternative for many AI models that is gaining traction in the market. We'll also be talking to many silicon startups that are delivering unique solutions for AI trying to capture some of the growing logic TAM in this space. The real question is if the large players will move off of NVIDIA for any of these alternatives or simply continue to advance their internal designs and if the real market for new entrants to AI processors are aimed at the next layer of cloud providers and in the enterprise data center and edge.

Will 2025 Usher in AI Fabric Innovation?

While much of the industry attention has been focused on performance drivers of compute acceleration, the AI fabric cannot be ignored. This is a two-part target for us. First, we are keen to check in to viable competition to InfiniBand for offering the latency and scale required for AI training clusters. Today, NVIDIA also has a corner on this market with their tech acquired from Mellanox. While last year saw the announcement of the Ultra Ethernet consortium and a flurry of new specification development to bring Ethernet closer to the capabilities of InfiniBand, we may see other technologies emerge at the conference delivering an alternative for high-end fabric capability. The second target is a look at fabric connectivity and the age-old question of transition to optical. Copper continues to gasp out a life within the data center, but its limitations and power costs are putting increased pressure on providers to migrate more of their connectivity to optical. We'll be checking in with optical providers for the latest innovations in this space to see if 2025 is the year that optical will finally take over.

What About All That Power?

While the allure of AI is unquestioned, we may actually tap the planet's resources at the rate we're moving before we reach the age of AGI. Fundamental change to design principles including more efficient hardware innovation within every element of the data center is required. We will be looking to the vendor community to bring news of advancements on energy efficiency, embedded carbon, and circularity of designs, talk to operators at the show on how their organizations are tackling the challenge of power delivery and management, new cooling technology options, and more, and checking in with venture capital leaders to see how the energy quotient will help shape investments moving forward.

Got more things you'd like covered from next week's show? Please connect with us to share. And if you're going to be at the Summit and would like to connect, please hit me up on LinkedIn.

I’ve been active with hardware infrastructure since the Windows NT era, when Novell with SFT level III and Pentium Pro was all the craze and networks were built of coax cables running token ring. (Yes, I’m that old).

Generations and generations of technology innovation later, the data center industry is bumping up against the amount of energy available worldwide, and cooling and space are challenging to deal with in any data center. We must pivot. It’s essential that we re-think how we design, run, operate and optimize data centers.

Since my days in Intel corporation, I’ve had a front seat to technology transitions and adoption of innovation in the compute, network and storage space. Data centers are at the heart of digital transformation, which is touching almost every industry and person on the planet. All the way from the massive hyperscale data centers to the tier-2 Cloud service providers and co-location partners and enterprises. We’ve built an enormous and diverse ecosystem of industry players around data centers that cover any conceivable topic – from the financial investments and insurance down to the network and power equipment and connector specifications. All these industry players work hand-hand based on strict standards and protocols to ensure smooth data center operations.

The data center space has always been a hotbed of innovation, and the continuous digitization of society will continue this trend for some time to come. The demand for capacity is likely to soar, but it’s challenged by the availability of locations to build data centers, the cost and available supply of energy and skilled staff to build and maintain these critical facilities, to name a few. At the same time, the data centers are under increasing scrutiny and regulations. And the mass adoption of AI is driving an increased set of demands for the data center, at a speed we’ve not seen before.

Because of these competing dynamics, it’s an industry that naturally fosters innovation to overcome these challenges. And over time, the business of building and running a data center has been standardized with modular data center filled with 19-inch racks and x86 infrastructure building blocks complimented with adequate cooling, fiber and power networks. We industrialized it to a level where a have a good understanding of the business model, the available solutions, the leading indicators and how to align supply and demand.

However, the complexities described above are quickly compiling as the demand is outstripping capacity and the cost and availability of energy is no longer endless. The requirements are increasing to a level where it’s unsustainable. In addition, we all want to be sustainable, but the speed of innovation is driving new technology adoption at an increased pace.

When we consider innovating the data center ecosystem, it’s key to get as many creative ideas as possible from a wide range of players so we can filter and fine-tune to find the best solution. The OPEN innovation model is a great way to do that. I believe this mechanism will be pivotal for this industry to leverage to out-innovate ourselves of the data center infrastructure challenges we are facing.

Luckily, the data center space is no stranger to OPEN. A lot of the innovative services delivered out of data center are based (in part) on open-source software. OPEN is an innovation engine, I was once told, and I continue to see this statement being validated in our industry. I’m going to abstract a few things here to a high level to give a few examples.

Let’s take a look at some of the most recent technological adoptions that effected data centers:

• HPC matured because it leveraged standardized X86 compute and ethernet networks with open-source operating system + tools. It was a cheap alternative to the big scale up & proprietary HPC solutions.
• Once we complemented those ingredients with ADSL & Fiber and 3G as a ubiquitous and affordable last-mile connectivity solution for companies and homes and storage cost lowered in part in due to opensource solutions levering storage in a new way, we got cloud computing. We now had a cheap & reliable alternative to local storage.
• BigData was a combination of relatively cheap storage and compute and allowed us to do analysis on fast data lakes of information, and we had a lot of open-source tools such as data sciences benches developed which evolved to assist us in this analysis.
• Now add the open-source frameworks and tools for AI to the mix above and we’ve arrived in the age of Artificial intelligence.

As you can see, Open Source had a hand in all these major tipping points and while software is an innovation engine, it needs hardware to make it work to deliver its promise.

What if we could take the power of OPEN innovation and apply it to the data center IT infrastructure space?

The Open Compute Project does exactly that. It takes this OPEN approach so common in software development and applies this framework to the development of future data center IT infrastructure. This community-led effort spans many different domains, allowing all the different ecosystem players to come together and align on innovation efforts in the data center space. From the concrete used to build data center down to the chiplets on the board, there is a sub-project anyone can join to consume, contribute or learn about the latest developments.

Another exciting thing about OCP is its origin. It was instantiated by META (formerly Facebook) who in 2011 foresaw this need to redesign the IT infrastructure. Based on their scale and complexity, it’s only natural they had this foresight. But it also drove the innovation coming out of OCP.  Because of their scale and size, the hyperscale’s engineering team had the manpower, skills, experience and the resources to come up with a different formfactor and power delivery, which still fits in the so loved 19-inch rack outer dimensions but with a 21-inch inner width in the rack. This way there is more physical real estate to house components and move sufficient air for cooling through the unit. And by centralizing power distribution for the whole rack via a busbar, a saving up to 45% power consumption can be achieved per rack! These are just some examples of the innovation that META brought to OCP, and there are many more.

Many companies soon joined the ranks to consume, contribute or adopt OCP in their offerings. The most recent one was Nvidia, whose DGX infrastructure now leverages OCP for power and liquid cooling purposes. Both Jensen Wang and Mark Zuckerburg even commented about the flexibility and cost savings of OCP in a keynote during SIGGRAPH 2024.

Unknown to many, a number of OCP innovations are most likely already in your existing infrastructure. Per example the OCP3 nic’s are widely adopted by many traditional vendors in their current product designs and have even become the defacto standard. Also the recent modular redesign of motherboards to a more modular design which is known as the Data Center Modular Hardware System (or DC-MHS for short) is also making its way into a number of new products entering the market. Examples are the recently announced Dell PowerEdge R670 CSP Edition and R770 CSP Edition servers or the Supermicro X14 servers.

Open is an innovation engine and the OCP foundation offers a community for all the IT infrastructure industry players to align and build actual solutions to face the challenges this industry is up against. It’s an exciting community that is gaining more momentum and one to watch or even better, participate in!

‍