Welcome to
the Forum

One of the most compelling factors driving autonomous vehicles is the reduction of fatalities and accidents. More than 90% of all car accidents are caused by human failures. Self-driving cars play a crucial role in realizing the industry vision of “zero accidents.” Beyond addressing the high power consumption typically associated with the high-performance AI computing required to achieve “just” level 3 ADAS, (the power goes up considerably more for L4 & L5), the fully self-driving car must address some additional really hard challenges: It needs to be able to learn, see, think and then act. And it must do so in a wide range of weather conditions, road conditions, lighting conditions, and also do so flawlessly in a wide range of traffic scenarios.

For the vehicle to “see” (perception) and ultimately understand its surroundings, the vehicle employs a mix of different sensors which includes cameras, radar, and LIDAR (light detection and ranging). While controversy exists in the industry regarding the need for LIDAR, reduced price points and learnings from many millions of miles of trials are leading to broader industry adoption. The reason different types of sensors are employed is to offset the limitations inherent to a given sensor. - i.e. cameras don’t see well in the dark, so by creating a composite image by combining the LIDAR sensor (which works well at night) with the camera sensor, the vehicle can see at night. Unlike cameras and LIDAR, radar is not affected by fog. When radar is combined with the other sensors, the vehicle can now “see” in a wider range of driving conditions. The combining of data from the different sensors is known as sensor fusion.

The AI processing performance required for sensor fusion is relatively modest when compared to the AI performance required for perception. The Convolutional Neural Network (CNN), which is typically employed in sensor fusion, is a form of a deep learning neural network commonly used in computer vision. While the AI processing required to address aspects of sensor fusion may be lightweight, addressing other aspects of sensor fusion are quite complex.

To illustrate that point, today there are vehicles on the road that employ 11 cameras, 1 long-range radar, and 1 LIDAR, in addition to 12 ultrasonic sensors (radar). It’s important to note this collection of sensors and sensor types is employed just to achieve Level 2+ ADAS. Successfully creating a composite image through the fusion of this large number of disparate sensors is a very challenging task. Not only does the aggregated data rate of all these different sensors present significant dataflow and data processing challenges, but the cameras typically have different resolutions, frame rates, and operate asynchronously. These challenges are further compounded when fusing the LIDAR and radar images with the camera images.

4 approaches to sensor fusion commonly employed to yield a composite image include:

• Early fusion
• Mid-level fusion
• Sequential fusion
• Late fusion

Each approach describes the point when the fusion of the different sensors occurs within the fusion signal processing chain. Early fusion typically delivers higher precision at the expense of completeness of scene coverage whereas late fusion offers more coverage of the environment at the expense of precision. Mid-level and sequential fusion offers more balanced trade-offs between coverage and precision. Once fusion has been completed, the composite image is sent to the Perception engine which is responsible for making sense of the fused image.

‍

As shown, the autonomy workload consists of 4 major tasks: perception, prediction, planning, and control. Level 4 autonomy, where a vehicle can operate in self-driving mode with minimal human interaction, requires autonomous driving tasks (e.g. scene understanding, motion detection, image inference tasks, etc.) to be executed continuously with significant consideration for functional safety. While the above signal chain shows a linear signal flow from left to right, functional safety considerations drive the need for redundant signal paths, leading to even further complexity both to the overall signal chain architecture and the underlying computations.

Except for the motion control block, deep-learning neural networks are employed across the entire ADAS signal chain. Each block has different demands regarding the required AI computing performance and the neural nets that are employed. There is an ever-growing population of neural networks that tend to differ in structure, network connectivity, and their target end application while offering different trade-offs in complexity, accuracy, and performance. What they all have in common is that the networks mirror the human brain, (hence the name neural networks), in their approach to solving a very different class of problems than traditional CPUs have for many generations. This field is rapidly evolving and hit its stride when research led to the development of neural networks that could more accurately perform object detection and recognition than a human. Hence why the application of AI in autonomous systems is very logical.

The Convolutional Neural Network CNN, which was widely employed in ADAS perception for object detection, was quickly displaced by the advent of region-based CNNs (RCNNs). RCNNs marked a significant improvement in perception performance, by focusing primarily on just a region of interest within the image vs the entire image.  However, this drove up a corresponding increase in AI computational demands. Since then, newer models including Single Shot Multibox Detector (SSD) and "You Only Look Once" (YOLO), which deliver even better efficiency and improved accuracy for object detection have gained industry momentum.

Most recently, vision transformers, which borrow from neural network concepts used in natural language processing (NLP) – think Siri – are now beginning to emerge as a leading neural network considered for image classification. Vision transformers surpass the performance of Recurrent Neural Networks (RNNs), in accuracy however, this is at the expense of extended training times – which we haven’t discussed yet. But just like learning how to ride a bike, there is a training phase required before you can ride any distance without training wheels. Networks need to be trained to be able to accurately detect and recognize objects in the vehicle’s surroundings.

As another benefit, vision transformers are also less sensitive to visual occlusion. Here again, with this more advanced and accurate neural network, there is an increased cost associated with the AI computing performance required to achieve similar frames-per-second performance when compared to earlier networks.

Suffice it to say, it’s awe-inspiring to witness the almost breakneck pace at which the automotive industry is embracing state-of-the-art technologies, including neural networks, that, in some cases are not quite off the drawing room table yet, but are being embraced ultimately to improve the safety of automotive transportation.

As someone who has spent a quarter of a century immersed in the technology sector, I’ve witnessed seismic shifts in how this disruptive force has changed how we work, communicate, and innovate. But never have I seen a moment quite like this. 

The convergence of AI, cloud and edge computing and advanced network technology is reshaping the world around us at a pace that’s sometimes difficult to comprehend, let alone keep up with. We are at a critical juncture in human history, where technology doesn’t just support daily life — it defines it. IT leaders are being called to quickly harness this technology to fuel massive organization transformation or be left behind. And industry technologists are being challenged to innovate every element of computing to deliver to this north star of opportunity.  

We are all challenged to keep up with the pace of innovation and change. We need connection with those at the center of the storm and guidance about how to harness new opportunities ahead. Innovators have stories to tell about their unique delivery of technology that will advance AI’s influence on our industries and society, will make hyperscalers’ quest for AGI more attainable, and will tip the industry landscape to deliver new market introduction.  

 That’s why I’m thrilled to announce the launch of TechArena 2.0. — an enhanced media platform designed to fill today’s tech news and information void and connect the architects of technology deployment with the engineering gurus driving its creation. This isn’t just another tech blog or news site; it’s an unparalleled hub of innovator connection dedicated to bringing you in-depth coverage and expert perspectives on the topics that matter most: AI, data centers, edge computing, networking, and compute efficiency. Our foundation is based on the premise that direct access to those driving industry innovation is your best bet to charting a successful course in next-generation technology adoption. Our content features direct access to the brightest minds in the tech landscape with an arena filled with tech titans representing >$9T in market capitalization and scrappy startup CEOs and founders sharing their disruptive visions for the future. 

Why Now? Because Timing is Everything. 

 The truth is, tech storytelling has not evolved fast enough to keep pace with the innovations reshaping our world. Traditional media often falls into the trap of either overwhelming audiences with jargon or caring more about winners, losers and earnings, and less about sound guidance on deployment strategies. In this landscape, TechArena is stepping up to do what no other platform is doing: combining tech domain expertise with a clear-eyed, authentic approach that connects innovators directly with the audiences they care about. Our new platform features exclusive interviews, technology deep dives, and expert analysis — delivered in a way that cuts through the noise and speaks to the heart of what IT organizations really need to track. 

Unmatched Expertise, Unfiltered Voices 

We’re bringing together an editorial team that isn’t just reporting on the industry — they are the industry. From seasoned technologists to data center architects, our contributors bring unparalleled expertise and a deep understanding of the solutions that are redefining our world. This isn’t content created from the sidelines pontificating about what may happen; it’s first-person perspectives from those who have lived in the arena itself and know the stakes of tech decision making. 

I started TechArena in 2022 with a vision for a hub for tech innovation in mind. Today, we move closer to that vision’s realization, and I couldn’t be prouder of the entire TechArena team with its delivery. In order to tell this story authentically, I’ve brought in tech content expert Rachel Horton – a longtime tech editor and journalist – to lead our editorial, and Chad Koontz, a digital swami with years of tech sector expertise, to head our digital practice – because I believe our job is not done until content published is delivered to its intended audience every time. These seasoned leaders will help ensure that the TechArena platform hums with valuable stories that educate our audience and influence technology deployments. 

Covering the Critical Issues in Compute Efficiency 

In a time when energy use is as much a technological challenge as it is an ethical imperative, our platform will also focus heavily on how innovations in AI, data center, edge, and networking can drive more efficient solutions. We know that the future of technology cannot exist in isolation from the future of our planet, and we actually believe the tech sector will play a leading role in helping mitigate our climate crisis ahead. TechArena will continue to explore the intersection of tech and energy utilization with the urgency and depth that the topic demands, and we welcome stories from innovators focused on efficient and sustainable technology across the cloud to edge landscape. 

Building a Network for You 

Our goal is not just to inform, but to engage. We’re building a community of like-minded professionals, engineers, and business leaders who are passionate about tech and pushing the boundaries of what’s possible. Through live online discussions, community forums, and face-to-face network engagements at industry events, we’ll offer you more than just articles — we’ll provide engagements for meaningful connection and collaboration that can help reshape your business opportunity.  

Join Us on This Journey 

TechArena’s vision will be fully realized with your collaboration, whether that be engaging in consumption of our content, contributing your voice to our platform, or connecting with us to drive content creation. We want to hear from you. Thank you for being a part of the TechArena community, and I look forward to having you join us on this journey as we continue to shine a light on the voices of innovation.

Allyson Klein
Principal and Founder, TechArena

‍

Guest Gayathri “G” Radhakrishnan, Partner at Hitachi Ventures​, joins host Allyson Klein on the eve of the AIHW and Edge Summit to discuss innovation in the AI space, future adoption of AI, and more.

Join Allyson Klein and Jeniece Wnorowski as they chat with Rita Kozlov from Cloudflare about their innovative cloud solutions, AI integration, and commitment to privacy and sustainability.

To say that the TechArena team is excited for next week's AIHW and Edge Summit is an understatement. With 2024 being completely dedicated to all things AI, this event is a fantastic opportunity to check in on disruptive innovation to fuel the next wave of AI buildouts from hyperscale to the enterprise.

And with industry conversations focused on supply chain constraints of critical technology like GPUs and HBM, it's clear to see that innovation is required to keep up with customer demand and introduce more choice of solutions into the market. We will be coming at you with a deluge of content featuring bright stars in the AI hardware development arena which we recently kicked off with our conversation with Arun Nandi from Unilever. Here's what we will be tracking at the event next week:

Will NVIDIA GPUs Finally Face Real AI Competition in 2025?

The year has been filled with stories of NVIDIA AI Factories packed with Blackwell platforms and broad deployments of GPU clusters scaling to 100,000 nodes. While the extreme shortages of GPUs of the first half of the year have eased, the cost and power requirements to fuel these performance engines are eye-opening. AMD, the event's primary sponsor, has delivered the MI300X accelerator to the market, a viable performance alternative for many AI models that is gaining traction in the market. We'll also be talking to many silicon startups that are delivering unique solutions for AI trying to capture some of the growing logic TAM in this space. The real question is if the large players will move off of NVIDIA for any of these alternatives or simply continue to advance their internal designs and if the real market for new entrants to AI processors are aimed at the next layer of cloud providers and in the enterprise data center and edge.

Will 2025 Usher in AI Fabric Innovation?

While much of the industry attention has been focused on performance drivers of compute acceleration, the AI fabric cannot be ignored. This is a two-part target for us. First, we are keen to check in to viable competition to InfiniBand for offering the latency and scale required for AI training clusters. Today, NVIDIA also has a corner on this market with their tech acquired from Mellanox. While last year saw the announcement of the Ultra Ethernet consortium and a flurry of new specification development to bring Ethernet closer to the capabilities of InfiniBand, we may see other technologies emerge at the conference delivering an alternative for high-end fabric capability. The second target is a look at fabric connectivity and the age-old question of transition to optical. Copper continues to gasp out a life within the data center, but its limitations and power costs are putting increased pressure on providers to migrate more of their connectivity to optical. We'll be checking in with optical providers for the latest innovations in this space to see if 2025 is the year that optical will finally take over.

What About All That Power?

While the allure of AI is unquestioned, we may actually tap the planet's resources at the rate we're moving before we reach the age of AGI. Fundamental change to design principles including more efficient hardware innovation within every element of the data center is required. We will be looking to the vendor community to bring news of advancements on energy efficiency, embedded carbon, and circularity of designs, talk to operators at the show on how their organizations are tackling the challenge of power delivery and management, new cooling technology options, and more, and checking in with venture capital leaders to see how the energy quotient will help shape investments moving forward.

Got more things you'd like covered from next week's show? Please connect with us to share. And if you're going to be at the Summit and would like to connect, please hit me up on LinkedIn.

I’ve been active with hardware infrastructure since the Windows NT era, when Novell with SFT level III and Pentium Pro was all the craze and networks were built of coax cables running token ring. (Yes, I’m that old).

Generations and generations of technology innovation later, the data center industry is bumping up against the amount of energy available worldwide, and cooling and space are challenging to deal with in any data center. We must pivot. It’s essential that we re-think how we design, run, operate and optimize data centers.

Since my days in Intel corporation, I’ve had a front seat to technology transitions and adoption of innovation in the compute, network and storage space. Data centers are at the heart of digital transformation, which is touching almost every industry and person on the planet. All the way from the massive hyperscale data centers to the tier-2 Cloud service providers and co-location partners and enterprises. We’ve built an enormous and diverse ecosystem of industry players around data centers that cover any conceivable topic – from the financial investments and insurance down to the network and power equipment and connector specifications. All these industry players work hand-hand based on strict standards and protocols to ensure smooth data center operations.

The data center space has always been a hotbed of innovation, and the continuous digitization of society will continue this trend for some time to come. The demand for capacity is likely to soar, but it’s challenged by the availability of locations to build data centers, the cost and available supply of energy and skilled staff to build and maintain these critical facilities, to name a few. At the same time, the data centers are under increasing scrutiny and regulations. And the mass adoption of AI is driving an increased set of demands for the data center, at a speed we’ve not seen before.

Because of these competing dynamics, it’s an industry that naturally fosters innovation to overcome these challenges. And over time, the business of building and running a data center has been standardized with modular data center filled with 19-inch racks and x86 infrastructure building blocks complimented with adequate cooling, fiber and power networks. We industrialized it to a level where a have a good understanding of the business model, the available solutions, the leading indicators and how to align supply and demand.

However, the complexities described above are quickly compiling as the demand is outstripping capacity and the cost and availability of energy is no longer endless. The requirements are increasing to a level where it’s unsustainable. In addition, we all want to be sustainable, but the speed of innovation is driving new technology adoption at an increased pace.

When we consider innovating the data center ecosystem, it’s key to get as many creative ideas as possible from a wide range of players so we can filter and fine-tune to find the best solution. The OPEN innovation model is a great way to do that. I believe this mechanism will be pivotal for this industry to leverage to out-innovate ourselves of the data center infrastructure challenges we are facing.

Luckily, the data center space is no stranger to OPEN. A lot of the innovative services delivered out of data center are based (in part) on open-source software. OPEN is an innovation engine, I was once told, and I continue to see this statement being validated in our industry. I’m going to abstract a few things here to a high level to give a few examples.

Let’s take a look at some of the most recent technological adoptions that effected data centers:

• HPC matured because it leveraged standardized X86 compute and ethernet networks with open-source operating system + tools. It was a cheap alternative to the big scale up & proprietary HPC solutions.
• Once we complemented those ingredients with ADSL & Fiber and 3G as a ubiquitous and affordable last-mile connectivity solution for companies and homes and storage cost lowered in part in due to opensource solutions levering storage in a new way, we got cloud computing. We now had a cheap & reliable alternative to local storage.
• BigData was a combination of relatively cheap storage and compute and allowed us to do analysis on fast data lakes of information, and we had a lot of open-source tools such as data sciences benches developed which evolved to assist us in this analysis.
• Now add the open-source frameworks and tools for AI to the mix above and we’ve arrived in the age of Artificial intelligence.

As you can see, Open Source had a hand in all these major tipping points and while software is an innovation engine, it needs hardware to make it work to deliver its promise.

What if we could take the power of OPEN innovation and apply it to the data center IT infrastructure space?

The Open Compute Project does exactly that. It takes this OPEN approach so common in software development and applies this framework to the development of future data center IT infrastructure. This community-led effort spans many different domains, allowing all the different ecosystem players to come together and align on innovation efforts in the data center space. From the concrete used to build data center down to the chiplets on the board, there is a sub-project anyone can join to consume, contribute or learn about the latest developments.

Another exciting thing about OCP is its origin. It was instantiated by META (formerly Facebook) who in 2011 foresaw this need to redesign the IT infrastructure. Based on their scale and complexity, it’s only natural they had this foresight. But it also drove the innovation coming out of OCP.  Because of their scale and size, the hyperscale’s engineering team had the manpower, skills, experience and the resources to come up with a different formfactor and power delivery, which still fits in the so loved 19-inch rack outer dimensions but with a 21-inch inner width in the rack. This way there is more physical real estate to house components and move sufficient air for cooling through the unit. And by centralizing power distribution for the whole rack via a busbar, a saving up to 45% power consumption can be achieved per rack! These are just some examples of the innovation that META brought to OCP, and there are many more.

Many companies soon joined the ranks to consume, contribute or adopt OCP in their offerings. The most recent one was Nvidia, whose DGX infrastructure now leverages OCP for power and liquid cooling purposes. Both Jensen Wang and Mark Zuckerburg even commented about the flexibility and cost savings of OCP in a keynote during SIGGRAPH 2024.

Unknown to many, a number of OCP innovations are most likely already in your existing infrastructure. Per example the OCP3 nic’s are widely adopted by many traditional vendors in their current product designs and have even become the defacto standard. Also the recent modular redesign of motherboards to a more modular design which is known as the Data Center Modular Hardware System (or DC-MHS for short) is also making its way into a number of new products entering the market. Examples are the recently announced Dell PowerEdge R670 CSP Edition and R770 CSP Edition servers or the Supermicro X14 servers.

Open is an innovation engine and the OCP foundation offers a community for all the IT infrastructure industry players to align and build actual solutions to face the challenges this industry is up against. It’s an exciting community that is gaining more momentum and one to watch or even better, participate in!

‍

Allyson Klein and Jeniece Wnorowski chat with Kelley Osburn of Graid about SupremeRAID™ and its role in tackling high-performance storage challenges in data-driven environments.

There can be an attitude that people who are partner managers are not as sensitized to achieving revenue, developing products and services or the challenges of taking them to market. Don’t ever believe that industry partnerships have little real effect on company revenues or market successes. The people in those roles must know their own corporate business and their partners' business drivers for anything of consequence to be accomplished. Let me walk you through the “why” behind that claim.

Everyone has a boss with authority to “fire” them.

I will start with the argument I’ve heard many times: “I can’t wait until I save up enough money so I can run my own company, no one can fire me!” Unless you are wealthy enough to pour money into a company that can’t cover its operating costs, your customers are your boss. They get to vote with their wallets and “fire” you by not buying from you at all.

A Board of Directors has “bosses” – shareholders, stakeholders and regulators. Shareholders can “fire” directors in a few ways – hostile takeovers, purchasing enough stock to have a voting seat on the board then voting everyone else off the board, or dramatically selling off the company’s stock from their investment portfolios. There are three primary duties for BOD members: Duty of care (diligence in being informed and involved), Duty of Loyalty (prioritize the corporations’ best interests along with its shareholders) and Duty of Obedience (personal and corporate actions comply with local, state and federal regulations).

CEOs have a boss – The Board of Directors. Two top failures by a director in exercising their fiduciary responsibilities are failing to oversee management and making decisions without adequate information. The implication is that a good director prioritizes preventative involvement with a CEO.

It is not “just business.”

Humans are social/relational – even engineers! Regardless of the title or level, humans all have similar fears, hopes and ambitions. In many cases, the fear of failure or being exposed as an imposter is highest in the C-suite compared to the working level.

Everyone has goals set by their “bosses” that advance personal success within the company or cause personal loss of capital with their management chain. Are you approaching a given partnership as if more than your goals matter?

Ask yourself if there is a way to help increase your counterparts’ chances of success while executing the work you want to accomplish with them. Understand the business they are in, how they are working to improve their profitability, and their brand value to their customers. Know whether your success results in their success – at both an individual level and a corporate level.

Do the math. Literally.

Over a decade ago I was responsible for executing a previously negotiated agreement between my company execs and a critical partner’s executives. My counterpart and I started hashing out how to adhere to the agreement while advancing the goals each company had for the partnership.

Just one tiny innocuous comment from my GM caused me to dig a little deeper after some puzzling behavior from my counterpart at our partner: “Don’t worry – they make so much more money with us; they have no interest in stalling our partnership in favor of other actions.”

I did the math. I truthfully hoped to calculate how much more money they would make from our partnership to present that in ongoing negotiations when I instead confirmed they did NOT make more money in our partnership as written. This explained why my counterpart, who like me was part of the pre-close negotiations, was so uncomfortable with the suggestions I made. Inadvertently, I was asking that individual to expend significant capital with their management chain, who were accountable to the CEO, who was accountable to their BOD, to shareholders, on down the line.

As much as my counterpart appreciated me as a person and valued the way I collaborated, they were unable to execute against many of the items my company hoped for. An action from my counterpart that either hurt or simply failed to advance their company’s overall business interests would ultimately be unwound.

Concluding thoughts

Industry partnerships are some of the most rewarding experiences I have had, because the sense of joint accomplishment in executing something that moved everyone’s business forward has very few equals. To take it to the next level, be a student of your own company’s business as well as that of your partner’s. You would be truly astounded at how often these simple concepts are overlooked, and surprised by what you discover when applying them to your own situation.

‍

Arun Nandi of Unilever joins host Allyson Klein to discuss AI's role in modern data analytics, the importance of sustainable innovation, and the future of enterprise data architecture.

Join Allyson Klein as she welcomes former colleague/ industry innovator Jen Huffstetler. Jen shares her extensive experience driving advancements from client devices to the data center, including groundbreaking technologies like Centrino and 3D packaging.

With GPU-driven AI training ruling the moment, we have finally come to the asymptotic moment for liquid cooling to overtake air cooled data center infrastructure for many environments. Consider, for a moment, that NVIDIA Blackwell-based racks are drawing from 60kW to 120kW per rack, a dramatic shift from the historic 5-10kW per rack delivered to fuel general purpose applications. When you extrapolate that power across football fields of racks for a hyperscale training cluster, you realize that there’s a LOT of heat to extract. The debate has quickly shifted from air vs liquid to what type of liquid to utilize, opening the door for market disruption and new player entry.

This is why I was so excited to talk to Dr. Kelley Mullick, vice president of technology advancement at Iceotope. Kelley joined Iceotope, a Sheffield, England-based immersion cooling startup, last year, bringing with her a technology leadership pedigree and the notable achievement of having delivered the first industry liquid cooling warranty while at Intel in 2022. Her PhD in chemical engineering and lengthy engagement in industry standards work places her squarely in the middle of liquid cooling advancement.

So why liquid cooling? Kelley confirmed that AI is the primary driver for urgency in transition to liquid cooling due to its serial computing nature, but also stated that broader commitments to sustainability have driven hyperscalers to consider liquid alternatives. She outlined the three alternatives in play in the liquid market: cold plate, tank immersion and precision liquid cooling. While all are more effective and efficient than air, each of the alternatives offer different advantages for consideration. Cold plate has the advantage that it has been widely deployed in HPC environments and utilizes air to cool parts of the chassis where liquid plates are not uniquely targeted, supporting retrofit opportunities for existing infrastructure. Tank immersion delivers a solution where heat can be captured for secondary usage but is also delivered at a weight that requires reinforcement of flooring in existing data center tile flooring, likely limiting to greenfield buildouts. Finally, precision liquid is somewhat of a hybrid, offering advantages of immersion cooling with alternative chemistries to water and similarities to cold plate, offering deployment in existing vertical racks.

If this complexity wasn’t enough, there’s also the topic of chemistry, and it’s here that Kelley really lit up. To start, the options for liquid cooling are water (used in cold plate designs) and dielectric fluid (used in cold plate, immersion, and precision designs). Dielectric fluid is composed of hydrocarbon or fluoridated hydrocarbon fluid with most vendors targeting hydrocarbon options because of its non-toxic composition and ability to be recycled. For two phase cooling solutions, however, only fluoridated hydrocarbon solutions can be used, introducing toxic chemicals into the data center and representing increased challenges from a circularity perspective.

Iceotope is delivering a pretty special chemistry within this landscape. Kelley explained that solutions are delivering precision cooling at up to 1500 watts with thermal resistance 0.037 Kelvin/watt, at par with fluoridated solutions with a sustainable and environmentally friendly chemistry. This technology is delivered in adaptable form factors including racks, power shelves and more, enabling customers to deploy across data center and edge environments. Kelley also noted that different types of infrastructure from GPUs and CPUs to storage JBODs can be submerged in dielectric fluid. Iceotope has done extensive testing of material compatibility to ensure customer deployments will keep cool without reliability erosion.

What’s the TechArena take? We were delighted that we were able to feature this story on our Data Insights series sponsored by Solidigm as cooling is critical to delivery of the data pipeline. Iceotope is delivering disruptive technology in this space, and I expect to hear much more about their solutions as we head into the OCP Summit this fall. If liquid cooling is not on your radar today…put it on your radar. With hyperscalers moving rapidly to liquid alternatives, we expect solutions to scale to meet edge requirements and broader scale AI configurations in data centers. To learn more, check out the interview and visit Iceotope’s site.

‍

Why Wait? Sometimes That’s the Only Choice

“All good things arrive unto them that wait - and don't die in the meantime.”

-   Mark Twain

Memory is always a tricky thing. And we’re not just talking about trying to find the TV remote for five minutes only to finally discover that it’s in the fridge next to the five-pack. Don’t judge until you’ve done your 1.6 km in a certain pair of size 14’s, thank you. Anyway, Your Humble Author (YHA) remembers many a discussion on system memory, cache topologies, drive characteristics, and all those other fun things that are associated with an engineer’s most-dreaded, four-letter “L” word: latency. During one particular discussion with a senior software VP at a large OS company, he opined, “All processor architectures wait equally fast.” So, so true.

Pretty much every one of those computer architectures arranges the logic units to operate on a series of data registers. We promise this is going somewhere… and we’re also simplifying a lot so everyone can grab a bone and pick. Data registers load and store from memory. Some architectures will also allow for direct memory operations. But which memory? This is where waiting comes to play.

The following analogy has been changed to protect the innocent, but YHA thanks the smart people for the concept. Let’s say the desired memory is stored in the nearest cache (with typical latencies in the one nanosecond range these days). Let’s also say that acquisition is the equivalent of finding the TV remote again next to the three-pack. Easy. Processor architectures typically have a series of larger caches or on-chip memories these days, followed by DRAM on the motherboard. It could take a walk to the neighbor’s house for a quick wave while opening the fridge in the garage to accomplish that particular DRAM “fetch.”

If that particular data happens to be stored on some of the latest and best solid-state memory on a coherent bus, the equivalent would be a short drive to the local convenience store. Leave the remote at home, please. A slightly older SSD configuration – or a poor driver routine (processor people always blame the software) – could mean a slightly longer trip to the grocery store in town. The average AI cluster data transfer across the mesh would involve stopping for a sit-down dinner before heading home. That’s a lot of work for one individual piece of data, all while the particular system thread is waiting.

One more. The best network topologies in the datacenter these days usually guarantee a maximum of 5 milliseconds latency from any server to your register. THE COMPANY where YHA used to work offered the promise of an 8-week sabbatical. Were that entire sabbatical devoted to the acquisition of one beverage unit, we’d be in the ballpark of the latency we’re discussing. That sucker better not be an over-hopped IPA. And what’s the system thread doing that whole time? To use the Yiddish: bupkis.

Clearly, those decades of work between hardware and software geeks on multi-threading, data-ordering, data promotion, appropriate memory and SSD sizing, (keep going, it was decades) were all pretty vital. The modern datacenter rack topology is now largely being driven by collaborative, hive-mind organizations like the Open Compute Project (OCP). And even with all that effort, many of those cores are still essentially thirsting for that next drink of data. There’s a very cogent person a few TechArena articles away from this one that noted it can be very disappointing to pay $1B for some AI hardware only to see $500M of work happen. These are not inconsequential decisions.

So what to do? First, don’t panic (towel and exploding planet optional). There are plenty of resources out there that can help optimize whatever you’re building. Look for similar organizations in your region or technology circle and ask questions, many will be very open if they’re not competing with you, some even if they are. Look to organizations like OCP as a guide for setting a proper configuration. Also, your software providers likely have a much better view of configurations that best run their work, mostly because of those decades of work. Finally, look to cloud options, since your provider will then be taking the risks.

And keep the analogy going. Maybe in the flat world of data analytics we’ll collectively find out what data fetch has us all going to the moon and back. By the way, is anyone else thirsty? And where the heck is the remote?

‍

“If a machine is expected to be infallible, it cannot also be intelligent.”

-   Alan Turing

“Shall we play a game?”

-   Joshua, War Games

Total aside that will become common in this space. We’ve been out camping to celebrate our 28th anniversary. How the heck she’s tolerated me that long is a story in itself, THE COMPANY only stood me for 26.5!

We were short on a bottle for the pre-dinner drink. Thirty minutes to the closest liquor store (relax, we needed groceries, too), and we found a Rebel 100 (Lux Row Distillers, Bardstown, KY). It’s made using a traditional wheated recipe. The correspondent and the lovely bride both have a fondness for the grassiness wheat adds, and there’s a little beyond-bourbon sweetness that you normally don’t get in a 100 proof taste. The smokiness might come from the barrel, or it might be that the AQI is well into the 200 range because half of Oregon is on fire. Anyway, cheers.

Your Humble Author (YHA) has a fascination with the stock market, mostly because compound interest provides to us, the willfully underemployed. One particular amusement these days is watching all the little Whos in Whoville screaming, “But we have AI!” when being dragged to the dust speck boiler of earnings results. First off, a quarter of the time it’s not AI, stop lying to your marketing department. We get that Machine Learning is esoteric and hard to explain, but it also generates results you can audit. Another quarter involves using someone else’s chatbot to produce not very meaningful results. That remaining half? The development department was looking for some “me time” and figured that a 60% accuracy rate was better than the dartboard they usually get from upper management, so they did an implementation hoping to get Friday afternoons off. There’s some good, but there’s a lot of not good.

AI has been around for a long time, the Turing Test is named after a really smart guy who passed in the 1950’s, after all. Some of us would probably say that the current rush on AI is due to the accumulation of data, but that’s a little misleading. There’s always been data sloshing around the world. It’s just that very little of it was in the right place, and it certainly was not in the right structure to be scrutinized. If some government-funded project wanted to study global temperatures, it had to do the work of getting the data from every ledger to one computer storage spot, in one format, with some serious programmers to write serious code to produce meaningful results.

The PC/Internet revolution of the 90’s started the ball rolling on creating a “place” for everything. After a couple decades of upper management demanding that everything go up in the cloud, we finally solved the access problem, but not the structure one. If that now bigger government-funded project wanted to take a look now, it’s not only got the old temperature data, but access to a billion backyard weather stations, all the articles about the prior art, and a serious amount of social media memes vamping on a repetitive Nelly tune. So now it’s down to figuring out how to access all the slosh in that giant container.

A more relevant aside than usual: YHA remembers being at an IBM Conference a few years back where a Watson executive told the audience that they were tracking potential flu epidemic outbreaks in part by monitoring Twitter for people posting the equivalent of, “I feel like crud today.” They were finding outbreaks about ten days faster than the CDC who was tracking their standard measures like hospital data and pharmacy orders. So this AI stuff CAN work if you know what you’re doing.

Anyway, back when we were all calling it Big Data ten years ago, the structure problem was still a limiting factor in the analytics discussion. We were all about the V’s: Volume, Variety, Velocity. The real winning formula in analytics at that point was addressing the Variety by referencing more and more data types in their natural form, which eliminated the need for costly (money and time) Extract/Transform/Load (ETL) routines. We were right back to those serious programmers, serious code, meaningful if directional results.

Artificial Intelligence is a fundamentally lazy approach to the data problem. Lest you think that’s a condemnation, any good strategist will tell you that the best strategies are ones that are lazy, the hard work is in the tactics. The basic approach of AI is to throw it at all the data and see what patterns it finds. Structure is less of an issue when your algos are seeking repetition independent of prior relevance. So all those companies claiming that AI will solve your problems are likely correct in the long term. Except…

All you data nerds have been thinking: “Dude, you forgot a V a couple paragraphs ago.” You’d be correct. As the data in the cloud continued to grow, we added Veracity to the equation. The sober geeks out there are still sifting the data before they attack it with AI, knowing that garbage in still produces garbage out. Those less experienced, or less aware that tactics are where the hard work hits, are likely to slow their actual decision processes by getting inconsistent results. If you’d like a practical example, hit up any chatbot or AI search engine with a conspiracy theory that goes against your grain, and discover the chaos it creates.

So financial results won’t actually just come at the behest of, “We have AI!” They’ll come when companies build in the tools and training to sort the data and make the sober decisions. We’ll all be able to see this shake out in front of us, even as the new types of data hitting the cloud create a need to do it all over again in a more complex way. As always when dealing with your own personal compounding, caveat emptor, and happy hunting.

‍

In a recent episode of the TechArena Data Insights series, my co-host Jeniece Wnorowski and I had an insightful conversation with Ariel Pisetzky, the Vice President of Information Technology and Cyber at Taboola, about the transformative impact of data and AI on ad placement. Our discussion revealed how these advanced technologies are redefining the advertising landscape, making ad placements more efficient and targeted to business objectives.

The Shift to AI-Driven Advertising

Ariel emphasized that at Taboola, the mission is to "connect people with content they may like but never knew existed." This mission is powered by sophisticated algorithms that analyze user behavior, preferences, and context to deliver highly personalized content. Ariel noted, "Our systems are designed to process enormous amounts of data in real-time to understand user intent and deliver the most relevant ads." As someone whose business in part is driving micro-targeted paid media, I was delighted to learn from Ariel about what he and the team at Taboola are delivering.

I was also delighted to hear that AI is at the core of Taboola's strategy for ad placement. By utilizing machine learning models, Taboola can predict which ads are most likely to engage individual users. Ariel explained, "Our AI systems analyze vast amounts of data in real-time to understand user intent and preferences. This allows us to serve ads that are not only relevant but also engaging."

These AI algorithms take into account various factors, including browsing history, time of day, and even the type of device being used, ensuring that ads are placed in the optimal context, thus maximizing user interaction.

Dynamic and Contextual Ad Placements

One of the key innovations we discussed with Ariel is Taboola's approach to dynamic and contextual ad placements. Traditional ad placement strategies often rely on static parameters, but Taboola's AI-driven platform can adapt in real-time. For instance, if a user frequently reads tech blogs in the evening, Taboola's system might prioritize tech-related ads during that time frame.

Ariel highlighted this capability, stating, "Dynamic ad placements allow us to adjust the content based on immediate user context. This not only improves the user experience but also enhances ad performance for our clients."

Predictive analytics is another area where Taboola excels. By analyzing historical data and user behavior patterns, the platform can forecast future actions and preferences. This predictive power enables advertisers to stay ahead of trends and tailor their campaigns accordingly.

As Ariel mentioned, "Predictive analytics gives us a glimpse into what users might be interested in next. This foresight is invaluable for creating timely and relevant ad campaigns that capture user interest before it peaks."

Optimized Platforms Deliver to Taboola’s Vision

In our discussion. Ariel highlighted the collaboration between Taboola and Solidigm, focusing on how their combined efforts are enhancing data management and AI capabilities. Ariel mentioned that Solidigm’s advanced data storage solutions play a crucial role in supporting Taboola's AI infrastructure. He noted, "Solidigm's innovations in data storage technology have allowed us to manage and process vast amounts of data more efficiently, which is essential for our AI-driven ad placement systems."

Ariel further explained that the high-performance and reliability of Solidigm's storage solutions ensure that Taboola's AI models can access and analyze data in real-time, leading to more accurate and timely ad placements. "With Solidigm, we're able to scale our operations and maintain high performance even as our data needs grow," he added. This partnership exemplifies how cutting-edge storage technology can support the demanding requirements of modern AI applications, enabling more effective and personalized advertising strategies.

Challenges and Ethical Considerations

Despite the collective advancements Taboola has made to its platform, Ariel acknowledged the challenges and ethical considerations involved in using AI and data for ad placement. Privacy concerns and data security are paramount, and Taboola is committed to maintaining high standards in these areas. "We are constantly evolving our practices to ensure user data is handled with the utmost care and transparency," he emphasized.

Looking ahead, Taboola plans to further integrate AI capabilities to refine ad placements. The company is exploring the use of deep learning models to enhance content recommendations and improve ad targeting accuracy. Ariel shared, "Our goal is to push the boundaries of what's possible with AI, making our ad placements smarter and more intuitive." I, for one, cannot wait to see what is on the horizon as Taboola continues to spearhead innovation in this important arena for marketers.

For more insights from this episode, you can listen to the full conversation on TechArena's podcast.

About Solidigm:

Data storage requirements are evolving rapidly with the explosion of the AI era, and it's important to find the right partner that can provide the flexibility and breadth for each specific AI application. Solidigm is a global leader in innovative NAND flash storage solutions with a comprehensive portfolio of SSD products based on SLC, TLC, and QLC technologies. Headquartered in Rancho Cordova, California, Solidigm operates as a standalone U.S. subsidiary of SK hynix with offices across 13 locations worldwide.

In this Tech Arena episode, Allyson Klein interviews Mehdi Daoudi, CEO of Catchpoint, on internet monitoring, observability innovations, and AI's impact on automation. Discover their latest tools enhancing performance.

For those who listen to the TechArena podcast, you may recall an episode featuring Urban Machine, a leading startup in the construction reclamation space whose solution creatively combines advanced robotics and AI to “clean” used lumber salvage and ready it for a second use in new building construction. When you consider that up to 25% of landfill is derived from construction materials, it is apparent that solutions that embrace circularity are rapidly adopted. Urban Machine has garnered a lion’s share of attention in this arena given accolades at SXSW and more recently named as a top 10 startup to know by Climate Insider. Urban Machine also collected recognition from the US Green Building Council California with a 2024 Mighty Materials Award underscoring the power of reclaimed materials to green construction practices.

We were curious to check in with Urban Machine on the progress it has made in delivering this breakthrough tech to construction sites. The eager engineers at the company report that they are hard at work readying to deliver their first production machine onsite to customers. Check out this video to check out the latest in deployment advancement. I wouldn’t be surprised at all to see this solution garner more than its unfair share of the >$20 million in US EPA funding for clean manufacturing materials.

Want to learn more about Urban Machine? Check out my podcast with Co-Founder and CTO Andrew Gillies as he describes how the machine has come into being, the technologies behind being able to pinpoint fastener locations in wood, pry fasteners out with minimal wood damage, and deliver a safe product back into construction sites, and how developers can engage with the team to get multiple generations out of lumber.

In this podcast, learn about the challenges of silicon advancement, the importance of advanced packaging, and how Lam is driving breakthroughs to support the future of AI and chiplet ecosystems.

The TechArena are fans of what WEKA is delivering in the market, and we’ve covered their data platform since last year as the company has unveiled innovations to help speed enterprise adoption of AI in market. It was, therefore, no surprise to us to see Contextual AI select WEKA as a strategic partner for delivery of enterprise AI services on Google Cloud this week. Contextual AI has made a name for itself with RAG 2.0, delivering enterprises fine-tuned models that provide foundational tools for enterprises wanting to build and customize specialized AI applications.

When you consider the application of contextual language models, a step beyond traditional RAG with integration across pre-training, fine tuning, and alignment with human feedback. Traditional RAG models uses an off-the-shelf model embedding, vector database for retrieval, and a distinct language model for generation, stitched together through an orchestration framework and therefore limiting both the final value of the application and difficulty and efficiency in delivery of the model.

RAG 2.0 was delivered, you guessed it, by the same leadership team that first delivered RAG at Facebook AI Research, so they know a bit about the RAG approach and how to make it better. RAG 2.0 has delivered proof of benefit across various industry benchmarks showcasing improved accuracy vs. RAG and Vanilla RAG models based on GPT-4 and Mixtral). That’s cool stuff. 

When you consider how much enterprises are focusing on RAG implementations to bring AI to the mainstream, you can understand why Contextual AI is garnering significant attention for delivery of its solution to customers. And when you consider the scale of data required for RAG 2.0, WEKA emerges as a perfect partner to support data pipeline requirements.

WEKA has delivered a fantastic data management platform to optimize GPU utilization delivering peak efficiency in model performance as well as value to the customer funding the Google Cloud instance. Contextual AI has deployed a total of 100TB thus far of WEKA data platform capacity to fuel data requirements and has seen increased developer productivity and faster model training times with the new solution. The delta in performance is eye-opening with a stated 3X performance gain across key AI use cases and 4X faster AI model checkpointing. They’ve done all this while reducing cost 38% per terabyte.

What’s the TechArena take? We love stories about taking technology innovation and actually implementing to customer value. What Contextual AI and WEKA are delivering here are useful tools that enterprises can tap today for real adoption of AI within their environments. We can’t wait to see more advancement on the contextual language models as the industry at large continues its bandwagon of support of RAG type frameworks for AI fine tuning and deployment. And we’re delighted to see WEKA’s data management solutions continue to garner momentum in market for this once-in-a-lifetime moment of AI-era computing advancement.

While Level 3 ADAS (conditional automation) requires the driver to be present and engaged, the resultant heterogeneous workload, a mix of compute and AI processing, is driving the need for new system-level architectures and state-of-the-art system-on-chips (SoCs) based upon leading-edge semiconductor processes and packaging technologies.  More specifically, solutions for Level 3 ADAS and above are driving the automotive industry to embrace chiplets to most efficiently and effectively address the demanding workloads by allocating the right task to the right compute engine optimized for a given task in a footprint that is most efficient from both a power and area perspective.  It is also the case that chiplet-based solutions can be more cost-effective when compared to an equivalent solution based on monolithic technologies.

Chiplet technology, still in its relative infancy, enables disparate technologies and semiconductor dice to be combined in a single package through the use of die-to-die interconnect which is physically connected via a package substrate while enabling performance that is equivalent to that of a monolithic device.     

In March 2022, a universal interface for the interconnection of chiplets was released. Dubbed UCIe 1.0 (Universal Chiplet Interconnect Express), the introduction of this standard, in part, reflects the broader industry awareness that the traditional benefits of scaling through process technology are being challenged. To be clear, the major semiconductor foundries continue to invest heavily in developing advanced process nodes which continue to offer improved power, performance, and area benefits. However, the economics of these advanced nodes present significant barriers to adoption for a large percentage of the ASIC / ASSP design community. Additionally, the benefits of scaling associated with migrating to the most advanced semiconductor process node do not apply uniformly across all circuit types - most notably analog circuits.

As such, while scaling through Moore’s law is still a very important vector for the semiconductor industry, the move to chiplets and the use of advanced packaging technologies are rapidly becoming an important vector for the industry - and in fact, hold the promise of spawning a new industry where chiplets from different vendors can be readily interconnected via UCIe to rapidly build new products that combine best in class technologies into a single package delivering fast time to market, low risk, and low development cost. The benefits of chiplets are quite meaningful enabling the following benefits:

• Ability to build SoCs that are larger than the reticle of the semiconductor manufacturing equipment
• Building SoCs with higher yield and correspondingly lower cost vs. monolithic solutions
• Reduced R&D cost vs. employing advanced nodes
• Accelerated time to market
• Smaller footprint - vs. discrete two-chip solution
• Higher performance vs. discrete two-chip solution
• Lower power vs. discrete two-chip solution 

UCIe is an open specification that defines the interconnect between chiplets within a package, enabling the formation of a chiplet ecosystem with a common interconnect footprint at the package level.  In effect, through industry standardization, the equivalent of a common footprint, similar to Lego®, is being established.  The UCIe standard is endorsed by 72 contributing members, and 26 adopting members at the time of this writing.  

The initial focus of the UCIe 1.0 specification was in the following areas   

• Physical Layer: Die-to-Die I/O with industry-leading Key Performance Indicators (KPIs)
• Protocol: CXL/PCIe for near-term volume attach
• Well-defined specification: ensure interoperability & evolution

The UCIe physical layer supports I/Os that can provide up to 32 Giga Transfers per second (GTs) with 16 to 64 lanes and uses a 256-byte Flow Control Unit for data, similar to PCIe 6.0.  The protocol layer is based on Compute Express Link (CXL).  In short, UCIe is leveraging tried and tested technologies that have a strong legacy of adoption across many markets.

As a testament to the importance of UCIe to the automotive industry, In August of 2023, the UCIe consortia introduced a 1.1 version of the specification with a focus on the following areas:

• Enhancement for automotive - including runtime health monitoring and repair for high-reliability applications
• New usages for streaming protocols with full UCIe stack
• Protocol support with end-to-end link layer functionality
• Cost optimization for advanced packaging resulting from new bump maps
• Enhancements for compliance testing

While all of these components of the 1.1 specification reflect a keen focus on the critical next-level details associated with chip-to-chip interconnectivity and device-to-device communication, the specific considerations for automotive applications in the UCIe 1.1 specification underscore the strong anticipated adoption of chiplets in safety-critical automotive applications. 

Addressing those safety concerns, the UCIe 1.1 version of the specification includes preventive monitoring to ensure that the die-to-die signaling  “eye” height and width are optimal and can be re-trained as needed. (As a note, the “eye” diagram is a way to measure the signal integrity of the link.  The more open the “eye” is, the greater the signal integrity).  There is also the addition of run-time testability of link health which includes the periodic parity Flit injection checking for the health monitoring of each lane with the ability to repair as required. 

The adoption of state-of-the-art technologies marks a sea change in the automotive industry where historically, automotive electronics employed mature products that were typically based on mature semiconductor process technologies.  With the advent of ADAS and AD, this has now changed. AI performance requirements that can reach PetaOPs levels for Level 5 ADAS are best addressed via heterogeneous computing solutions employing high energy-efficient, high-TOPs AI accelerators in a chiplet form factor. 

At the time of this writing, the UCIe consortia is preparing to release a 2.0 version of the specification.  When the results are made publicly available, we will take a look and see what additional features and considerations have been made to the specification and place a focus on those that specifically address the automotive market.

Learning to learn in a structured manner is the only way to maintain a long career in technology. I tell those I mentor they should settle into the reality they will likely have 5+ careers in this industry.

“Don’t be scared, just stay curious,” I say. “Do everything in your power to be the positive influence on every collaboration you engage in.”

Technology domains are small communities, and being the individual who consistently demonstrates high EIQ, dignifies vs. degrades others (regardless of the role they play) is a superpower for the long haul.

Many of us with a longer tenure in technology see patterns in industry transformations, both for good and for ill. Example: years ago, I was informed by a long tenured mainframe/UNIX expert that virtualization was a mainframe feature for decades. If a server solution was considered to be equivalent to “…a toy you find at the bottom of a cereal box”, why bother with virtualization technology that limits performance? (Extra credit: Name the CEO that gave us the soundbite comparing servers to Fruit Loops without doing a web search!)

To kick off my blog series with Tech Arena, I decided to share three patterns I have consistently seen across ~30 yrs of technology inflections.

The “best” solution in the eyes of a customer rarely offers the best raw performance.

As a hardcore engineer fresh out of college, I got my hands dirty coding some of the most complex SOCs with firmware and compiled languages. I was convinced RISC was the ‘best’ architecture. I was so excited to promote the first >100MHz System on Chip (SOC) for the ARM ISA, thinking it would be easy to sell ‘better technology.’ I instead learned about legacy code and ‘good enough’ x86/68K/PowerPC incumbency. “The best technology” may cause such an operational hassle, customers will admire, but never purchase “the best.” A company can’t maintain a profitable business on glowing reviews from technology analysts or journalists.

Corollary Lesson: Being boring, reliable, easy to debug, offering ways to sample operational data without impacting run time performance is surprisingly underrated by technology providers. To service providers, this list is REALLY important to deliver a good customer experience. Using a car analogy, a Lamborghini may be the “best” auto on a racetrack as measured by its top end speed, but when driving grade school carpools, a Lamborghini is limited to the same 25 MPH zones that a Toyota Corolla or Minivan comfortably achieves.

“What compute performance giveth, the network can taketh away.”

Maintaining optimal compute/throughput/storage is a never-ending battle. Equilibrium with enough compute power that processes data, balanced by enough storage in the right tiers, plus the right network capacity to keep the data moving – it just never lasts for long.

There are so many examples, I’ll just point you to two deeper articles to explore. Both The Register on high core count CPUs and GPUs, and Broadcomm’s article with Meta’s data on AI instance efficiency offer depth on the consequences of imbalanced systems. Now, imagine the ‘inside voice’ conversations within infrastructure operators who pay ~$1B for AI processing hardware and only see ~$500m of work done on it.

“Immutable ‘laws’ vs. Ideology and narrative”

A favorite comic in my family included something related to exercise in every routine. A classic quote is “I don’t do ups, I do DOWNS, because gravity is the law, and I obey the law!” In commercial airplanes, applied physics temporarily suspend the effects of the law of gravity, but eventually gravity wins, and the plane must land. Similarly, in every technology era, exuberance for a hot technology seems to temporarily suspend the laws of economics and the principles of supply and demand. Countless breakthroughs have promised a new utopia, to replace or better humanity, and to keep humanity from…well, being itself! The internet evangelists in 2001 claimed ‘borders didn’t matter’ and ‘information wants to be free!” Here in 2024, how many paywalls have you run into this week? Economics clapped back on utopia, because it turns out that you need to make more money than you spend to stay in business. There are a few hard lessons in the fact that customers/consumers will not spend at the level of the difficult engineering that built the product. Do consumers save more money? Do enterprises see benefits well beyond what they must spend to afford the product?

Concluding thoughts

Many of these ‘rules of thumb’ don’t need to be stated when business and technology together offer a solution that legitimately solves more problems than it causes. That often happens years after the discovery of a new technology innovation – so stay curious, keep learning and share your thoughts in the comments!

Lynn A. Comp Bio

Lynn A. Comp is a Vice President and Director of the Microsoft Global Account Team in the Intel Sales and Marketing Group. Lynn’s mission is to align the unique benefits that Intel and Microsoft offer their customers through access to unparalleled technology at scale, enabling the broadest ecosystems to innovate solutions to the most challenging business problems. Lynn returns to Intel from AMD where she served as the VP and GM of the EPYC CPU Cloud Business and the VP of EPYC Product Marketing.

Lynn has a wide range of experiences spanning her ~30 years in the tech industry, from strategic planning and go to market of RISC SOCs for both communications infrastructure and mobile phones, to software offers laying the groundwork for rapid video-based services innovation, to pioneering the foundational libraries that paved the way for ‘software defined’ networking with telecommunications operators.

Lynn has extensive experience in marketing, product management, product planning, and strategy development across software, hardware, cloud, and communications service providers (CoSPs).

Lynn has a Bachelor of Science in electrical engineering from Virginia Technology and an MBA from University of Phoenix. 

TechArena's Allyson Klein and Jeniece Wnorowski from Solidigm sit down with Kelley Mullick, Vice President Technology Advancement and Alliances, from Iceotope to discuss the latest in data center cooling technology. They dive into the role of liquid cooling in supporting AI workloads, the sustainability benefits of advanced cooling solutions, and the future of edge computing.

While no day in tech is technically slow, we’ve hit mid-summer, which means that conferences have slowed, Europeans are jetting off to holiday, and everything else is feeling a bit like wading through molasses in terms of the regular maelstrom of advancement that we’ve come accustomed to in the AI era. A day like today makes me look forward to what is on the horizon, and there is plenty to look forward to in the months ahead. Today’s focus is the OCP Summit coming up October 15-17 in San Jose, and if you have peered into the roster of industry sponsors like I have you’ve noticed that OCP has…well…super-sized.

I remember the days that this Summit would feature about three top tier sponsors vying for the spotlight, and this year features nineteen. NINETEEN. This is just the top tier players, and as you scroll through the list of companies spending major money to feature their tech at the show you realize that OCP has become the nexus of what is happening in AI infrastructure. Silicon providers, infrastructure builders, power and cooling entities, oil companies – all showing up to talk about the future of the data center. All likely engaged in OCP configuration delivery and participating in conference proceedings in a major way.

The TechArena is excited to be part of the OCP Summit this year again as a media sponsor, and we are especially excited to dip into conversations regarding the advancement of sustainable infrastructure, how the fabric of the future will be shaped including advancement of Sonic (which we covered extensively in Lisbon this April), what’s next in memory advancement, and of course power and cooling innovations required to fuel these ridiculously hungry racks. We’ll be podcasting daily, engaging with vendors in video discussions from the show floor, and writing insight stories for those unable to make it to San Jose. We’re keen to know what is on the top of your mind in regards to hyperscale infrastructure priorities as well! Please take part in our daily polls on my LinkedIn feed this week to share your views on what will grab the world’s attention during the summit in October.

“We need NASA, because we need a tower of flame 60 miles high burning at temperatures hotter than the surface of the sun.” - Nova, Sharknado III

Technology: it’s all about the sublime new features. That intricate blend of performance and programming, design and development, brilliant insights leading to giant leaps of innovation. Right? We’re talking about the best and the brightest innovators, those ultra-cool kids who opted to take all those math classes, knowing that the Psychology majors were already hitting quarter beer night while they wrote up their fourth page of derivatives. But it was all worth it, because someday…some wonderful day…the world would recognize that Linear Algebra would make things safe for, um, whatever they’d get paid a lot to algebrate.

As cool as technology is, the reality of any good high-tech solution involves more boring logistics than elegant engineering. For all the results, the hours of sweat are more about jamming six new ideas in a space that holds three and hoping that something doesn’t break once the test suite is finished. Innovation is built on the shoulders of a lot of swearing in small rooms with smudged whiteboards. And maybe the archaeology of those smudges reminds some about how it was done before.

The modern data center began sometime in the early ‘00s, when commodity servers began to fill the racks, displacing RISC machines. (Sometime later, we’ll talk about how it was a stock market bust that revolutionized the datacenter.) Virtualization hadn’t hit volume yet, so most servers were single-task machines. The really innovative IT organizations were just starting to think at a rack level with stateless transactions.

The move from computer rooms to data centers was only twenty years old or so. The common architecture was a raised floor that delivered power and pulled air downward (it was fun to watch the airflow using the cigarette smoke back in the day). We were moving on from a legacy of big servers that filled a tile or two and pulled in the range of 6-8kW of power, and now facing a world where evolving commodity servers in a rack were - gasp - demanding 10kW, maybe 12kW, per tile. This was also when the data center was generally located in prime real-estate on the buffer floor in between IT and management, so forgive us for the infrastructure costs back then. We thought local, not global.

The conversations back then went to two simple vectors. One: make servers with less power. It would take a bit for us to figure out how to turn off portions of silicon that weren’t doing anything, and performance dictated that we’d probably claim that power back anyway. Two, and usually prevalent: brute-force the data center to adapt. In some cases, that meant that a rack or two of servers on a tile would be isolated by a few blank tiles to keep up with the limited cooling capacity in the room. It was not entirely uncommon to find a box fan or two on top of a rack to shove air over to another air intake.

The next innovations focused on logistics, mechanics, and air flow over server engineering. The data center began to evolve as the cloud companies led the charge to dedicated buildings with specific architecture to optimize power delivery and air movement. Hot/cold aisles started to appear, as did plenums and upward air flow. That alone bought a few kW of breathing room, even as the racks began to evolve to support a mix of compute, network, and storage to handle virtualized workloads. These days, it’s not uncommon to see 20kW racks in a vanilla data center, standing on those shoulders of innovation.

Of course, the demands never end. The commodity server isn’t as cool (pun intended) as it used to be, we processor folks blame the memory. And the rise of dedicated GPU workloads is back to driving power of individual rack units to new heights. It’s not uncommon for a full rack of AI/ML servers to demand up to 60kW of power. As before, the brute-force solutions appear to be back in vogue. Depopulating space to support the boutique solutions is back, minus the box fans. We don’t allow humans in the data center anymore, they’re just wasted heat. It’s likely that commodity will start to pull in AI innovation, and a bit of re-architecture will save some power, but the end result is likely the same. It’s likely time for another innovation in the actual buildings to accommodate the new rack reality.

It’s not hard to imagine another twenty years when we’re all chuckling about those old days where 50kW created a week of meetings in the conference room, and maybe the AI can peer back through the smudges to realize the old solutions might be useful again. So for now, here’s to the next generation of geeks skipping out on the vape party to do homework. We’ll need you then like we always did.

The world is rapidly transforming with new applications of AI across numerous industries. I’ve been fascinated by the advancement of AI in the world of pharma research in particular, given the complex process to identify an early drug candidate, conduct preclinical research and run phases of clinical trials before reaching mass market introduction.

We recently discussed pharma within our paper on semiconductor requirements for AI advancement. I am reminded of this quote from Oii’s CEO and founder Bob Rogers:

“For each application area in drug development, the speedups reported by AI vendors are 3x to 10x. These accelerations by themselves are significant, but the real magic lies in the fact that every step of the drug development process is currently built from interlocking, inefficient human tasks. Replacement with AI tooling will result in wholesale reductions in the time it takes to propose, test, and report on new drugs in the market.” 

So where is the low-hanging fruit? I’d argue that one is the world of microscopy, where early stage research is performed under electron microscopes to seek information about the chemical structures of compounds for potential efficacy to target disease. This stage of the process is incredibly human intensive with work that – frankly – can be done faster and with more precision by AI visualization techniques.

Enter Health Technology Innovations, Inc. (HTI), a National Science Foundation-funded AI operation that is targeting the space with collaborations with the leading microscope vendors and research institutions. Led by industry veteran Tuan Phamdo, HTI has built a comprehensive Cryo-FAST software platform that helps accelerate pharma research. According to the company’s claims, HTI has driven average screening time from six hours to 5 minutes, halved scope collection time, and cut annual lab costs for this function by 50%. Today, the folks at HTI announced the fantastic news that their software is ready for mass market adoption as a 1.0 offering.

This advancement has gotten the attention of large players in the sector. Gatan, JEOL, and Thermo Fischer – all major players in electron microscopy – have collaborated with HTI on the delivery of this solution. The Northwest Cryo-EM center at Oregon Health Sciences University have also deeply engaged, and I’d expect to start seeing major research labs deploying HTI’s software in tight integration with their scope of choice. 

What’s the TechArena take? Well, we’ll go where this blog started. AI is a massive disruptive force, and solutions like HTI’s will help advance scientific discovery at record pace. We’re excited to see this as a proof point of the larger transformation happening in pharma and can’t wait to see societal benefits from solution adoption. We also expect HTI to become much better known quickly in the world of pharma research as a player that is delivering real technology to customers with the help and support of industry leaders. We can’t wait to hear more about solution deployments in the months ahead.