Welcome to
the Forum

AI advancement is a global pursuit, and Canada has accelerated its core capabilities with a new collaboration with industry leaders Hypertec and VAST Data.

While we have written about the AI's geo-political underpinnings and posturing of late, it's terrific to see a focus on real-world advancement of technology to serve pragmatic application innovation. In the agreement, Hypertec Cloud will boost compute capacity for AI through their proven cloud offerings, enabling researchers and government agencies access to compute cycles to fuel their application development.  

In order to deliver the data pipeline required, Hypertec will utilize the VAST Data Platform, known for its efficiency and scale and delivering seamless management and access across diverse environments.

Part of this agreement will include delivery of up to 100,000 GPUs across North America early this year, growing the companies’ collaboration that already supports mainstream deployments from Together AI, Zyphra and other leading AI adopters. The collaboration taps VAST's large-scale AI cloud infrastructure to support customers who choose to deliver solutions based on diverse data locations and combination of structured and unstructured data sets to train AI models uniquely suited for their applications.  

We were delighted to catch up with Hypertec Cloud at Supercomputing (SC ’24) to learn more about this long-standing collaboration between the two companies. Check out additional insights here, and be sure to check out TechArena's content contributions to VAST's Cosmos community.

Untether AI's Bob Beachler explores the future of AI inference, from energy-efficient silicon to edge computing challenges, MLPerf benchmarks, and the evolving enterprise AI landscape.

‍

In this video, learn why Arm is investing in chiplets as a way to build dynamic options for compute in the AI era, and hear more about their new Chiplet Standard Architecture Specification.  

To learn more about the CSA, check out Arm’s blog.

The evolution of software has experienced a few consistent patterns over the years, with a few disruptors here and there that drive a new methodology, or new economics. One consistent trend has been that the introduction of a useful technology, such as a virtual machine monitor, often generates significant enthusiasm. However, it frequently ends up relegated to middleware, which either requires a solution higher up the stack (like Broadcom/VmWare VSphere) to maintain market momentum and customer retention, or gets absorbed into the lower layers of the platform or hardware. The base technology is useful, but doesn’t have a direct touch to the customers/users, and ends up bundled with other innovations that do.

With that background, everyone has been watching the battle for domination of worldwide AI capital investments amid claims that “agentic AI” may hollow out the inherent value of SaaS services. Others continue the never-ending hype surrounding new model development to justify ‘build it and they will come’ data center investments. This news cycle, there has been a lot of traffic on Deepseek, small language models, and OpenAI’s SUPER agents against their “Benchmarkgate.” That is a lot of jargon for Wall Street begging AI providers “take my money, please,” and the largest model competition between a handful of players who want an unfair share of that capital/investment. It seems to be independent of the revenue base of paying customers who are still working to find the AI 10x ROI to their businesses. There are other economic challenges – if OpenAI is losing money on every query after seemingly randomly selecting $200/month as their Pro business model, how long can their business model hold before prices must change? Does an enterprise need a model that can answer world history questions, or quotes from Shakespere, or just a trained-on-that company-data model that can accelerate time to value for their specific enterprise processes? If a business does not need a model that can answer world history questions, does it need “superintelligence”?  

There is a clear and practical application for the automation and streamlining that AI, as part of a business process, can bring. The challenge for the underlying model itself is that another process has to invoke it, whether a browser/app (ChatGPT), or the front end of a business process/query (ex:ETL). For consumer facing businesses, LLMs function as middleware that drives revenue growth in advertising and shopping businesses. The helpful system recommending things based on my browsing or shopping history? AI functions and models running in the background. This business isn’t new and radical – different types of applied AI have driven shopping operations for years. In other enterprises, for significant value for AI to be realized, the debate on agentic/ SaaS and the importance of a superintelligence model comes down to what interacts with the AI function – customers or front-end processes. Agentic AI in particular breaks up workflows in ways that abstract which model is running a function or process. The model or more likely, models, underpinning the agentic AI workflows work in the background.

An analyst told me recently that “models don’t matter”, which may be an exaggeration since data bases aren’t sexy anymore, but are critical to getting great results from AI deployments.

One test on whether becoming middleware is a likelihood is how close the funding pitch sounds like the Southpark Underwear Gnome business plan:

Their business plan is broken into three simple steps:

• step 1:  Steal underpants
• step 2: ???
• step 3: Billions and billions of profit.
  ‍

While I see signs that models themselves are becoming background functions using the Southpark test, the SaaS/Agentic debate where both options offer user facing features and AI capabilities “in the workflow” is the real battlefield for companies wanting to capitalize on the AI transformation. My theory on which model wins comes down to “users likely do not care how the underlying architecture was constructed”, so the advantage goes to ease of use.

What are your thoughts about where SaaS and Agentic have advantages or disadvantages?

And just like that, the world of artificial intelligence (AI) model training was thrown into turmoil. In case you were vacationing or otherwise occupied late last week, China introduced DeepSeek, a free-to-the-public generative AI model that is outperforming Chat GPT.

While the performance is stunning, what's most notable about DeepSeek is how this elegant solution was purportedly trained with a fraction of the resources required by leading models today. And this news emerged literally days after OpenAI announced a $500 billion investment with Microsoft, Arm, NVIDIA and others to drive U.S. AI superiority and the U.S. delivered the first meaningful threat to China’s TikTok access to U.S. markets.

Chatter across social networks dubbed the makers of DeepSeek geniuses, heralding an end to U.S. AI leadership and calling NVIDIA stock deeply overvalued. Let’s unpack what we know and what we suspect as the dust settles on this massive shock to the AI world.

DeepSeek appears to have taken some innovative approaches to training its algorithm, including utilizing less precise math (eight decimal places vs 32) and processing larger groups of words, driving down precision. But it’s also created more efficient multi-token ingest and allocated training across multiple experts like a group of smaller, smart models working in tandem. All of these examples are innovations that will likely get attention from across the AI community and come at a time when the field was seeking disruptive approaches to delivering AI models more efficiently, something we’ve discussed at length on TechArena in 2024.

What is raising questions among many experts whom I chatted with over the weekend is the full transparency in the cost of training the model. One pegged the true cost at 1.5-to-2 orders of magnitude higher than what the startup has stated, with disclosed costs solely focused on knowledge distillation and fine-tuning of the algorithm. They point to the fact that this version of the model has had the benefit of training of previous iterations, similar to the investment alternatives like ChatGPT, Llama, Gemini and others have based on iteration of versions of models over time. The truth is likely that the cost of the model we’re looking at today is much higher than the $5 million that promoters are claiming. Yet, this should not discount the value of a competitive model and the overall performance it’s delivering.

Of course, the timing of DeepSeek’s release does give credence to this lack of transparency being a shot across the bow of the Stargate announcement, continued tensions on TikTok restriction from the U.S. market and U.S.’s focus on AI as a central policy imperative. And the well-timed emergence of DeepSeek to access U.S. data sources can’t be overlooked, given that China has historically used TikTok to collect data from Americans and this new generative AI model provides a much more powerful way to collect that data and deliver potential misinformation through content.

While the coming weeks will provide more clarity on the full truth of this model’s efficiency, one thing is clear: DeepSeek has absolutely captured the public’s attention, seizing leadership on AppStore downloads vs Chat GPT.

So, what’s the TechArena take?

We see the arrival of DeepSeek as a reminder that while the U.S. hyperscalers grab the headlines for AI model advancement, their Chinese counterparts have been dedicated to developing their own AI solutions for years. Bytedance alone has committed $20 billion in AI investment in 2025 ($12 billion earmarked for U.S. spending), and others such as Alibaba, Baidu, and Tencent have similar large-scale operations established. In this high-stakes realm of the race for LLM superiority, we can expect to see both dizzying announcements of innovation and overstated differentiation at both a corporate and geo-political scale. And while we applaud any advancement to drive efficiency into AI training and are seeking further clarity of the full veracity of what’s been delivered with DeepSeek, we aren’t yet ready to call an end to the Blackwell era before it’s really even begun.

We also see the public’s zeal for utilization of all of these LLM models igniting without a lot of thought about what happens to the data provided to the model owner, whether that be an enormous tech conglomerate in the U.S. or a Chinese startup with ties to the government. In this world of rapid adoption of LLMs, we wonder what defines truth in the future and will there be multiple definitions of truth depending on who controls the algorithm.

Share your thoughts with us on LinkedIn and expect more news to roll out quickly in this space in the coming days.

Innovators from around the globe gathered in Santa Clara this week for the third annual Chiplet Summit to learn the latest in artifical intelligence (AI)/ machine learning (ML) acceleration, the open chiplet economy, advanced packaging methods, die-to-die interfaces, and more. The TechArena team had courtside seats for the event, and with chiplets being the underpinning technology for continued advancements of Moore’s Law, we were keen to understand the state of industry innovation.

We have been talking about an open chiplet economy on the TechArena platform since we named UCIe one of the top innovations in tech in 2022. The speed of design, efficiency of multi-process solution delivery, and opportunity for best-in-breed chiplet integration offers unquestionable opportunity for the market. But while all major compute architectures embrace chiplet designs today, the open economy of chiplet design has been slow to emerge due to missing elements, such as form factor designs, and interoperability testing still needing to be addressed by the industry. Technologists attending the event this week enjoyed opportunities to learn from industry pacesetters about the current state of addressing these gaps as discussions centered on how to create the latest chiplet designs in less time, for less cost, and in a more scalable way.

The event featured a series of keynotes from semiconductor heavyweights, including  Alphawave Semi, Arm, the Open Compute Project Foundation, Synopsys, and Teradyne. While these companies provided incredible insight to the state of the industry, there were a few takeaways that rose above the rest:

1. ‍The industry is embracing chiplet technologies as the foundation for future AI scaling – In a keynote delivered by Tony Chan Carusone, chief technology officer at Alphawave Semi and professor of electrical and computer engineering at University of Toronto, Tony unveiled Alphawave’s vision for the future of AI scaling, connectivity, and custom silicon solutions. He discussed how groundbreaking advancements in chiplet technology are enabling the rapid evolution of AI systems while addressing critical challenges of sustainability, cost, and performance. This included three pivotal challenges driving innovation in AI: the insatiable demand for compute, the diversity of workloads requiring customized hardware, and the increasing complexity and cost of traditional monolithic chip designs. He emphasized how chiplet-based architectures are uniquely positioned to address these issues. By leveraging modular design principles and advanced packaging technologies, chiplets enable faster development cycles, greater scalability, and optimized performance tailored to AI workloads.
   Alphawave Semi showcased its AlphaChip 1600 IO, a multi-protocol chiplet that supports diverse connectivity requirements, including Ethernet and optical solutions, vital for enabling next-generation AI infrastructures.
2. ‍Collaboration is becoming increasingly critical across the semiconductor supply chain – Eddie Ramirez, vice president of marketing at Arm, delivered an insightful keynote outlining the pivotal role of chiplet technology and ecosystem collaboration in addressing the challenges of AI scaling and democratizing computing. With expertise spanning data centers, networking, and edge solutions, Arm is enabling partners to accelerate innovation while optimizing cost and performance, he said. We first covered Arm’s ecosystem programs in this arena in 2023 at the Open Compute Foundation Summit, and it was terrific to see the groundswell of support Arm has developed with partners reflecting a broader trend in chiplet advancement.
   In his talk, Eddie noted that 80% of AI workloads are centering on inference, a shift from a more training-centric focus in 2024, requiring specialized silicon and heterogeneous integration to meet the needs of both data center and edge environments. Chiplet-based architectures provide a solution to the rising costs and complexity of traditional monolithic designs, enabling reusable silicon and faster development cycles. Eddie highlighted examples of Arm’s ecosystem collaborations, including NVIDIA’s Grace Hopper system and Amazon’s Graviton processors. Grace Hopper showcased true heterogeneous integration, combining Arm Neoverse CPUs with NVIDIA’s high-performance GPUs to enhance AI pipeline efficiency. Meanwhile, Amazon’s accelerated delivery of four Graviton processor generations in five years highlighted how chiplets can streamline development and boost performance.
   Eddie also discussed Arm’s NeoVerse CSS compute subsystems as a catalyst for advancing custom AI silicon. By pre-integrating IP and reducing design complexity, the company is empowering partners to bring AI-enabled CPUs to market faster, he said. He pointed to the Arm Total Design initiative and open-source chiplet system architecture (CSA) specification as steps toward standardizing chiplet designs and fostering a more accessible computing ecosystem.‍
3. Multi-Die architectures hold increasing significance as critical to the future of AI-driven semiconductor growth – Abhijeet Chakraborty, vice president of engineering at Synopsys, highlighted the transformative role of multi-die and chiplet technologies in driving semiconductor industry growth. With global semiconductor sales projected to approach $700 billion in 2025, Abhijeet[AK1]  emphasized how advanced chip designs are addressing the surging demand for AI and high-performance computing (HPC) applications. Multi-die architectures are growing in significance as essential for overcoming the limitations of monolithic designs in AI chip development, he said. AI accelerators, tailored for training and inference workloads, require immense compute power, high memory bandwidth, and advanced interconnects. These demands have led to innovations like Synopsys' UCIe-compliant interface IPs, enabling high-speed die-to-die connections and ensuring reliability through monitor, test, and repair capabilities.
   Packaging technology serves as a critical chiplet enabler, with advancements like hybrid bonding facilitating higher interconnect densities and cost-efficient stack-die solutions, he said. Abhijeet also underscored the role of AI in electronic design automation (EDA), leveraging AI for optimization, prototyping, and multi-physics analysis to tackle the complexity of multi-die systems. Looking ahead, Synopsys anticipates rapid growth in multi-die adoption, with over 50% of HPC designs expected to utilize the technology in 2025.

UCIe – Enabling Widespread Chiplet Adoption Across Industries
‍

As the industry races to keep up with the pace of change, the UCIe (Universal Chiplet Interconnect Express) Consortium has gained momentum tackling chiplet integration challenges. During UCIe’s presentation at the summit, Brian Rea, marketing workgroup chair for the consortium, emphasized that the group is driving the future of chiplet integration by establishing open industry standards and fostering collaborative innovation. Milestones include the Pike Creek project, demonstrating multi-vendor interoperability, and UCIe 2.0, released in 2024, featuring enhancements for 3D integration, interconnect performance, and advanced packaging. These advancements address critical industry needs, such as scalability and seamless integration.

TechArena’s Take
‍

So what is TechArena’s take on the state of chiplets? Like many industry-wide innovations, growing the foundational standards, IP, and tools are the first steps to massive adoption of technology. What we saw this week at the Chiplet Summit underscores industry advancement on all fronts, and examples of market traction felt more substantial as compared to last year’s keynotes. We expect to see broader application of multi-vendor chiplet solutions for custom chip delivery to fuel the insatiable compute demands of AI in 2025, bringing us closer to an open chiplet economy.

We can’t wait to hear more about standards development to support various aspects of multi-vendor delivery, such as common form factors. We believe the industry is aligned on these challenges being key gaps for urgent collaboration this year. Finally, we expect that 2025 editorial will feature a lot of this and more, as large providers continue to leverage this foundational ecosystem to drive the hundreds of billions in data center buildout expected globally.

In this In the Arena episode, Winbond’s Jun Kawaguchi discusses their industry-leading strategies for tackling next-gen cybersecurity threats, ensuring robust protection for the future.

In a previous article, I used the story of Alex–a busy salesperson targeted by a phishing scam–to illustrate the sophisticated nature of today’s phishing attacks, as cybercriminals work to stay one step ahead of end users. I also shared some general strategies to avoid falling victim to phishing.

While these methods are broadly applicable, understanding phishing’s evolving techniques and deceptive tactics is crucial—because spotting these scams is often harder than it seems. So, what exactly is phishing?

Understanding Phishing
‍

Phishing is, fundamentally, an identity cyberattack that uses communication (emails, texts, calls) to trick targets into providing access to sensitive data or installing malware. The target takes an action based on mistaken identity and misplaced trust in that identity. Because the target is unwittingly providing access, the malicious actor does not need to use technical means to gain access to the computer system.

The most basic phishing attack occurs when the malicious cyber actor sends a link to the target, who erroneously trusts the link and enters their credentials.  The malicious cyber actor then uses the credentials to commit fraud. A variation of phishing, known as smishing, uses SMS text rather than email to transmit the message.

Spear-phishing is a more sophisticated form of phishing. Unlike generic phishing emails sent to a large group, malicious actors specifically target a particular individual in a spear-phishing attack. By using this personalized information, such as Alex’s friend Jade’s phone number, spear phishing attacks appear credible, and are more likely to be acted upon by the intended targets. In the future, AI-powered synthetic data, such as deepfakes, will become increasingly prevalent in spear-phishing attacks, making it harder to trust even the most trustworthy voices and faces.

Phishing as an Attack Vector
‍

The big secret of cybersecurity is that it is not especially challenging to get sensitive personal or organizational information, but it is difficult to monetize that access. Phishing offers a straightforward solution. Phishing thrives, because it offers a comparatively low-effort, high-reward pathway to monetization. Cybercriminals can use stolen personal information, such as Social Security numbers, credit card details, and bank account information, to commit identity theft and financial fraud. They can make fraudulent purchases, take over accounts, open new accounts, or even take out loans in the target’s name.

Additionally, creating a reverse proxy can serve as a command and control (C2) channel, enabling the malicious cyber actor to remotely control the compromised system to install malware, pivot to other systems on the network, and launch further attacks.

Phishing attacks can go beyond simple credential theft. They can also function as a delivery mechanism for more sophisticated malware, blurring the line between phishing and other cyberattacks. These types of advanced attacks should be classified as something other than phishing. For example, surveillance tools, such as those from NSO Group, Candiru, Sourgum, and Intellexa, are four of the commercial tool makers who have delivered cyber tool payloads via phishing. Security researchers have linked these companies to sophisticated phishing or smishing cyber tools that use zero-day or zero-click exploit payloads. They can combine the social engineering tactics of smishing with well-executed sophisticated technical exploits to achieve their goals.

Phishing's popularity as a cyberattack technique stems from its ease, flexibility, and low risk for perpetrators. The low barrier to entry is a significant factor; unlike complex malware development or network intrusions, phishing attacks need minimal technical expertise and resources, making them accessible to a broad spectrum of cyber actors. Phishing’s remarkable adaptability amplifies this accessibility. Attacks can be meticulously tailored (spear phishing) to target specific individuals with personalized lures, or scaled up to target entire organizations (whaling) by impersonating high-ranking executives. Even more broadly, business email compromise (BEC) attacks the target’s supply chains. Phishing attackers exploit our human nature. They meticulously study how we think and make decisions, leveraging our cognitive biases to craft phishing lures that trick us into making irrational choices.

The diffuse nature of phishing attacks, often targeting individuals, makes it difficult to trace and prosecute, creating a low-risk environment for cybercriminals. These attacks often go unnoticed, or are difficult to trace back to the perpetrators across international borders, further reducing the perceived risk for malicious cyber actors, and solidifying phishing's position as a highly favored attack vector.

In 2023 alone, phishing attacks accounted for over $2.9 billion in losses, according to the FBI’s Internet Crime Complaint Center (IC3) (2023 Internet Crime Report). When combined with data breaches, credential theft, and the operational expenses tied to MFA and password-based systems, the aggregate cost of phishing losses is significant.

Phishing attacks often have a direct and immediate financial motive. Cybercriminals that use phishing seek to steal sensitive financial data, such as usernames, passwords, and credit card information. This stolen data grants them direct access to victims' accounts, allowing them to withdraw funds or make fraudulent purchases. This direct access eliminates the need for complex and costly money laundering schemes, which aim to disguise the origin of illicit funds. The malicious cyber actor can make fraudulent purchases, transfer funds, or withdraw cash using the stolen credentials. There is no need to obscure the origin of the funds or move them through accounts to hide their tracks. The costs associated with money laundering can significantly reduce the profitability of cybercrime, making direct access to accounts provided by phishing more lucrative. Phishing provides a straightforward pathway to immediate financial benefits, making it a highly efficient and appealing method for cybercriminals seeking rapid low-risk returns.

The increasing use of multi-factor authentication (MFA) and “advanced email security” systems can make it more challenging for malicious cyber actors to use stolen credentials. Phishing attacks that rely on stolen passwords are rendered ineffective with MFA, but MFA implementations come with their own associated costs, including financial, operational, and potential hidden expenses. This is an active area of development, with innovative approaches emerging.

Advanced email security systems, leveraging AI, behavioral analytics, and sandboxing, can identify and neutralize most phishing attempts. Additionally, the adoption of zero-trust security models can detect and block such attacks. Users can overlook this sophistication, because it operates in the background without the user’s knowledge. Companies such as Microsoft, Darktrace, and Proofpoint are at the forefront of these innovations, indicating a broader industry trend toward more secure and resilient systems that challenge the dominance of phishing as the most popular cyberattack technique.

While sophisticated email security solutions have made significant strides in blocking phishing attempts, they are not perfect. Implementing and maintaining advanced email security solutions can be expensive and complex.

See the previous article in this series to learn 10 things you can do to protect yourself from this cyberattack.

Phishing attacks are increasing in sophistication, and the signs of phishing that many people learned early in the lifespan of this criminal activity have changed.

Today, phishing is the most popular cyberattack technique because it offers a straightforward way for cybercriminals to monetize unauthorized access into quick cash.

Here’s an example scenario:

Alex, a busy salesperson with thousands of contacts, received a text from her friend Jade, an Apple developer rep, offering a 20% discount on AirPods Pro, Alex's favorite. The text looked genuine, even coming from Jade's phone number, and her iPhone identified and labeled the text as coming from Jade (caller ID spoofing).

Eager for the deal, Alex clicked the link in the text. (Click!)

The link led to a website that looked identical to the brand's official website. It had the same layout, color scheme, and product images. The website even had a limited-time banner ad for partners – the same ad Alex had seen on Apple’s site recently.

The site displayed a small padlock icon in the address bar; the URL started with https, indicating that the connection was “secure.” Confident that the website was legitimate, Alex entered her ApplePay payment credentials, but the site rejected them. She reasoned that her corporate VPN account could be blocking the purchase, so she entered her credit card details instead.

Once Alex entered her credentials, they were immediately visible to the malicious actors. With access to Alex's account, they made increasingly large fraudulent purchases totaling $2,600, mostly in gift cards, then sold both her account credentials on the dark web, for $4 USD each (yes, that is a going rate), as well as her address and other information valuable in the criminal economy for more sophisticated identity thefts.

The ease with which cybercriminals tricked Alex underscores the effectiveness of phishing engineering tactics. Alex discovered the breach months later, before a flight, when she was unable to access her ApplePay account. She never realized the malicious actors had also compromised her credit card. The advice given to her was to change her password, use MFA, and monitor her credit report.

What Alex Missed

In the scenario above, the padlock gave Alex a false sense of security. The truth is that most phishing sites use TLS (Transport Layer Security), meaning they would have that padlock. Many users, like Alex, see the padlock and assume the site is safe. They do not understand that the padlock only means the transmission is encrypted, not that the website is trustworthy. This misunderstanding is what phishing attackers exploit.

Here’s a little background on this point: Public Key Infrastructure (PKI) provides the foundation for secure communication by managing digital identity through certificates, while TLS leverages these certificates to establish encrypted and authenticated connections between devices. While PKI has been a cornerstone of digital security for decades, it has become increasingly clear that it is not the ideal solution. PKI is often difficult for average users to utilize correctly.

10 Ways to Protect Your Accounts

The story of Alex, a target of a phishing text message, exemplifies the ease with which cybercriminals can trick unsuspecting individuals. Phishing tactics prey on human trust and exploit emotional triggers, making them particularly dangerous.

Here are ten things you can do to help protect your accounts:

• Do not click on links or open attachments in unsolicited emails or texts, regardless of their apparent legitimacy.
• Hover over links to reveal the actual URL and use link expander services to verify their authenticity.
• Scrutinize email addresses and phone numbers for any discrepancies or anomalies.
• Contact the sender directly through a separate, trusted communication channel (e.g., company phone line, internal messaging system, Signal) to confirm the message's authenticity.
• Employ unique, complex passwords for each online account. 
• Utilize an open-source self-hosted password manager to generate, store, and securely manage your passwords.
• Enable MFA for all financial accounts with hardware keys (e.g., YubiKeys).
• Recognize the limitations of training and individual actions. Leverage enterprise-level security measures such as Zero Trust Architecture (ZTA) and advanced email security solutions.
• Consult with people that can help.
• Stay informed and vigilant! By staying vigilant and taking proactive steps, you can reduce your risk of falling victim to phishing attacks.

‍

‍

Artificial Intelligence (AI) is everywhere. It's at the center of conversations about technology, business, and even our daily lives. But when you see "AI" in the news, advertisements, or policies, do you really understand what’s being discussed?

Here’s why it matters. The way we define and use AI terms isn’t just about tech jargon. It influences how products are marketed, how policy is drafted, and how funding is allocated. If AI is shaping our world, we need to grasp what it means—and what it doesn’t.

This blog dives into why clarity around AI terms is essential, from its economic and political impact to how it’s reshaping technology and culture.

How AI Is Driving Economics and Politics

Every day, people search for “AI” and related terms over 6.5 million times. The AI market is booming—between 2017 and 2024, the number of AI companies more than doubled to 70,000. According to a 2024 Databricks survey, companies are projected to spend between $1 million to $10 million this year on generative AI alone.

The total addressable market (TAM) for artificial intelligence is staggering. Back in 2018, McKinsey pegged the TAM for AI at $13 trillion by 2030. By 2023, generative AI—just one subset of AI—was predicted to add $4.4 trillion annually to the economy.

AI isn’t just an economic game; it's a geopolitical one as well. Countries are racing to lead the AI revolution. For instance, the U.S. Congress has been urged to create a Manhattan Project-level initiative to reach artificial general intelligence (AGI) before nations, such as China, do. Why? Because AI leadership isn’t just about innovation; it’s about economic dominance and global influence.

When AI has this level of impact, understanding what we mean when we say “AI” becomes not just important—but essential.

AI is Changing How Computing Works

At CES this year, NVIDIA CEO Jensen Huang discussed AI's recent history and NVIDIA's significant role in it:

Artificial intelligence has changed how computing works. AI isn’t an ephemeral term, or an application, but an important evolution in computer science.

Artificial intelligence depends on lots and lots of data. Once the data is digitized and stored, you need lots of high-performance computing available to process it. That’s where NVIDIA comes in. Their products can run compute for machine learning and deep learning, the techniques used to train machines to do a task.

We Have a Cultural Bias for the Term AI

“Artificial intelligence” is a very loaded term. It was coined in 1955 by Stanford professor John McCarthy. He defined it “the science and engineering of making intelligent machines”.

Let’s be honest. AI is just a marketing term.

However, our collective consciousness is already filled with imaginations of what AI is. The oldest is HAL 9000 from 2001: A Space Odyssey.

Or maybe your mind takes you to Robocop, the story of crime-eradicating cyborgs.

If you’re a comic lover like I am, maybe you think of Ultron, an intelligent AI that was created by Dr. Henry Pym to keep global peace. But it got weird when Ultron decided there could never be peace among humans.

The list goes on and on. The Jetsons, Star Wars, Wall-E, the Matrix, Blade Runner, and The Terminator all feature some sort of futuristic AI.

These shows are part of our culture. It’s what we’re bound to think AI will be. Combine that with the gold rush to cash in on the AI craze, it’s important to know what people mean when they say “AI”.

Types of AI

If AI is a marketing term, it only makes sense that the types of AI can be separated into different types:

• Narrow AI: We are here. This includes things like voice assistants, facial recognition, recommendation algorithms, language translation, and even generative AI. 
• Artificial General Intelligence (AGI): This is the future. Sam Altman claims the latest version of OpenAI is at this level (although that’s up for debate). This type of AI will have the ability to understand, learn, and apply knowledge in a generalized manner.
  In an interesting sidenote, most of the media examples given above would classify as AGI.
• Artificial Super Intelligence (ASI): This is the future. It is theorized that this form of AI will surpass human levels of intelligence, capable of solving problems and creating solutions beyond what we can even imagine.
  Ultron is the only example above that meets the criteria for ASI. I mean, he was smart enough to hypnotize Pym to forget about him!

Instead of AI types, I like the way Huang marked out the stages of AI:

• Perception AI: Speech recognition, medical imaging, recommendation engines
• Generative AI: Digital marketing, content creation
• Agentic AI: Coding assistant, customer service, patient care
• Physical AI: Self-driving cars, general robotics

From a technology perspective, we have a handle on perception and generative AI. Those were building blocks to get us agentic AI, software that can interact with data and other tools. They can create lists of steps and then perform them with minimal human intervention (via RedHat).

The lessons learned from agentic AI are what will get us to Physical AI.

The More You Know…

Let’s practice seeing all types of AI for what it is – the newest evolution of computer science. Instead of types of AI, let’s see where things are on the evolution of AI as a science scale.

Don’t be afraid to question what you read, that is how you can defeat AI FUD. If the claims sound like they came out of a movie, dig deeper. What is meant by AI? Does that make sense based on where AI technology is today? If not, maybe you’re being sold the dreams of our childhood.

Let’s not be afraid of what the future holds, because AI definitely will open doors to amazing things. But first we’ll have to build the technologies needed to make it real.

In this report, discover the 2025 predictions of TechArena’s Voices of Innovation – industry experts who forecasting what’s on the horizon in AI, data centers, edge, network, sustainability and more.

In this Great Debate, moderator Allyson Klein is joined by a panel of industry leaders to discuss the future of AI in the data center. Jen Huffstetler, Chief Sustainability Officer at HP Inc.; Rebecca Weekly, VP of Infrastructure at GEICO; and Stephanie Schmidt, VP of Strategy at Flex, delve into enterprise adoption of AI in 2025, and the compute infrastructure needed to support its scale-out.

As artificial intelligence (AI) and high-performance computing (HPC) workloads continue to grow bigger, hotter, and more power-intensive, players like DUG – a pioneer in HPC solutions – focus on marrying sustainability with cost efficiency.

DUG, originally a geoscience data processing company, has been perfecting the use of liquid cooling technologies for HPC infrastructure for 15 years. Their latest innovations, including the Nomad—a data center in a shipping container—are designed for scalability and efficiency, offering HPC-as-a-service to clients who value sustainability, affordability, and simplicity over costly on-prem solutions.

The company’s liquid-Cooled HPC-as-a-service offers scalable, cost-effective, sustainable solutions to customers who are looking to access bare-metal HPC machines at a fraction of the cost of building on-premise solutions.

As the demand for AI and machine learning (ML) solutions continues to surge, DUG is observing a transition from traditional HPC-as-a-service to AI-as-a-service and ML-as-a-service, said Executive Vice President Ron Schop.

AI workloads also amplify the need for storage and heat management. DUG has doubled its cooling capacity through advanced Fluid Control Modules (FCMs), Schop said. And as storage demands continue to grow, partnering with storage leaders like Solidigm is critical, bringing large-capacity drives to provide the density required to meet modern AI and HPC needs.

So what’s the TechArena take? DUG brings a commitment to sustainable HPC solutions with an emphasis on creativity and innovation and more than a decade of experience with liquid cooling, presenting an interesting alternative for enterprises weighing new approaches to their data needs.

To learn more about DUG’s sustainable, liquid-cooled HPC and AI/ML solutions, visit dug.com.

SC ‘24 brought together innovators, thought leaders, and cutting-edge technologies to push the boundaries of high-performance computing (HPC).

During the event, Solidigm’s own Yuyang Sun had the chance to speak with Chen Lee, VP of Sales, HPC, Data Center and Enterprise for Giga Computing, about Gigabyte’s pivotal role in the server and AI market and the transformative impact of high-density SSDs.

Gigabyte’s Evolution: From Gaming to Global Server Leader

Gigabyte is widely recognized for its gaming and consumer electronics. Perhaps less widely known is that the company established a server division in 1989 to address the growing demand for enterprise and data center solutions. Today, Gigabyte ranks among the world’s top white-box server manufacturers, with a portfolio tailored for HPC, AI, and enterprise workloads.

Chen shared an anecdote about Gigabyte’s early collaboration with Google during the inception of open compute, underscoring the company’s focus on innovation, foresight, and willingness to embrace challenges.

“Game-changing” new 122 TB SSDs

Just before SC ‘24, Solidigm unveiled its groundbreaking 122 TB SSD, a high-density storage solution designed to supercharge AI workloads and data-intensive applications. Chen praised the SSD’s ability to deliver unprecedented storage capacity and speed, essential for the seamless integration of data with CPUs and GPUs in AI systems.

“The density and performance of this SSD are game-changing,” he noted. “It allows organizations to store more data with a smaller physical footprint, reducing infrastructure costs and energy consumption.

“With the emergence of the AI market, it becomes necessary to be able to react fast along with the GPU, the CPU, and the data needs to be seamless with the systems. So being able to have something with this kind of capacity, plus the speed, is amazing.”

Efficiency Meets Performance

High-density SSDs not only boost performance, but also address critical challenges in modern data centers. Compared to traditional 3.5-inch SAS drives, these compact SSDs dramatically reduce weight and space requirements, enabling enterprises to optimize their hardware investments.

Chen highlighted how Solidigm’s innovations align with Gigabyte’s vision of delivering efficient and reliable server solutions.

“Imagine holding a petabyte of data in just eight of these drives—it’s a testament to how far we’ve come. High-density storage is not just about capacity; it’s about enabling faster, smarter, and more sustainable computing.”

Gigabyte and Solidigm share a commitment to advancing AI and HPC. By integrating leading-edge technologies like high-density SSDs into their server solutions, Gigabyte empowers customers to tackle complex workloads with ease, efficiency, and scalability.

So what’s the TechArena take? As AI evolves and the infrastructure to support it becomes more advanced, hats off to Gigabyte for doubling down on its strengths—high-performance GPU servers, innovative cooling technologies, and partnerships with leading hardware and storage vendors.

At SC ‘24, Solidigm’s Jeniece Wnorowski caught up with NASA's Laura Carriere to discuss the space agency’s high performance computing (HPC) capabilities and the role of supercomputers in advancing Earth sciences, climate research, and more.

Laura, the high-performance computing (HPC) lead for NASA's Center for Climate Simulation (NCCS) in Greenbelt, Maryland, discussed how the smaller of NASA’s two supercomputing facilities focuses on Earth sciences while also supporting astrophysics, heliophysics, and planetary science research.

NASA’s Discover supercomputer plays a pivotal role in climate modeling, helping scientists tackle challenges like air pollution, aerosol movement, and hurricane formation, Laura said. With 60 petabytes of storage, Discover integrates powerful CPUs and GPUs to process massive datasets. Complementing Discover is the Prism system, a GPU-dense supercomputing environment that supports artificial intelligence (AI) and machine learning (ML) applications for tasks such as atmospheric simulations and space studies.

Visualizing Climate Change

One standout area of focus for NASA is aerosols—tiny particles that affect air quality and weather patterns. Using 3D climate models, NASA’s scientists can visualize how dust travels from the Sahara Desert to Florida or how sea salt is picked up at the poles. These dynamic visualizations provide insights into air pollution, emissions, and the impacts of climate change, helping policymakers and researchers develop data-driven strategies.

Power and Storage Challenges

While NASA’s supercomputers are powerful, they are also constrained by power and cost limitations. Discover’s modular design allows for incremental upgrades, decommissioning older components as new ones are added, Laura said. This ensures optimal performance but necessitates careful resource planning.

On the storage side, NASA maintains datasets on spinning disks, balancing cost-efficiency with performance. Laura noted that transitioning to high-capacity solid-state drives (SSDs) could drastically reduce power consumption and physical footprint, enabling more efficient operations. However, the highly compressed nature of NASA’s datasets poses unique challenges for cost-effectiveness in SSD adoption.

AI Meets Space Exploration

NASA is increasingly incorporating AI into its operations. The Prism system has enabled researchers to develop foundation models for Earth sciences and accelerate tasks such as binary star detection. Laura shared a captivating example of using HPC and ML to identify sextuple star systems—three binary star pairs gravitationally bound together.

NASA also used its supercomputers to generate awe-inspiring visualizations of black holes, giving the public a glimpse into the mysteries of the universe. These groundbreaking projects demonstrate the synergy between HPC, AI, and scientific discovery.

So what’s the TechArena take? As NASA’s awe-inspiring work continues to push the boundaries of science and technology, the agency’s commitment to HPC and data-driven research remains steadfast. As the agency continues to make incremental improvements in its HPC hardware and capabilities, we’ll be watching closely to see which of the great mysteries of Earth and space they solve next.

During SC ‘24, Yuyang Sun of Solidigm caught up with Joseph Lu, Director of Storage Technology at ASUS, to discuss ASUS’s role in advancing HPC and AI server solutions.

While ASUS is widely recognized for its gaming and consumer electronics products, Joseph highlighted the company’s extensive history in server technology. Since establishing its server division in 1989, ASUS has continually evolved to address the growing demands of enterprise, data center, and HPC markets.

“We’ve been in the server business for over 25 years,” he said, emphasizing ASUS’s commitment to innovation and focus on delivering comprehensive, customizable server packages. The company provides consultation, installation, and after-service support tailored to meet diverse customer needs.

At SC ‘24, ASUS demonstrated its latest AI supercomputing solutions, featuring NVIDIA, Intel and AMD GPU servers. Joseph emphasized that ASUS’s broad portfolio enables the company to optimize solutions for various workloads and customer preferences, empowering organizations to tackle complex challenges in AI and HPC.

“Our offerings cater to diverse workloads, providing customers with the flexibility and performance they need to succeed,” he noted.

High-Density SSDs: A Game-Changer for AI Workloads

A highlight of the discussion centered on Solidigm’s recently launched 122TB SSD and the transformative potential of high-density storage for data-intensive applications.

The new SSD is a great option for ASUS customers, Joseph said, because AI workloads require high-speed data transmission between GPU and storage servers.

“High-density SSDs help overcome bandwidth bottlenecks, enabling seamless data flow and non-blocking infrastructure,” he said.

The combination of high capacity and performance offered by such SSDs not only supports massive data volumes, but also enhances infrastructure efficiency. Joseph highlighted the benefits of reduced physical footprint, lower infrastructure costs, and improved energy efficiency—key considerations for modern data centers.

As AI data volumes continue to grow exponentially, scalability remains a top priority. ASUS’s commitment to integrating high-density storage into its offerings ensures customers can scale without worrying about future limitations.

“With the increase in AI data, it’s hard to predict how massive the storage requirements will be in a few years,” Joseph noted. “By adopting high-density storage solutions early, customers can scale effortlessly, ensuring they’re prepared for future demands.”

He also shared insights into ASUS’s ongoing efforts to develop software-defined storage solutions in collaboration with independent software partners. These innovations aim to provide customers with flexible, high-capacity storage options for both core and edge workloads.

“High-density storage is not just about capacity—it’s about driving innovation in AI and HPC,” Joseph said. “It empowers organizations to store more data, run diverse workloads, and achieve unprecedented performance.”

So what’s the TechArena take? ASUS and Solidigm’s collaboration underscores the importance of leading-edge storage technologies in unlocking the full potential of AI applications. High-density SSDs represent an important advancement, enabling faster, smarter, and more sustainable computing.

As 2024 draws to a close, I began to reflect both on the year behind us and the transformative changes ahead. Below are my five major predictions for AI in 2025.

Prediction 1: AI is going to move past simply using previous linguistic patterns to generate text and will begin to use a framework for associating facts and knowledge with linguistic patterns. Today, AI talks a good game, but doesn’t know anything. I predict that this will change in 2025, probably as the result of a marriage between knowledge graphs and LLMs. It will require a new architecture to make it work, so it's not as simple as combining knowledge graphs and LLM in a graphRAG type of application. 

Prediction 2: 2025 will be the year of the Chief AI Officer. Enterprises large and small will hire someone responsible for ensuring that AI projects make sense, that security and regulatory structures are in place, and that use guidelines are published and enforced. Chief AI Officers will be responsible for minimizing the risk of organizational damage caused by hallucination and will need to ensure that enterprises are not investing in products that don't actually work. I recently heard a Chief AI Officer say that they are looking to block “fly-by-night” AI products.

Prediction 3: We will see a proliferation of locally-deployed small GenAI models along the lines of Llama 3 that can be tuned or pre-prompted for very specific use cases. These applications will be especially important for use cases that require access to highly sensitive data or secure mobility. We are seeing an increasing number of “LLMOps” products that address the need to seamlessly train, test, deploy, and update such local models. Infrastructure costs to deploy local models have come down, making the economics of local models more sensible, but the risk is that without an integrated LLMOps pipeline, locally deployed models could become hopelessly out of date every few months.

Prediction 4: Enterprises will move beyond simple AI guidelines to begin to deploy real regulatory and governance frameworks and tooling that allows them to monitor model performance, data quality, regulatory compliance, security, and network connectivity. Many of these monitoring activities on their AI infrastructure will be done by agents.

Prediction 5: Agentic AI, which I view as using an AI quarterback to help automatically pick which specific AI tool or service is best for each enterprise application, will begin to become more commonplace. This is a natural evolution of the current practice of using ChatGPT, Claude, Gemini, Co-pilot and other tools to generate content and solve problems, and then comparing the quality and validity of each before choosing which output to use.

“The future bears a resemblance to the past, only more so.”

- Faith Popcorn (of course)

‍

Out here in the mountains, the days – well, the days really aren’t. As the TV tunes to the morning market opening, some of us still can’t see if that lump in the back yard is a deer or coyote, requiring more attention than typical with the fuzzy creatures and morning routines. It’s great standing out here when the Fahrenheit scale is in the teens. We’re also realizing that our last calendar item of the day will end after twilight. Thus, Prediction 0000: I predict there will be no changes to Daylight Savings to give we Northerners a glimpse of the sun on either side of a workday and provide cheer to our lives.

As the days become shorter and the holidays near, it’s the season to posit about the year ahead, so here we go. The C-suites are restless. Execution has not been great in technology this year, even though the market has responded well. That pressure to keep the market momentum could be the final straw to improve the execution through observation. That will mean increases in costs in some areas, which will result in the push to decrease costs elsewhere. Thus:

Prediction 0001: The back-to-the-office push will ramp significantly, with many tech and tech-centered companies reverting to five days of cubicle labor. This will mirror the general global industry. Time to wear shoes, folks. Additionally, there will be a frankly unsurprising lack of movement of the employee base who have been threatening to switch jobs for a couple years if they have to go back. 

Prediction 0010: The general tech industry trend will be to do more with less. Lack-of-skilled-workforce claims aside, the industry will contract globally with the influx of graduating and new employees, displacing significant experience that is still slow to adapt to new methodologies. We, the former planners, will tell you that it’s not about the people, it’s about the people required to do the job.

Corollary to the above, not a prediction, just an observation: M&A in the tech industry will be very noisy. Second Corollary to the above, the vocal global momentum on a four-day work week will grow with no action.

Corollary that leads to an actual Prediction 0011: Global industry will see a strong resurgence in the workspace client market. Hey, The Management might be cheap at heart, but they know that cool office schwag gets people back to said offices. This is as much a software trend as it is a hardware one, created by new PCs and tablets, wireless infrastructure, collaboration tools, and the like. It will even extend to furniture and commercial real-estate pricing. New cubes and new whiteboards for the peeps! West Coast commute times will be hardest hit.

Oddly enough, however, this rising tide won’t lift all newer and heavier boats. Prediction 0100: By about mid-year, the market will hammer more than a few companies who admit that the monetization of their new AI services (is AIaaS in use yet? If not, consider it ™-d here) is significantly lower than expected. Basic economics applies: while demand is high, supply is, well, over-supplied. This won’t (yet) extend to the high-end hardware providers. It also will not decrease the momentum in software and services to create newer and shinier AI tools, perhaps to the derision of the analyst crowds. Hey, live by the, “We have AI!” die by the, “We have AI!”

Corollary Prediction 0101: The AI services that buck the trend will be client-based. This will create an opportunity for continued AI development on single-system images that will have a longer-term impact on hardware; a topic for a future predictions column. The real result will be a problem that will start, but not totally come to the fore next year. While clients individually will have some cool new search and usability tools, clients as a whole will have fragmented individual services. Productivity results will vary. Literally. No back-end services will resolve this next year, though bets are that there will be speculation on how to do so in those shiny new conference rooms.

A few more off the cuff:

Prediction 0110: Not only will there be more announced government investment in technology infrastructure globally, there will be SIGNIFICANTLY more announced government investment in technology infrastructure globally.

Prediction 0111: Tariffs will not have an impact on the movement of global technology, though some companies will probably try to blame them in an earnings report around Q4.

Prediction 1000: Hardware innovation in the datacenter will continue to be in a stall. The industry as a whole will admit it’s up against a wall that’s a conglomeration of power consumption, bandwidth limitations, delayed products, and the intransigence of one or two major players to participate in general solutions. It’ll get fixed, just not next year.

Prediction 1001: Noted above. There will be at least two positive M&A or fragmentation deals in the tech industry next year. We mean big ones, in the multi-billions.

Prediction 1010: There will also be at least one pretty grim security breach that will create a good couple months of debate on whether the as-a-service industry has fundamental flaws.

Prediction 1011: We will review all of these predictions next year. Yer’ Humble Author (YHA) is not afraid of admitting mistakes. 

And proof that this entire column was conceived to deliver a geek joke: There was no way we were going to take this to 1111 predictions. That would give this column an F. Have a good set of holidays, everyone.

In the world of artificial intelligence (AI), the quest to recognize value upside can be compared to the wide divide between elite athletes and basic fitness for the masses. Just as the most elite athletes work tirelessly to improve their physical strength, stamina, speed and mental game, the most elite AI researchers and data scientists are pushing the limits on super intelligence.

While the race for Artificial General Intelligence (AGI) might be akin to breaking world records in an Olympic event, the real transformation in AI is happening incrementally—much like the steady progress made by weekend warrior athletes striving to maintain their physical fitness. In 2025, applied AI – the kind that has the potential to return more value than is spent in achieving outcomes from it will resemble a regular fitness tune-up.

Key Trends Shaping the Future of AI

As we look ahead, the AI landscape is evolving rapidly, with several emerging trends that promise to reshape the industry. Much like fitness routines that adapt to new goals, making progress in the AI journey requires effort in data strategies, new business models, and evolving user experiences. Here are three key trends to watch for in 2025:

1. Data Structure Becoming Crucial

Much like how ‘great abs begin in the kitchen’ rather than through long, daily ab workouts, great AI results start with high-quality, well-structured data. With people, as with AI, it’s a matter of garbage in, garbage out. In 2025, the importance of data management, security, and governance will be more critical than ever. A key learning from listening to industry leaders at the recent AI Summit NYC is that customization to your business workflows using your data drives the specific results you need. Don’t be fearful of the tuning necessary to extract the best of what AI offers your enterprise – think of it as the normal process to integrate other more mature enterprise software platforms such as Salesforce.

This is no small challenge as there are no automatic tools that simply fix years or decades of neglect – whether on our eating habits or on enterprise data structures and silos. The upfront investment is an absolute must however. According to a blog from Pure Storage, data bloat, mismatched formatting and incompatible structures can result in both a reduced return from the investment in new AI platforms, and may balloon costs if a usage-based provider is used for AI training or inferencing.

2. Business Model Innovations – Freemium Tools on Open-Source Models

The biggest challenge I have heard from IT Directors trying to figure out how to offer AI platforms and services to their internal customers is that commercial off-the-shelf AI platforms with upfront fees come with the disclaimer “…you have to work with it to find the value to your enterprise.” Because a specific enterprise has its own data and data structures, AI platforms that rely on unique data require self-assembly, self-discovery and experimentation to find the nuggets of value for a specific business. While assembling a program requires significantly less time for an AI-partnered software expert, the majority of users may lack interest or incentive to go through the process of self-assembly just to discover if AI has value in a subset of their specific workflows. The upfront cost of entry for commercial AI platforms combined with the vague upsides a user must discover on their own through trial and error will limit the deployments to early adopters and enthusiasts if it continues.

As a result, the introduction of more freemium business models will ensure that there isn’t an adoption stall from the upfront cost of off the shelf tools combined with the treasure hunt required to find clear and specific business value.

3. The “Pixar effect”

There is a lot of debate about what creative industries AI could destroy and, at the same time, more content consumers are resisting and pushing back on AI-generated content. Whether videos, automated comments on social media platforms that reward ‘clicks’, or even the first two drafts of this blog – there is a spark missing and difficulty connecting to concepts that present like a chatbot reading a manual back to us.

When Pixar first began using computing technology to push the bounds of animation, many believed the art that previously underpinned animated film making was dead. Pixar’s end product and emphasis on story crafting, using technology to rapidly iterate on the art of what was visually possible changed animated film making disruptively. Like the technology Pixar relied on, AI amplifies what’s already there offering insight potential from underutilized data to do more, better.

Applying AI will only amplify what’s already there – great insights, great products, robust data. Don’t lose the importance of improving your differentiation, and weighing the potential of AI against staying true to your core mission in your business.

References:

• Pure Storage (2024). “Improve Data Hygiene, Overcome the AI “GIGO” Problem”

‍

Explore how OCP’s Composable Memory Systems group tackles AI-driven challenges in memory bandwidth, latency, and scalability to optimize performance across modern data centers.

Since today’s modern car is a data center on wheels, it should come as no surprise that the neural networks that appear to be gaining all the attention these days are also deployed in the automobile, and have been for some time now.

In a previous blog titled, “Why a Self-Driving Car Might Run a “STOB” Sign: To ViT or Not To ViT,” I wrote about some of the different classes of neural networks addressing vision processing. In this blog, I’m going to discuss another class of neural networks employed in the vehicle, referred to as Large Language Models (LLMs).

As a brief genealogy, an LLM is a type of neural network referred to as a transformer model, which falls under the larger generative AI umbrella. Keeping up with the latest developments in these models is difficult, as the number of neural networks being introduced into the public domain is growing at an explosive rate. As a proxy for this growth, over 100 machine learning (ML) white papers are being published daily.

LLMs used for natural language processing (NLP) most likely first caught the awareness of the broader population with Apple’s introduction of Siri. Suddenly, the futuristic concept of computers recognizing speech had gone from being an idea right out of Star Trek to a hand-held reality. And while these initial versions still had room to improve, Siri’s capabilities were nonetheless quite impressive. From personal experience using an early speech-to-text program after a bicycle accident left me unable to type for several months, the improvements were profound.

More recently, the release and growth of ChatGPT, which provides an accessible frontend for various OpenAI LLMs, has marked the next major milestone in raising the broader population’s understanding of the power, potential, and capabilities of AI. This, in turn, has caused a frenzied uptick in the pace of AI research, both in areas of neural networks and semiconductor architectures that improve power efficiency and performance.  It has also given rise to the explosive growth of AI in the data center and the “AI bubble” that we are currently experiencing.  The power of AI, and of LLMs, in particular, is so profound that governments are aggressively evaluating different approaches to potentially manage and regulate this technology.

So how does this all fit into the automobile? NLP had its humble beginnings a few decades ago with the introduction of simple keyword recognition, that when properly enunciated, would typically allow for control over the body control or stereo system.  Similar to my early experiences with speech-to-text processing, the capabilities were clumsy, yielded inconsistent performance, and at some point, it was easier to simply adjust the car’s thermostat manually vs. continuing to repeat keywords over and over while waiting for a response.

Fast forward to today, where NLP in the car is on par in capability with Alexa-enabled devices. In fact, Alexa is finding its way into the car, as there is a land grab underway to capture as much information regarding user behavior as possible. Beyond Alexa, multiple other digital assistants are typically resident in the high-end vehicles, allowing for voice control over the vehicle, searching the user manual, in addition to voice-controlled navigation, etc. Despite all this original equipment manufacturer (OEM) attention to NLP, however, it’s been shown that these features don’t see much use today.

One of the greatest challenges that manufacturers face in deploying NLP to accomplish more useful tasks stems from onboard compute limitations. The compute demands for NLP are so significant that the data is generally sent to the cloud for processing, avoiding the need for very high-performance and power-hungry AI processors to reside in the vehicle. All is well when connectivity to the cloud can be guaranteed, but when driving in remote locations where cell service may be spotty or non-existent, the NLP will fail. Hence, there is a race underway to design more power-efficient compute architectures that can harness local compute to run LLMs.

While NLP accounts for the majority of the use cases for LLMs in today’s automobile, there are other key areas where they will soon be employed to take on more significant tasks that will yield greater value than voice control over the car’s stereo or heater.  Through the combination of computer vision and other AI technologies, LLMs can be used for scene understanding, which can lead to greater levels of driver confidence and safety in ADAS systems. This is similar to more advanced adaptive cruise controls that employ the heads-up display to communicate with the driver that the cruise control does indeed “see” the car that it is following and will keep a safe distance. It does this by providing insights through image captioning on a central display so the driver can have confidence that the vehicle understands the surroundings and is explaining the rationale behind a given action that is going to take place.

This is typically referred to as contextual analysis, and the concept is similar to the way a human typically takes in the entire environment, pedestrians, street signs, road construction, road hazards, etc., and takes action accordingly. Contextual analysis, in conjunction with NLP, will also allow for more human interaction with the vehicle – such as, “What was that building we just passed,” or, “What brand of car is in front of us?” The driver could then ask “How much does this car cost, and where can I purchase it?”  (That’s Alexa’s fantasy, anyway.) I’m sure you get the picture by now (pun intended).

Contextual awareness can also enable the vehicle to more closely mimic a driver’s behavior, showing caution when entering an intersection that has a dense population of pedestrians, despite having the full right-of-way. And while V2X (vehicle-to-vehicle communication) offers the promise of understanding traffic ahead etc., to date, deployment has been spotty. With contextual awareness, the driver can be alerted that there is a traffic jam ahead. When V2X is fully deployed, it is not only belt-and-suspenders, but it can also “see traffic” in zero visibility. Here again, if the driver is suddenly experiencing extreme deceleration with no understanding as to why, contextual awareness, in conjunction with V2X, can communicate to the driver that there is a pile-up ahead and that’s why the brakes are being aggressively applied.

So will LLMs end up in the car? I believe so. I also believe that, as they become more integral to the safety of the vehicle, compute architectures that can deliver the requisite performance in vanishingly small power envelopes will come to market to address this need – necessity being the mother of invention.

For now, relying on a connection to the cloud with unpredictable, spotty coverage and indeterminate latency for such an integral function will be a show-stopper. Power is a topic that the AI community, in general, likes to avoid talking about, but it’s currently an “inconvenient truth.” 

Based upon current AI compute architectures, GPT-3, which is based on an LLM, requires roughly 1,300 megawatt-hours of electricity for training, and a single query is estimated to consume almost 3 watt-hours of energy. These are astonishingly high levels of energy and hence why data centers are now being considered to be immersed in the SF bay to secure a low-cost means of cooling and the 3-mile island nuclear power plant is being spun up to power a data center. As is happening in the memory industry, it is said that automotive is now the technology driver for memory, it can well be the same for future AI compute architectures.

The incoming U.S. administration’s position on climate change, regulation, and Environmental, Social and Governance (ESG) disclosures is poised to wreak havoc on environmental progress. The potential rollback of US-based commitments may force multinational corporations (MNCs) to navigate a complex regulatory landscape, balancing compliance in international markets while avoiding lawsuits from boardroom activists, shareholders, and domestic consumers. This "Not In My Back Yard" (NIMBY) mentality could lead to a fragmented approach to emissions regulations, further complicating the path to sustainability.

Following are three things I see on the horizon for ESG and sustainability in 2025.

Prediction 1) Innovation At Risk

The confusion of the new regulatory environment promises to stifle ESG software investment, innovation and strategic flexibility – with a potential domino effect to follow: Companies would opt to redraw their plans at the expense of pursuing innovative sustainability initiatives that fit better within the prescribed frameworks. This would result in a more rigid approach to sustainability, where businesses prioritize regulatory compliance over genuine environmental and social impact. And ultimately, progress would slow down in corporate sustainability efforts, as businesses become more conservative in their ESG communications and disclosures.

Prediction 2) Evolving ESG Efforts & the Role of AI

Those who take on ESG efforts without having boardroom and executive sponsorship risk failure, simply because ESG reporting can be very disruptive to a company’s workflow across all departments, including externally with suppliers and customers. Company-wide support and engagement is crucial.

However, companies who opt to integrate artificial intelligence (AI) with advanced technologies in their workflows have an opportunity to revolutionize data collection, analysis and governance to create true business value. Automation will be a driver for data gathering from a wide range of sources; improving the accuracy of ESG metrics will result in more reliable and timely reporting. Using AI to follow the ‘digital thread’ to enhance end-to-end traceability and auditability of ESG data will create a fast ROI.

Prediction 3) Companies to Expand Data Science, Analytics Requirements

As ESG culture becomes mainstream for companies, scalability and flexibility should become table stakes for any ESG reporting. This will become critical given the diversity and large volumes of ESG data to be able to identify both sustainability and business trends, patterns, insights and opportunities. AI’s ability to apply advanced analytics and insights will become a daily outcome. This will drive companies to expand their data science and analytics requirements leading to additional staffing skills, software innovation and services opportunities.

Looking Forward

MNCs will still need an ESG disclosures strategy that is truly global. While they will still invest in collecting emissions data, they should also focus on planning to implement a double materiality assessment (DMA). The outcome of a DMA is a list of material important sustainability-related Impacts, Risks, and Opportunities (IROs). These IROs then determine the specific disclosures and data points the company must include in its final sustainability report.

AI and advanced technologies for ESG should be in every CEO’s top 10 strategic goals. Investment in building out the company’s AI roadmap with ESG as a leading workload will release unknown business outcomes and accelerate business process re-engineering. Companies who adopt AI into ESG processes will also become more efficient at producing their goods and services, be more competitive than their peers and be more sustainable – something that will be evident in their ESG reporting.

What if your biggest business challenges could be solved before you even identified them? That’s the potential of Artificial Intelligence (AI). The enterprise world is beginning to accept AI as more than a buzzword. They can see value in harnessing their data to solve some of their biggest business problems.

One thing is for sure: data is the key ingredient that enables AI. The more data you can use, the better your insights will be. So where will we see innovations? Will the AI-washing fizzle out? And are we even thinking about this new technology in the right way?

1. We will continue to see infrastructure innovations that support AI.

The digital universe continues to grow at an astonishing pace, hitting an estimated 147 zettabytes by the end of 2024 (via Statista). For perspective, that’s 147 trillion gigabytes—a staggering amount of data. This explosion underscores the need for AI to process and create actionable insights from such a vast ocean of information.

Thanks to advancements in GPUs and faster CPUs, algorithms can now process enormous datasets in record time. At the same time, storage technologies—like higher-capacity flash drives—are enabling faster, more efficient consolidation of data, setting the stage for the next evolution in AI-driven insights.

These are exciting advancements, but infrastructure only exists to support the data. If we are creating information in new ways, will there be new ways to consume these insights?

2. Information delivery will be reimagined.

AI computing advancements such as transformer models have given us a glimpse of what is possible with large language models (LLMs). Om Malik, founder of GigaOM and long-time tech reporter/investor wrote an article about how AI will change the browser as we know it.

He points us in the right direction about what information delivery will look like in the future:

Malik envisions a future where AI doesn’t just passively gather data—it actively connects the dots, delivering insights seamlessly. It’s easy to see this trend developing in tools we already use today. He gives an example of having a DietBot to deliver a data stream customized to you that includes an analysis of your eating patterns, health goals, and dietary requirements that match restaurants in real time.

My Samsung watch does a lot of that now, but I have to manually provide much of the information. I have to track my meals and water intake with a different app (Yazio) that I connected to my watch’s app. I weigh in with a digital scale that I also connected to the watch app. The watch can calculate my steps and heart rate.

But my blood pressure monitor can’t connect to my watch, so I have to enter that reading manually. It also can’t connect to my Kardia (used to check for atrial fibrillation). How cool would it be to have a DietBot to do all the work for me? Even better – what if I could tap my watch to my doctor’s tablet and she could get all my latest readings? What if she could subscribe to my data stream when I felt sick, for a better diagnosis?

That’s what is coming – data streams we’ll need to subscribe to in some way. The question becomes how will we consume it? Will it be a new type of browser, an app on your phone (or watch, or TV)? I think we’ll see innovation in how we consume these data streams in 2025.

3. AI hustlers will be everywhere.

With the AI market predicted to hit $1 trillion by 2027, it’s no wonder the bad actors are beginning to circle like sharks.

AI-washing, where companies exaggerate the AI capabilities of their products, is already rampant. Maybe you’ve seen an “AI” version of a product that has merely added a chatbot. The Federal Trade Commission (FTC) is cracking down on exaggerated AI claims, such as the infamous 'world’s first robot lawyer,' which failed to deliver on its promises—mainly because it lacked real lawyers!

Cybercriminals aren’t far behind, using generative AI to launch scams that are more sophisticated than ever. Ransomware gangs are already on the AI bandwagon, probably looking for better ways to a big payout (up to 75% of ransom payments are over $1 million).

AI has lowered the barrier to entry to new hackers. Generative AI has helped these bad actors create more believable text, images, audio, and video to hook their victims. Once in, they are able to pinpoint the high-value data for exfiltration (and higher ransoms).

The bad guys are usually first adopters of new technology. And with so much money on the line, they are only going to double down on their AI efforts.

Adapting to the Changing AI Landscape

The basics of data management are not going to change. The Association for Computing Machinery (ACM) lays out the 7 Principles of an Organizational Data Strategy as governance, data content, data quality, data access, data management, data informed decision making, and analytics.

These steps won’t change because AI processes are using the data. But it will be easy to forget the things we learned as we get caught up in the excitement of new technologies to deploy. So remember your training!

The AI revolution is here, and it’s changing the way we work, think, and interact with information. Living through technological changes can be really exciting. We are bearing witness to new technologies that are driving an information revolution. The best way to enjoy this ride is to learn more about these technologies and explore new ways to consume data streams.

After all, the ride is only beginning.

Got ideas for how AI will reshape the way we consume data? Are you building one? Join the conversation on the Tech Aunties podcast—reach out on LinkedIn!

When you think of Gigabyte, gaming hardware probably comes to mind. But this Taiwan-based computer hardware manufacturer develops much more than motherboards and graphics cards – they provide a spectrum of computer hardware as well as liquid and immersion cooling and have a long history of contributing to open standards and advancing server technologies.

I recently had the pleasure of chatting with Chen Lee, VP of Sales, HPC, Data Center and Enterprise for Giga Computing and learned a fascinating tidbit about how the company became involved in the Open Compute Project Foundation (OCP).

“Around 2004, this very little-known company came to us and said, ‘We’ve got a search engine, and we want to build this motherboard and this thing called OpenRack,’” Chen explained.

The little-known company was Google, he said.

“So that's how we got into OCP,” Chen said. “(Gigabyte was) actually the first company to help Google develop open compute.”

Gigabyte’s collaboration with Google on OpenRack marked the company’s entry into the open infrastructure movement, making them one of the initial contributors to OCP standards.

Today, Gigabyte’s portfolio extends beyond Intel and AMD servers — they also produce Arm-based solutions using Ampere technology and specialize in advanced cooling systems like immersion and direct-to-chip liquid cooling. With this holistic approach, they continue to drive efficiency and performance in the data center space, reflecting their adaptability and forward-thinking approach.

Embracing AI: Gigabyte's Focus on GPU Servers

Artificial Intelligence (AI) has reshaped the demands on data centers, particularly in terms of computing power and infrastructure. Chen discussed how Gigabyte has been positioning itself in the AI hardware game, particularly through high-density GPU servers. He shared a pivotal moment for Gigabyte in 2010 when they introduced a 2U server that could support eight double-wide, dual-link GPUs, which at the time was the highest density on the market.

Today, Gigabyte’s expertise in GPU servers continues to be an asset, providing systems for AI model training and inferencing, using cutting-edge GPUs like Nvidia’s H100 and soon, Blackwell. As AI shifts towards edge deployments, Gigabyte is also preparing for the growing importance of edge inferencing, which Chen predicts will be a significant area of growth in the near future. Industries such as medical, finance, and retail are moving fast to adopt AI solutions at the edge, from convenience store smart shelving to real-time customer analytics. Gigabyte is ready to meet these needs with high-performance, scalable server technology that suits the unique challenges of edge computing.

Liquid Cooling and Efficiency in Data Centers

The demand for powerful servers to support AI training and inferencing has pushed energy consumption to unprecedented levels, making cooling a top priority. Chen highlighted how immersion and direct liquid cooling are allowing Gigabyte to manage energy efficiency better while meeting the needs of customers working on advanced AI projects. It’s a testament to the company’s adaptability and focus on sustainable solutions—aligning well with the OCP’s values of open innovation and energy efficiency.

AI Beyond the Data Center: The Future of Inference

Chen and I also discussed moving from centralized data center training to inferencing at the edge. Today, most inferencing still happens within large data centers, using high-power systems designed for training. But Chen believes that as AI technologies mature, edge inferencing will become critical—allowing smaller, more efficient hardware to perform tasks where the data is generated, such as in retail stores, hospitals, and banks.

Chen shared an interesting example involving a convenience store, where AI systems can detect customer behavior in real-time and use edge servers tucked away in the back to provide analytics directly to the headquarters. The potential for rapid, on-site AI-driven insights will push industries to adopt smaller-scale AI inferencing solutions—a market that Gigabyte is well-positioned to serve.

This shift to the edge will transform how AI is implemented across industries, bringing smarter technology closer to users and changing how data centers interact with local environments. Chen also shared that, in his view, AI isn’t just a passing trend—it’s a new wave that’s here to stay.

So what’s the TechArena take? As AI evolves and the infrastructure to support it becomes more advanced, hats off to Gigabyte for doubling down on its strengths—high-performance GPU servers, innovative cooling technologies, and partnerships with leading hardware and storage vendors.

Thanks to Solidigm for sponsoring this delightful Data Insights discussion. In case you missed it, check out the full episode here. As AI and edge computing continue to advance, the innovations coming from companies like Gigabyte are paving the way for the data centers of tomorrow.

Explore how Accenture and Oracle are revolutionizing the telco industry with their new all-in-one BSS solution, enabling cloud-native innovation, AI integration, and new revenue opportunities for service providers.