From Experimenting to Measuring: Intel's Lynn Comp on AI

The AI Infra Summit is just around the corner, and TechArena is proud to return as a media sponsor. This summer, we've been trading notes with the companies building the AI stack, from silicon to systems, to hear where AI infrastructure requirements are really heading.

I had the pleasure of sitting down with Lynn Comp, Head of Global Sales and GTM for Intel's AI Center of Excellence. We covered what enterprises actually need from their infrastructure as they move from experimenting to measuring, why the real case for AI gets made on total cost and value per useful output rather than peak performance, and who answers for it when an AI system gets something wrong. Here's what I learned.

Q1: You’ve argued the hard problem in AI isn’t the model, it’s getting the same answer twice: determinism and time-based predictability. What does infrastructure have to deliver for an enterprise to trust agentic AI in production, and where does it fall short today?

A: Infrastructure needs to have very tight design constraints for a given AI based agent, avoiding the ability of an agent to exercise functionality beyond its intended purpose. Many of the exploits that have been uncovered come from an agent supporting prompts that go well beyond the intended purpose of the agent’s function. Simple, straightforward functional design and thinking of agent design like building an appliance helps keep the architecture clean, which then has the byproduct of being more deterministic and predictable.  

Q2: You spend your days with enterprises operationalizing AI. What do they most need from their infrastructure to make it pay off, and what has shifted this past year as they move from experimenting to measuring?

A: The baseline need is to have a data architecture that can support AI operations – both using business data as a baseline input as well as during AI-based workflows. The AI is generic until it applies your business information and context, but it is difficult to see ROI from AI that spends the majority of tokens on the data input processes rather than getting insights from the underlying data. Beyond that, the next most important question relates to enterprise governance since AI operations can change the company posture related to copyright, underwriting and regulatory compliance questions.

As companies shift from experimenting to measuring, I’m seeing harder questions being asked about private and hybrid AI as the true costs of frontier models hosted in hyperscaler data centers become more obvious. No CFO is happy when they are told “I don’t know how much budget I will need for AI Ops at scale, nor can I guarantee I can hit that budget” by their sysadmins and IT architects.

Q3: Much of the AI conversation fixates on the biggest, most expensive hardware. Where does the real case get made on total cost and value per useful output, rather than peak performance?

A: One of the hardest benefits to articulate and demonstrate on a balance sheet is better operational efficiency, since it’s not a hard cost you can show gains or reductions in a way similar to when purchasing hard goods or services. Because of that, buying new expensive hardware that has overhead just from having to deploy new cooling strategies is very difficult to pencil out against “went from 7 days to 5 minutes in developing dashboards that report on overall fleet health”. The current approach on explaining the value in that example is to claim fewer employees are required, which is counterbalanced by increasing token costs from suppliers.

I hear from multiple enterprises that they’ve applied the multi-cloud skills developed over the last 10 years to AI. They are learning to mix and match privately optimized AI and small language models that have been fine-tuned with judicious application of frontier models, hyperscaler services and neoclouds. For the health-meter dashboard example, prototyping is done on a frontier model and then the workflow is moved to a local SLM to improve operational economics.

Q4: The industry is moving from buying components to running one integrated machine. From where you sit with enterprises, is that the real shift, and what does it change about what they buy and how they operate?

A: The combinatorial explosion with agentic AI between discrete agents, frameworks, disparate API ecosystems and overlapping capabilities is resulting in security gaps, version conflict and unpredictable TCO. At the same time, market changes on the virtualization front are causing enterprise decision makers to be very cautious about anything that resembles vendor lock-in since even stable, trusted suppliers can suddenly be acquired by a larger company with very different customer management principles. I have already seen enterprises creating their own interfaces and allowing model selection under their internal APIs so model-independence is built in from the start, and I suspect similar architecture principles will be carried forward where there are important control points in an enterprises’ tech stack.

Q5: You’ve pointed out that most enterprises still can’t answer a basic question: when an AI system produces a harmful or biased result, who is accountable? What has to live in the infrastructure itself, the telemetry, the provenance, the controls, before anyone can answer that honestly, and how much are enterprises still taking on faith today?

A: The telemetry and the provenance for data as well as decision traceability have to be operational code running within the AI systems and the overall orchestration of the AI operations. Any decision that would be difficult to defend in a current regulatory framework needs to have a human in the loop, and at the same time, a human has to have the right insights support from the AI Operations so the human isn’t overwhelmed with overwhelming amounts of data in the process.