Question 1

Do enterprises actually use LLM benchmarks to choose a model?

Accepted Answer

Rarely as the deciding factor. Among the 93 leaders who described where benchmarks sat in their last model decision, 44% called them background context or ignored them and 41% said real-workload testing was the decisive evidence. Only 15% raised vendor claims failing in production as a benchmark-related concern. Asked how much benchmark discourse helps, 55% of 92 said it has limited or background value and 42% prefer pilots and direct testing.

Question 2

Are DeepSeek and other non-US LLMs replacing US models in enterprises?

Accepted Answer

No, not in this sample. 75% of 101 leaders said non-US models such as DeepSeek, Kimi and Qwen are excluded or have never been tested in their organization, and 93% have limited or no comparative production evidence. Only 27 of the 102 interviewed have run one on a real workload. Where they did, 5% found lower cost and adequate performance on simple workloads and 2% preferred US frontier models for complex, consistent work.

Question 3

How do enterprises evaluate LLMs before buying?

Accepted Answer

By testing them on their own workloads. 61% of 100 leaders are involved in hands-on model evaluation and selection, and 41% of those who discussed evidence named real-workload testing as decisive. The criteria weighed most were task capability, output quality and accuracy (42% of 96), then security, privacy and enterprise integration (29%) and cost, speed and hands-on workflow testing (29%). Security and data exposure was the dealbreaker for 59% of the 75 who named one.

Question 4

What percentage of companies use more than one LLM provider?

Accepted Answer

63% of 101 enterprises interviewed run multi-provider environments spanning several frontier labs. A further 20% are actively evaluating Claude, Gemini, open-weight or embedded tools alongside their current provider, and 17% run OpenAI or Microsoft Copilot alone. Routing between providers is manual for 80% of the 84 who described it, and 46% of 72 are building for portability rather than leaning on one provider.

Question 5

Is cost per token the right way to measure LLM spend?

Accepted Answer

Buyers say no, but few have replaced it. 60% of 98 leaders compare models on downstream ROI and labor savings rather than sticker price per million tokens, and only 23% track aggregate spend and token efficiency. Yet 53% of 96 have no formal cost-per-outcome metric; 29% measure per case, ticket, document or task and 18% count time saved. Budget owners are twice as likely as hands-on evaluators to have a unit (50% of 18 against 25% of 61).

Question 6

Why do companies switch LLM providers?

Accepted Answer

Performance first, cost second. Among the 84 leaders who described their last model change, 57% said it was driven by performance or capability, 27% by an IT, engineering, business or executive decision, and 15% by cost, pricing or redundant spend. Of the 49 who weighed switching overhead, 55% said meaningful performance, cost or security gains justify a switch, while 33% said incumbent ecosystems, contracts and licensing keep them where they are.

Question 7

What blocks enterprises from adopting a new LLM?

Accepted Answer

Security and data exposure. 59% of the 75 leaders who named a near-dealbreaker cited security, privacy or data-exposure risk, ahead of failure on real-world task quality (28%) and unfavorable pricing (13%). Day to day, 43% of 98 said access, permissions and sensitive-data governance is the main work of running a model, and security, privacy, legal and compliance teams hold the veto for 44% of 77.

Question 8

Do LLM updates break production systems?

Accepted Answer

Sometimes, and quietly. 40% of 97 leaders said their last bad surprise in production was an output regression, hallucination or silent model change, and 34% cited outages, integration failures or unexpected cost spikes; 26% reported no material incident. Asked directly about provider updates, 21% of 78 said silent updates or version changes caused regressions, 72% reported no material incident, and 8% have testing, monitoring, rollback or fallback protections in place.

Make Your Category the Next G2 Report.

Not Benchmarks, Not Tokens: Enterprise LLM Buying 2026

The Connected Mandate 2026: UK AI Readiness Gap

State of AEO 2026: AI Outpaces Measurement

Make Your Category the Next G2 Report.

Keep Reading

Not Benchmarks, Not Tokens: Enterprise LLM Buying 2026

The Connected Mandate 2026: UK AI Readiness Gap

State of AEO 2026: AI Outpaces Measurement