AI market shifts from biggest models to cheaper, smarter systems and open-weight alternatives

The AI industry's scorecard is changing. After two years of chasing bigger models and better benchmarks, companies are now optimizing for routing, cost, control, and compute efficiency. Perplexity CEO Aravind Srinivas told CNBC this week: 'The model alone is no longer the product. It is the harness, the orchestration system that puts the model inside a very capable harness and pairs the model with a lot of tools.' Perplexity announced a system for its computer-use product built around GLM 5.2, an open-weight model from China's Z.ai, designed to route cheaper models for routine tasks and escalate only when needed.

Open-weight models are becoming more capable and far cheaper to run than proprietary frontier models. Benchmark general partner Peter Fenton said this week: 'A maybe contrarian view that is becoming consensus is our belief that 90-plus percent of the tokens created will come out of open-weight models over the next 18 to 24 months, possibly even by the end of the year.' Ollama, which helps developers download and run open models, claims adoption by 85% of the Fortune 500, including regulated industries like aviation, insurance, and health care. The rise of open models puts direct pressure on the inference margins of OpenAI, Anthropic, and other frontier labs.

Corporate AI spend is tightening as budgets mature and ROI pressure mounts. Companies are increasingly moving from experimentation to production, where cost per token becomes a first-class concern. The shift also fuels a strategic challenge for the U.S.: most competitive open-weight models now come from Chinese labs like Z.ai and DeepSeek. Srinivas argues the U.S. should support open models to make AI more affordable and accessible, but policy and national security concerns complicate that stance.

For architects: the shift from 'best model' to 'best model for the task' is reshaping infrastructure decisions. Heterogeneous stacks—mixing small tuned models with occasional calls to frontier models—are becoming standard. This creates demand for inference optimization, model routing, and local execution capabilities, potentially shifting capex from hyperscaler data centers toward edge and on-premise deployments. Open models also raise defensibility questions for labs betting on pricing power: as Fenton notes, 'when you can run those without the markup that they're providing, inference margins…are going to come under pressure.'