Enterprises Pivot to Open-Weight Models to Curb Escalating AI Token Costs

Enterprise artificial intelligence strategies are undergoing a structural shift as organizations move away from exclusive reliance on frontier models in favor of open-weight alternatives to manage spiraling operational costs. Sachin Chitturu, a partner and Southeast Asia leader at QuantumBlack, AI by McKinsey, highlighted this trend at Tech Week Singapore, noting that the rapid adoption of agentic systems has fundamentally altered the economics of large-scale model deployment.

While per-token pricing has dropped by approximately 90% since 2023, total AI expenditure continues to outpace corporate budgets. Gartner estimates that agentic models require five to 30 times more tokens per task than standard LLM interactions, effectively negating the benefits of falling unit costs. A recent survey on enterprise AI financial operations revealed that 93% of organizations have exceeded their allocated AI budgets within the last six months.

McKinsey’s latest global research indicates that one in five organizations has already restricted AI usage due to these mounting expenses. The technology sector reports the highest incidence of budget-related curtailment at 25%, closely followed by financial institutions at 24%. Despite widespread reports of improved individual productivity, only 37% of users claim that AI has contributed positively to their organization’s earnings before interest and taxes.

The current market reality suggests that frontier models from providers like OpenAI and Anthropic are increasingly reserved for high-order reasoning tasks where failure carries significant risk. For the vast majority of enterprise activities—which Chitturu characterizes as requiring little more than simple arithmetic or routine automation—organizations are moving toward open-weight models. These models can be downloaded, fine-tuned, and hosted in-house to provide predictable cost structures.

Engineers are now tasked with building sophisticated software harnesses that act as a dynamic routing layer around LLMs. This architecture allows companies to intelligently select between costly frontier APIs and cheaper, self-hosted open-weight models based on the specific requirements of each task. By implementing these harnesses, firms can maintain control over context windows and token expenditure without sacrificing performance on domain-specific workloads.

Self-hosting open-weight models introduces significant MLOps complexities that organizations are only beginning to address. Managing GPU cluster utilization requires precise orchestration to ensure that latency remains within acceptable bounds for real-time applications. Engineers must also account for the overhead of maintaining model weights, managing inference server configurations, and ensuring that local infrastructure can scale to meet spiky demand without incurring the high costs of over-provisioning hardware.

The transition toward open-weight architectures is supported by the narrowing performance gap between proprietary frontier models and their open counterparts. Chitturu noted that the time required for open-weight models to match frontier quality has compressed from years to mere months. This shift enables mature enterprises to run core, routine traffic on self-hosted infrastructure while reserving frontier APIs for complex, high-stakes reasoning.

The economic viability of this approach depends heavily on the volume of tasks and the engineering maturity of the organization. While self-hosting introduces fixed infrastructure and machine learning operations costs, these expenses are increasingly justified as organizations scale their AI footprint. High-performing firms are distinguishing themselves by fundamentally redesigning workflows rather than simply layering AI over existing processes.

The challenge for executive teams remains the lack of clear attribution metrics for AI investments. Many organizations struggle to track the tangible value generated by their platforms, leading to a focus on optimizing existing deployments rather than expanding into new, unproven use cases. Future success will likely depend on the ability to treat AI as a transformation of organizational structure rather than a standalone technology procurement.

Projections indicate that enterprise AI architecture will eventually stabilize with a maximum of 10% to 15% of workloads running on frontier models. The remaining 80% to 85% will migrate to open-weight solutions as enterprises prioritize cost-efficiency and data sovereignty. Success will require a dedicated think tank approach to continuously evaluate solution architecture as model capabilities and economic conditions evolve every few months.