AI Strategy & Business Case

Controlling AI Costs: Licenses, Usage, Infrastructure

Marin Niehues14 April 202512 min read

Taking notes on a pad next to a laptop

AI costs become controllable as soon as every cost type has a metric and an accountable owner. Spending arises in three places: as licenses for assistants, as usage-based billing for models by token and as infrastructure for proprietary applications. Each follows its own cost logic and needs its own control loop. This article works through what a request to a language model costs, why costs rise after the pilot and which control model lets executive management and business-unit heads run all three in one portfolio.

Why AI costs behave differently from traditional IT costs

Traditional enterprise software follows the contract: a number of licenses is negotiated, an annual fee is paid, and operations run. With AI, the moment of decision separates from the moment costs arise. A business unit enables an assistant, and whether the seats create value is then determined by the behavior of hundreds of employees; under consumption-based models that behavior also determines the invoice. An integration into a specialist system sends requests to a model for every transaction, and billing follows the volume of text processed. A prototype runs on a platform that has to be sized differently in regular operation.

This places a requirement on control. It has to start at the source of the costs, before the monthly invoice arrives. Finance sees the invoices, the business unit sees the value and IT sees the capacity. Without a shared view, savings land in the wrong place: canceled licenses endanger a well-adopted tool while unused seats keep running elsewhere. Where this control sits within AI management as a whole is set out in the article AI management: what executive leadership must steer.

Licenses: allocate seats by value

Licenses are the most visible cost block and the easiest to plan. They are billed per seat, and costs grow with every activation. The control problem lies in the gap between paid and used seats. A staged procedure has proved its worth: a defined pilot group first, with usage measured weekly, then roles with documented value. Seats without usage over an agreed period are withdrawn and reassigned.

The decisive element is the metric. The number of seats issued says little. The number of actively used seats per role says more, and the share of usage that leads to a measurable work step says most. Contract terms and notice periods determine how quickly the organization can react to these figures, which is why they belong in the negotiation.

How much does a request to a language model cost?

A single request to a language model costs between a fraction of a cent and a few cents, depending on the model and the amount of text; the expense comes from volume. Billing is by token, that is by the amount of text processed, with input and output priced separately and output costing a multiple of input. A token is a fragment of text that the model processes as one unit. In English a token corresponds to roughly four characters or three quarters of a word; languages with long compound words, such as German, generally need more tokens per word.

List prices are quoted per million tokens. Between small models and frontier models lie roughly two orders of magnitude: small models charge cents per million input tokens, while output from frontier models costs double-digit dollar amounts. This spread is the single most important lever in cost control.

Exhibit 1 · Schematic

The choice of model changes the cost by more than an order of magnitude

The choice of model changes the cost by more than an order of magnitudeThe choice of model changes the cost by more than an order of magnitude. Bar chart. 3 values. Highest: Mid-range model, 3,000 input tokens (around €1,650). Lowest: Small model (around €75). Schematic illustration, not measured data.Mid-range model, 3,000 input tokensaround €1,650Mid-range model, shortened contextaround €1,050Small modelaround €75The choice of model changes the cost by more than an order of magnitudeThe choice of model changes the cost by more than an order of magnitude. Bar chart. 3 values. Highest: Mid-range model, 3,000 input tokens (around €1,650). Lowest: Small model (around €75). Schematic illustration, not measured data.Mid-range model, 3,000 input tokensaround €1,650Mid-range model, shortened contextaround €1,050Small modelaround €75
Schematic illustration, not measured data.
Source: worked example in the article
Note: assumed values from the article
Data and download
Download
Data as table
LabelValue
Mid-range model, 3,000 input tokensaround €1,650
Mid-range model, shortened contextaround €1,050
Small modelaround €75

Graphic and data: Niehues Consulting, CC BY 4.0. Free to use with attribution and a link.

The worked example shows where control applies. The cost of an application depends on four variables: the number of transactions, the volume per transaction, the choice of model and the share of repeated requests. Each of these can be shaped, and the choice of model moves the result by more than an order of magnitude.

Why do AI costs rise so sharply after the pilot?

AI costs rise after the pilot because several drivers multiply at once: users, transactions per user, text per transaction and model calls per transaction. A pilot with twenty users and selected cases captures none of these drivers in full.

Five patterns recur in the transition to regular operation. First, the context grows: short test questions become complete case files, histories and attachments. Second, calls per transaction increase as soon as an application acts as an agent that executes several steps itself, queries tools and checks interim results. Third, repetitions are added, through failed attempts, users rephrasing their requests or automatic review loops. Fourth, line items appear that were absent in the pilot, such as monitoring and logging or quality checks with model calls of their own. Fifth, the pilot often stays on the most expensive model because that is where quality was reached fastest.

Anyone who measures cost per transaction during the pilot and projects it onto the target volume sees the jump before it appears on the invoice.

How can the cost of language models be reduced?

The cost of language models falls most through model routing and context discipline, supplemented by prompt caching. Each lever acts on a variable of the worked example.

Model routing directs every request to the smallest model that meets the quality requirement. Simple classification or extraction tasks do not need a frontier model. An upstream rule or a small model decides which requests are passed on; in a cascade, the larger model only handles the cases the smaller one cannot solve reliably. A study from Stanford University shows that such a cascade, on the tasks examined, matches the performance of the best individual model at up to 98 percent lower cost.

Context discipline means that anyone who sends entire documents where an excerpt suffices pays for every superfluous page. A targeted search for the relevant passages, as retrieval-augmented generation provides, shortens the input and improves accuracy at the same time.

Prompt caching stores recurring parts of an input with the provider, such as long instructions or reference documents that stay the same in every transaction. The major providers bill cached input at substantial discounts, between half and nine tenths of the regular input price depending on the provider. The lever takes effect as soon as more than half of the input is identical across many transactions.

In addition, every application receives a budget limit with a warning threshold, so that a faulty workflow cannot consume a monthly budget overnight. Ongoing measurement belongs to the operation of the application.

Infrastructure: consolidate line items per application

Infrastructure costs arise where organizations run their own models, prepare data for applications or deploy applications through cloud services. They are less visible because they are spread across several cost centers. Compute capacity sits in IT's cloud account, storage and data pipelines sit in the data function, and operating effort sits in the application team. A reliable picture emerges only when these items converge per application. Consistent tagging of resources by application and a monthly consolidation with the usage costs are sufficient for this.

A question of principle comes first: self-hosting or consumption as a service. Self-hosting exchanges variable costs for fixed costs and operating responsibility, and it pays off at consistently high utilization. Consumption as a service keeps costs variable and makes usage control all the more important. As long as an application has not reached its target volume, consumption as a service is the more economical choice, because unused compute capacity is paid for in full under self-hosting.

A control model for the AI portfolio

A control model for total AI spending consists of four elements. The first is a cost model per application that relates license, usage and infrastructure to one common unit, such as cost per transaction processed. The second is accountability: every application receives a named product owner who answers for costs and value together. The third is a monthly rhythm in which executive management sees the portfolio metrics. The fourth is a decision rule for scaling and shutdown: an application grows when the cost per result falls and adoption rises, and it is reviewed as soon as either fails to materialize. The model follows the FinOps approach for cloud spending, with one addition for AI: the cost per transaction changes with every choice of model.

Niehues Consulting as a partner for cost control

Niehues Consulting supports organizations in building this control from a delivery perspective. The basis is the Niehues Delivery Model, with three phases and fixed review questions. Three of them matter most for cost control: What is the baseline, and how do we measure success? Is the business case confirmed with real values? Which metric shows after 90 days that the solution is landing?

In AI management, the business case and the prioritization of applications are developed first, followed by control of the portfolio through metrics. Niehues Consulting moves prototypes into operation in such a way that cost per transaction is recorded from day one and the value is demonstrable in the real workflow. The choice of model is tested for both cost and quality along the way. As delivery manager or product owner, Niehues Consulting takes over the control itself on request. The entry point is a free initial consultation of 30 minutes.

Frequently asked questions

What is a token and how much does it cost?

A token is the billing unit of language models, a fragment of text of roughly four characters in English. Prices are quoted per million tokens, separately for input and output. Small models cost cents per million tokens, while output from frontier models costs double-digit dollar amounts.

How can AI costs be reduced without losing value?

AI costs fall without loss of value when the cost per application is made visible first and targeted levers follow: unused licenses are withdrawn, and requests are routed to cheaper models. Costs are reduced where the cost per result exceeds the value. Applications with documented value receive more capacity instead.

What is prompt caching and how much does it save?

Prompt caching stores unchanging parts of an input with the provider so that follow-up requests are billed at a lower rate. The major providers grant discounts of between half and nine tenths of the input price on cached input. The overall saving follows from the share of recurring input per transaction.

What is model routing?

Model routing distributes requests across different models according to difficulty and quality requirement. Simple requests go to small, inexpensive models and only difficult ones to frontier models. With a price gap of roughly two orders of magnitude between model classes, it is the most effective single cost lever.

What is FinOps for AI?

FinOps describes the practice of managing cloud spending jointly across engineering, finance and the business according to its business value. For AI the approach carries over to model calls and licenses: spending is made visible per application, assigned to an accountable owner and reviewed regularly against the value delivered.

How do you budget for AI applications?

The cost of an AI application is planned with a cost model that maps transaction volume and tokens per transaction for the chosen model, supplemented by a budget limit with a warning threshold. The values come from measurement in the pilot, projected onto the target volume. The model is adjusted each quarter against measured values.

Let’s talk about your initiative.

In 30 minutes we clarify where your organization stands and which step will pay off most now.

Request a consultation

30 minutes · free · no obligation