From Pilot to Scale: Embedding AI in Regular Operations

AI moves from pilot to regular operations when an owner in the business unit and a metric with a baseline are settled before rollout. Most pilots show under pilot conditions that the technology works. That no operation follows is mostly down to the organization around the application and its integration into the workflow, rarely to the model itself. This article describes five decisions that the executive team and divisional leadership take before scaling and gives benchmarks for duration and cost.
Why do AI pilots fail to reach production?
AI pilots fail at the transition because scaling removes exactly the special conditions under which the pilot worked. The scale of the problem is considerable. A RAND Corporation study cites estimates that more than 80 percent of AI projects fail, twice the rate of IT projects without AI. The study's interviews with experienced data scientists and engineers show that the most common causes lie outside the model: a poorly understood business problem and inadequate data. Just as often, the infrastructure for operation is missing.
A pilot runs under special conditions. A small group of selected users works with the application, the project team answers questions the same day, and the data has been prepared for the test. In regular operations each of these conditions disappears. Many users work with different levels of prior knowledge, the data arrives as the business units produce it today, and the project team has long since moved on to the next initiative.
The decisive question is therefore which of these conditions must be created permanently as part of the organization's structure. Anyone who asks it only after rollout ends up with an application that is formally in production and in practice bypassed. The causes in detail are covered in the article Why AI pilots do not make it into operation. This article concentrates on the decisions before rollout.
Proof of concept, pilot and MVP are three different tests
A proof of concept tests whether the technology can solve a task in principle, usually with sample data and without real users. A pilot tests whether the application delivers the expected benefit in the real workflow with real data and a limited group of users. A minimum viable product is the smallest version that already runs in regular operations and is extended from there. Mixing up these stages means scaling a proof of concept that never showed its benefit in the workflow.
An AI pilot should run for eight to twelve weeks in the real workflow. The duration follows two criteria. The pilot covers at least one complete business cycle, such as a month-end close or an ordering period, so that seasonal peaks become visible. And it generates enough cases for processing time or error rate to be compared robustly with the baseline. A pilot that allows no statement on benefit after twelve weeks is stopped or re-scoped.
Five decisions before rollout
From a delivery perspective, the transition can be condensed into five decisions. Each has a clear addressee and a verifiable result.
- Owner: a named person in the business unit is accountable for the result and the budget of the workflow in which the AI operates. The application thus has a counterpart who decides in daily business.
- Operation: it is settled who handles incidents and who reviews model changes together with quality deviations. This is an internal team or an external provider, defined with named responsibility and a deputy.
- Workflow: the changed process is documented in writing. It describes which steps the AI takes over, where a person checks or approves, and what happens in the event of deviations.
- Measurement: a metric with a baseline from before the pilot, such as processing time per case or the rework rate, makes the benefit demonstrable.
- Enablement: users are trained, and there are contacts in the business unit. The handover from the project team to the line is scheduled and documented.
The owner comes first, because he or she carries the other four decisions forward in daily business. For operation: an operating model for AI applications differs from classic IT operations, because model behavior changes with updates and inputs.
How long does the move from pilot to regular operations take?
The move from pilot to regular operations takes three to nine months. The lower end applies to one business unit at one site without deep system integration, such as an assistant based on existing documents. The upper end applies with several sites and with applications that write into legacy systems. Most of the time goes into integration and enablement; the model itself is largely finished after the pilot.
The transition follows three phases. In the Understand phase, business analysis describes the current workflow and its bottlenecks. This is where the metric and its baseline are set. In the Build phase, the prototype runs in the real workflow, with real data and users from the business unit, until the readiness criteria are met. In the Roll out phase, internal or external teams take over operation, and users are enabled.
How much does scaling an AI pilot cost compared with the pilot?
Scaling costs more than the pilot, because integration and operation are added and usage costs grow with every user. The pilot essentially funds the test of the benefit. Scaling funds four blocks that are missing from the pilot or solved provisionally.
External support is billed in the German market at roughly 1,000 to 2,500 euros per consultant day, according to the fee study of the Federal Association of German Management Consultants. How benefits and costs come together in one calculation is described in the article AI business case: calculating benefits and costs robustly.
Steering after rollout
Steering begins with rollout. The owner reviews the metric and the usage rate at a fixed rhythm. Deviations lead to a decision on improvement or extension, and where the benefit fails to materialize, on decommissioning. Treating decommissioning as a legitimate decision keeps the portfolio clean, because applications without measurable benefit tie up operating capacity that is missing elsewhere.
At portfolio level, the executive team uses the same metrics to decide which application is scaled next. An application that holds its metric after 90 days in operation sets the standard for the next one.
How Niehues Consulting delivers the transition
Niehues Consulting supports organizations from the first analysis through to adoption in operation. Marin Niehues comes from a delivery perspective: he leads programs as delivery manager or product owner and designs the structures and decision paths in which an application later operates. Projects range from VC-funded scale-ups to DAX 40 groups, with assignments in 20+ countries on six continents.
The approach follows the Niehues Delivery Model with the phases Understand, Build and Roll out. Three checkpoint questions matter for the move from pilot to scale. Does the prototype prove its benefit in the real workflow with real data? Who is accountable for operation from day one? Which metric shows after 90 days that the solution is taking hold? This covers AI management with the business case and the sequence of applications, AI prototypes with operationalisation through to adoption by the business units, and delivery management for reliable responsibilities.
The entry point is a free initial consultation of 30 minutes. It clarifies which of the five decisions a program is currently stuck on.
Frequently asked questions
Why do AI pilots fail to move into production?
AI pilots mostly fail because of the organization and the integration into the workflow, rarely because of the model itself. The pilot had special conditions such as selected users and prepared data, which are missing in regular operations. Without an owner in the business unit and without a metric with a baseline, nobody develops the application further, and usage declines.
What percentage of AI pilots fail?
According to estimates cited in a RAND Corporation study, more than 80 percent of AI projects fail. That is twice the rate of IT projects without AI. The most common causes lie in how the problem is understood and in the data.
Who owns an AI application in regular operations?
Accountability sits with the business unit in whose workflow the application operates. A person with responsibility for results and budget runs the application as part of their process. Technical operation is handled by an internal or external team, and both responsibilities are named before rollout.
Which metrics show that an AI application is scaling?
Metrics from the business unit's workflow show scaling, such as processing time per case or the rework rate, supplemented by the usage rate. The baseline from before the pilot is decisive. Two metrics per application are enough if an owner reviews them regularly.
How long does it take to move an AI pilot into production?
The move takes three to nine months. It is shorter for one business unit without deep system integration and longer with several sites and legacy systems.
How long should an AI pilot run?
An AI pilot should run for eight to twelve weeks in the real workflow. It covers at least one complete business cycle and generates enough cases for a robust comparison with the baseline.
What is the difference between a proof of concept, a pilot and an MVP?
A proof of concept tests technical feasibility with sample data. A pilot tests the benefit in the real workflow with a limited group of users. An MVP is the smallest version that already runs in regular operations.
How much does scaling an AI pilot cost compared with the pilot?
Scaling costs more than the pilot, because four cost blocks from integration to enablement are added. The budget for the first year of operation is derived from these blocks and set against the extrapolated benefit before rollout.
Let’s talk about your initiative.
In 30 minutes we clarify where your organization stands and which step will pay off most now.
Request a consultation30 minutes · free · no obligation



