AI Adoption & Operations

Building an AI Center of Excellence

Marin Niehues10 November 202511 min read

Marin Niehues gesturing as he speaks on a dark stage next to a table with a laptop.

You build an AI center of excellence by fixing the mandate first and then staffing a core team that takes applications into operation together with the business units. The most common mistake is the reverse order, in which team and tools are settled first and the tasks are looked for afterwards. The result is prototypes without a customer. This article sets out the build-up from the delivery perspective, with starting size and reporting line as well as a timeline for the three build phases.

According to official statistics, 20 percent of German companies with ten or more employees use AI, rising to 48 percent among companies with 250 or more employees. Larger companies therefore run several AI initiatives side by side in different units, and that is where the task of a center of excellence arises: a shared route from idea to measured value in operation. How the center fits into AI management as a whole is set out in the article AI management: what executive leadership must steer.

What is an AI center of excellence and what mandate does it need?

An AI center of excellence is a unit that pools knowledge and delivery capability for AI and thereby enables the business units to take applications into regular operation. It sets standards, accompanies applications into operation and makes results comparable. For the executive team, the center thus becomes the steering instrument for the AI portfolio: every application follows the same path, and every decision to continue or stop rests on the same metrics.

The decisive step before any build-up is the mandate. It answers four questions. Which applications does the center build itself, and which does it only accompany? Who decides on priority and budget? Which standards are binding for all business units? How does the executive team recognize that the center is fulfilling its brief? Where these answers are missing, the result is an advisory body without authority, one that issues recommendations and waits for agreement.

The mandate also determines the set-up. The delivery-oriented center of excellence works in a hub-and-spoke model: a central core, with responsibility for value in the business units. A central team that delivers everything itself is the right entry point while no more than two units pursue their own initiatives. From three units onwards, the center is the better form, because otherwise parallel solutions and inconsistent standards emerge. The selection criteria are set out in AI organization: central team, center of excellence or decentralized.

Roles and staffing

Staffing follows the mandate. Five roles form the starting point:

  • Head of center: owns the AI portfolio in the manner of a product owner and reports to the executive team.
  • Business analysis: translates needs from the business units into requirements and a business case.
  • Engineering and data: build prototypes and transfer them into operation.
  • Rollout and enablement: ensures that processes and responsibilities come into force together with the application.
  • Delivery management: steers timelines and handovers to the operating teams.

The head of center needs access to the executive team and a budget that can be used without case-by-case approval. In addition, each participating business unit names a person with decision authority who owns the value of an application and sits in the center's prioritization round. Value thus stays in the line where it will later be measured.

How large does an AI center of excellence need to be?

To start, a core team of six to eight people is sufficient, in which each of the five roles is filled full-time by a named person. The size follows from a simple calculation that aligns the center with its portfolio.

Part-time assignments without firm responsibility lengthen the lead time of every application. External support for engineering and data is common during the build-up. In the German consulting market it is typically billed at around 1,000 to 2,500 euros per consulting day, depending on seniority. Fill the head of center and business analysis permanently from within your own company, because that is where knowledge of processes and decision paths resides.

Where should an AI center of excellence sit in the organization?

The center reports to the executive who holds overall responsibility for AI, chosen by the focus of the portfolio. If most initiatives sit in operational processes such as service or production, the executive responsible for operations is the right sponsor. If the focus lies on data and a shared platform, the executive responsible for digital or IT fits. There is at most one level between the head of center and the executive team.

From around ten parallel initiatives across several units, the coordination effort justifies a Chief AI Officer; below that, a head of center with direct access to the responsible executive is sufficient. The mandate and metrics of the role are described in the article on the role of the Chief AI Officer.

Operating model and metrics

The operating model defines how a request reaches the center and what happens next. A fixed sequence of four stages creates comparability:

  • Intake: the business unit describes the problem and the expected value and names a responsible person.
  • Assessment: the center reviews value, feasibility and risk against a uniform grid.
  • Prototype: a limited prototype proves the value in the real workflow.
  • Transfer: the application moves into regular operation with an operating team.

How committees and policies develop around the center is described in Building AI governance.

The impact of a center shows in four metrics:

  • Applications in regular operation
  • Lead time from request to prototype
  • Usage rate in the business units
  • Realized value compared with the business case

A metric that measures only activity, such as the number of workshops or prototypes, points in the wrong direction.

How long does it take to build an AI center of excellence?

It takes nine to twelve months until the center is fully operational and the first applications run in regular operation. The build-up follows three phases, each of which ends with a result the executive team can review.

The Understand phase takes six to eight weeks. It establishes existing AI activities, demand and available data. From this emerge the mandate, a target picture and a prioritized list of first applications with a business case.

The Build phase takes three to four months. The core team is staffed, the sequence from intake to transfer comes into force, and two to three applications are built as prototypes in the real workflow. The outcome is prototypes with demonstrated value and a handover concept. If key roles still have to be recruited, plan for four months.

The Roll-out phase takes three to four months. The applications go into operation with internal or external teams. The business units take over processes and responsibilities, users are enabled, and the center's metrics flow into the executive team's reporting.

Why the transition into operation is the critical point

Many AI initiatives stall at the transition from prototype to operation, because operation requires a team with a budget and adapted processes in the business unit. A center that plans this handover from the outset brings more applications into regular operation than one that negotiates it at the end. How scaling is organized is shown in From pilot to scale.

Niehues Consulting as a partner for the build-up

Niehues Consulting supports the build-up of an AI center of excellence from clarifying the mandate to transfer into operation. Founder Marin Niehues is a management consultant working from the delivery perspective: he builds organizations that deliver and brings AI into operation. His projects range from VC-funded scale-ups to DAX 40 groups, he has worked in 20+ countries on six continents, and his home base is the DACH region.

The approach follows the Niehues Delivery Model with the phases Understand, Build and Roll out. Three of its review questions matter most when building a center. In Understand: who decides, who delivers, who uses? In Build: which roles, decision paths and interfaces does operation require? In Roll out: who owns operation from day one? Once these questions are answered for the center and for each of its applications, staffing and tools follow from the answers.

AI management provides business analysis and the business case and sets the sequence of applications. Prototypes and operationalisation take the first applications into operation. Delivery management and organizational design define the center's roles and decision paths, with Marin Niehues as delivery manager or product owner. The entry point is a free 30-minute initial consultation: Request an initial consultation.

Frequently asked questions

What is an AI center of excellence?

An AI center of excellence is an organizational unit that pools knowledge and delivery capability for AI and makes them available to the business units. It prioritizes applications, accompanies them into operation and reports the value achieved to the executive team.

How long does it take to set up an AI center of excellence?

Setting up an AI center of excellence takes nine to twelve months until the first applications run in regular operation. Understand takes six to eight weeks, Build and Roll out three to four months each.

How large should an AI center of excellence be?

Six to eight people are enough to start: four for the fixed roles from head of center to delivery management, plus two to four for engineering and data. Each additional initiative built in parallel adds two people.

Who should an AI center of excellence report to?

The center reports to the executive who holds overall responsibility for AI. If the focus is on operational processes, that is the executive responsible for operations; if it is on data and platform, the executive responsible for digital or IT. There is at most one level between the head of center and the executive team.

When does an AI center of excellence make sense?

A center makes sense once three or more business units pursue their own AI initiatives, because from that point parallel solutions and inconsistent standards emerge. It creates shared prioritization and a shared route into operation.

What is the difference between an AI center of excellence and a central AI team?

A central AI team builds all applications itself, whereas a center of excellence sets standards and enables the business units, which own the value of their applications. The central team suits an entry with few initiatives. The center is the right form once several units run their own applications.

What is the difference between a center of excellence and a shared service?

A shared service delivers standardized services at scale against agreed service levels, whereas a center of excellence sets standards and builds capabilities in the units. For AI, both often exist side by side: the center defines how applications are built and assessed, and a platform team runs the shared technology.

Does an AI center of excellence need a Chief AI Officer?

A Chief AI Officer becomes worthwhile from around ten parallel initiatives across several units. Below that, a head of center with an own budget and direct access to the responsible executive is sufficient.

How much does an AI center of excellence cost?

The costs consist of the core team's staff, the budgets of the individual initiatives and the running of the applications. For the core team, multiply the number of full-time positions by your fully loaded cost rate. External support during the build-up is typically billed in the German market at around 1,000 to 2,500 euros per consulting day.

Let’s talk about your initiative.

In 30 minutes we clarify where your organization stands and which step will pay off most now.

Request a consultation

30 minutes · free · no obligation