AI Adoption & Operations

Choosing an AI Provider: Criteria for Procurement and Business Units

Marin Niehues17 March 202512 min read

Automated assembly line

An AI provider is well chosen when procurement and the business unit have assessed it against the same criteria and it meets the requirements in the actual workflow. The most common mistake in selection processes lies in the sequence: a list of providers comes first, and the requirements are sought afterwards. This article describes a four-step approach with six decision criteria and twelve questions for every provider. It touches on contracts and data processing. This article provides an overview and does not constitute legal advice.

The article AI management: what executive leadership must steer describes the selection, sequencing and steering of AI applications as a whole. The selection question arises in more and more companies: a survey by the German Federal Statistical Office shows that in 2024 one in five companies with ten or more employees used AI, up from 12 percent a year earlier.

Procurement and business units assess by different standards

Procurement checks whether a provider can be controlled contractually and commercially. The business unit checks whether the tool takes work off its hands day to day. Both questions are legitimate, and both lead to different results when answered separately. The business unit then favors the product with the most convincing demo. Procurement favors the offer with the most favorable terms. IT learns of the decision once it has already been made.

The process therefore needs a common starting point, and that is the use case. It describes which work step is to change and who carries it out. It also defines which result counts as good and by which measure this can be assessed. The article Prioritizing AI use cases shows how to rank use cases by value, feasibility and risk. A prior question is whether a provider is needed at all. The decision grid for this is in the article Make or buy.

How does the selection of an AI provider work?

Selecting an AI provider runs in four steps: define the use case, shortlist providers against weighted criteria, run a final round with a test on your own material, and decide together with an implementation plan. The sequence is binding, because each step supplies the yardstick for the next.

  1. Use case: work step, user group and target metric are fixed. From these you derive must-have requirements and desirable criteria. A must-have requirement excludes providers, a desirable criterion weights them.
  2. Shortlist: the business unit and procurement assess providers against the six criteria in the next section. Two or three providers remain.
  3. Final round: each provider works through the same test set of real cases.
  4. Decision: the weighted overall score determines the provider, and the implementation plan is in place with the decision.

Six criteria for the assessment

An assessment matrix with six criteria covers both perspectives. Three criteria are assessed mainly by the business unit, supported by IT:

  • Output quality on your own material: the provider processes real documents and cases from the business unit, and expert users assess the result.
  • Fit with the workflow: the solution engages at a defined point in the existing workflow, and it is clear which steps fall away.
  • Integration: sign-on and the permissions model can be connected without custom builds. IT assesses this criterion.

Three further criteria sit with procurement:

  • Contract and pricing model: license type and usage dependency as well as term and price adjustment add up to a predictable figure over the planned period of use.
  • Data handling and evidence: storage location, use for training, retention periods and subprocessors are governed by contract and backed by certifications such as ISO/IEC 27001, SOC 2 or C5. ISO/IEC 42001 demonstrates structured AI management.
  • Provider stability and exit: data export, replacement of the underlying model and notice periods are agreed.

Each criterion receives a weighting derived from the use case. For an assistant producing sales materials, output quality weighs more heavily than for a summary of internal minutes, where data handling is decisive. The business unit and procurement set the weightings together. The article Data protection and AI describes what belongs in the data processing agreement with an AI provider.

What evidence should an AI provider supply?

An AI provider should evidence its information security with ISO/IEC 27001, a SOC 2 report or a C5 attestation and disclose its data processing in the contract. ISO/IEC 27001 is the international standard for information security management systems. SOC 2 is an audit report based on the criteria of the American Institute of Certified Public Accountants (AICPA). The C5 criteria catalog of Germany's Federal Office for Information Security (BSI) addresses cloud services and is widely used in the German market. Since 2023, ISO/IEC 42001 has described a management system for AI and shows that the provider manages the risks of its models systematically.

The role under the AI Act also matters. The provider states whether its product falls into a high-risk category and which documentation it makes available to the company as deployer. A certificate does not replace a contract: storage location, use of inputs for training and subprocessors belong in the written agreement.

Testing on your own material and deciding

The matrix produces a final round of two or three providers. Each of them works through the same test set of real cases, and the evaluation is made against the target metric of the use case. Expert users assess the results without knowing the provider, where the tool allows this. In parallel, procurement negotiates the contract terms of all finalists and brings the terms into the overall score only after the functional assessment.

The test set needs substance. It contains typical everyday cases and a selection of difficult ones, for example incomplete documents or special cases that expert users know from practice. For each case, the business unit defines a target result in advance. This creates a benchmark that holds regardless of how the provider presents itself. In addition, procurement holds reference calls with customers running a comparable use case. The answers feed into the criteria of provider stability and integration. The result is a ranking that can be justified to the executive team with data.

A proof of concept in the final round should be paid for as soon as it runs on your own data in the real workflow and ties up more than a few days of the provider's work. A paid test with a fixed scope commits the provider to the agreed result and gives the company the rights to the test results. Free trial access is suitable for the shortlist.

The decision follows the weighted overall score and already includes the implementation plan: who owns the tool, which user group starts first, and which metric shows after the first quarter whether the benefit materializes. Without this plan, the selection ends with a contract that has no effect.

Providers change their models in quick succession. The architecture should therefore allow a change of model or provider without the workflow having to be rebuilt. Three safeguards protect against lock-in: a dedicated layer between application and model, prompts and test sets owned by the company, and contractually guaranteed data export.

How long does it take to select an AI provider?

Selecting an AI provider for a clearly defined use case takes eight to twelve weeks. The time required follows from the four steps: two to three weeks for the use case and requirements, two weeks for market screening and shortlisting, three to four weeks for the test in the final round, one to three weeks for negotiation and the implementation plan. Formal public procurement procedures follow their own deadlines and take longer. Skipping the use case costs the time later in the final round, because the yardstick is missing.

Niehues Consulting as a partner for provider selection

Niehues Consulting supports provider selection as part of AI management, following the Niehues Delivery Model with the phases Understand, Build, Introduce. In the Understand phase, business analysis clarifies the use case and requirements together with the business unit, guided by the test question of which business result should change measurably. In the Build phase, a prototype proves whether the solution delivers the benefit in the real workflow with real data before any contract is signed. The business case is confirmed with real values along the way.

In the Introduce phase, Niehues Consulting introduces the solution with processes and responsibilities and enables the users until the business unit adopts it and operations can take it over. Marin Niehues works from a delivery perspective: what matters is what reaches users in their daily work and functions there. His projects range from VC-funded scale-ups to DAX 40 groups in 20+ countries. The free initial consultation lasts 30 minutes and clarifies which phase your selection is in.

Frequently asked questions

What are the most important criteria when choosing an AI provider?

The most important criteria are output quality on your own material and data handling, with the weighting following the use case. Fit with the workflow also determines the benefit, and exit options determine the risk. The business unit and procurement set the weighting together before the first demo.

Who should be involved in selecting an AI provider?

The business unit as the future user and procurement as the owner of contract and terms form the core. IT assesses the integration, and for questions on data processing and contracts you obtain legal advice. A member of the executive team or divisional leadership owns the decision.

Is a product demo enough to make a decision?

No, a demo only shows what a product achieves under favorable conditions. Whether it works with your documents and cases only becomes clear in a test with your own material. That is why expert users assess the same real cases for every provider in the final round.

How many providers should make the final round?

Two or three providers belong in the final round. Fewer providers allow no robust comparison, while more multiply the testing effort without improving the decision. The matrix of must-have requirements and weighted criteria reduces the provider list to this number.

How long does AI vendor selection take?

For a clearly defined use case, the selection takes eight to twelve weeks. The largest block, at three to four weeks, is the test in the final round. Formal public procurement procedures take longer.

Should a proof of concept with an AI vendor be free?

A proof of concept on your own data in the real workflow should be paid for, while free trial access is sufficient only for the shortlist. Payment commits the provider to scope and result and secures the company's rights to the test results. Agree the price in advance and negotiate whether it counts towards the later contract.

Which certifications should an AI provider have?

An AI provider should hold ISO/IEC 27001, a SOC 2 report or a C5 attestation from the BSI. ISO/IEC 42001 additionally demonstrates a management system for AI. Certifications complement the contract, which governs storage location, use for training and subprocessors.

How do you avoid vendor lock-in with AI?

Companies avoid vendor lock-in with a dedicated layer between application and model, their own prompts and test sets, and contractually guaranteed data export. This allows a model to be replaced without rebuilding the workflow. Notice periods and export formats belong on the table before the contract is signed.

Let’s talk about your initiative.

In 30 minutes we clarify where your organization stands and which step will pay off most now.

Request a consultation

30 minutes · free · no obligation