AI Adoption & Operations

Data quality: why it decides the success of AI

Marin Niehues19 May 202511 min read

Robot arm working on a table

Data quality decides the success of AI because an application in operation is only as reliable as the data it draws on. A prototype can mask this, since a cleansed extract is enough. In regular operation the application reaches the entire data estate, and every gap becomes a source of error. This article shows why AI initiatives fail on data, how to measure quality with five metrics, what poor data costs and how an organization secures quality over time.

What data quality means in the context of AI

Data quality describes the fitness of a data set for a specific purpose. The same data set can be adequate for quarterly reporting and unsuitable for a customer service assistant. A controller tolerates a discrepancy in master data as long as the totals are right. A language model that summarizes contract clauses, by contrast, treats every outdated version of a document as valid and builds a convincing answer from it.

This gives rise to the notion of AI-ready data. It refers to a data set whose fitness for a specific AI use case has been demonstrated: it reflects the cases that occur in operation, has a named owner, is described in terms of meaning and origin, and is checked continuously. A data set is always AI-ready for a purpose. There is no company-wide seal of approval.

The assessment therefore starts with the application. First it is clear which decision or task is being supported. Then it can be determined which quality is required and which defects actually put the benefit at risk. This sequence prevents organizations from spending years cleansing data that no application will ever need. It is also the core of any robust data strategy as a prerequisite for AI. Where this article touches on legal requirements such as the AI Act: this article provides an overview and does not constitute legal advice.

Why do AI projects fail because of poor data?

AI projects fail because of poor data because defects only become visible in operation, when budget and trust are already committed. An interview study with 65 experienced data scientists and engineers names missing or unsuitable data as one of five root causes of AI project failure. The large analyst and consulting firms likewise see a lack of AI-ready data as the main reason companies abandon AI initiatives.

How widespread the problem is emerges from a study in which 75 executives each checked the last 100 data records of their unit for critical errors. Only 3 percent of the units reached an acceptable score. For AI this weighs more heavily than for reporting. In classic analytics, defects show up in totals and averages and partly cancel each other out. An AI application, by contrast, processes individual cases. A wrong record produces a wrong answer for exactly one customer or one case handler, and nobody spots the error in a statistic.

How do you measure data quality for AI?

You measure data quality by testing a sample from the real sources against a minimum standard set by the application, with a separate metric for each dimension. The international standard on data quality describes fifteen characteristics. For steering AI applications, a grid of five dimensions has proved its worth:

  • Completeness: are all fields the application needs present? Metric: share of records with mandatory fields populated.
  • Accuracy: does the stored value match reality? Metric: error rate of a sample checked against the original document or by query.
  • Timeliness: does the data reflect the current state? Metric: share of records whose last check is older than the pace of the decision.
  • Consistency: do terms and formats mean the same in every system? Metric: share of records that match across two source systems.
  • Uniqueness: does each object, such as a customer or an item, exist exactly once? Metric: duplicate rate.

The business function sets the minimum standard for each application. For an application that answers customers directly, the thresholds for accuracy and uniqueness are higher than for one whose suggestions an employee checks before use. Repeating the measurement monthly produces a metric for management reporting.

Where defects show up first

A typical pattern appears in the transition from prototype to operations. The prototype ran on a carefully selected sample and convinced the business units. In operation, the same application meets legacy data with duplicate entries and fields nobody has maintained for years. The hit rate falls, the business units correct results by hand, and trust in the application is lost before the benefit arrives.

Three signs point to a quality problem early on:

  • The business units regularly correct results without the correction flowing back into the data.
  • Two systems give different answers to the same question.
  • Nobody can name who is responsible for a data source.

Each of these signs can be evidenced in an inventory within a few days. That is one reason to check the data situation as early as the business case.

What does poor data quality cost?

Poor data quality costs a company above all working time spent on rework, and you can determine this amount for each application with a simple calculation. Added to this are the consequential costs of wrong decisions and lost trust in the application, which are harder to quantify. The calculation multiplies the number of cases by the error rate, the correction time per error and the fully loaded hourly rate.

The calculation shows two things. It gives the maximum a cleansing effort may cost, and it shows which dimension causes the largest share. An error prevented at the point of capture is always cheaper than one corrected by hand in operation.

A four-step approach

Data quality can be built systematically along the application:

  1. Define the requirement: for each prioritized application, a minimum standard is set per dimension, for example the permissible rate of incomplete records.
  2. Measure the data: a sample from the real sources shows where the data deviates from the standard.
  3. Rank defects by impact: priority goes to defects that noticeably distort results or jeopardize adoption by the business units. Deviations with no effect on the result are deferred.
  4. Embed assurance in operations: automated checks at data entry, thresholds with alerts and a fixed route for the business units to report and resolve errors.

The fourth step decides how long success lasts. A one-off cleansing decays at the pace at which the sources create new data, for customer and contract data often within the first year. Quality is maintained when checking is part of ongoing operations.

Responsibility belongs in the line

Technical tools for data checks are available. The constraint lies in responsibility. Data quality is created where data is captured, in sales or in accounting. A central data function can set rules and provide tools, but it does not own the capture itself.

Two roles in the business function have proved effective. The data owner is a manager who decides on the meaning and use of the data and answers for the standard. The data steward knows the fields in detail, checks quality values in daily business and initiates corrections. Their contribution becomes visible once a quality metric per application enters executive reporting. Once the figure appears in monthly management and has an owner with time for upkeep, data quality becomes a management task like any other metric.

How Niehues Consulting builds data quality into delivery

Niehues Consulting treats data quality as part of delivery and works according to the Niehues Delivery Model, with the phases Understand, Build and Roll Out. In Understand, the data requirements of each application are defined together with the business units. The test question is: what is the baseline, and how will success be measured? For data quality, this means capturing the five metrics before building.

In Build, a prototype proves the benefit in the real process with real data and reveals which gaps in the data put adoption at risk. In the transfer to operations, responsibilities and checking routes are established with internal or external teams. The test question here is: who is accountable for operations from day one? Data quality metrics move into the AI portfolio reporting of the executive team. The path from the first use case to regular operations is described in the article From pilot to scale.

Founder Marin Niehues is a management consultant who works from the delivery perspective, in projects ranging from VC-funded scale-ups to DAX 40 corporations. The entry point is a free initial consultation of 30 minutes, in which the data situation of a prioritized application is assessed. You can arrange an appointment via the contact page.

Frequently asked questions

What is data quality?

Data quality describes how well a data set is suited to a specific purpose. It is usually assessed along completeness, accuracy, timeliness, consistency and uniqueness. For AI applications working with documents, version and validity are added.

What is AI-ready data?

AI-ready data is data whose fitness for a specific AI use case has been demonstrated. It reflects the real cases, has an owner, is described and is checked continuously. A data set is therefore always AI-ready for a purpose.

Does a company have to cleanse all its data before starting with AI?

No. It is enough to cleanse the data that a prioritized application actually uses. Cleansing the entire estate ties up capacity for years, and its benefit for AI only shows once the first application is running.

Who is responsible for data quality?

The business unit that creates and uses the data is responsible. A central function provides standards and tools. The executive team anchors this responsibility by including a quality metric per application in ongoing management.

What is a data steward?

A data steward is the specialist who secures the quality of a data set in daily business. The steward maintains definitions, checks quality values and initiates corrections in the source systems. Decisions on the meaning and use of the data lie with the data owner.

What does the AI Act require for data quality?

For high-risk AI systems, the AI Act requires that the data used for training and testing fits the purpose, is sufficiently representative and is examined for bias. For most everyday business applications these obligations do not apply directly, yet they are a useful benchmark. The classification is covered in the article on high-risk AI under the AI Act.

How often should data quality be measured?

At least monthly for every application in operation, and continuously through automated checks at data entry for data with a high rate of change. The measurement before building provides the baseline, and every further one shows whether assurance in operation is working.

Let’s talk about your initiative.

In 30 minutes we clarify where your organization stands and which step will pay off most now.

Request a consultation

30 minutes · free · no obligation