AI Ethics in Practice: Identifying and Limiting Bias

An organization identifies bias in AI systems by measuring results for each affected group, and it limits bias through fixed rules for the data base and for release. Ethics thus becomes a management task with metrics and clear owners. It arises wherever a system assesses or prioritizes people, for example in job applications or credit requests. This article provides an overview and does not constitute legal advice.
What is bias in AI, and where does it arise?
Bias in AI is a systematic distortion through which a system delivers worse results for certain groups than for others, without the task justifying it. It arises at several points in the delivery chain of an AI system, and each point needs its own owner.
The best-known example comes from recruitment. Amazon developed a model to rate CVs and trained it on the applications of the preceding ten years, most of which came from men. The model learned to rate CVs containing references to women lower. The company abandoned the project, as became public in 2018. The case shows the basic pattern: the model worked correctly in technical terms and faithfully reproduced the bias of the past.
- Data base: historical data reflect earlier decisions. A model that learns from past hiring reproduces its patterns.
- Target variable: what a system optimizes determines whom it favors. Minimizing processing time alone systematically pushes complex cases to the back.
- Model: language models adopt patterns from their training texts, such as role stereotypes in wording.
- Context of use: a system that works well in one market delivers a different quality in another country or customer segment.
- Use: employees often follow recommendations unchecked when the system conveys an impression of certainty.
Review starts at each of these points. For executive management this means: before launch, it names an owner for each point and specifies which evidence it wants to see.
What types of bias are there?
Research distinguishes types of bias by the stage in a system's life cycle at which they arise. A widely used taxonomy by Suresh and Guttag names seven sources, which for practical purposes can be grouped into five.
Two widely cited studies show how strong such effects can be. A study of commercial facial analysis systems found gender classification error rates of up to 34.7 percent for darker-skinned women and 0.8 percent for lighter-skinned men. An analysis published in Science of an algorithm in US healthcare revealed measurement bias: the system used past treatment costs as a proxy for treatment need. After correction, the share of Black patients receiving additional care rose from 17.7 to 46.5 percent.
Exhibit 1
Gender classification error rates ranged from 0.8 to as much as 34.7 percent depending on the group
Note: Error rate of commercial facial analysis in gender classification
Risk tiers: how deeply to review
The depth of review follows the impact on people. A grid with three tiers matches the effort to that impact. It follows the risk-based approach of the EU AI Act and of the frameworks NIST AI RMF (2023) and ISO/IEC 42001 (2023).
- Tier 1, low impact: the system touches no decision about individuals, for example when summarizing internal documents. A documented purpose and spot checks suffice.
- Tier 2, medium impact: the system prepares decisions, for example when prioritizing customer requests. Results are compared per group before launch and at fixed intervals thereafter.
- Tier 3, high impact: the system influences decisions on employment or credit. Here, approval by executive management and a final human decision apply. Continuous monitoring is added.
Tier 3 coincides with the high-risk areas of the AI Act: systems for recruitment, promotion and creditworthiness fall under Annex III, and Article 22 GDPR and the German General Equal Treatment Act also apply. The legal department reviews the individual case. For high-risk systems, the AI Act also requires data governance that examines possible biases. What this means in detail is described in the article High-Risk AI under the AI Act. Organisationally, classification comes at the start of every initiative, and executive management knows every tier 3 application. The wider framework is described in the article AI Risk Management: A Framework for Decision-Makers.
How do you measure bias?
Bias is measured by setting comparison groups and a metric in advance and comparing results per group. Common metrics are the selection rate and the error rate per group. As soon as the base rates of the groups differ, equal selection rates and equal error rates cannot be achieved simultaneously, as a matter of mathematics. Kleinberg and colleagues and Chouldechova proved this independently. Which equality should apply is therefore a business decision that the responsible management takes and documents before testing.
A practical method is the paired comparison. The team builds test cases that are identical in all relevant characteristics and differ only in the group attribute, for example the name or country of origin. If the result differs, there is a finding. With language models the same method works with identical prompts and varying personal details, repeated several times per pair so that random fluctuation is not mistaken for a finding.
The organization also sets tolerances in advance. A proven rule reads: if the error rate between two groups differs by more than a defined value, the system does not go into operation. The business unit sets the value together with risk management. The method for ongoing quality assurance is described in the article Assuring the Quality of AI Results.
Limiting bias: measures along the delivery chain
Each point in the delivery chain has its own lever. For the data base, the team checks representativeness: are all relevant groups included in sufficient numbers, and do the data reflect earlier decisions that themselves needed correcting? Missing groups are added, overrepresented ones reweighted.
For the target variable, an adjustment is often enough. Measuring solution quality per case alongside processing time changes the system's behavior without any technical intervention.
In use, the design of collaboration between person and system is decisive. A recommendation with reasons invites scrutiny, while a proposal without reasons encourages rubber-stamping. Room for decision and time for review belong in the process, as does a fallback route in case the system fails or is suspended.
In operation, data and usage change. Monitoring compares the agreed metrics at fixed intervals and triggers escalation when thresholds are exceeded.
Responsibility and escalation in operation
An AI system that affects people needs three clearly named roles in operation:
- Business owner: accountable for purpose, benefit and tolerances.
- Technical operator: accountable for model, data and monitoring.
- Independent review body: assesses findings without any interest in the initiative's success.
Added to this is an escalation path with fixed deadlines. A finding goes first to the business owner, and to executive management if the tolerance is exceeded. The metrics belong in regular management reporting so that they are visible alongside cost and benefit. Where a system assesses employees, the participation rights of employee representatives must be clarified early. How committees and policies are set up for this is shown in the article Building AI Governance.
How Niehues Consulting delivers such initiatives
Niehues Consulting treats bias risk as part of delivery from the outset. The approach follows the Niehues Delivery Model with three phases:
- Understand: business analysis of the use cases and classification by risk tier. The review question: what is the baseline, and how do we measure success, including per group?
- Build: prototypes that show in the real workflow how results behave per group, before any operation is set up. The review question: does the prototype prove its benefit in the real workflow with real data?
- Introduce: transfer into operation with clear processes and responsibilities, plus enablement of the business units until they adopt the system and monitor it themselves. The review question: who is accountable for operation from day one?
Marin Niehues works from the delivery perspective. He builds organizations that deliver and brings AI into operation, in projects ranging from VC-funded scaleups to DAX 40 corporations and with assignments in 20+ countries on six continents. On request, Niehues Consulting takes on steering as delivery manager or product owner. How AI ethics fits into AI management as a whole is described in the article AI management: what executive leadership must steer. The entry point is the free 30-minute initial consultation.
Frequently asked questions
What is bias in AI?
Bias is a systematic deviation through which an AI system delivers worse results for certain groups than for others. It arises in the data base, in the target variable, in the model, in the context of use or in how employees use the system.
What are examples of AI bias?
The best-known example is Amazon's abandoned recruiting model, which rated women's CVs lower because it had learned from predominantly male applications. Facial analysis systems showed error rates for darker-skinned women many times higher than for lighter-skinned men. An algorithm in US healthcare underestimated the needs of Black patients because it used cost as a proxy for illness.
What types of bias are there?
The main types are historical bias, representation bias, measurement bias, deployment bias and automation bias. They differ in whether the distortion arises in the data, the target variable, the context of use or the behavior of users. Each type calls for its own countermeasure.
What is fairness in AI?
Fairness is the equality, agreed in advance, that a system should maintain between groups, such as equal selection rates or equal error rates. With differing base rates, both goals cannot be achieved at once. Which form applies is decided and documented by the responsible management before testing.
How do you detect bias in an AI system?
Bias is detected by comparison: the team measures metrics such as selection rate and error rate per group and uses paired comparisons in which test cases differ only in the group attribute. Tolerances are fixed before testing, and measurement continues at fixed intervals after launch.
Does a company need an AI ethics board?
A dedicated committee pays off once a company runs several tier 3 applications or uses AI in products for customers. For individual applications with lower impact, a named, independent review body with a fixed escalation path to executive management is sufficient. In both cases the body needs a mandate to halt applications.
Who in the company is responsible for AI ethics?
Overall responsibility lies with executive management. Operationally, it names a business owner and a technical operator for each application and adds an independent review body for findings.
Can bias be eliminated completely?
It can rarely be eliminated completely, because data reflect reality with its inequalities. It can be limited to a level defined in advance, and that level is reviewed continuously in operation and adjusted where needed.
Let’s talk about your initiative.
In 30 minutes we clarify where your organization stands and which step will pay off most now.
Request a consultation30 minutes · free · no obligation



