AI Law, Risk & Governance

Data Protection and AI: Using Language Models in a GDPR-Compliant Way

Marin Niehues13 January 202512 min read

Padlock on a white background

Language models can be used in line with the GDPR when companies settle purpose and data classes before deployment, conclude a suitable vendor contract and manage operations in a way they can evidence. The decision is made in the organization around the model: who may enter which data, which contract applies, and who can demonstrate this if asked? This article describes a review model that lets the executive team and department heads answer these questions in a fixed order. This article provides an overview and does not constitute legal advice.

Is ChatGPT GDPR-compliant?

ChatGPT can be used in compliance with the GDPR when the company uses a business plan or the API with a data processing agreement and training on inputs is excluded. Personal accounts do not meet these requirements for personal data, because no contract exists between the company and the vendor.

The distinction applies to every large language model, whether from OpenAI, Microsoft, Google or a European vendor. In data protection terms, the access route is what counts. An enterprise contract governs data processing together with retention periods, and also the use of inputs for training. With a personal account, the individual decides on these settings, and the company can evidence none of it.

Vendors based in the United States require a legal basis for the transfer.

Configuration matters as much as the contract. A tool with a sound contract loses its value for data protection if employees use personal accounts alongside it. Controlling usage therefore belongs to vendor selection. The article on shadow AI shows how companies detect uncontrolled usage.

Which data may be entered into a language model?

Any data may be entered into a language model for which purpose and legal basis are settled and for which a suitably approved tool is available. Every review therefore starts with the data flow. As soon as a name or an email address appears in a prompt, the company is processing personal data. This also covers data that an assistant retrieves from connected documents, and inputs that the vendor logs.

For control purposes, a division into three data classes is sufficient, and it fits into any existing information classification:

  • Class A: content without personal data, such as published texts, product descriptions or general research questions.
  • Class B: ordinary personal data, such as contact details and business correspondence.
  • Class C: special categories under the GDPR, such as health data, and any data whose processing carries a high risk for the individuals concerned.

Each class receives one rule. Class A is permitted in every approved tool; confidential company information additionally follows the company's information classification. Class B applies only in tools with a reviewed contract. Class C requires a separate review for each application. The rule can be adopted word for word into the employee policy.

Is anonymizing or pseudonymising data before input enough?

Genuine anonymization is enough, pseudonymisation is not. Anonymous data falls outside the scope of the GDPR because no individual can be identified any longer. Pseudonymised data remains personal data as long as the company or a third party can restore the link with reasonable effort. Anyone who replaces the name in a customer email with “Customer 1” but leaves contract number, location and the description of the claim in place has pseudonymised the text. In practice, therefore, pseudonymisation lowers the risk and is a good safeguard, yet it replaces neither the contract nor the legal basis.

Every processing activity needs a defined purpose and a legal basis under Article 6 GDPR. For internal text assistance, legitimate interest under Article 6(1)(f) GDPR is available if the balancing test is documented. For employee data, a ruling of the Court of Justice of the European Union in 2023 restricted the German special provision; companies therefore rely on Article 6 GDPR or a works agreement.

For operations this results in a clear sequence. The purpose is recorded in one sentence for each application. The legal basis goes into the documentation, and privacy notices for employees or customers are adjusted where the new processing affects them. The business unit that owns the application creates the entry in the record of processing activities. The company takes the decision as controller; the data protection officer and the legal department assess the individual case and advise.

At this point data protection meets the EU AI Act. Article 4 obliges companies that deploy AI systems to ensure a sufficient level of AI literacy among their staff. A policy with data classes and a short training session on applying it meets both requirements at once.

The review model in five steps

Every new application runs through the same five steps, before the business unit starts using the tool. The fixed order prevents the business unit from starting with the tool and asking data protection for approval afterwards.

  1. Describe the application: purpose, user group, expected data classes.
  2. Clarify the legal basis and the information owed to the individuals concerned.
  3. Review the vendor: data processing agreement, training on inputs, storage location, sub-processors.
  4. Carry out a risk assessment and decide on a data protection impact assessment, in particular for applications that evaluate people.
  5. Grant approval, name owners, set a review date.

Step four has a clear criterion. A data protection impact assessment under Article 35 GDPR is mandatory where processing is likely to result in a high risk for the individuals concerned. The German supervisory authorities maintain a list of processing operations for which it is always required. If an AI application evaluates applications, performance or behavior of employees, or processes Class C data on a larger scale, carry out the impact assessment. For text assistance that processes only Class A and Class B data, a documented risk assessment is sufficient.

The fifth step decides durability. Models and vendor terms change continuously, so every approval receives a date for renewed review: at least once a year for tools with Class B data, and additionally whenever the contractual terms change. The employee policy makes the review model workable in daily operations; the article on the AI policy for employees describes content and rollout. The article From pilot to scale shows how data protection fits into the wider rollout.

What fines apply to data protection breaches involving AI?

For breaches of the processing principles, the legal basis or data subjects' rights, the GDPR provides for fines of up to 20 million euros or 4 percent of worldwide annual turnover, whichever is higher. For breaches of organizational obligations, for example on the data processing agreement or the impact assessment, the ceiling is 10 million euros or 2 percent.

No special rules apply to AI; supervisory authorities apply the existing framework consistently. The Italian data protection authority imposed a fine of 15 million euros on the provider of ChatGPT, among other things for lacking a legal basis for training and for insufficient information to users. For companies that deploy AI, a second point matters just as much: when setting the amount, the authority expressly takes into account which technical and organizational measures were in place. A documented review model therefore lowers both the risk of a breach and the level of any sanction. Alongside fines, data subjects can claim compensation, including for non-material damage.

Niehues Consulting as a partner for implementation

A review model works when business units and IT apply it together with data protection in the daily process. This is where Niehues Consulting comes in. The firm works according to the Niehues Delivery Model with the phases Understand, Build and Roll out, and three of its checkpoint questions matter most for data protection. In Understand: who decides, who delivers, who uses? In Build: which roles, decision paths and interfaces does operation require? In Roll out: who is accountable for operation from day one?

In AI management, requirements and the business case emerge together with a sequence of applications in which data classes are considered from the outset. In prototyping, operationalisation and rollout, the review model is transferred into processes and responsibilities, and business units are enabled to apply it. The legal assessment in the individual case remains with the data protection officer and the legal department. Niehues Consulting provides the framework in which this assessment takes place on time and repeatably, with named owners and metrics from which the executive team can read the status.

Founder Marin Niehues is a management consultant who builds organizations that deliver and brings AI into operation. His projects range from VC-funded scale-ups to DAX 40 corporations in 20+ countries. The entry point is a free initial consultation of 30 minutes.

Frequently asked questions

Is ChatGPT GDPR-compliant?

ChatGPT can be used in compliance with the GDPR when the company uses a business plan or the API with a data processing agreement and training on inputs is excluded. Personal accounts are unsuitable for personal data. Transfers to the United States additionally require a legal basis for the transfer.

Can employees put personal data into ChatGPT or other language models?

Employees may enter personal data only into tools with a reviewed contract and for a defined purpose. Content without personal data is permitted in every approved tool. Special categories such as health data require a separate review for each application. A written policy makes these rules binding.

Does ChatGPT use my data for training?

With business plans and the APIs of the major vendors, training on inputs is excluded by contract. With personal accounts it is switched on by default at some vendors and can only be switched off by the individual user. Check the setting during vendor selection and record it in the approval list.

Is pseudonymised data still personal data under the GDPR?

Pseudonymised data remains personal data, whereas anonymized data falls outside the GDPR. Data is anonymous only when the person cannot be identified even with additional knowledge, which free text rarely achieves. Pseudonymisation is a sensible safeguard but replaces neither the contract nor the legal basis.

How high are GDPR fines for AI-related breaches?

The GDPR provides for fines of up to 20 million euros or 4 percent of worldwide annual turnover, and up to 10 million euros or 2 percent for organizational obligations. The higher figure applies in each case. Authorities take the safeguards in place into account when setting the amount, so a documented review model has a mitigating effect.

Do AI applications need a data protection impact assessment?

A data protection impact assessment is mandatory where an AI application is likely to result in a high risk for the individuals concerned, for instance because it evaluates people or processes special categories on a large scale. For pure text assistance without such data, a documented risk assessment is sufficient. The company takes the decision and records it.

What should a data processing agreement with an AI vendor include?

The agreement covers subject matter and duration of processing, the vendor's duty to follow instructions, technical safeguards and rules on sub-processors. For AI, the exclusion of any use of inputs for the vendor's own purposes and clear retention periods are added. Data protection and the legal department review the detailed wording in each case.

Who is responsible for data protection when a company uses AI?

The company is responsible, represented by the executive team, while the business unit that uses an application takes the decision on it. Data protection officers advise and monitor but do not decide. Name an accountable person for every approved application.

Let’s talk about your initiative.

In 30 minutes we clarify where your organization stands and which step will pay off most now.

Request a consultation

30 minutes · free · no obligation