Artificial intelligence has moved from promise to reality in thousands of companies: customer service chatbots, credit scoring systems, recruitment tools, and predictive models that recommend products. Without exception, all of these involve processing personal data. Regulation (EU) 2016/679 —known as the GDPR— governs that processing regardless of whether the tool is a human, an algorithm, or an artificial intelligence model. This article analyses the most common friction points between AI and the GDPR: automated decisions under Article 22, the legal basis and information obligations for chatbots, data minimisation in AI models, and the intersection with Regulation (EU) 2024/1689 —the AI Act—.
Why does artificial intelligence pose specific challenges to the GDPR?
The GDPR was adopted in 2016 and does not expressly mention language models or generative AI, but its principles apply in full force. When a company deploys an AI model that processes data of natural persons, it must comply with the same principles that govern any other processing activity: lawfulness, fairness and transparency; purpose limitation; data minimisation; accuracy; storage limitation; and integrity and confidentiality (Art. 5.1 GDPR). In addition, it is subject to the accountability principle in Article 5.2: the organisation must be able to demonstrate compliance at any time to the Spanish Data Protection Agency (AEPD — Agencia Española de Protección de Datos).
The distinctive feature of AI is that it intensifies risks in two dimensions. First: the volume and variety of data it consumes to operate — conversation history, metadata, behavioural data, and sometimes special-category data. Second: the opacity of outputs, which makes it difficult to explain why the system reached a particular decision and how it got there.
Article 22 GDPR: automated decisions and profiling
What Article 22 prohibits and what it permits
Article 22.1 of the GDPR states that «every data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her». This is the central rule affecting AI systems that make decisions without genuine human involvement.
Two elements must be present for Article 22 to apply:
- The decision is based solely on automated processing, without effective human review capable of challenging the outcome.
- The decision produces legal effects — denial of a contract, termination of an employment relationship — or significantly affects the data subject — denial of access to a service, discriminatory pricing —.
It is not sufficient for a human to formally approve the decision: if that person cannot — or in practice does not — question the algorithm's output, the AEPD may consider that the decision is still «based solely» on automated processing for the purposes of Article 22.
Exceptions and minimum safeguards
Article 22.2 provides three exceptions that permit this type of automated decision-making: (a) where it is necessary for the entering into, or performance of, a contract between the data subject and the controller; (b) where it is authorised by Union or Member State law; or (c) where it is based on the data subject's explicit consent.
In cases (a) and (c), the controller is required to implement suitable measures to safeguard the data subject's rights: at minimum the right to obtain human intervention, to express his or her point of view, and to contest the decision (Art. 22.3 GDPR). These are not optional: they must be genuinely exercisable in practice, not merely stated in the privacy policy.
Where the automated decision involves special categories of data — health, racial or ethnic origin, sexual orientation, trade union membership, among others — only exceptions (a) or (b) with enhanced safeguards are admissible, or the explicit consent referred to in Article 9.2(a) GDPR (Art. 22.4 GDPR). In these cases it is recommended to carry out a Data Protection Impact Assessment (DPIA) pursuant to Article 35 GDPR.
| Case | Enabling basis | Mandatory safeguards |
|---|---|---|
| Decision to enter into or perform a contract | Contractual necessity (Art. 22.2.a) | Human intervention + right to express view + right to contest |
| Decision authorised by EU or national law | Express enabling rule (Art. 22.2.b) | Those established by the enabling rule |
| Decision based on explicit consent | Consent (Art. 22.2.c) | Human intervention + right to express view + right to contest |
| Decision involving special-category data | Only (a) or (c) with enhanced safeguards | Human intervention + DPO informed + DPIA recommended |
Chatbots and data protection: legal basis and information obligations
Legal basis for data processing in chatbots
A typical customer service chatbot processes, at minimum, the content of the conversation and session metadata. If integrated with a CRM, it may cross-reference those data with purchase history, previous incidents, or profile data. The most common legal basis will be performance of a contract (Art. 6.1.b GDPR) when the user is already a customer, or legitimate interests (Art. 6.1.f GDPR) in the pre-contractual stage, provided that basis passes the prior balancing test.
Consent (Art. 6.1.a GDPR) is the least advisable basis for a service chatbot because it must be freely given: if the user cannot access the service without interacting with the chatbot, that consent can hardly be considered free within the meaning of the GDPR. Consent may, however, be the appropriate basis where the chatbot collects data for additional purposes — personalised marketing, model improvement using user data — that go beyond the primary service.
What information must be provided to the user
Article 13 of the GDPR, which applies when data are collected directly from the data subject, requires information to be provided at the time of collection. In a chatbot context this means that, before or during the first exchanges of the conversation, the user must be informed of:
- Who the data controller is and how to contact them.
- The purpose for which their data are processed and the legal basis.
- Whether the data will be shared with third parties, including the AI model provider.
- The retention period for the conversation data.
- Their rights of access, rectification, erasure, restriction, portability, and objection.
- Whether the chatbot's responses involve an automated decision affecting them, the logic applied, and the envisaged consequences (Art. 13.2.f GDPR).
This last point is particularly important: if the chatbot can, on its own, deny a request, apply differential pricing, or escalate or not escalate a complaint according to an algorithmic criterion, the user must be informed that such automated processing exists and of the logic applied. A mere reference in the terms and conditions does not satisfy this obligation when it is not prominently displayed at the point of data collection.
Data minimisation in artificial intelligence models
The minimisation principle (Art. 5.1.c GDPR) requires that data processed be «adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed». In the AI context, this principle operates at three distinct stages of the system's lifecycle.
At model training. If the company develops or fine-tunes its own model, the training data must have been obtained lawfully and must be the minimum necessary. Using production data — real customer conversations — to train the model requires a specific legal basis for that purpose, which normally differs from the basis of the original processing. Incompatible use of data without an independent legal basis may constitute an infringement of Article 5.1.b GDPR.
At inference. The model should not receive more personal information than is strictly necessary to provide the requested response. A chatbot that handles billing disputes does not need access to the customer's medical history or their real-time geolocation. The technical design must reflect this minimisation principle from the outset — privacy by design, Art. 25 GDPR —.
In log and conversation retention. Conversation transcripts are personal data. Their indefinite storage, which is common practice in many CRM and chatbot systems, conflicts with the storage limitation principle (Art. 5.1.e GDPR). Retention periods justified by reference to the processing purpose must be established, along with procedures for effective erasure or anonymisation at the end of that period.
The intersection with the AI Act (Regulation EU 2024/1689)
Regulation (EU) 2024/1689, known as the AI Act, entered into force in August 2024 and its application is phased according to the risk category of the system. It does not replace the GDPR: both regulatory frameworks apply simultaneously when an AI system processes personal data. The AI Act adds a layer of obligations based on the risk level of the AI system, irrespective of whether it processes personal data.
High-risk AI systems — listed in Annex III of the AI Act, including systems for recruitment, credit scoring, decisions in essential public services, and the administration of justice — are subject to specific requirements: documented risk management, training-data governance, transparency, mandatory human oversight, and registration with the competent authorities. Many of these requirements overlap with GDPR obligations: the DPIA under Article 35 GDPR and the AI Act conformity assessment cover partially the same ground.
Limited-risk AI systems, such as general-purpose chatbots, have lighter transparency obligations: informing the user that they are interacting with an AI system, unless this is obvious from the context.
Our AI Act compliance service provides a roadmap for companies to comply simultaneously with the GDPR and the AI Act, identifying which AI systems are in use, which risk category they fall into, and what technical and organisational measures must be implemented to demonstrate compliance to the AEPD and to the Spanish Artificial Intelligence Supervisory Agency (AESIA — Agencia Española de Supervisión de la Inteligencia Artificial), operational since September 2024.
| Aspect | GDPR (EU 2016/679) | AI Act (EU 2024/1689) |
|---|---|---|
| Object of protection | Personal data of natural persons | AI systems according to risk level |
| Competent authority in Spain | AEPD | AESIA (operational since 2024) |
| Automated decisions | Art. 22: right not to be subject to them without safeguards | Mandatory human oversight for high-risk systems |
| Transparency | Information to the data subject (Arts. 13–14) about the logic applied | Inform the user that they are interacting with AI |
| Impact assessment | DPIA where high risk exists (Art. 35) | Conformity assessment for high-risk systems |
| Maximum penalty | Up to €20 M or 4 % of total worldwide annual turnover | Up to €35 M or 7 % of total worldwide annual turnover |
Practical obligations for your company
If your company uses or plans to deploy AI systems that process personal data, the starting point is a structured assessment that answers, at minimum, these questions:
- What personal data does the system process and for what exact purpose?
- What is the legal basis under Article 6 GDPR for that processing?
- Does the system produce decisions that affect data subjects within the meaning of Article 22 GDPR?
- Does the system fall into any risk category under the AI Act — in particular Annex III?
- Have data subjects been properly informed about the use of AI and the logic of the decisions?
- Are there effective mechanisms for human intervention and for contesting decisions?
- Has a DPIA been carried out pursuant to Article 35 GDPR where the processing so requires?
- Is there a data processing agreement with the AI system provider?
The Summum Consultoría team, with a presence in Castilla y León — Valladolid, Burgos, Palencia, Aranda de Duero — and the Canary Islands — Las Palmas —, supports SMEs and larger organisations in bringing their artificial intelligence systems into compliance with the GDPR and the AI Act. From the initial analysis through to the implementation of technical and organisational measures, the goal is for the company to demonstrate documented compliance. Consult our AI Act compliance service for an initial assessment of the regulatory impact of AI on your organisation.
Frequently asked questions
Does a chatbot that only collects a name and email address require a legal basis under the GDPR?
Yes. Any processing of personal data — even if limited to a name and email address — requires a legal basis under Article 6 of the GDPR. For a chatbot that captures enquiries, the most common basis is legitimate interests (Art. 6.1.f GDPR), provided the balancing test is passed, or consent (Art. 6.1.a GDPR). The legal basis and purpose must be communicated to the user at the time of collection, in accordance with Article 13 of the GDPR.
If the chatbot provider accesses my customers' data, what obligations apply?
The system provider acts as a data processor within the meaning of Article 28 of the GDPR. It is mandatory to enter into a data processing agreement with the provider that governs: processing instructions, applicable security measures, the sub-processor chain, audit rights, and the procedure for returning or erasing data at the end of the contract. If the provider is located outside the European Economic Area, the safeguards of Chapter V of the GDPR also apply — standard contractual clauses, adequacy decisions —.
When is a DPIA mandatory for an artificial intelligence system?
Article 35 of the GDPR requires a Data Protection Impact Assessment where processing is likely to result in a high risk to the rights and freedoms of natural persons. The AEPD has published a list of types of processing that always require a DPIA; these include systematic profiling, large-scale processing of special-category data, and automated decision-making with legal effects. An AI system that combines two or more of these criteria will almost always require a DPIA before it is put into production.
Does the AI Act replace the GDPR for artificial intelligence systems?
No. The AI Act and the GDPR are complementary regulatory frameworks that apply simultaneously. The AI Act governs AI systems according to their risk level; the GDPR governs the processing of personal data. The obligations of the AI Act — transparency, human oversight, technical documentation, conformity assessment — do not replace those of the GDPR — legal basis, information to the data subject, rights, DPIA —, but rather add to them. The LOPDGDD (Organic Law 3/2018 of 5 December on the Protection of Personal Data and guarantee of digital rights) supplements the GDPR in the Spanish legal order without altering this framework.