ICO publishes report on gen AI, foundation models and data protection

The IC has published "Building trust and transparency into generative AI development: our work to create regulatory certainty," setting out the regulatory expectations arising from its three-year supervision programme covering the 11 largest foundation model developers active in the UK (Amazon, Anthropic, Apple, Cohere, DeepSeek, Google, Meta, Microsoft, OpenAI, Stability AI and X.AI). While framed around foundation model developers, the report is directly relevant to any organisation deploying, fine-tuning or procuring generative AI tools, since many of the compliance expectations flow down the supply chain.

Headline message

The IC’s position is unambiguous: where a developer processes personal data to train models, it must:

• identify a lawful basis
• provide meaningful transparency
• enable people to exercise their rights, and
• demonstrate safeguards that materially reduce risk.

The IC recognises that applying the GDPR to the processing of personal data (particularly special category data) in the context of AI is sometime complex but stresses that complexity is not an excuse for non-compliance. Companies must comply with the IC guidance on data protection and AI.

Key compliance themes and ICO expectations

1. Transparency.

The IC is concerned about lack of transparency in relation to the use of personal data in the development of generative AI. It perceives that there are continuing, worsening transparency shortfalls across the industry with vague language, information scattered across multiple notices, and overly technical drafting. The IC is also concerned that the disproportionate effort exemption (applicable when data is gathered from sources other than the data subject) is being used too liberally rather than assessed on a case by case basis. Developers must now determine, and be able to evidence, the extent to which they process personal data in all forms (including indirect identifiers, special category data and pseudonymised data) and clearly distinguish transparency information for first-party data from third-party / scraped data. Reliance on the disproportionate effort exemption must be evidenced on a case-by-case basis and not used as a blanket position.

2. Lawful basis — legitimate interests.

Legitimate interests remains, in the IC's view, realistically the only available lawful basis for web-scraped training data, but the IC found gaps in developers' assessment of the application of the legitimate interests lawful basis. Following the IC (and indeed the EDPB guidance for the EU) developers must document an assessment that evidences:

• a valid and specific interest for processing each type of personal data (including first-party data, third-party data, and special category data) at each stage of development. Broad expressions of interests such as "improving our products" or "benefitting humanity" will not suffice
• the necessity of using the relevant data for the identified purpose (including analysis of less privacy intrusive alternatives)
• an assessment of the impact on individuals’ rights and freedoms (including their ability to exercise their rights and not be subject to “invisible” processing) and how risks are mitigated and that mitigation evidenced.

3. Special category data.

The IC's research indicates foundation model developers are likely processing special category data, intentionally or not, unless they can positively demonstrate otherwise — and many developers were not actively engaging with the GDPR requirements for that processing (in Article 9). Only two Article 9 conditions are realistically available. The first is that the data has been "manifestly made public" by the data subject (Article 9(2)(e)) but this requires verified evidence of the data subject's intent to share the data publicly and this is unlikely, in the IC’s view, to be the case for scraped third-party data. And the second is "scientific research" (Article 9(2)(j)), which will not cover all training activity. Where no condition is available, the IC's expectation is that organisations prevent collection of, filter out, or otherwise avoid processing special category data.

4. Data subject rights.

The IC identified poor practice including inaccessible or absent mechanisms for access, erasure and objection requests (especially for third-party data subjects), "privacy maze" layering, and blanket refusals without adequate explanation. Developers must provide clear, accessible routes for rights requests (including third-party data), explain technical limitations affecting outcomes, and justify any refusal with sufficient detail to allow meaningful challenge, including information on the right to complain to the IC or pursue court enforcement.

5. Do models themselves contain personal data?

The IC maintains its 2020 position that personal data extractable from a trained model means the model itself can contain personal data and fall within scope of data protection law — a position now also supported by the European Data Protection Board's December 2024 opinion. This requires case-by-case assessment of memorisation risk (influenced by model size, data duplication, training epochs, and similar factors), and the IC expects testing for "singling out" and "linkability" risk using its anonymisation guidance methodology.

Supervisory outcomes and enforcement signal

Following IC reviews and industry engagement, leading developers have made or have committed to make specific changes to their privacy compliance programmes. These include updating legitimate interest assessments, standalone model-training privacy notices, clearer retention and international transfer disclosures, and improved rights-request mechanisms.

Most significantly for enforcement risk: the IC has opened a formal investigation into X Internet Unlimited Company and X.AI LLC over Grok's processing of personal data and its potential to generate harmful sexualised content and has suspended X.AI's participation in the voluntary supervision programme as a result. The outcome of this investigation will further inform the market on the IC’s approach to generative AI development and privacy risks.

What's next from the IC?

The IC intends to develop a statutory code of practice on AI and automated decision-making and engage in continued monitoring of foundation model developers (including on how they justify reliance on legitimate interests, provide meaningful transparency, enable exercise of rights and mitigate risks). It will also engage with the UK Government to discuss how special category data may be used to promote innovation but in a way that protects individuals.

The IC will also have a focus on increasing personalisation of consumer services / engagement through the use of AI.

Practical actions

• Developers of AI models: Revisit (or complete) LIAs ensuring that they robustly and specifically address the LIA tests (identification of specific, not generic, interests; necessity of processing; impact assessment and evidenced risk mitigations). Map how the company meets first- vs third-party transparency obligations separately and stress-test any reliance on "disproportionate effort" exemptions on a case-by-case basis rather than as a blanket policy.
• Deployers of AI models: The IC expects accountability across the AI model / system supply chain. Deployers should conduct due diligence on a developer's transparency and lawful basis position as part of AI system procurement.
• All organisations processing special category data via AI: Given the limited availability of Article 9 conditions, prioritise technical filtering / exclusion measures over attempting to construct a condition that may not hold up.
• Rights-handling teams: Review request-handling mechanisms (access, erasure, objection etc) to ensure that it is easy for individuals to engage in the process of seeking to exercise their rights. Ensure refusal decisions are documented and explainable.
• Watch for further developments: The outcome of the Grok/X.AI investigation and the forthcoming statutory AI/ADM code of practice will be important developments in the IC’s guidance on development and deployment of AI.