Privacy International on September 25, 2026 published its evidence to the UK Department for Science, Innovation and Technology (DSIT), arguing that artificial intelligence erodes the transparency duties and individual rights on which British data protection law depends, and that five categories of AI processing carry risks too high to permit. The 29-paragraph document is dated September 9, 2026, the day the department's call for evidence closed. It also contends that the Data (Use and Access) Act 2025 moved the burden of policing automated decisions away from organisations and onto the people those decisions affect.

In Short

Privacy International, a UK charity, told the British government that data protection law was built on the idea that companies know what personal data they hold, and that AI systems break that idea at every step. This affects anyone whose information trains an AI model or gets guessed by one, because you cannot ask to see or correct a guess about you that you do not know exists. Nothing has changed in law yet, but the government is now weighing whether to issue guidance, make targeted fixes or rewrite the rules more fundamentally.

A call for evidence framed around adoption

DSIT published its call for evidence on data regulation in the age of AI and other data-intensive technologies on July 15, 2026, and closed it at 11:59pm on September 9, 2026, according to the department. The exercise ran across five themes: accessing and using data; data quality, accuracy and downstream impacts; governing data use across organisations; transparency and rights in complex data environments; and the effectiveness of data frameworks in regulating AI. Copyright and intellectual property were left out, having been the subject of separate consultations.

The government's framing left little doubt about its priorities. According to DSIT, ministers aim to make the UK the G7 economy that takes up AI fastest, and evidence already gathered through the department's AI Growth Lab call had surfaced industry difficulties with lawful bases for large-scale personal data use, data minimisation, purpose limitation, data subject rights and the allocation of responsibilities across supply chains. The department cited its own figures: 83% of firms handle data, 73% analyse it and 15% share or sell it, while data-driven companies generated £85 billion in gross value added in 2022 and employed 1.5 million people in 2023. It said the responses would help it judge whether further guidance, targeted changes or more fundamental reform were needed.

Where the call sought evidence of friction, the answer from Privacy International (PI) concentrates on the opposite risk: protections that exist on paper but have become difficult to exercise.

The organisation is a registered charity in England and Wales (number 1147471) and a company limited by guarantee (04354366), based at 62 Britton Street in London. According to the submission, it researches and campaigns globally against government and corporate abuses of data and technology, and has examined AI in intelligence agencies, immigration and recruitment. Two pieces of that work are cited: a complaint to the UK regulator over two Home Office immigration algorithms, and a July 9, 2026 long read titled "Humanless Resources? Uncovering AI recruitment software". The submission dates the press release on the immigration complaint to August 15, 2025, whereas the charity filed the complaint over the IPIC and EMRT tools with the Information Commissioner's Office (ICO) on August 18, 2025. The two dates may describe different steps; neither source reconciles them.

Of DSIT's five themes, the charity answered two questions. Under Theme 4, on transparency and rights, it responded to prompt question 2, which asks how effective the framework's transparency requirements and data subject rights are for data-intensive technologies, and whether they will stay fit for purpose as technology advances. Under Theme 5, it took prompt question 1: what is working, where barriers have appeared, and where risks to data protection have arisen.

The premise that controllers know what they hold

The argument starts from a structural observation. Data protection law, the submission states, "is structured around the premise that data controllers know their own processing: what personal data they hold, where it came from, the purposes and legal basis for processing it, and the consequences of that processing for data subjects."

Article 5(1)(a) of the UK GDPR requires personal data to be processed transparently, and Articles 12, 14 and 15 give that principle effect. The rights of access, rectification and erasure assume the controller can answer the questions those articles pose. PI's claim is that AI makes the assumption unreliable at three stages of the lifecycle: training, where data enters a system; process, where the system uses it; and output, where it produces something new.

Training: scale, ingestion and the mosaic effect

Scale comes first. According to PI, AI systems are typically trained on large datasets that are often merged and unstructured, drawn from wide sources that can include data scraped from the internet, data collected for one purpose and repurposed for another, and data bought from a private company specifically for training. That last category now carries a price tag. The charity's footnote points to reporting on Google's plan to train AI with Spirit Airlines data, and Google won the bankrupt carrier's data for $10 million in a US bankruptcy auction in August 2026, with Mercor.io named as alternate bidder.

The consequence, in PI's telling, is that a controller may be unable to establish what personal data a training set contains, about whom, from which source, or whether the people concerned were ever notified. It also flags the risk of breaching data minimisation where data is collected and processed "just in case".

Nor does data simply pass through. Drawing on the European Data Protection Board's Opinion 28/2024, adopted on December 17, 2024, PI says personal data can be "ingested" into a model, memorised and retained inside it. Ingested data may be "represented through mathematical objects", yet will likely still constitute personal data, and any claim that a model holds only anonymous data has to be assessed case by case, the submission states, citing paragraphs 31 and 34 of the opinion. That opinion established that AI models trained on personal data cannot automatically be considered anonymous. How much gets retained is still argued over; University of Tübingen researchers estimated that large language models memorise between 0.1% and 10% of training data. If training data is not stored in its original form, PI argues, it becomes difficult to identify, isolate, trace or remove, which undermines responses to requests for access, rectification and deletion.

The boundary between personal and anonymous data blurs further. Data that looks innocuous or anonymised in isolation can be combined with other datasets to re-identify people or expose more sensitive information, which the submission calls the "mosaic effect". AI, PI argues, strengthens that capacity because it can analyse different sources together and find correlations no single dataset would show. A controller may therefore hold data it believes to be anonymous that is in fact personal. The charity cites paragraphs 83 to 93 of the EDPB's Guidelines 02/2026 on Anonymisation, which the board adopted on July 7, 2026 and opened for consultation until October 30, 2026. Section 171 of the Data Protection Act 2018 makes it a criminal offence, in certain circumstances, to re-identify de-identified data. That offence, in PI's words, "polices misconduct: it doesn't restore the controller's knowledge."

The submission then turns to Brussels. It treats onward identifiability - data anonymous for one actor but re-identifiable by another - as a live risk, and says proposals in the European Commission's GDPR omnibus to amend Article 4 "would significantly narrow the definition of personal data, therefore creating loopholes in data protection law, and undermining people's rights." It adds, without qualification: "They should be avoided." The charity relies on paragraph 17 of EDPB-EDPS Joint Opinion 2/2026, the text in which the two authorities rejected the Commission's plan to narrow GDPR protections in February 2026. The UK is not bound by the omnibus. In a submission addressed to a UK department, the reference reads as a warning against a comparable change in British law.

Purpose is the next difficulty. Article 5(1) of the UK GDPR requires collection for explicit and specified purposes, and Article 8A confines further processing to compatible uses. General-purpose models, PI notes, are built precisely to be turned to tasks that cannot be specified in advance, and a lax approach could normalise large-scale data accumulation in breach of purpose limitation. Greater upstream transparency and participation of data subjects in model design, including through Data Protection Impact Assessments (DPIAs), "may be one way to help ameliorate these risks, if done in a meaningful way."

Notification is where the argument sharpens. Article 14 of the UK GDPR requires controllers to inform people when personal data was not obtained from them directly, but Article 14(5)(e) lifts that duty where doing so is "impossible or would involve a disproportionate effort." A controller that does not know whose data sits in its training set cannot notify those people, and PI expects the exemption to be relied on excessively as scraping and vast data collection become normal. If that is permitted, the rights of access, rectification, erasure and objection are "effectively rendered meaningless", because people who do not know their information is being processed are unlikely to use them. DPIAs do not fill the gap, the charity argues: they are not mandatory in every instance, need not be published, and are produced without data subjects' involvement. The EDPB only adopted its first standardised DPIA template on March 10, 2026, almost eight years after the GDPR became applicable.

A related risk concerns special category data under Article 9. A controller unaware that it is processing such data may breach those provisions unwittingly, and PI argues that the notification exemption must not become a loophole as companies build AI models into products and services through large-scale scraping and combination of data.

Process: the black box and the logic involved

The second stage concerns what happens inside the system. According to PI, an AI system's immediate inputs and outputs can be observed, but the process that turns one into the other is often too complex, opaque or uninterpretable for humans, developers included, to understand. That collides with the requirement in Article 5(1)(a) that data be processed "in a transparent manner", and with Articles 13(2)(f), 14(2)(g) and 15(1)(h), which entitle people to "meaningful information about the logic involved." The charity's conclusion is blunt: "Information that a controller cannot itself understand or interpret is unlikely to be meaningful to a data subject."

European courts have covered similar ground. The Court of Justice ruled in Dun and Bradstreet (C-203/22) on February 27, 2025 that complex algorithmic descriptions alone do not amount to a comprehensible explanation, a line of reasoning the Dutch data protection authority built on when it opened a consultation on explaining automated decisions to individuals in April 2026.

The UK's own changes to automated decision-making (ADM) heighten the concern, PI writes. The submission quotes the ICO's characterisation that "[t]he amendments reframe the ADM provisions from a prohibition with exceptions to a right of challenge with safeguards." That quotation is sourced, in PI's footnote, to a Bird & Bird briefing dated May 5, 2026 on the ICO's consultation on draft ADM guidance, rather than to an ICO publication directly. Under Article 22C(2) of the UK GDPR, the safeguards require controllers to notify people of ADM decisions, let them make representations, obtain human intervention and contest decisions. The submission's footnotes cite paragraph (c) for both human intervention and contestation, an apparent duplication the document does not explain. Such safeguards, PI argues, work only if people understand enough about a decision: "It is difficult to challenge reasoning that is opaque and incomprehensible, and a reviewer would need to understand it enough to intervene."

Personalisation features draw separate attention. PI names "memory" functions and retrieval-augmented generation, both of which augment prompts with additional information to personalise responses, as processing that may be unexpected or poorly understood by the people whose data is involved. "Users must retain control over this further ingesting of their data," the submission says, in line with data minimisation and transparency. It also warns that concentrating large amounts of personal data, and access to user accounts, in one place creates "an attractive target for malicious attacks and/or inappropriate government use." These are not hypothetical products in Britain: Google brought Gemini's memory setting and chat import tools to UK users on April 29, 2026. The Council of Europe's draft guidelines on large language models, which address chatbot memory and retrieval systems, were scheduled for review by the Convention 108 Bureau on September 16 and 17, 2026, ahead of planned adoption in November.

Agentic systems raise two further questions. Tools that look up and process personal data in ways their developers did not anticipate may require a fresh assessment of compliance, including of the legal basis, and may trigger transparency duties; yet, the submission notes, "it is unclear how that can be done in the middle of an agentic operation." Systems that chain several models together may also re-identify previously anonymised data, or combine it into inferences, in ways never explained to the people concerned. Four UK regulators - the Competition and Markets Authority (CMA), the Financial Conduct Authority, the ICO and Ofcom - published a joint foresight paper on agentic AI on March 31, 2026, and Spain's AEPD had already issued a guide on the GDPR risks of agentic AI in February.

Output: data that was made, not given

The third stage may matter most to advertisers. Data protection law, PI argues, assumes controllers can anticipate the results of their processing. AI systems instead generate new personal data in the form of regurgitations or inferences, including incorrect outputs known as hallucinations. Inferences are predictions or assumptions about a person derived from data already held. The submission's examples are specific: a name used to infer religion or ethnicity, or shopping habits used to infer pregnancy or illness.

Such inferences can themselves be personal data. They are probabilistic, they can be wrong, and through the mosaic effect they can turn innocuous data into personal or special category data. Because they are created after collection, they may arise after the moment at which Article 13 requires purposes and lawful bases to be fixed and explained. "Accordingly, the data subject will be told or aware of only the data they gave, not the data that was made," the submission states. An inference can shape a decision even when the controller does not know it exists.

The rights meant to catch such data struggle with it. A person "cannot request access to an inference they don't know exists." Rectification under Article 16 assumes data can be shown to be "inaccurate", and PI questions whether a probabilistic estimate, as opposed to a fact, clears that threshold. Erasure assumes data can be located and deleted, yet personal data absorbed by a model may be impossible to make it forget, and may be reproduced from the model even after the input is deleted. Campaigners are not alone in saying so: a lawsuit against the notetaking company Granola cited the company's own materials acknowledging that data used in training cannot be isolated or removed afterwards. Separately, researchers tested eight models to see whether ordinary people can find out what a language model has learned about them.

Two further points close the section. "Guardrails are inevitably limited in their ability to prevent an AI from outputting personal data," PI writes, and a disclaimer warning that AI may make errors "does not mitigate the underlying data protection risks" of inaccurate personal data being generated or relied on. The charity also objects to recent changes that limit the right of access to what a "reasonable and proportionate search" can find, under Article 15(1A). Combined with AI's technical characteristics, it argues, such changes "should not be interpreted in a manner that creates practical exemptions from data protection obligations."

Automated decisions after the 2025 Act

The Data (Use and Access) Act 2025, which received Royal Assent on June 19, 2025, rewrote the ADM rules through section 80, replacing Article 22 of the UK GDPR with Articles 22A to 22D. Before the change, according to the submission, ADM was permitted only in narrow circumstances: where necessary for a contract with the data subject, authorised by law, or based on consent. Such decisions are now generally permitted, and only significant decisions involving special category data remain prohibited, subject to exceptions and conditions under Article 22B(1). The Act was presented at the time as a more permissive framework for solely automated decisions, with safeguards attached.

PI sets out three objections. The first concerns marginalised groups. The government itself, in a DSIT impact assessment dated October 23, 2024, acknowledged that "those with protected characteristics such as race, gender, and age are more likely to face discrimination from ADM due to historical biases in datasets." Citing a December 2024 Big Brother Watch briefing, PI argues that the remaining prohibition on ADM involving special category data does little to reduce that risk, because ordinary personal data can act as a proxy for protected characteristics, and biased training data can reinforce discrimination.

The second is a shift in the burden of proof. Rather than controllers having to justify ADM, such processing is now generally presumed compliant unless it can be shown that the required safeguards were not followed, according to PI. That places the onus on individuals who, if the safeguards were skipped and they were never notified, may not know a decision was taken at all. The third objection returns to opacity. Safeguards that depend on both the individual and the reviewer understanding the reason for a decision are difficult, if not impossible, to use when the underlying reasoning cannot be followed.

European practice points the other way. The Dutch data protection authority fined Uber 824,990,000 euros on August 21, 2026 for automatically deactivating drivers without human involvement between 2018 and 2022, a decision Uber has appealed. And the Council of the EU's September 3, 2026 compromise on the Digital Omnibus restores Article 22 as a right not to be subject to solely automated decisions, undoing the Commission's permission-based redraft. That text is a negotiating draft, not law, but it marks a clear divergence from where the UK landed in 2025.

Five uses the charity wants prohibited

The Theme 5 answer starts from a structural claim. Data protection laws, PI writes, citing a 2025 book by Brendan McGurk KC and Joe Tomlinson, are currently the most comprehensive frameworks regulating AI in the UK, yet they do not address every AI-related risk. The most significant gap, in its view, concerns uses of AI that "pose such a high-risk of violating human rights that they should not be permitted." Such prohibitions would be consistent with the EU AI Act, the charity argues, and would reflect UN General Assembly resolutions A/RES/78/265 and A/RES/80/215 as well as the views of independent experts such as the UN Special Rapporteur on counter-terrorism and human rights, whose office published a position paper on the subject on December 15, 2025.

The list opens with sentiment analysis, defined as processing personal data to detect, measure or infer a person's emotional state. Behavioural prediction covers processing personal data to predict the likelihood that a person will commit a crime or threaten national security, based solely on AI profiling or on an assessment of personality traits and characteristics. Profiling based solely on protected activities would bar profiling, targeting or tracking a person solely because they exercise human rights such as freedom of expression and assembly. The remaining two concern biometrics: live identification, meaning the processing of biometric data for real-time remote identification of a person, and scraping to create biometric databases, meaning the use of AI to build or expand such databases through untargeted scraping of biometrics and personal data from the internet or CCTV footage.

Several of these overlap with EU law already in force. The AI Act's Article 5 prohibitions, including a ban on AI systems that infer emotions in the workplace, have applied since February 2, 2025; Italy's data protection authority has since cited that provision in warning a startup over a Slack plug-in built to detect stress. The scraping category describes a business model that has drawn repeated European enforcement. Clearview AI, which assembled more than 60 billion facial images, faced a criminal complaint from noyb in Austria on October 28, 2025 after ignoring roughly 100 million euros in fines.

PI's version of the emotion category is wider than the AI Act's context-specific prohibition, because it is not tied to any particular setting. For advertising, the precise wording matters. In media buying, sentiment usually describes the tone of a page or video, scored by contextual and brand safety vendors to decide where an ad runs. PI's definition instead targets personal data processed to infer a person's emotional state. On its wording, classifying content rather than people would appear to fall outside the proposed prohibition, while systems that infer an individual user's mood for targeting or measurement would fall inside it. The submission does not discuss advertising directly.

Competition, intelligence services and blind spots

PI identifies three further gaps. The first lies between competition and data protection. Dominant firms, it argues, can use existing market power to shape the development of the AI market and gain more control over, and potentially misuse, people's data, which may reduce the availability and viability of privacy-protective alternatives, limit consumer choice and produce poorer privacy outcomes. The submission names no company. The UK's competition regime has been active in adjacent territory: the CMA designated Google with strategic market status in general search services in 2025, and imposed its first binding conduct requirement on June 3, 2026, covering the use of publisher content in generative AI features, with the substantive obligations taking effect on December 3, 2026.

The second gap is narrower and highly specific. Section 108(1) of the Data Protection Act 2018 requires the intelligence services, when acting as controllers, to notify the Information Commissioner of serious personal data breaches. They need not do so "if the breach also constitutes a relevant error within the meaning given by section 231(9) of the Investigatory Powers Act 2016." That section defines a relevant error as one "by a public authority in complying with any requirements which are imposed on it by virtue of this Act or any other enactment, and which are subject to review by a Judicial Commissioner". The Investigatory Powers Commissioner's Office (IPCO) summarises the term as "an error made by a public authority when carrying out activity overseen by IPCO."

The effect, according to PI, is that such breaches can bypass the ICO altogether, while IPCO has no data protection functions and no relevant enforcement or remedial powers. Who, then, polices them? IPCO raised the question itself in its 2024 annual report, quoted in the submission: at present, "UKIC could commit a serious personal data breach which might not come to the attention of the competent supervisory authority for data protection unless we refer the matter." The commissioner added that it was not best placed to make such referrals, and that "this may leave a gap which could be contrary to the public interest." PI expects the gap to widen as the intelligence community uses more AI, since AI can amplify the scale and consequences of breaches involving large volumes of personal data.

The third is a governance point. The lack of a clearly defined purpose for AI models creates problems beyond purpose limitation, the charity argues: "Proper scrutiny of its development and deployment may be evaded if there are not adequate frameworks and institutions" able to weigh risks, harms, public benefits and costs, and to act where necessary.

How the argument lines up with EU drafting

Timing gives the submission an unusual counterpart. Six days before PI's document was dated, the Irish Presidency of the Council of the EU circulated a 158-page Digital Omnibus compromise, dated September 3, 2026, which noyb made public on September 21. Several provisions in it touch the questions PI raises. The Commission's proposed additions to the personal data definition in Article 4(1) are struck, with the question moved into a new Article 25a on pseudonymised data. A new Article 9(2)(k) addresses sensitive data that ends up in AI development unintentionally and residually, requiring erasure once identified, or protection where erasure is technically impossible or would take disproportionate effort.

Germany's written comments on an earlier Presidency text, dated August 17, 2026, went in a different direction on notification. Berlin proposed that the Article 14 information duty and the rights of rectification, erasure and restriction in Articles 16 to 18 would not apply where compliance proves impossible or would involve disproportionate effort, replaced by a published notice before training. That is close to the scenario PI describes, in which an exemption for disproportionate effort absorbs the rights that depend on notification. None of this binds the UK, but it shows how closely the two debates now track each other.

Why the marketing community is watching

Advertising is barely mentioned in the submission, yet much of it describes practices at the centre of data-driven marketing. Inferred interests, propensities and life events are the raw material of audience segments, and PI's example of shopping habits used to infer pregnancy describes the kind of output that retail and audience modelling routinely produces. If such inferences are personal data, as PI and the ICO guidance it cites maintain, access, rectification and erasure rights attach to them, whether or not the people concerned know they exist.

UK advertising rules are already moving on a separate track. The ICO advised the government on May 18, 2026 to relax consent requirements for lower-risk advertising such as contextual targeting, frequency capping and measurement, while keeping behavioural advertising inside the consent regime. The political range on UK data law is wide as well. Reform UK has pledged to repeal GDPR in a contract aimed at 5.7 million small firms; PI wants tighter statutory safeguards; and the business evidence DSIT itself cited leans towards loosening. Which way the department leans will shape the rules for UK audience data, AI training sets and automated campaign tools alike.

Agentic buying adds a further layer. PI's concern about agents processing personal data mid-operation, without a fresh legal assessment, applies to any system permitted to query customer records or audience segments on its own initiative, a category that now includes media-buying agents.

What happens next

DSIT has not published a date for its response. The department says it publishes a list of responses within a summary of responses received and in any subsequent review reports. Until that appears, the UK's direction on AI and personal data remains open, while the EU's own rewrite of the GDPR continues through Council negotiations that precede trilogue talks with the European Parliament.

The charity's own closing line is modest. It hopes DSIT "reviews and responds to the data protection challenges posed by AI." The paragraphs before it are less so. Where existing safeguards are uncertain or ineffective in the AI context, PI argues, the gaps require legislative intervention so that technological complexity does not erode them. Whether a government that wants AI adopted quickly agrees is the question DSIT now has to answer.

Timeline

Summary

Who: Privacy International, a London-based charity registered in England and Wales, responding to the UK Department for Science, Innovation and Technology.

What: A 29-paragraph submission arguing that AI undermines transparency and data subject rights at the training, process and output stages; that the Data (Use and Access) Act 2025 shifted the burden on automated decisions onto individuals; and calling for prohibitions on inferring people's emotional states, behavioural prediction, profiling based solely on protected activities, live biometric identification and scraping to build biometric databases. It also identifies gaps concerning competition and intelligence-service data breaches.

When: The submission is dated September 9, 2026, the closing date of DSIT's call for evidence, which opened on July 15, 2026. Privacy International published it on September 25, 2026.

Where: The United Kingdom, under the UK GDPR, the Data Protection Act 2018, the Data (Use and Access) Act 2025 and the Investigatory Powers Act 2016, with repeated reference to EU instruments including the AI Act, EDPB guidance and the Digital Omnibus proposal.

Why: DSIT is deciding whether data regulation needs guidance, targeted changes or deeper reform to accommodate AI. Privacy International argues that the law's core assumption - that controllers know what personal data they hold and what it produces - no longer holds for AI systems, leaving rights that exist in principle but are difficult to exercise in practice.