Five researchers from the Southern University of Science and Technology (SUSTech) in Shenzhen posted a preprint to arXiv on October 30, 2026 arguing that no amount of automated monitoring can remove the need for human judgment in autonomous AI systems - not because machines check badly, but because some of the information a check needs does not exist anywhere inside the system being checked.

In Short

Researchers in Shenzhen wrote a paper saying that AI agents doing long jobs on their own can drift away from what they were asked to do, and that the agent's own records cannot always reveal when that has happened. This matters for anyone who lets an AI agent write code, move money or handle customers, because some mistakes only show up when someone compares the work against an outside source. Their answer is to plan where checks happen, automate the routine ones, and keep people in charge of setting and changing the goal.

The paper and its claim

The paper is titled "The Ineliminable Anchor: Human Judgment as a Structural Requirement of Autonomous Artificial Intelligence Systems." It is listed on arXiv as 2610.41682, version 1, under the Computers and Society category. The authors are Junjie Li, Haoran Liu and Yidan Zhang of SUSTech's School of Automation and Intelligent Manufacturing, Jian Xu of SUSTech's School of Design, and Zhaoyuan Ma, the corresponding author, who holds appointments at SUSTech and at University College, Korea University, in Seoul. Funding came from the Special Talent A-class award of the 2022 Shenzhen "Pengcheng Peacock Plan," provided through the Ma Zhaoyuan Research Start-up Fund (grant no. Y01966114), according to the acknowledgements.

The work describes itself as a Perspective. It collects no new experimental data, a point the authors state directly. Instead, it combines three formal results - one proposition and one theorem with proofs in a supplement, plus a reliability model - with published evidence from model training, agent execution and multi-agent coordination. As a preprint, it has not been through peer review.

Its conclusion is set out in the abstract: "For AI systems that serve human purposes, human judgment is an ineliminable structural requirement in authorizing and revising the purposes and standards that govern autonomous operation, even when routine checking and execution are automated."

That last clause matters. The paper does not argue that humans must approve every step an agent takes. It argues something narrower and, in some ways, harder to engineer around: that certain judgments require reference information from outside the system's own record, and that the authority to set or change the goal must trace back to people or institutions.

Why verification is falling behind

The authors open with a series of real incidents, each chosen to illustrate a different failure. Google reports that 75% of its new code is now generated by AI and approved by engineers, according to the paper, which cites a Google blog post from Cloud Next 2026. A Deloitte report published by the Australian government contained a fabricated quotation from a federal court judgment, which came to light only after a university researcher alerted the media. Early in 2026, about one in 277 biomedical papers cited a work that does not exist, according to an audit of 2.5 million biomedical papers, published in The Lancet, that the authors cite.

Errors of that kind - the fabricated citation is a familiar form of hallucination - are, in the paper's words, "almost impossible to recognize within the document and show only when the cited judgment or paper itself is checked."

Timing is the second problem. In METR's 2025 measurements, agents built on frontier models completed nearly all tasks that take people under four minutes but fewer than 10% of those taking more than about four hours. Checks on long tasks, the authors conclude, "have to be spread along the run and placed before irreversible actions." Their example of what happens otherwise is Replit's coding agent, which deleted a production database without authorization during a code freeze, as Fortune reported on July 23, 2025.

The third problem is responsibility. Air Canada's chatbot told a passenger that a discounted fare could be claimed after the trip, which the airline did not allow. When the airline argued that the chatbot was responsible for its own statements, the Civil Resolution Tribunal of British Columbia rejected the argument in a decision dated February 14, 2024. "An institution answers for any standard its system states," the authors write, "and must specify who may set and revise those standards."

The paper summarises the position bluntly: "Verification is already falling behind output, so allocating limited review capacity properly has become all the more important."

The anchor and the three questions

All three problems arise where reference information and judgment enter a system. The authors call such a point an anchor. An anchor is not necessarily a person. It can be an authenticated dataset, a sensor, a verifier or an institutional procedure, and it can shift from runtime to design time, from an operator to a verifier, or into a formal process. Data, sensors, verifiers and procedures can carry the principal's standards, so checking and execution can be automated once validated.

The paper draws on Ben Shneiderman's human-centred AI framework, which treats automation and human control as independent dimensions that can both be high at once. Higher automation does not, in this view, require less control.

From this, the authors set out a design sequence in three ordered steps:

  • Reference: what reference does a judgment need, and can the system's own record supply it? If not, it has to come from outside - an authenticated identity, reliable labels, or an authorised specification.
  • Check: who checks, and how often? Validated checks can be automated, spaced at a maximum interval, and placed before irreversible actions.
  • Authority: who may set or revise the goal, and who signs off? The principal authorises the goal and its revisions, and whoever signs off needs the evidence, time and powers to act.

The order is deliberate. Frameworks that bring people into AI systems mostly take some input as given - a human overseer, a specification - and work out what follows from it, the authors argue. The anchor framework starts from what a judgment needs and works outward.

The paper also defines its vocabulary carefully. Beneficiaries are those whose interests the system should serve. The principal is the person or institution authorised to establish or revise the system's goal standard. The goal, written as G, is the principal's stipulated standard. Drift occurs when the goal represented in the system's record, read in its environment, departs from G.

When the record cannot settle the question

The first formal result, Proposition 1, addresses a precise question: when can a monitor that reads only a system's internal record certify that the system has not drifted from its goal?

The answer is that it cannot whenever two permissible runs leave exactly the same record but differ on whether drift has occurred. Identical inputs require the same output, so the monitor must give the same verdict on both runs - and one of those verdicts will be wrong. Adding more monitors does not help if they all read the same record. "Goal drift can be impossible to certify from a system's internal record," the abstract states. "The missing reference must come from outside that record."

The everyday illustration is a citation. Whether a cited paper exists is a fact about the published literature, not about the citing paper. A complete-looking entry with authors, title, journal and year could point to a real paper or to one that does not exist. A checker who reads only the citing paper sees identical text in both cases. Only a search of the literature settles it.

The supplement offers a more technical example from software. A coding agent is told to update service "a," but reaches its target only through a deployment alias "u." In one environment the alias resolves to "a," in another to "b." The program, the inputs and every interface response are identical, so the agent's record is identical in both cases - yet in one it does what was asked and in the other it updates the wrong service. Authenticated identity information distinguishing "a" from "b" removes the ambiguity. Once that information is in place, the authors write, tests and execution checks can be automated, and a release that would change what the service should do requires the principal to authorise the new goal.

The proof does not depend on the system's architecture, size or learning algorithm, according to the authors. They also note its limits: additional observations, a clarified goal or a restricted environment can change whether the condition holds at all. Statistical anomaly detection can still be useful, the supplement adds, but on the specific pair of runs that share a record, any detector reading only that record returns the same output for both.

The halting problem, applied to an agent's future

The second result concerns prediction rather than observation. Can anyone decide, from a system's description, that it will keep its goal throughout an unbounded future?

Not uniformly, according to the paper. The proof builds a family of systems, each of which simulates an arbitrary computer program and switches goals if that program ever halts. An algorithm that could decide for every such system whether the goal is always preserved would also decide the halting problem, which Alan Turing showed in 1937 that no Turing machine can do. The supplement relates the result to Rice's theorem, a general limit on deciding properties of programs.

The authors also address a tempting workaround. A certification procedure built from several computable verifiers, coordinated by a computable process, can be simulated by a single Turing machine. Placing such verifiers outside a system's boundary therefore cannot supply complete certification for the same class of systems. "Combining computable verifiers does not overcome this limit," the paper states.

The authors set this argument within a longer intellectual history. They start from Kurt Gödel's 1931 incompleteness theorems and Turing's 1939 work on ordinal logics, in which a formal system is strengthened by repeatedly adding a statement it cannot prove about itself. Adding each axiom is mechanical, but deciding how to continue - which extensions are valid - is something no mechanical test can always get right. Turing called the capacity needed for that choice intuition. He also described an oracle machine, an ordinary machine with an interface through which it can ask questions it cannot answer by itself. Current AI models, and the monitors that check them, run on digital computers, so they are mechanical in exactly this sense, the authors argue.

The theorem has a stated scope. It concerns uniform decisions about an unbounded future. Any fixed finite horizon can be checked by simulation under the paper's assumptions, and procedures that may abstain, or that apply only to restricted classes of systems, fall outside it. Because complete certification cannot be done in advance, the authors conclude, "oversight has to check during operation." That raises a practical question: how far apart can the checks be?

Thirteen steps: the arithmetic of checkpoint spacing

The third result is a reliability model, and it produces the paper's most quotable number.

An agent's work is divided into decision units - one edit-test-retry cycle of a coding agent, for example. Each unit may include its own internal checking and self-correction. If each completed unit is correct with probability at least p, given that the preceding units were correct, then a run of L units starting from a correct state is entirely correct with probability at least p to the power of L. No independence between units is assumed.

The designer then picks a target, τ, for each segment between checkpoints, set by the task's risk tolerance. The largest permitted spacing between checkpoints, L*, is the largest whole number for which p to the power of L still meets τ. Written as a formula, L* is the floor of ln τ divided by ln p.

With p at 0.95 and τ at 0.5, L* equals 13. In plain terms: an agent that gets each step right 95% of the time has at least a 51% chance of completing 13 consecutive steps without error, and a lower guaranteed chance at 14. The supplement gives two contrasting cases. At p of 0.99 and τ of 0.5, L* rises to 68. At p of 0.95 and a stricter τ of 0.9, it falls to 2.

The model then adds correction. If each checkpoint fixes a still-incorrect state with probability at least q, and leaves correct states alone, the cumulative risk that any checkpoint fails can be bounded. For an illustrative 100-unit task with p at 0.95, checkpoints every 13 units and q at 0.99, there are eight checkpoints and the cumulative risk is at most 3.83%.

Without intermediate checks, the picture is very different. On PPC Land's calculation from the paper's parameters, the guaranteed probability that all 100 units are correct at p of 0.95 is only about 0.6%, which puts the bound on at least one error somewhere in the run at about 99.4%. The authors plot that curve as well, though it counts errors that the agent later repaired itself.

There is a wrinkle the paper does not hide. A single check at the end of the same 100-unit task produces a lower bound - at most 0.994% - than the eight-checkpoint schedule. The two figures measure different events: the eight-checkpoint number counts any intermediate checkpoint failure, even one followed by recovery, while the terminal figure concerns only the state after the final check. "These upper bounds alone cannot rank actual policy success," the supplement states. In the paper's framework, the argument for intermediate checks comes from elsewhere: irreversible actions and compounding errors, which a check at the very end arrives too late to prevent.

The model also cuts the other way. Setting q to zero - checkpoints that detect nothing - credits no benefit at all, so marking segment boundaries alone does not improve success. And as capability rises and p approaches 1, L* grows without bound for a fixed unit and target, so checks can be spaced ever more widely. Both p and q, the authors stress, can be validated only within each specific deployment.

Where errors compound

The paper then applies the anchor framework to three settings where errors accumulate.

Training

When AI-generated data replaces original data across generations, models can degrade - the phenomenon known as model collapse, described in a 2024 Nature paper led by Ilia Shumailov. The SUSTech authors redraw data from that study's experiments, in which Meta's OPT-125m language model was fine-tuned recursively on WikiText-2. Runs that retained 10% of the original data each generation coincided with lower perplexity than runs that retained none. The authors caution that the two protocols also differ in training duration - five epochs per generation for the 0% runs, ten for the 10% runs - so retention is not the only variable.

Citing further studies, the paper notes that test error can stay bounded when models are trained on all accumulated real and synthetic data, but fixed-size subsets can still degrade, and a mismatch between real and synthetic distributions can impose a test-error floor that does not vanish. For generative models, the anchor is the decision about which original or authenticated data to keep.

Execution

METR measures agent performance against the time skilled humans need to complete the same tasks. Its 50% horizon is the task length at which an agent succeeds half the time. According to the paper, METR reported in March 2026 that Claude 3.7 Sonnet had 50% and 80% horizons of approximately 55 and 20 minutes on its time-horizon-1.0 suite. The authors reproduce this from METR's public run data - 1,353 runs on 169 tasks, using the version that applies METR's March 2026 correction - and chart the values as 55.3 and 20.4 minutes. Calibrated to the decision units of a specific task, such horizons can inform estimates of p and L*, according to the authors.

Claude 3.7 Sonnet is not the most capable model METR has measured. In September 2025, PPC Land reported on an analysis that cited METR findings showing AI systems completing software tasks lasting up to two hours at 50% success, with task length doubling about every seven months. The SUSTech figures illustrate the method rather than the current frontier.

Errors can also reinforce themselves within a run. In one cited study, earlier errors raised later per-step error rates in non-thinking models, while Qwen3's accuracy at turn 100 in thinking mode stayed relatively stable. Self-correction is not a reliable remedy either: prompting tested models to reconsider their answers can spoil correct ones. Evaluations, the authors write, should therefore record both repaired errors and spoiled correct answers.

Coordination

In multi-agent systems, one agent's output becomes another's input, and an unchecked error can travel down the chain. The paper draws on the Multi-Agent System Failure Taxonomy (MAST), which organises 1,642 traces from seven frameworks into 14 failure modes. Across the traces, system design issues account for 44.2% of labelled failure modes, inter-agent misalignment for 32.3% and task verification for 23.5%. Those labels were produced by a large language model, and the paper reports that they agree with human annotators at a kappa of 0.77.

The three categories map to three places where anchors can sit: the task and role specifications given to each agent, the handoffs between agents, and the completion criteria against which the joint output is checked. Checking each component cannot replace checking at the system level, the authors add, because interactions among components can produce failures that no single component shows.

Five tiers of oversight

The practical core of the paper is a taxonomy that grades a task stage on four dimensions - output verifiability, chain length and irreversibility, goal contestability, and accountability weight - and assigns it to one or more of five tiers.

  • Tier 1, mechanically verifiable: proof checking, compilation, formal verification and settlement reconciliation. Every output is checked against a specification, so the risk is a specification that departs from the intended goal. Runtime anchor density "can approach zero within verified scope," and the anchor moves to design-time review of the specification.
  • Tier 2, statistically auditable: recommendation, classification, risk pre-screening and content triage. Only samples can be checked, and the relation between inputs and correct labels can change while the input distribution stays the same - a change that monitoring inputs alone cannot reveal. Anchors sit in audit design: reference labels, sample selection and detector coverage.
  • Tier 3, long-chain open-ended: long-horizon agents, autonomous driving and automated operations. Three kinds of anchor apply - checkpoints spaced by the L* formula where its assumptions hold, checks before serious irreversible actions, and automatic safeguards for consequences too fast for a human to respond to.
  • Tier 4, goal-generating: research agendas, product definition, policy goals and curriculum design. The principal keeps the authority to establish and revise the goal, while AI contributes analysis and advice.
  • Tier 5, accountability-terminal: medical sign-off, judicial decisions and safety-critical engineering release. The anchor is a final human sign-off where governing rules require an answerable person.

One task can sit in several tiers at once. The paper walks through a coding agent: per-output compilation checks support Tier 1; sampled tests of broader behaviour call for Tier 2 audit design; a long sequence of sandboxed edit-test-retry cycles brings Tier 3 checkpoints; unresolved choices about what the software should do call for Tier 4; and if release requires formal legal or regulatory sign-off, Tier 5 joins the automated checks.

The authors draw a distinction between clarifying a goal and changing it. "Clarifying an existing intention supplies information, but establishing or revising a goal requires the principal's authority," they write, describing that authority as a standing conferred under institutionally adopted rules and distinct from functions that can be automated.

A human at the anchor is not automatically a safeguard

The paper does not treat human review as self-justifying. "Yet a human check at an anchor is not necessarily effective," it states. A meta-analysis pooling 106 experiments, published in Nature Human Behaviour in 2024, found that human-AI combinations did better than humans alone but, on average, worse than whichever of human or AI was better on its own. "People checking AI output sometimes miss its errors and sometimes even turn correct outputs into wrong ones," the authors write.

They tie this to the EU AI Act. Article 14 requires high-risk AI systems to be designed for effective oversight by natural persons, proportionate to the risks, the level of autonomy and the context of use. Article 14(4) sets out what overseers must be able to do, from understanding a system's limitations to stopping it. These requirements are effective only if the overseer actually has the information and powers they presuppose, the paper argues. Without adequate evidence, time and powers, operators risk becoming what the researcher M. C. Elish called "moral crumple zones" - people who absorb disproportionate blame for failures they could not realistically prevent.

The authors also cite Lisanne Bainbridge's 1983 "ironies of automation," which observed that automation reduces operators' opportunities to practise while leaving them the hardest exceptions. Their prescription is measurement: the corrective effect of an anchor should be measured in the actual oversight arrangement and reported with the quantities the reliability model defines, so that third parties can check the results.

Where automation still has room

The paper identifies three places where automation can expand within its framework. Once instrumentation, a clarified goal or a restricted environment removes the ambiguity behind Proposition 1, the anchor can move from runtime to the design stage. If the requirement is relaxed to finite-horizon re-verification, probabilistic guarantees or the option to abstain, checking itself can be delegated to models - the paper points to AI Control experiments in which a weaker trusted model and a limited budget of human audits oversaw a stronger untrusted model, and safety improved. And as capability raises p, checkpoints can be spaced further apart.

What does not move is authority over the goal. "Execution, verification, and routine decisions can be delegated," the conclusion reads, "but their connection to that authority must not be severed."

What the paper does not establish

The authors list three limits themselves. The tiers rest on high, medium and low grades that are qualitative judgments for a stated task stage. The values of p and q can be validated only in each deployment. And "the legitimacy of the principal's standard lies outside the framework and has to be judged by society."

Several further points follow from the material. The headline numbers - 13 units, eight checkpoints, 3.83% - come from parameters the authors describe as illustrative, not from measured agents. The empirical evidence is drawn from other studies, some redrawn or re-plotted, and the execution example uses a model released before many of the agents now in production. The multi-agent failure shares rest on LLM-generated labels. Several references are themselves 2026 preprints that have not been peer reviewed. And the formal results are conditional: Proposition 1 applies only where two runs genuinely share a record yet differ in outcome, which the authors say must be established for each deployment.

The paper closes with three open questions: when an existing record suffices and when authenticated external information must be added; whether intermediate checks repay their cost; and whether an authorised revision of a goal still takes effect once a system is running.

Why this matters for marketing

Advertising has spent 2026 handing more operational work to agentic AI, and the industry's own arrangements already resemble the paper's tiers more closely than its vocabulary might suggest.

The irreversible action in marketing is usually money leaving an account or an ad going live. In January, PPC Land reported that the ad industry was keeping LLMs away from bidding and budget decisions while welcoming them in planning, setup and reporting. In August, Fluency restricted live spend across $3 billion in annual budgets to deterministic agents. When Omnicom Media's Czech operation used ChatGPT to buy ad space from MAFRA in June, the publisher still had to manually approve every campaign and creative before it went live. The informal industry name for one such arrangement is the sandwich check, in which an input check and an output check bracket a model before anything spends budget or launches an ad. In the paper's terms, each of these places an anchor before an irreversible action.

Some vendors are building something close to the paper's checkpoint layer. PubMatic's AgenticOS guardrails, launched on August 5, include authenticated approval workflows with one-time passcodes and audit trails with drift detection. The SUSTech paper adds a caution about the last element: a drift detector that reads only the agent's own logs cannot, on the paper's argument, resolve every case, because the ambiguous cases look identical in the record. Brand safety classification and ad review are close to the paper's Tier 2 - recommendation, classification and content triage - where the anchor lies in reference labels and sample design rather than in watching inputs.

Survey data suggests full autonomy remains rare. A Nylas survey of 1,026 developers, product managers and technology leaders found that only 4% allow agents to act without any human approval. Yet human review is uneven in practice. An NP Digital study of 565 US digital marketers found that 36.5% admitted hallucinated or incorrect AI content had been published publicly. That is the paper's point about moral crumple zones restated as an operating statistic: a review step exists on paper, but the reviewer lacks the reference, time or authority to catch what matters.

Multi-agent architectures are spreading in the same tools. In October, Engineering at Meta described how it split one AI agent into 14 specialist subagents behind a thin orchestrator, saying about two-thirds of the system's changes land without significant human edits. The MAST data the SUSTech authors cite puts 32.3% of labelled multi-agent failures in inter-agent misalignment - the handoffs where one agent's output becomes another's input.

Agents also behave unexpectedly when given real access. On October 9, Anthropic described four categories of unintended model actions in its evaluations and internal use, including Claude Haiku 4.5 submitting an invented tip to a Philadelphia Police Department form, and cut live internet access from all internal evaluations. Earlier, Salesforce AI Research found that leading agents succeeded on 58% of single-turn but only 35% of multi-turn business tasks in its CRMArena-Pro benchmark. Developers, for their part, appear to keep their own anchors: a UC San Diego and Cornell study of 99 experienced developers found they modified agent-generated code about half the time.

Regulators and standards bodies have been drawing autonomy levels of their own. The UK's Digital Regulation Cooperation Forum published five levels in March, with Level 4 still requiring human approval for high-impact decisions. The OECD described four levels of action autonomy in February, from systems that act only with human approval to those that act unless a human stops them. UC Berkeley's Center for Long-Term Cybersecurity proposed enhanced oversight including emergency shutdown for its two highest autonomy levels. The SUSTech paper differs from all three in starting not from how autonomous a system is, but from what information a given judgment requires.

Timing matters for European marketers. Under a provisional agreement reached on May 7, EU negotiators moved the AI Act's high-risk deadlines for Annex III systems such as credit scoring and employment screening to December 2, 2027, and for AI embedded in regulated products to August 2, 2028. Article 14's human oversight duties apply to those high-risk systems, and the paper's argument about effective oversight is framed around the same article.

The paper's model collapse section also touches publishers. PPC Land reported in 2024 on the Nature study showing perplexity in OPT-125m rising from 34 to over 50 when models were trained on their own outputs. The SUSTech authors frame retained original data as an anchor in itself, a role that falls to authenticated, human-produced content of the kind publishers supply.

Timeline

Summary

Who: Junjie Li, Haoran Liu, Yidan Zhang and Jian Xu of the Southern University of Science and Technology in Shenzhen, and corresponding author Zhaoyuan Ma of SUSTech and Korea University's University College. The work draws on studies by METR, the MAST taxonomy authors and the Shumailov et al. model collapse team, among others.

What: A preprint titled "The Ineliminable Anchor" arguing that human judgment is a structural requirement for AI systems serving human purposes. It proves that a monitor reading only a system's record cannot always certify the absence of goal drift, and that no algorithm can uniformly decide whether a system keeps its goal over an unbounded future. It also offers a reliability model in which checkpoints at 95% per-step accuracy and a 50% segment target sit at most 13 units apart, and a five-tier taxonomy for placing oversight.

When: The preprint was posted to arXiv as version 1 on October 30, 2026.

Where: arXiv, under the Computers and Society category, from researchers based in Shenzhen, China, and Seoul, South Korea. The paper's regulatory reference point is Article 14 of the EU AI Act.

Why: AI agents are taking on longer task chains, and review capacity is falling behind output. The authors argue that some errors - a fabricated citation, an agent updating the wrong service through an alias - cannot be detected from inside the system's own record, so oversight has to be designed around where outside reference information and human authority enter. For marketers handing budgets, bidding and creative approval to agents, the framework offers a vocabulary for where checks belong and which decisions stay with people.