A computer use agent is an artificial intelligence (AI) system that operates a computer the way a person does. It looks at the screen through screenshots, decides what to do next, and then moves a cursor, clicks, scrolls and types. The idea exists because most software, and most of the web, was built for human eyes and hands rather than for programs. Instead of waiting for every application to offer an application programming interface (API), a computer use agent works through the graphical interface that is already there. Anthropic's computer use, OpenAI's Operator and Google's Project Mariner are the best-known examples, and the term now covers both agents that control a full desktop and narrower ones that control only a web browser.

How the loop works

Every computer use agent runs the same basic cycle: observe, reason, act, repeat. OpenAI described its version, the Computer-Using Agent (CUA), as three stages. In perception, screenshots are added to the model's context; CUA "processes raw pixel data" to understand the screen, according to OpenAI. In reasoning, the model considers current and past screenshots and actions to track where it is in a task. In action, it clicks, scrolls or types until the job is done or it needs the user.

The plumbing is visible in Anthropic's developer documentation. A developer sends the model a task and a computer tool definition. The model replies with one or more tool_use blocks, each naming an action. The developer's own code, not the model, executes that action on a virtual machine or container, then returns a tool_result, usually a fresh screenshot. The loop continues until the model stops asking for actions. As of October 2026, Anthropic's current version, computer_toolset_20260801, exposes 17 member actions, including screenshot, zoom, left_click, double_click, left_click_drag, scroll, type, key and wait. Coordinates are pixels from the screenshot's top-left corner.

Those pixels matter. Anthropic said in October 2024 that Claude "counts how many pixels vertically or horizontally it needs to move a cursor", and that training it to do so accurately was critical. The documentation recommends modest screen sizes - 1024x768 (XGA) or 1280x720 for desktop tasks, 1280x800 or 1366x768 for web applications - and advises against anything above 1920x1080. Each screenshot costs roughly 1,000 to 1,800 input tokens, and the tool definition itself adds about 4,500 more, according to Anthropic. A 50-step task can therefore consume at least 50,000 tokens in images alone, which is one reason inference cost shapes how these agents are deployed.

Google's version follows the same pattern. Its Gemini 2.5 Computer Use model, released on October 7, 2025, is called through a computer_use tool in the Gemini API. Each request carries the task, a screenshot and recent action history; the client executes the returned function call and sends back a new screenshot plus the current URL. Google said the model was optimised for browsers and "not yet optimized for desktop OS-level control".

Not every agent relies only on pixels. Many browser agents also read the accessibility tree, the browser's structured map of buttons, links and form fields, or the page's HTML. Google's May 2026 guidance named screenshots, HTML source and the accessibility tree as the three ways agents perceive a site. Pure screenshot agents are the most general but also the most fragile.

Who runs them

On the supply side of the technology sit model developers: Anthropic, OpenAI, Google, Alibaba and open-weight builders. They sell the capability through APIs to developers who wrap it into products, and through their own consumer tools - Claude in Chrome, ChatGPT's desktop app, Google's Chrome features and Perplexity's Comet browser.

On the receiving end sit websites. A site sees a computer use agent as a browser session, sometimes inside a user's logged-in account. Google added a dedicated Google-Agent fetcher to its documentation on March 20, 2026, citing Project Mariner as an example. Because a human triggers the request, Google-Agent ignores robots.txt. Google is also experimenting with signed requests under Web Bot Auth so sites can verify the agent rather than trusting a spoofable user agent string.

Origin and evolution

Software that drives a graphical interface is not new; robotic process automation (RPA) tools have scripted clicks for years. What changed was generality. In April 2024, researchers led by Tianbao Xie published OSWorld, a benchmark of 369 tasks on real operating systems; humans completed about 72% of them, and GPT-4V managed 12.2%.

Anthropic released computer use in public beta on October 22, 2024, with an upgraded Claude 3.5 Sonnet, through its API, Amazon Bedrock and Google Cloud's Vertex AI. Claude scored 14.9% on OSWorld in screenshot-only mode, against 7.7% for the next-best system, according to Anthropic. Its announcement called the feature "slow and often error-prone" and noted that Claude saw the screen as a "flipbook" of stills, so it could miss brief notifications.

Google followed on December 11, 2024 with Project Mariner, a Chrome extension prototype built on Gemini 2.0 that worked only in the active tab and scored 83.5% on the WebVoyager browsing benchmark. OpenAI launched Operator on January 23, 2025, powered by CUA, which scored 38.1% on OSWorld, 58.1% on WebArena and 87% on WebVoyager, according to OpenAI. On March 11, 2025, OpenAI made a computer use tool available in its new Responses API.

Products then multiplied and merged. OpenAI folded Operator into ChatGPT agent in July 2025 and shut Operator down on August 31, 2025. Perplexity launched Comet that July. Anthropic opened Claude for Chrome to 1,000 Max plan users on August 26, 2025. OpenAI's Atlas browser arrived on October 21, 2025. On July 9, 2026, OpenAI said it would retire Atlas and fold it into ChatGPT Work, whose desktop mode can click, type and move files in the background.

Scores rose steeply. A repaired benchmark, OSWorld-Verified, appeared in July 2025. As of October 13, 2026, the top eight entries on the BenchLM aggregation of that leaderboard all exceeded 82%, led by Alibaba's Qwen3.8 Max at 86.1%, above the original human baseline. BenchLM notes, however, that 34 of its 37 rows are self-reported by model providers.

Why marketers should care

Computer use agents change who, or what, arrives on a website. HUMAN Security tracked a 7,800% increase in agentic traffic during 2025, and by September 2026 found that Meta's Muse accounted for 72% of AI agent requests, with 84% of them aimed at product and search pages. In May 2026, Comet held about 47% of measured agentic traffic against 20.3% for Atlas, according to HUMAN data cited by PPC Land.

That traffic does not behave like a shopper. An agent that skips sponsored listings threatens retail media models, a concern behind Amazon's legal fight with Perplexity. Amazon sued on November 4, 2025 over covert agent access to its marketplace; on August 4, 2026, the Ninth Circuit vacated the injunction, holding that the user, not the agent's maker, accesses the site.

Measurement is affected too: agent sessions can register as visits or skew conversion paths, while blocking them may also block the customer who sent them.

Limitations and disputes

Reliability remains the main weakness. Dynamic seat maps, prices that appear only after a click and hover effects defeat agents, and researchers watching Atlas attempt a booking saw it take far longer than a human and end in errors. Google's Chrome team described screenshot-and-click approaches as computationally expensive and brittle when layouts change, the problem its WebMCP proposal tries to remove.

Security is unresolved. Instructions and data arrive through the same screen, so hidden text can hijack an agent - the attack known as prompt injection. Brave's researchers demonstrated it against Comet. Anthropic measured a 23.6% attack success rate across 123 test cases before mitigations and 11.2% after. Its documentation still recommends a dedicated virtual machine, no credentials and human confirmation for payments or accepting terms.

Benchmarks are contested. Results depend on the harness, step limits and whether a provider or an independent team ran the test, and the near-total reliance on self-reports makes leaderboard comparisons loose.

Legal standing is unsettled. Platform terms often bar robots, and the Amazon-Perplexity ruling addressed only a preliminary injunction.

What it is not

  • Browser agent or agentic browser. A subset: a computer use agent restricted to a web browser, such as Project Mariner or Comet. Desktop agents can also operate spreadsheets, file managers and native apps.
  • API or MCP agent. These agents call structured endpoints instead of clicking. When Ads Uploader found that 17.5% of 36,289 Meta ad batches came from AI agents in September 2026, they arrived via its MCP server and command line, not through the screen.
  • RPA. Robotic process automation replays fixed scripts against known screen positions. A computer use agent decides its next step from what it sees.
  • Personal agent. A personal agent describes whose interests an assistant serves; computer use describes how it acts. Many personal agents use computer use, but not all.

Recent developments

The direction of travel in 2026 has been away from pure pixel-reading where structured alternatives exist. Chrome's WebMCP origin trial lets sites expose tools directly to agents, and Google's Chrome auto browse brought form-filling agents into the browser itself. Computer use has meanwhile become a default feature rather than a standalone product: OpenAI's Operator, ChatGPT agent and Atlas have all been absorbed into ChatGPT Work, and Anthropic pairs Claude in Chrome with Cowork for tasks spanning files and the web. Benchmark scores now sit above the human baseline that seemed distant two years ago, while prompt injection, verification and the legal right of agents to act inside logged-in accounts remain open.

Timeline

  • April 2024 - Researchers publish OSWorld, with a human baseline of about 72% and 12.2% for GPT-4V
  • October 22, 2024 - Anthropic releases computer use in public beta with Claude 3.5 Sonnet; 14.9% on OSWorld
  • December 11, 2024 - Google announces Project Mariner on Gemini 2.0; 83.5% on WebVoyager
  • January 23, 2025 - OpenAI launches Operator and its Computer-Using Agent; 38.1% on OSWorld
  • March 11, 2025 - OpenAI adds a computer use tool to its Responses API
  • July 2025 - OpenAI folds Operator into ChatGPT agent; Perplexity launches Comet; OSWorld-Verified released
  • August 26, 2025 - Anthropic opens Claude for Chrome to 1,000 Max plan users
  • August 31, 2025 - Operator shut down
  • October 7, 2025 - Google releases the Gemini 2.5 Computer Use model
  • October 21, 2025 - OpenAI launches the Atlas browser
  • November 4, 2025 - Amazon sues Perplexity over Comet
  • February 10, 2026 - Google's Chrome team announces WebMCP
  • March 20, 2026 - Google adds the Google-Agent fetcher to its documentation
  • July 9, 2026 - OpenAI announces Atlas's retirement and launches ChatGPT Work with desktop computer use
  • August 4, 2026 - Ninth Circuit vacates the injunction against Comet
  • October 13, 2026 - Top self-reported OSWorld-Verified scores exceed 86%

Summary

Who. Model developers including Anthropic, OpenAI, Google and Alibaba build computer use agents; Perplexity, Meta and others ship them in consumer products; websites, retailers and publishers receive their traffic; HUMAN Security and similar firms measure it.

What. An AI agent that operates a computer or browser through screenshots, cursor movements, clicks and keystrokes, in a loop of observing, reasoning and acting, rather than through an API.

When. OSWorld set the benchmark in April 2024. Anthropic released computer use on October 22, 2024, Google announced Project Mariner on December 11, 2024 and OpenAI launched Operator on January 23, 2025. By 2026 the capability had moved into ChatGPT Work, Claude in Chrome and Chrome itself.

Where. In virtual machines and containers run by developers, in browser extensions and agentic browsers, and on users' desktops; its effects land on any website the agent visits.

Why. Most software and websites were designed for people, not programs. Computer use lets an agent act on any interface without waiting for an API, at the cost of speed, reliability and exposure to prompt injection.