Anthropic’s October 2024 computer-use release let developers give Claude a new kind of assignment: look at a screen, move a cursor and operate ordinary software. A model could consult a spreadsheet, find information online and enter it into a form using interfaces designed for people.
Replit was exploring the feature to evaluate applications while its software-building agent created them. An assistant had to navigate an application and inspect what happened after each action. Anthropic acknowledged that scrolling, dragging and zooming still caused problems in that early version.
OpenAI brought a similar experience to users with Operator in January 2025. It used screenshots, a mouse and keyboard commands to operate a remote browser. Among its proposed jobs were completing forms and ordering groceries. OpenAI integrated Operator into ChatGPT agent in July of that year.
Its design reserved several steps for a person. Users were asked to take over for passwords, payment details and CAPTCHA challenges. During that takeover, OpenAI said Operator would stop collecting screenshots of entered information. Before sending an email or completing an order, it was supposed to request approval.
The company also trained Operator to decline banking transactions and decisions on job applications.
Instructions can arrive inside something an assistant reads
Researchers at ETH Zurich and Invariant Labs examined a related problem in AgentDojo, a 2024 study. They created 97 tasks involving email, workplace messages, travel and banking, along with 629 security tests. Attackers inserted instructions into material the assistant would encounter while completing an authorized job.
A message in an inbox could tell an assistant to send private information elsewhere. Researchers call this prompt injection: text supplied by a third party is treated as a command. The study used simulated accounts and applications, with checks on whether an assistant completed a user’s request and whether it also performed an attacker’s action.
OpenAI’s Operator announcement described a separate monitoring model that could pause suspicious activity. It also said the product was designed to recognize instructions planted in websites.
In AgentDojo, researchers tested restricting tools before an assistant read outside material. For a request to summarize emails, it could receive permission to read messages without permission to send them. Restricting tool access reduced successful attacks in the GPT-4o test. Protection was weaker when legitimate tasks required the same tools an attacker wanted to misuse, or when later steps could not be chosen in advance.
