In February 2026, METR reported that its second programming experiment faced a recruitment difficulty: some developers refused to participate because they did not want to work without AI. Others withheld particular tasks for the same reason. Among participants surveyed, 30 to 50 percent said they had left some tasks out.
A year earlier, experienced programmers in a METR experiment expected AI tools to help them finish their work 24 percent faster. Even after completing assignments, they believed the tools had saved them about 20 percent. Researchers measured a different outcome: tasks took 19 percent longer when AI was allowed.
The study involved 16 developers working on 246 real issues in open-source software projects they knew well. Researchers randomly allowed or prohibited AI for each task. Participants recorded their screens and reported how long implementation took; most AI-assisted work used Cursor with Claude models available in early 2025.
Completing an assignment meant producing code a developer considered ready for review, with the project’s requirements for testing, style and documentation. Those conditions required knowledge of existing software and its conventions. METR described its findings as a snapshot of experienced developers working in familiar projects.
AI was already becoming a regular habit elsewhere. In nationally representative U.S. surveys analyzed by Alexander Bick, Adam Blandin and David Deming, 23 percent of employed respondents said they had used generative AI for work during the preceding week in late 2024. Nine percent reported using it every workday. Their February 2025 paper estimated that between 1 and 5 percent of all working hours were assisted by the technology.
Newer employees received more help
In a 2025 study, Erik Brynjolfsson, Danielle Li and Lindsey Raymond followed an AI assistant introduced to customer-support staff in stages. Among 5,172 workers, access increased successfully resolved issues per hour by 15 percent on average. Less experienced and lower-performing employees improved by about 30 percent; experienced, higher-performing colleagues gained little. These findings came from workplace records, allowing researchers to compare employees before and after access with colleagues still waiting for the assistant.
METR is changing its experiment
METR’s follow-up covered 57 developers and more than 800 tasks. Raw estimates suggested faster work with newer tools, with uncertainty ranges that included no improvement. Researchers said participation and task choices probably excluded work expected to benefit most. Tracking time while several agents ran at once added complications.
METR proposed shorter, more intensive experiments, tasks assigned by researchers and designs that give each developer AI for all their work or none of it. That last approach would reduce opportunities to choose which individual tasks entered a comparison, though it would still depend on recruiting people willing to work without the tools.
