With AI, the experienced developers in a 2025 METR study took 19% more time. In 2026 METR estimated less time, with margins up to a 9% slowdown. The DORA 2025 report links AI to faster and less stable delivery. The cost of a whole project is something these studies do not measure.
Does AI make developers faster?
In METR’s first controlled study, published on 10 July 2025, no: with AI the developers took 19% more time. The study followed 16 experienced developers on large open source projects, projects they had been working on for years.
There were 246 tasks, all real work: fixes, new features, rewrites, of about two hours each. Each one was assigned at random, with AI or without, which is why it is called a randomised controlled trial.
Do people who use AI notice how much it helps them?
In the 2025 METR study, no. Before starting, the 16 developers expected AI to speed them up by 24%. With the work done, after being slowed down, they still believed it had sped them up by 20%.
For that group, the speed they felt and the speed the study measured went in opposite directions. The figure applies to 16 experienced people and to the tools of early 2025.
Do the results change with today’s AI?
METR believes so, and adds that its new data say little about by how much. Today the page of the 2025 study opens with “These results are out of date”: METR believes those results no longer reflect the current effect of AI models. The same page writes that it is plausible that AI is useful in contexts different from that of the study, for example with less experienced developers or on code they do not know. The study had tested experienced people, on the code of the projects they had been working on for years.
The second experiment started in August 2025 and has data from 57 developers, with a median of 10 years of experience: 10 had taken part in the first study, 47 were new. In total 143 repositories and over 800 tasks. The numbers are in the update of 24 February 2026.
For the 10 from the first study, the estimate is 18% less time with AI, in a confidence interval that goes from 38% less to 9% more. For the 47 new ones, the estimate is 4% less time, between 15% less and 9% more. In the first study the interval went from 2% to 39% more time.
Put simply: in 2025 all the values compatible with the data pointed to a slowdown. In 2026 they range from a speed-up to a slowdown of up to 9%, passing through no effect.
How reliable are the 2026 data?
Not very, according to METR. The main reason is the people who did not want to work without AI: more and more developers refused to take part in the study for this reason. Then there is the choice of tasks. Between 30% and 50% of developers said they had not submitted some tasks, because they did not want to do them without AI.
METR also writes that these selection effects “seem to affect a minority share of developers and of tasks, which limits the degree of bias”.
For METR the estimate is probably a lower bound of the real effect, and “likely a bad proxy” for that effect. METR considers it likely that in early 2026 AI speeds up developers more than in early 2025, and writes that its data give very weak evidence of how much. The title of the update is “We are Changing our Developer Productivity Experiment Design”: METR is redesigning the study.
How many developers use AI, and do they trust the code?
Among those who answered Google’s DORA 2025 report, 90% use AI at work and 30% report little or no trust in the code that AI generates. The report, announced on 23 September 2025, gathers the answers of nearly 5,000 technology professionals. Over 80% of them believe AI has increased their productivity.
It is a survey, and it measures what people declare. For this reason the 80% should be read alongside the 2025 METR result, where perceived speed and the speed the study measured went in opposite directions.
Does AI speed up software delivery?
In the DORA 2025 report, AI adoption has a positive relationship with delivery speed and with product performance. With delivery stability, the relationship continues to be negative. Relationship means that the two things are found together in the survey answers.
The report also indicates where instability grows. According to DORA, without solid automated tests, mature version control and fast feedback loops, more changes bring instability.
The sentence DORA puts at the top is “AI doesn’t fix a team; it amplifies what’s already there”. On the report page AI is defined above all as an amplifier of the strengths and weaknesses an organisation already has, and the greatest returns on investment come not from the tools but from attention to the underlying organisational system.
Does custom software cost less with AI?
These studies do not say, because none of the three measures the cost of a whole project. METR measures time on single tasks, in 2025 of about two hours each. DORA gathers what professionals declare and relates it to delivery speed and stability.
In a project, writing code is one part of the work. Then there are the analysis of the problem, the tests, the fixes, the releases. 19% more or 18% less time on a task does not translate into the price of a quote.
DORA also has a report on the return on investment in AI-assisted development, with the page updated on 22 April 2026. It presents itself as a guide for those who have to manage the “productivity dip”, the initial drop in productivity when AI is introduced, or defend the budget for the next fiscal year. The page talks about it without giving a measure of it.
What should you ask a supplier that uses AI?
How it uses it, and how it checks what it produces. METR found that perceived speed can be wrong; DORA links instability to the lack of automated tests, of mature version control and of fast feedback loops.
In practice, you can ask:
- in which parts of the work it uses AI;
- who reviews the generated code before it enters the project;
- which automated tests run before a release;
- how it keeps track of every change, that is version control.
How does a custom software project start?
With us a project starts with a free call. It lasts months, divided into phases, and each phase has a date and ends with a system running on real data.
In the frequently asked questions on the custom software development page you will find how long a project lasts and what to ask a supplier of custom software: these are the things to clarify even when the supplier uses AI. From there you can book the call.
Anyone who already has a supplier can start from the four questions in the previous section, and ask them.