Is AI Coding Worth It? Follow The Money And Measure The Results

🔍 Read the full analysis: Is AI Coding Worth It? Follow The Money And Measure The Results on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get smart everyday buys delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

Engineering teams are spending $150 to about $3,000 per developer per month on AI coding tools, but a survey of 27 workplaces found that many companies still cannot say what those budgets deliver. The findings arrive as firms set 2027 budgets and some discounted subscriptions approach renewal, putting pressure on engineering leaders to compare costs with accepted work, rework and software quality.

The survey, published October 20 by Gergely Orosz in The Pragmatic Engineer, gathered accounts from engineering leaders and senior engineers at 27 workplaces, along with aggregate figures from code-review vendors CodeRabbit and Gitar. The article’s author, Thorsten Meyer, said his account draws on the freely readable portion of Orosz’s report. The spending range included a publicly traded company that caps monthly use at $150 per engineer and a two-person European agency spending about $3,000 per developer.

Reported output measures do not point to a single result. CodeRabbit says weekly pull requests on its platform rose 113% year over year, while a Microsoft study of tens of thousands of engineers found adopters merged about 24% more pull requests over four months than they otherwise would have. Those figures track different populations and measures; CodeRabbit’s data comes from users of its review platform, and Microsoft’s researchers cautioned that a merged pull request does not establish the value delivered.

Across the surveyed workplaces, pull-request frequency ranged from flat to increases of five to ten times. In one company, the number of pull requests stayed level while their average size tripled. That variation makes raw activity an unreliable stand-in for business outcomes. Meyer recommends tracking cost per accepted change, rework, change failure rates and whether teams close more user stories than arrive.

Budgets Need Outcome Measures

The cost question is becoming more pressing as billing shifts toward usage. The source report says spending on API plans peaked in June and July and later roughly halved for some companies, which Orosz attributed to a move to open models and the arrival of Opus 5.5. It also reports that several firms expect discounted subscriptions to end. Some said that, without changes, their bills could rise five- to tenfold; that is a reported forecast, not a confirmed increase across the industry.

The benefits described are often specific rather than broad. At one software company, a staff engineer said customer-issue triage that could take a day sometimes now takes minutes of agent work and up to an hour of human checking. Code archaeology and prototypes were also reported as faster. At Ix Technologies, which uses a uniform C# and Blazor stack, the founders said AI helped them work through a long-running backlog. These accounts suggest where time savings may be tangible, but they are individual company reports rather than controlled comparisons.

Costs also include checking what AI produces. Veracode’s latest test found that 44% of AI coding tasks produced a known vulnerability. That result is from a test and does not establish the vulnerability rate for every company or tool. It does reinforce why teams should measure security findings and review effort alongside code volume: faster generation may not reduce total delivery time if validation and repair expand.

Mixed Evidence Behind Productivity Claims

Other research cited in Meyer’s report points to uncertainty about the size of productivity gains. METR found experienced open-source developers took 19% longer with AI in an early-2025 study. A February follow-up estimated an 18% speedup for returning developers and a 4% speedup for new developers, but both confidence intervals included zero. METR also described the data as unreliable because an estimated 30% to 50% of participants held back tasks they did not want to attempt without AI.

Self-reported benefits were higher, but METR cautioned against taking them at face value. In a survey of 349 technical workers, the median respondent reported a 1.4-to-2-times change in the value of their work. METR said participants in its 2025 study overestimated AI’s effect by 40 percentage points. The contrast between self-reports, vendor platform data and controlled studies means estimates should be read with their methods and limitations attached.

Billing changes add a near-term reason for companies to scrutinize use. GitHub moved all Copilot plans to usage-based billing on June 1, charging tokens at each model’s API rate and removing the fallback to a cheaper model, according to the source material. GitHub said the change reflected rising demand from agentic coding, which it said uses more compute and inference. That is the company’s explanation for its pricing decision, not evidence that every customer’s costs will rise by the same amount.

““a question that few companies seem able to answer – at least for now.””

— Gergely Orosz, The Pragmatic Engineer

The ROI Gap Remains

The available figures do not settle how much AI coding improves delivery across the industry. The workplace survey is based on reports from 27 organizations, vendor data covers users of particular products, and the cited studies use different tasks and methods. The source does not provide a common baseline that would allow the reported pull-request increases to be compared directly.

It is also unclear how much of the reported output becomes useful, maintainable software after review, rework and security checks. Some companies described clear time savings on selected tasks, while others reported flat pull-request counts. The evidence provided does not establish a single average return on investment, nor does it show that the spending patterns in the survey represent all engineering teams.

Track Results Through 2027

Companies setting 2027 budgets will need to decide whether to retain caps, change models or accept usage-based bills as discounts expire. The most useful next step, according to Meyer’s analysis, is to record a baseline and compare it over time: spending per accepted change, time spent on rework and review, change failures, and the balance between user stories completed and added.

Those measures can show whether gains in triage, code discovery or backlog work translate into faster, reliable delivery in a particular team. The source material does not identify a shared industry standard or provide a confirmed timeline for one. For now, reported results remain uneven, and companies will have to test their own workflows rather than infer returns from tool usage or code volume alone.

Key Questions

How much are companies spending on AI coding tools?

The 27-workplace survey cited in the report found budgets ranging from $150 to about $3,000 per developer per month. The figures describe the organizations surveyed, not a universal market average.

Do the reported data show that AI coding makes developers more productive?

Some measures show increases, including more merged pull requests at Microsoft and more weekly pull requests on CodeRabbit’s platform. Other companies reported flat activity, while controlled-study results cited in the report were mixed. The measures do not establish one productivity gain that applies across teams.

Where do teams report the clearest benefits?

Accounts in the report point to customer-issue triage, understanding unfamiliar code, building prototypes and clearing backlogs on simple, uniform technology stacks. These are company reports and may not apply to complex legacy systems or other teams.

What should companies measure to judge AI coding spending?

The report recommends looking beyond usage and code volume. Suggested measures include cost per accepted change, rework, change failure rates and whether completed user stories outpace incoming work.

Is the reported 44% vulnerability result an industry-wide rate?

No. Veracode’s cited figure comes from its latest test of AI coding tasks. The source does not establish that 44% of AI-generated code in everyday company use contains a known vulnerability.

Source: ThorstenMeyerAI.com