Possibility and Probability

A Python programmer with a personality thinking about space exploration

21 November 2026

Results > efforts

by Nick

At work one of the biggest shifts I think I’m seeing is around Pull Requests (PRs). In the past a PR was a good proxy for where our effort was going.

They could be used to measure progress on delivery of a project, or as a proxy for the number of defects in a delievered feature. But now with AI generated code getting radically better every day, these measures are approaching meaninglessness.

For example, a feature estimated to be “a sprints’ worth of work” can now be coded and in review in under an hour. And assuming the description used to kick off the agent was good (e.g. the problem description was good and there’s references that the agent can check to verify its assumptions) the end result if probably going to have really good quality. As in there probably won’t be any “dumb” errors when it deploys: the errors will probably be more related to the business logic that was in the ticket description.

So using PRs to measure a developer’s work… probably not a good idea. Likewise, reviewing PRs line-by-line is probably a losing proposition because who is going to be able to do a thourough job on that quickly, especially compared to an agent’s ability to do the same job?

So we are quickly getting to the point where everyone involved needs to care deeply about the results, and put way less on the efforts. Traditionally, as empathetic humans, we have geen efforts a lot a weight: we recognize some tasks are hard from some people and we try not to judge harshly.

But now that the machine can spit out near-perfect features… we have to spend more time juding the results. As in we need to define our rubrics around the results of the work.

This is quite a shift. I am and will struggle with this for a while.

tags: software - development