What the Machine Cannot Want

Last week Anthropic released Claude 4, and a few days before that OpenAI launched Codex as a coding agent. The headline capability for both is the same: they can work on their own for long stretches. Anthropic described a customer running their model on a coding task for around seven hours, unattended. You give it a goal, go to sleep, and wake up to a pull request.

Two years ago I wrote that agents mostly went around in circles. That’s no longer true. These systems plan, act, notice their mistakes, and recover. They are, in a narrow but real sense, doing the work.

So I’ve been asking myself a question I didn’t expect to be asking this soon: if the machine can do the work, what’s left for us?

Capability is not intention

Here’s where I keep landing. The machine can now do a great deal. But it doesn’t want anything.

It doesn’t want this product to exist. It doesn’t lie awake worrying that the customers will be disappointed. It doesn’t care whether the feature is good, only whether it matches the description. It has no stake in the outcome. Give it a different goal tomorrow and it will pursue that one with exactly the same energy.

That’s not a flaw to be engineered away. It’s what a tool is. A hammer doesn’t want the house.

The “why” comes from somewhere

Every piece of software that has ever mattered started with someone wanting something. A founder who couldn’t stand how a thing was done. A developer who wanted their mother to be able to use the app. A team that cared, sometimes irrationally, about getting a detail right that no customer would ever consciously notice.

That wanting is where direction comes from. Without it, you get output - possibly enormous amounts of output - without purpose. Code that does exactly what was asked, for a goal nobody examined.

As the machines get more capable, I think the scarce thing in software stops being the ability to build, and becomes the ability to want well: to choose worthwhile goals, to care about the right things, to notice when what was asked for isn’t what’s needed.

Responsibility cannot be delegated

The other thing the machine cannot do is be responsible.

When the seven-hour agent ships a bug that loses a customer’s data, the agent doesn’t feel anything. It doesn’t apologize to the customer, and it doesn’t carry the lesson forward into the next decade of its career. A person does that. Someone signed off, someone owns it, someone answers for it.

I think this is why experienced developers are becoming more valuable even as the machines become more capable. The job is quietly changing from doing the work to owning the work: setting the direction, holding the standard, and being accountable for the result. That was always the senior part of the job. Now it’s becoming most of the job.

A strange kind of freedom

I find this, unexpectedly, freeing.

For most of my career, a lot of my energy went into the mechanics: typing, wiring, debugging the same kinds of problems over and over. That’s increasingly handled. What’s left is the part I always cared about most, and had the least time for: understanding people, deciding what’s worth making, and making it well.

There’s a teaching I’ve always loved: it is not your duty to finish the work, but neither are you free to desist from it. The machines may finish more and more of the work. The duty - to care, to choose, to take responsibility - is still ours.

The machine can do almost anything you ask. The question is still what to ask for.