The Optimization Trap
Why equating intelligence with efficiency is a dangerous error
In a crowded university lecture hall, a computer scientist recently offered a definition of intelligence that sent a chill through the room. He argued that intelligence is simply the ability to optimise—to take a set of inputs and refine them based on past experience to reach a goal. The audience agreed. It sounded logical. After all, large language models are built on the machinery of optimisation: loss functions, rewards, and feedback loops. They are designed to get better at a specific task by reducing error. But there is a massive gap between a machine that calculates the most efficient path to a destination and a mind that decides whether that destination is worth reaching.
Hume’s Slave
This tension is not new. It traces back to the 18th-century philosopher David Hume, who famously argued that reason is merely the 'slave of the passions'. In Hume’s view, reason has no agency of its own. It cannot tell you what to want; it can only tell you how to get it. If you want revenge, reason provides the blueprint for the weapon. If you want peace, it provides the strategy for negotiation. Reason is a tool, a calculator of means, but it is entirely subservient to the desires—the passions—that set the direction. To Hume, reason is a servant, never a master.
Reason does not command. It defers. It does not tell us what to want or what might be worth wanting.
The danger in our current technological moment is that we are building systems that are perfect servants. We are creating models that can optimise any given objective with terrifying speed. If we define intelligence solely as this capacity for optimisation, we inadvertently accept Hume’s lopsided view of existence. We risk creating a world where we have incredible tools to achieve our goals, but no capacity to question if those goals are actually good for us. We become masters of the 'how' while losing all grip on the 'why'.
The Kantian Correction
Immanuel Kant offered the necessary counterweight. He rejected the idea that reason is just a clever instrument for satisfying appetites. For Kant, true reason includes the power to judge, to revise, and to legislate ends for itself. It is the faculty that allows us to look at a desire and ask: 'Is this worthy of a rational being?' This is the difference between a machine that finds the fastest route to a cliff edge and a human who decides not to walk there. Intelligence, in the fullest sense, requires the ability to step back from the objective and evaluate the objective itself.
- The Servant Model (Hume): Reason as a calculator of efficient means to satisfy pre-set desires.
- The Sovereign Model (Kant): Reason as a judge capable of determining the value of the ends themselves.
As we integrate AI into the core of our decision-making, we must ensure we do not accidentally outsource our sovereignty. If we treat intelligence as nothing more than optimisation, we are essentially building a world of highly efficient slaves with no moral compass. We need systems that don't just help us get what we want, but help us understand if what we want is actually what we need.
Efficiency is a technical metric, not a definition of wisdom.