The Deep Feed

01 — Lenny's Newsletter

The Death of the Prompt Engineer

Moving from instruction to management in the age of autonomous agents

By Claire Vo · 12 min read

Editor's note: The role of the AI user is shifting from writing instructions to overseeing systems.

The era of the 'prompt engineer' is ending, replaced by something more demanding and more scalable: the agent manager. For the past year, the industry has obsessed over the perfect string of text—the magic incantation that coaxes a model into performing a specific task. But as agents become more capable of independent reasoning and tool use, the bottleneck is no longer the quality of the prompt, but the quality of the oversight. When an agent can write code, browse the web, and manage its own state, your job is not to tell it how to type, but to define the boundaries of its autonomy and manage the outcomes of its decisions.

The State Machine Approach

Alessio Fanelli demonstrates this shift through a workflow that treats project management software as a central nervous system for AI. By wiring OpenAI Symphony with Linear, he has turned a project management tool into a state machine. In this setup, the agent doesn't just sit in a chat window waiting for a command; it moves through a lifecycle of tasks, updating its progress and responding to changes in the environment. This removes the friction of local runtimes and the constant need for human intervention. Instead of babysitting a terminal, the manager checks a dashboard. The agent handles the heavy lifting, while the human ensures the trajectory remains correct.

The shift from 'agent prompter' to 'agent manager' is the unlock most people are still missing.

This management style requires a different kind of technical literacy. You aren't learning syntax; you are learning system architecture. You need to understand how to track token costs per task, how to purge outdated 'skills' files that confuse the model, and how to provide agents with better 'senses'—the ability to perceive the digital environment they are operating in. It is the difference between teaching a child to write a sentence and managing a team of specialists. One requires patience; the other requires strategy.

Key principles of agent management

Move away from local runtimes to cloud VPS for scalability
Use project management tools as the source of truth for agent state
Regularly purge instruction files to prevent context drift
Monitor token consumption as a direct metric of task efficiency

The implications for small business are massive. We are seeing the birth of a new category of micro-enterprises where a single person manages a fleet of digital workers. Fanelli uses this to hunt for underpriced collectibles on eBay, using Codex to browse, extract data, and flag deals. This isn't just automation; it is the deployment of a specialized workforce that operates while you sleep. The barrier to entry for complex, data-driven businesses has collapsed, leaving only the question of who has the managerial skill to direct the machines.

Key Takeaway

Stop trying to write better prompts and start building better systems for oversight.

02 — Lenny's Newsletter

The Taste Gap

Why automated benchmarks fail to capture human preference

By Lenny Rachitsky · 10 min read

Editor's note: As models improve, the gap between what an AI thinks is good and what a human actually likes is widening.

The industry is currently obsessed with benchmarks. We look at leaderboards and MMLU scores to decide which model is 'better'. But there is a growing divergence between mathematical performance and human utility. A model might pass every technical test with flying colours, yet produce work that feels hollow, robotic, or fundamentally misses the point of a design. This is the 'taste gap'—the space where an LLM's logic meets a human's aesthetic and functional requirements. If you rely solely on automated scores, you are flying blind.

The Failure of LLM-as-Judge

A recent attempt to benchmark the latest models revealed a startling truth: LLM judges are too polite. When asked to grade the outputs of other models, they tend to cluster around the middle of the scale, avoiding the 'spiky' judgements that humans make. An LLM judge might give a high score to a prototype that looks visually broken or ignores obvious constraints, simply because the code is syntactically correct. The model sees the logic, but it doesn't see the failure. It lacks the visual and intuitive grasp that a human brings to a task in seconds.

Models can’t yet see what the human eye catches in the first screenshot.

In a blind test, the results were telling. Automated judges ranked certain models as superior, while human testers—using a weighted index that prioritised actual usability and 'personality'—ranked them almost exactly the opposite. This suggests that the current metrics for 'intelligence' are measuring the wrong things. They are measuring the ability to follow rules, rather than the ability to solve problems in a way that feels right to the user. For builders, this means that the most important part of any evaluation is the human signal.

Model recommendations by task

GPT-5.5 for technical documentation and PRDs
Sonnet 4.6 for rapid prototyping and conversational agents
Opus 4.8 for complex, dense UI and codebase navigation

The lesson for developers is clear: do not trust the leaderboard. A model's 'personality'—the way it communicates, its ability to handle frustration, its tone—is often more important for long-term integration than its raw reasoning score. If a model is a joy to work with, it will be more productive in the long run than a technically superior model that is a chore to manage. Build your own benchmarks. Encode your own taste. The machines will follow the data, but they won't follow your standards unless you force them to.

Key Takeaway

Technical accuracy is a baseline, but human taste is the ultimate differentiator.

03 — Simon Willison

The $150 Bug Hunter

How AI agents are being used for high-stakes software maintenance

By Simon Willison · 8 min read

Editor's note: Software stability is being tested by a new kind of developer: the AI agent.

Software maintenance is often a thankless task, filled with the tedious work of checking edge cases and ensuring backward compatibility. However, a new method of development is emerging where the primary 'engineer' is an AI agent, and the human acts as a high-level reviewer. Simon Willison recently put this to the test, using Claude Fable to prepare a major release for his `sqlite-utils` library. The result was a demonstration of how agents can find critical, logic-breaking bugs that even experienced developers might overlook.

The Danger of Silent Failures

The most significant discovery was a bug in a `delete_where()` function. On the surface, the code appeared correct, but the agent identified that it failed to wrap the operation in an atomic transaction. This wasn't just a minor error; it was a 'poison' bug. Because the connection was left in a transaction state, every subsequent operation would fail to commit, leading to silent data loss. For a database utility library, this is a catastrophic flaw. An agent, working through a series of prompts, managed to reproduce the error end-to-end, proving the danger before the code ever reached a user.

The agent sometimes needs 10-15 minutes to churn away on a new task, providing an opportunity for the human to step away.

This workflow changes the rhythm of coding. Instead of the constant, micro-level focus required by manual debugging, the developer can set a high-level goal and allow the agent to work through the complexities. Willison describes a process of 'occasional check-ins'—prompting the next step from a phone while away from the desk. The agent handles the heavy lifting of code changes across dozens of files, while the human provides the strategic direction and the final, authoritative review.

Lessons from AI-led development

Agents excel at finding edge cases in transaction handling
The human role is shifting toward high-level architectural review
Multi-model verification reduces the risk of model-specific hallucinations
Documentation should be reviewed first to establish a baseline for changes

However, this is not a silver bullet. The human must still possess the deep expertise required to understand the implications of the agent's changes. The agent found the bug, but it was the human who understood why it was a 'release blocker' and how to integrate the fix into the existing library architecture. We are entering an era of 'augmented engineering,' where the speed of development increases, but the requirement for deep, fundamental knowledge becomes even more critical.

Key Takeaway

AI agents are excellent at finding bugs, but humans are still required to understand their consequences.

04 — Simon Willison

The Regression of Intelligence

Why state-of-the-art models are failing at basic tool use

By Simon Willison · 5 min read

Editor's note: As models get smarter, they are paradoxically becoming worse at interacting with the tools we build for them.

There is a strange phenomenon occurring in the development of large language models: as they become more capable in reasoning and knowledge, they are becoming less reliable in their ability to follow simple technical schemas. This is known as tool-call regression. Developers are finding that the newest, most powerful models—the ones touted as the state-of-the-art—are more likely to invent non-existent parameters or malform the structure of a tool call than their older, 'dumber' predecessors.

The Training Trap

The cause of this regression might be found in how these models are trained. Recent models undergo intense Reinforcement Learning (RL) to excel at specific, built-in tools—such as the search-and-replace functions used by Claude Code. While this makes them incredibly efficient within their own ecosystem, it appears to have a side effect. The models become 'overfit' to those specific patterns, making them less flexible when they encounter third-party tools that use different schemas. They try to force the logic of their native tools onto external systems, resulting in errors.

The SOTA models of the family are worse at this specific tool schema than their older siblings.

This creates a significant problem for the developers of coding harnesses and AI agents. If a model is trained to use a specific 'apply_patch' mechanism, it may struggle when a user switches to a tool that requires a different method of editing. This suggests that 'intelligence' is not a monolithic trait. A model can be a genius at reasoning while being a clumsy, unreliable operator of basic software interfaces. For the industry to move forward, we need models that are not just smart, but also architecturally compliant.

Implications for tool developers

Third-party tools may need to implement multiple versions of the same tool to accommodate different model biases
Schema strictness in models is a moving target
Reinforcement Learning for specific tasks can degrade general tool-following ability

The takeaway for those building the next generation of AI software is that you cannot assume a 'smarter' model will be easier to integrate. In fact, the more advanced the model, the more likely it is to behave in ways that defy your expectations. Reliability in tool use is a distinct skill from reasoning, and currently, the industry is trading one for the other.

Key Takeaway

Intelligence and reliability are not the same thing; do not mistake reasoning power for technical compliance.

05 — The Marginalian

The Hardest Battle

E.E. Cummings and the fight for the individual in an age of feedback

By Maria Popova · 12 min read

Editor's note: In a world of constant digital feedback, the poet's advice on being oneself is more urgent than ever.

We live in a Pavlovian ecosystem. Every action we take is met with a metric: a like, a view, a share, or a comment. This constant loop of feedback creates a powerful pressure to conform, to smooth out our edges, and to adopt the opinions that are most readily rewarded by the crowd. It is a system designed to turn individuals into echoes. In this environment, the act of being oneself is not just a personal choice; it is a radical act of resistance.

The Distinction Between Knowing and Feeling

The poet E.E. Cummings understood this struggle better than most. He argued that the most important distinction in human experience is between thinking, believing, and feeling. To think or to believe is to participate in a collective; when you adopt a thought or a belief, you are often just adopting the thoughts of others. But to feel is to be singular. As Cummings famously put it, the moment you feel, you are 'nobody-but-yourself.' Feeling is the only experience that cannot be taught or mass-produced.

To be nobody-but-yourself — in a world which is doing its best, night and day, to make you everybody else — means to fight the hardest battle which any human being can fight.

This battle is not easy. For Cummings, it meant facing intense criticism from the literary establishment for his unconventional style and his refusal to adhere to traditional poetic structures. He chose to live in defiance of those who ruled by fear, prioritizing his creative integrity over social or professional acceptance. He understood that the cost of being an artist—or simply being a person of integrity—is the constant struggle against the urge to be comfortable and predictable.

The components of creative integrity

The courage to reject established norms
The willingness to prioritize feeling over intellectual consensus
The discipline to work harder than those who follow the easy path
The resilience to withstand social and professional pressure

As we move further into an era of automated content and algorithmic curation, the temptation to surrender our individuality to the 'middle'—the most statistically probable opinion—will only grow. The machines are very good at helping us be 'everybody else.' The challenge for the modern individual is to find the courage to be the outlier, to protect that secret knowledge of self that no model can replicate and no algorithm can predict.

Key Takeaway

True individuality is found in feeling, not in the consensus of thought.