In Renaissance Florence, a patron commissioning a fresco from a master painter faced a peculiar problem. He could stand in the chapel every day and watch the artist work — the brush moving, the pigments mixing, the figures emerging from the plaster. He could ask questions. But he could not, in any meaningful sense, see what was happening. The decisions that separated a masterwork from a failure — pigment load, plaster timing, proportion adjustments — were invisible to him. He was watching, but he was not comprehending.
The patron had outsourced competence, not just execution. And the more skilled the painter became, the less the patron could evaluate the work. The gap between watching and understanding widened with every increase in the artist's capability.
This is the same paradox we are now living through with AI agents. Except we have built an entire industry to avoid admitting it.
The Inverse Relationship
Something strange is happening in the agent ecosystem. On one side of the market, a wave of tools is giving agents ever more autonomy — permission to execute multi-step plans, manipulate real-world systems, publish content, manage finances, and make decisions without human intervention at each step. The explicit goal is to collapse the distance between intention and outcome. You think it; the agent does it.
On the other side, an equally vigorous wave of tools is giving us ever more visibility into what agents are doing — streaming their terminal output, recording their decision logs, tracing their reasoning chains, building dashboards that claim to show us "what's really happening inside the agent."
These two trends are presented as complementary. In practice, they are in direct tension.
The relationship between agent capability and meaningful observability is inverse, not linear. The more competent the agent becomes, the more steps it can take autonomously, the more decisions it compresses into opaque internal representations — the less a human observer can actually understand from watching its logs.
A stream of tool calls and token probabilities is not insight. It is noise dressed as transparency.
We are building agents that think in high-dimensional spaces and then building dashboards that flatten that thinking into two-dimensional timelines, and we are calling this "observability." It is not. It is the illusion of observability — a window that shows us the glass, not the world beyond it.
Anxiety Management, Not Visibility
The observability industry is not solving a visibility gap. It is solving an anxiety problem.
When a powerful agent is set loose on a complex task, the patron — the human who triggered the execution — experiences a specific form of discomfort. They have delegated something important. They do not fully understand how the agent works. They cannot reconstruct its reasoning from its outputs. They feel, quite reasonably, that they have lost control.
The observability tool promises to restore that control. Here is the chain of thought. Here are the tool calls in order. Here is the confidence score at each step. The anxiety subsides. Not because the patron now understands what the agent did, but because they have something to watch.
This is the dashboard equivalent of checking your phone's battery percentage every five minutes. The activity provides a sense of agency without actually delivering any. You are not managing the battery by watching it. You are managing your anxiety about the battery.
The tools being built today — the terminal streamers, the log visualizers, the step-by-step replay systems — are anxiety management devices. They give the patron something to look at so they feel like they're still involved. But involvement is not understanding. And understanding is what trust actually requires.
The observability industry is not solving a visibility gap. It is solving an anxiety problem.
The Competence You Outsourced Is the Competence You Lost
The deeper shift — the one that makes this paradox structural rather than temporary — is that we have outsourced not just execution but the very framework for evaluating execution.
A Renaissance patron could not paint a fresco, but he could judge one. He understood proportion, perspective, iconography, the conventions of religious narrative. He had a framework. The painter operated within that framework, just at a higher level of skill. The patron's ignorance was quantitative, not qualitative.
Our relationship with agents is different. The framework itself is opaque. An agent's reasoning does not happen in a space we can intuitively navigate. It happens in a latent space of high-dimensional vectors, attention patterns across millions of tokens, and emergent behaviors that even the engineers who built the models cannot fully predict. When we look at a chain-of-thought trace, we are not seeing the real reasoning — we are seeing a post-hoc rationalization generated by the same model that produced the output. It is a performance of reasoning, optimized by RLHF to look plausible.
We have outsourced competence itself — the ability to know whether what just happened was good, correct, or safe. And then we built dashboards to pretend we hadn't.
This is not a failure of the tools. It is a structural consequence of the architecture. A transformer does not reason the way we reason, and it has no internal monologue we can tap into. The traces it generates for our benefit are translations into a language we can understand — but every translation is a lossy compression. The more complex the reasoning, the more is lost in the translation.
We are watching subtitled versions of a film playing in a language no human speaks. The subtitles are generated by the same system that made the film. And we are asking whether we can trust the film based on the subtitles.
The Performance of Progress
This leads to the most dangerous confusion of all: the inability to distinguish progress from performance.
An agent that produces a plausible chain of tool calls, hits all the right APIs, generates outputs that look correct, completes its task within expected parameters — is it genuinely making progress? Or is it performing a convincing simulation of progress?
We have no reliable way to answer this question for complex, multi-step tasks. Not because we lack the right observability tool, but because observability cannot solve an epistemological problem.
The question "is the agent actually doing what I think it's doing?" is not a logging problem. It is a theory-of-mind problem. We are trying to attribute intentions, understanding, and goals to a system that may or may not have any of those things. We are trying to read the agent's mind. The agent does not have a mind to read.
This is the difference between watching a human colleague and watching an agent. When a colleague explains their reasoning, we can evaluate it against our own understanding of the domain. We can ask follow-up questions that probe actual understanding. And we can detect when someone is bullshitting — because we share a model of what genuine competence looks like.
With agents, we have no such shared model. The agent's output is the only evidence we have. And that output is optimized to look like the right answer, not to reveal the process by which it was derived.
We have created a system that is extraordinarily good at performing competence. We build observability tools to watch that performance and mistake the watching for knowing.
The question "is the agent actually doing what I think it's doing?" is not a logging problem. It is a theory-of-mind problem.
The Historical Pattern
This is not the first time new tools have created new forms of ignorance. The printing press made memorization less essential and changed what it meant to know something. The industrial revolution made craft knowledge less valuable and shifted what it meant to be skilled. The internet made factual recall less important and redefined what it meant to be educated.
In each case, the expansion of capability came with a contraction of a different kind of knowledge. We gained reach and lost depth. Speed came at the cost of patience. Access to information cost us the ability to evaluate it.
But those were changes in what we knew. This time, the change is in how we know. The agent does not just give us answers faster — it performs the very process of reasoning that we previously used to evaluate those answers. It occupies the epistemic space that was once our own.
This is why no amount of observability tooling will solve the underlying discomfort. The tools address the wrong layer of the stack. They give us more data about the agent's behavior, but they do not give us back the competence to evaluate that behavior. They cannot. That competence was not lost to a visibility gap. It was lost to a capability gap between human and machine cognition.
What Trust Actually Requires
If observability tools cannot deliver genuine understanding, then what does trust in agents actually require?
The uncomfortable answer is that it requires something closer to faith.
Trust between humans is built on a shared model of the world, repeated evidence of reliability, and the ability to attribute intentions. We trust a surgeon because we believe they share our goal of our well-being, because they have trained for years, because their outcomes are good. But even then, trust is not total. We ask questions. We get second opinions. We maintain the ability to evaluate the surgeon's work against our own understanding.
With agents, we have none of these mechanisms. We cannot attribute intentions to a statistical pattern matcher. Cross-referencing its decisions against our own domain expertise is futile — because the decisions are increasingly made in domains where that expertise is insufficient or irrelevant. And a second opinion from another agent would carry the same structural opacity.
The patron's choice is stark: either rebuild the competence to evaluate what the agent is doing, or accept that you are operating on faith.
- Rebuilding competence means learning to think alongside the agent — to develop a new kind of literacy that maps between human intuition and machine output. This is possible, but it is hard work. It requires studying the agent's failure modes, building mental models of its blind spots, and developing the ability to recognize when a plausible output is actually wrong. The same work any expert does when learning to collaborate with a new tool — except the tool is now more capable than the expert in many dimensions.
- Accepting faith means acknowledging that you cannot know what the agent is doing, and deciding to trust it anyway. This is not irrational — we do it with human experts all the time. But it requires a different kind of relationship with the agent: treating it not as a tool to be monitored but as a delegate to be evaluated on outcomes, not process. It means accepting that observability is a ritual, not a control mechanism.
The worst outcome is the middle ground we currently occupy: the belief that our dashboards and logs are giving us genuine insight, when they are actually giving us the performance of insight. This is not trust and it is not understanding. It is self-deception.
Observability tools manage our anxiety, not our ignorance.
The observability industry is growing because the anxiety is real.
We have built systems that we do not understand, doing things we cannot fully evaluate, in domains where the stakes keep rising. The instinct to watch them is not wrong. The mistake is believing that watching is the same as knowing.
The real question is not "how do we see what agents are doing?" It is "how do we know that what we are watching is real?" And that question cannot be answered by building better dashboards, streaming more logs, or adding more traces.
It requires something harder: accepting that the gap between our understanding and our agents' capability is structural, not temporary. Observability tools manage our anxiety, not our ignorance. Trust, in the end, is not a technical problem. It is a choice about what kind of relationship we want to have with the minds we have made.
The patron in the Renaissance chapel could have learned to paint. He could have hired a second painter to watch the first. Or he could have accepted that his role was not to judge the craft but to judge the outcome — the finished fresco on the wall.
We face the same choices today. The worst option, then as now, is to stand in the chapel every day, watching the brush move, believing that you are seeing the art.
No comments yet