What Is an Agent Harness? A Security Perspective

There are many different definitions of agents that you'll find out there, but the standard definition almost always comes down to the same few elements: a model that's running in a loop, has access to tools, and is trying to reach a goal. Here's a good example from Simon Willison:

But I want to reach for a different definition that's more useful for us here, from Vivek Trivedy (an engineer at LangChain):

The same basic idea is captured in another definition from Justin Schroeder that I also want to share, because it's probably my favorite:

But you can only understand Vivek's definition if you know what a harness is, so let's now define that.

Picture yourself working with Claude Code (or Codex, or Cursor — any of them). You're on your local machine, running an application, whether a GUI or a TUI. When you ask it something, that question leaves your machine as an API call to a host in the cloud — Anthropic's clusters, running the model (say, Opus 4.8).

The model takes your text as a stream of tokens, runs inference, and predicts the stream of tokens that comes back. That is the model.

Now notice what the model can't do. It runs on someone else's hardware, in another building. It can't read a file on your machine, call an API, or run a command. All it can do is take text in and send text out. So if you want a model to have any effect on your system — to actually do something, rather than just describe what should be done — it needs a process on your machine that acts on its behalf: something that takes what the model says, carries it out, and reports back.

That process is the harness. It's the model's interface to your system — the thing that turns the model's output into actions and the results of those actions into the model's next input. Claude Code, Cursor, or Codex is exactly that: an application running on your machine that connects to the model and mediates the interaction between the model and your local system.

To see what that means in practice, take the exact same model — Opus 4.8 — and compare talking to it through a browser tab at claude.ai with using it through Claude Code. Those are vastly different experiences — one can do a fraction of what the other can — and yet it is literally the same model. The entire difference is the harness.

So yes, agent = model + harness, but what makes an agent an agent — what can turn a chat window into something that hunts — is the harness.

So if our goal is to build a security agent, and an agent is a model plus a harness, then it follows that we have two levers we can use to shape that agent: the model and the harness.

The model — you select it

The bad news first: with the model, we are limited to selection. Realistically, you will never train your own frontier model; 99.99% of us have neither the compute nor the expertise. (A toy model as a learning exercise, sure — a frontier one, no.)

So you pick the one that fits — a hosted frontier API, or an open-source model on your own hardware, whichever you can afford and from whichever provider you prefer. Open-source models can be fine-tuned, but for most people the juice isn't worth the squeeze. Once you've selected and set it up, that's essentially it — you have the weights and the architecture, and you change little beyond that.

The harness — you engineer it

The situation with the harness is very different because it is simply an application, usually written in Python or TypeScript, although you can build it in whatever language makes sense for the job, which means you get to decide how it works, what it can access, and how it shapes the behavior of the agent.

And it's not the consolation prize, because as we'll see next, the research keeps pointing the same way: the harness has the biggest impact on an agent's effectiveness. We're not making do with table scraps — we're taking control of the most powerful component in shaping an effective agent, and the one that's fully in our hands.

The natural objection is that the model must be the part that matters most because it is doing the reasoning, and from there it seems obvious that if you want a better agent, you should simply use a better model; that is the instinct almost everyone has, and it is one the vendors are happy to encourage, because a better model is the thing they sell.

The problem with that line of reasoning is that it leaves out everything that determines what the model receives and what happens after it responds. On any given turn, the model gets a context window containing some instructions, some history, whatever data was put in front of it, and a list of things it is allowed to do, and then it produces an answer.

Now consider who made all of those decisions: the harness determined what went into the window, which tools were available, and what happened to the answer — whether it was checked, acted on, or fed back into the next turn.

So a brilliant model handed the wrong evidence, or too much of it, or no way to act, will produce a poor result. A modest model handed exactly the right evidence, a small clean set of tools, and a check on its output will produce a good one. The harness is upstream and downstream of every single model call.

That's the argument, and it leads to a claim I'll state carefully: all else equal, on average, a mediocre model with a great harness will beat a good model with a mediocre harness.

CORE-Bench Hard tests whether an agent can reproduce the findings of a scientific paper when it is given both the paper and its code, and Princeton's leaderboard gives us a useful comparison because it ran the same model, Claude Opus 4.5, through two different harnesses.

With CORE-Agent it scored about 42 percent, while with Claude Code it scored about 78 percent, meaning the software around the same model nearly doubled its score on the same tasks. One caution before you lean on that number. The benchmark had grading problems, and the same leaderboard reports a much higher figure after manual validation, so treat this as an illustration of how much the harness can move a result, not as a perfectly controlled measurement of harness quality on its own.

A second measurement comes from LangChain's engineering team. They kept the model fixed, GPT-5.2-Codex, changed only the harness, and their coding agent went from 52.8 to 66.5 on Terminal Bench 2.0, a benchmark of tasks done in a terminal. That moved it from roughly thirtieth place to fifth. It is an engineering report from the team that built the agent, not a peer-reviewed paper, and both results are in the references.

So put the two halves side by side. The model is the half you mostly can't move: you pick one, and then it is what it is. The harness is the half you can move, and it turns out to be where the largest gains are. That is a very fortunate way for the world to be arranged, because it means the part you fully control is also the part that pays most.