Long-running AI agents: what you need before you let them loose

In July 2026, the headline is easy to find. Rakuten engineers put Claude Code to the test with a concrete task: implement an activation vector extraction method in vLLM, a 12.5-million-line open source library spanning multiple languages. The model finished the entire job in seven hours of autonomous work, in one continuous run, at 99.9% numerical accuracy against the reference method, according to Anthropic’s report^[1]. It’s the kind of stat that shares itself.

A few pages later, the same report says developers use AI for around 60% of their work but fully delegate only 0 to 20% of their tasks^[1]. That gap between what the model can do and what teams dare to hand off to it doesn’t close with a better model. It closes with a harness.

The senior question isn’t how many hours the model can hold up. It’s what you need to have built into your system so that letting a long-running AI agent work alone for hours is an engineering decision, not Russian roulette.

Do agents really work alone for hours already?

Yes, but it’s worth separating who’s saying it from how rigorously it’s measured. There are three tiers, and mixing them up is exactly what produces the hype.

The independent measurement comes from METR. They define the 50%-confidence task horizon: the task duration, measured by how long an expert human would take, that the model would complete with 50% probability^[2]. Their finding is that this duration roughly doubles every seven months^[3]. Models from March 2025 got nearly 100% of tasks under four human-minutes right, and dropped below 10% on tasks over four hours^[3].

The observation comes from Anthropic’s report. Early agents solved minute-scale tasks (fix this bug, write this function); by late 2025 they were shipping complete feature sets over several hours. For 2026 it predicts days — with a caveat that matters, “with periodic human checkpoints”^[1]. The days part is a prediction, not a measurement.

The third tier is the vendor’s own claims. Anthropic says it observed Sonnet 4.5 staying focused for more than 30 hours on complex, multi-step tasks^[4], and with Fable 5 (June 2026) describes a 50-million-line Ruby codebase migration done in a single day, versus more than two months by hand for a team^[5]. These are statements from whoever sells the model. Useful as a ceiling, not as a floor.

And here’s the detail almost nobody cites: METR itself warns that its measurements above 16 hours aren’t reliable with the current task suite^[2]. Not even the reference organization for measuring this can yet verify day-scale horizons. All the more reason for verification to be yours, not something you delegate to someone else’s benchmark.

Why does the harness grant autonomy, not the model?

Autonomy isn’t granted by trusting the model — it’s granted by the quality of the harness. That’s the thesis, and the very report celebrating the seven hours backs it up unintentionally.

The delegation gap (60% usage, 0 to 20% full delegation^[1]) isn’t irrational distrust. It’s the absence of the pieces that make letting the agent loose safe. The same document nails it: agentic work isn’t fully delegated, it’s highly collaborative^[1].

A long-running agent and a hung agent stuck in an infinite loop are, underneath, the same loop: read state, decide on an action, execute, repeat. The difference lies elsewhere. One has a designed stopping condition and ground truth at every step; the other spins with no external signal that it’s making progress. The long-running agent is the extreme case of the well-designed agentic loop: same mechanism, far more surface area to get it wrong.

What follows are six engineering pieces: design patterns you build into your system. If you want to practice them from the ground up, the AI Agent Design Patterns course walks through them one by one. And a good share of agent projects that end up canceled fail for lack of this scaffolding, not because of a weak model.

What executable acceptance criterion defines “done”?

An agent knows it’s done when something it can execute tells it so. Without that first prerequisite, none of the others matter.

The Claude Code docs put it plainly: give Claude a check it can run (tests, a build, a screenshot to compare); it’s the difference between a session you babysit and one you can walk away from^[6]. Without that check, “looks done” is the only signal available, and you become the verification loop: every error waits for you to notice it^[6].

This isn’t new. Since “Building effective agents,” Anthropic has insisted that the agent get ground truth from the environment at every step: tool call results, code execution, something real to check against^[7]. And the harness for long-running agents raises it to a process rule: mark a feature as “passing” only after testing it carefully^[8]. The acceptance criterion is a script that returns an exit code, not an impression.

# check.sh — la condición de parada que el agente ejecuta, no interpreta.
# Encadena verificación barata a cara y corta en el primer fallo.
set -euo pipefail

npm run lint            # estático: rápido, descarta lo obvio
npm test                # tests sobre el diff
npm run build           # que compile de verdad, no que "compile en su cabeza"
node evals/run.mjs      # tu eval de dominio: puntuación mínima o exit 1

echo "PASS"             # solo si todo lo anterior pasó

When there’s no deterministic test (a natural-language output, a design, a product decision), the secondary check is a model-as-judge with an explicit rubric. And if you’re defining which metrics count as “done” in production, that’s the same work I describe in how to evaluate AI agents in production. The underlying rule: your evals set the stopping condition, not a public benchmark measuring something else.

How does an agent with no memory resume?

A long-running agent remembers nothing between sessions: every start is from zero. The core problem, per Anthropic engineering, is that these agents work in discrete sessions and each new session begins with no memory of what came before^[8]. The model doesn’t persist. State has to live outside of it.

Their harness solves it with an initializer agent that builds the scaffolding the first time: an init.sh script, a claude-progress.txt file logging what the agents have done, and an initial git commit^[8]. Every subsequent session makes incremental progress and leaves structured updates. The key, in their words, is that a fresh context can understand the state of the work quickly, and that comes from the progress file alongside the git history^[8]. It’s the same principle that makes spec-driven development the team’s memory: whatever has to survive the session lives versioned in the repo, not in the context window.

# No es un script: es la estructura de ficheros que el arnés necesita en el repo.
# Estado reanudable: vive en el repo, no en el context window.
init.sh                 # levanta el entorno igual en cada sesión
claude-progress.txt     # log append-only: hecho / en curso / siguiente / decisiones
.git/                   # commits incrementales = puntos de reanudación reales

# convención de commit por sesión:
#   feat(step-7): extractor pasa; tests verdes; siguiente = normalizar salida

Claude Code added checkpoints with instant rollback to a prior state in September 2025, one of its most requested features^[4]. They’re useful within a session. But the docs themselves warn in a callout: checkpoints only record changes made by Claude, not external processes, and are not a replacement for git^[6]. Serious resumable state is still the repo.

Hard-cutoff budgets: tokens, time, actions

An agent without a hard-cutoff budget doesn’t work more: it burns more. “Building effective agents” recommends explicit stopping conditions, like a maximum iteration count, to keep control^[7], and names the risk that justifies them: autonomous behavior implies higher costs and compounding errors^[7]. An agent that’s been wrong for three hours isn’t making progress. It’s amplifying the error, and paying tokens to do it.

The hard cutoff runs along three measurable axes: tokens consumed, wall-clock time, and number of actions executed. Whichever runs out first kills the run. There are no public figures for the token consumption of a seven-hour session, so you don’t pull the budget from a headline — you calibrate it with your own short runs and their telemetry, the same discipline behind measuring every cycle to adjust the next one.

	Long-running agent	Hung agent
Stopping condition	designed and executable (check.sh green)	“until it looks done”
Progress signal	passing tests + commit + progress file per session	activity with no ground truth
Cost	hard-cutoff budget on tokens, time, and actions	grows as the error grows
Errors	caught at step N by the check	compounded silently for hours
When you come back	you audit a structured log	archaeology in the scrollback

The skeleton of an unattended run with a hard cutoff fits in a few lines of bash:

# Run desatendido: permisos acotados + reloj + tope de iteraciones + tope de tokens.
MAX_ITERS=40; iter=0
MAX_TOKENS=2000000; tokens=0     # tercer eje: corta por consumo acumulado

while [ "$iter" -lt "$MAX_ITERS" ] && [ "$tokens" -lt "$MAX_TOKENS" ]; do
  out=$(timeout 20m claude -p "Avanza el objetivo. Al terminar ejecuta ./check.sh." \
    --allowedTools "Edit,Bash(npm run lint),Bash(npm test),Bash(npm run build),Bash(./check.sh),Bash(git commit *)" \
    --output-format json)                                  # nada de acceso libre
  tokens=$((tokens + $(echo "$out" | jq '.usage.input_tokens + .usage.output_tokens')))
  ./check.sh && break            # condición de parada real: el check pasa
  iter=$((iter + 1))
done

[ "$iter" -eq "$MAX_ITERS" ]    && echo "CORTADO: presupuesto de iteraciones agotado"
[ "$tokens" -ge "$MAX_TOKENS" ] && echo "CORTADO: presupuesto de tokens agotado"

Least-privilege permissions and sandboxing: approval fatigue

An unattended run needs least-privilege permissions, OS-level isolation, and a record of what it did. Start with what looks like security but is actually design: approval fatigue. The more times a human approves, the less they read; oversight turns into theater and, according to Anthropic, makes development less safe^[9]. The answer isn’t to approve blindly. It’s that the agent shouldn’t have to ask about what’s already scoped.

Anthropic’s OS-level sandboxing cut permission prompts by 84% in their internal usage^[9]. It requires dual isolation: without network isolation, a compromised agent can exfiltrate sensitive files like SSH keys; without filesystem isolation, it can escape the sandbox and gain network access^[9]. One without the other doesn’t work. The runtime is open source (sandbox-runtime, Apache 2): Seatbelt on macOS, bubblewrap on Linux, denying both writes and network access by default^[10].

In batch execution, the --allowedTools flag restricts what the agent can do, and that matters precisely when you’re running unattended^[6]. Least-privilege permissions and a sandbox are guardrails applied to the highest-risk case: hours with nobody watching. And that perimeter includes whatever the agent has installed: every skill and every connected MCP server runs those hours with your keys — another supply chain you have to audit.

That leaves observability. When you come back, you need to be able to reconstruct what the agent did and why. The progress file, the git history, and the tool-call logs are the flight’s black box: without them, the post-mortem of a seven-hour run is impossible.

What’s your rollback plan for when you come back and it’s wrong?

Assume that some percentage of your long runs are going to end badly, and design for that case from the start. Isolating the work comes first: worktrees, CLI sessions in isolated git checkouts so edits don’t collide^[6]. One disposable branch per run. If the result doesn’t pass the check, you delete the branch and nothing of yours was touched.

Before you sign off on the work, run an adversarial review. The docs put it well: the longer the agent works unsupervised, the more an independent check matters before counting the work as done^[6]. A reviewer with fresh context sees the diff and the criteria, not the reasoning that produced the change, so it judges the work for what it is. Product checkpoints are the first layer; the real rollback plan is git.

And that closes the loop. The question was never how long the model can hold up. It was how long your harness can hold up. The Rakuten case worked for a reason you have to read between the lines to see: 99.9% accuracy against a reference method^[1] means a reference existed to verify against. There was an executable check. Without it, those seven hours would have produced only unverified code that someone would have had to review by hand, line by line, to know if it was any good.

Common mistakes

Letting the agent loose without an executable check

The root error that almost every other one grows out of. If you can’t write the check.sh, the task still isn’t delegable: “looks done” will be the only signal, and you’ll be the verification loop again^[6].

Treating product checkpoints as a rollback plan

They only record changes made by the agent itself, not external processes, and the docs say they’re not a replacement for git^[6]. They’re good for undoing within a session, not for returning to a known-good state.

Widening permissions “so it stops asking so much”

Approving blindly and giving free rein are the same mistake in different disguises: in both cases nobody’s really watching. The alternative to approval fatigue is a sandbox plus least-privilege permissions.

A budget without a hard cutoff

A hung agent doesn’t sit still: it spends. Without a cap on tokens, time, or actions, autonomous behavior translates into high cost and compounding errors^[7].

Extrapolating the vendor’s headline to your system

Anthropic saying 30 hours tells you nothing about your task. The independent measurement doesn’t even verify horizons above 16 hours with its current suite^[2]. Calibrate with your own runs.

Checklist before you let the agent loose

• A check.sh exists that returns pass/fail and defines “done” without your judgment call
• The agent gets ground truth at every step (tests, execution), not just self-assessment
• State survives the session: init.sh, a progress file, and incremental commits
• There’s a hard cutoff on tokens, wall-clock time, and number of actions
• The run executes with least-privilege permissions and a sandbox, with filesystem and network isolated
• You can audit afterward what it did: progress file, git, and tool-call logs
• The work lives in a worktree or disposable branch, with adversarial review before merge

Sources

1. 2026 Agentic Coding Trends Report — Anthropic — the Rakuten vLLM case (12.5M lines, 7 autonomous hours, 99.9% accuracy), the minutes→hours→days trajectory “with periodic human checkpoints,” and the delegation gap (60% usage, 0–20% full delegation).
2. Task-Completion Time Horizons — METR — the definition of the 50%-confidence task horizon and the warning that measurements above 16 hours aren’t reliable with the current suite.
3. Measuring AI Ability to Complete Long Tasks — METR — the task horizon doubling roughly every ~7 months and the success rates of March 2025 models (under 4 human-minutes versus over 4 hours).
4. Claude Sonnet 4.5 — Anthropic — the claim of more than 30 hours of focus and checkpoints with instant rollback as a product primitive in Claude Code.
5. Claude Fable 5 and Claude Mythos 5 — Anthropic — a 50-million-line Ruby codebase migration in a single day (vendor claim).
6. Best practices for Claude Code — Anthropic — “give Claude a check it can run,” the human as the verification loop, --allowedTools in unattended runs, worktrees, adversarial review, and the warning that checkpoints don’t replace git.
7. Building effective agents — Anthropic — stopping conditions (max iterations), per-step ground truth, and the risk of high cost and compounding errors.
8. Effective harnesses for long-running agents — Anthropic — memoryless sessions, the initializer agent (init.sh, claude-progress.txt, initial commit), and verification before marking a feature “passing.”
9. Claude Code sandboxing — Anthropic — the 84% reduction in permission prompts, dual filesystem+network isolation, and approval fatigue.
10. anthropic-experimental/sandbox-runtime — GitHub — the open source runtime (Apache 2) with Seatbelt on macOS and bubblewrap on Linux, denying writes and network by default.

Frequently Asked Questions

How long can a long-running AI agent work unsupervised in 2026?

It depends on who’s measuring. METR, which measures task horizons independently, puts the growth at roughly doubling every seven months, but warns that its measurements above 16 hours aren’t reliable with the current suite. Anthropic observed the shift from minutes to several hours in late 2025 and claims more than 30 hours of focus with Sonnet 4.5, though that last one is a vendor claim. In practice, how long a long-running AI agent can work alone is decided by your harness, not the headline.

What’s the difference between a long-running agent and a hung agent stuck in an infinite loop?

Underneath, they’re the same loop: read state, decide, act, repeat. The difference comes down to two things — a designed, executable stopping condition, and ground truth at every step. The long-running agent stops when a check comes back green; the hung one spins with no external signal that it’s making progress, burning tokens while it amplifies its own error.

Do Claude Code checkpoints replace git?

No. They only record changes made by Claude, not those from external processes, and the documentation itself says they’re not a replacement for git. They’re good for undoing within a session; the real rollback plan is git branches and worktrees.

Do I need a sandbox if I already review every permission by hand?

Yes. Manual review degrades on its own: after approving dozens of prompts you stop reading them and oversight turns into theater. OS-level sandboxing cut permission prompts by 84% in Anthropic’s internal usage without losing safety, precisely because it scopes what the agent can do instead of depending on you watching every single time.

How do I decide if a task is delegable to an agent for hours at a time?

Ask yourself two questions. Is there an executable check that defines “done” without your judgment call — tests, a build, an eval? Is there a reference to verify the result against? A migration with a test suite and a reference method to compare the output against satisfies both: the agent knows when it’s done and you can check whether it got it right, so it’s delegable. A UX refactor guided by “make it feel better” satisfies neither — you’re still the only judge — and stays on the human side. If the answer to either is no, the task is still collaborative, not delegable to a long-running AI agent for hours at a time. That’s consistent with teams fully delegating only 0 to 20% of their tasks.