The cockpit nobody designed on purpose

AgentOccam's claim fits in one sentence: leave the model alone, change what it reads and what it is allowed to say, and a browser agent's WebArena success goes from 16.5% to 43.1%. No second model, no search, no extra agent roles. Just a smaller cockpit.

I rebuilt the smaller cockpit, or the half of it that can be tested without a model in the loop, and ran eight synthetic business workflows through it. The observation-token proxy fell 46.0% and every control the tasks needed was still there afterwards. Then I sat with the result for a day and noticed that it had moved my production verdict by exactly nothing, which I half expected and still found deflating. A page the model can read is worth having. A page the model can read is also, on the open web, a page an attacker can write, and nothing in the compactor knows the difference.

That gap is what I can speak to, along with what the paper does earn. The prototype does not reproduce AgentOccam's WebArena numbers, it never calls a model, and it cannot tell you whether a smaller interface makes an agent choose better; it tests one property of the representation and stops.

If you only want the decision: build the compactor, run it against your verbose interface on frozen traces with the same model, then put it in a read-only shadow worker. Do not let it purchase, send, delete, approve, or update a system of record until a separate policy layer and a human have said yes. In my fixtures, mean observation size fell from 224.25 to 120.75 proxy tokens, the exposed action list went from 12 to 8, and all gold element IDs survived. A useful action can still be one the agent is not allowed to take, and a tidy observation can still carry a prompt injection. Neither is in the paper's scope, and the paper says so.

What the model gets handed for ticket T-390

Take a support workflow you have probably automated before: find ticket T-390, read the complaint, assign it to Payments, save. At each step a conventional browser agent receives some combination of a screenshot, a document object model, an accessibility tree, previous screenshots, past actions, tool instructions, and a long menu of commands. Out of all that, the model has to find the one control that matters, hold the goal in mind, emit valid syntax, and recover when the site changes under it.

More context feels safer, because nothing has been left out. In practice it hands the model more ways to be wrong. Duplicate labels compete for attention. Repeated navigation fills the prompt. Reversible actions like scrolling become somewhere to stall. A full history keeps abandoned plans alive long after they stopped helping.

AgentOccam calls this an observation and action-space alignment problem, and the reframe underneath it is the part I keep coming back to: the browser state and the model's world state are two different things. Somebody chose that representation, and somebody chose the action vocabulary too. If you built the agent, that somebody was you, and both are yours to redesign. A browser agent is an information bottleneck and a capability surface, and if you never designed either on purpose, you inherited someone else's defaults.

Three cuts the paper makes, and the one it doesn't

Hover goes, branch and prune arrive

WebArena's baseline action space hands the model noop, hover, press, scroll, tab management, forward navigation and direct URL navigation. AgentOccam takes away the ones that were rarely useful, frequently misused, or that assumed embodied knowledge a text model does not reliably have. Some interactions fold into higher-level operations, and the page is loaded in full so the model never has to scroll for it.

It also adds workflow actions the baseline lacked: note, stop, go_home. Two more, branch and prune, let the model open a subplan or abandon one. The planning still comes from the same model; there is no second planner service to run, pay for, or debug.

Here is the seam I want you to hold onto for the rest of the piece. Removing hover because it causes loops and blocking delete_account because policy forbids it both shrink the same list, and they are different acts. The first is capability design. The second is security. A production action set needs both kinds of reduction, and only the first is in this paper.

The smallest prompt that still holds the next decision

Raw HTML and accessibility trees mix useful semantics with formatting tokens, repeated roles and layout scaffolding. AgentOccam merges elements that describe the same function, compresses lists and tables, and strips redundant text. The target is the smallest prompt that still contains everything the next decision needs, which is a different thing from the smallest prompt, far harder to optimise for and far easier to get wrong.

Miss that distinction and you will prune on token count alone, and I would call that the trap. A control that appears exactly once can be the only route to finishing the task. Any metric worth having pairs compression with task-relevant recall, and reports both.

Memory keeps the active plan, not the whole past

While choosing an action, the agent marks page nodes it considers "pivotal". Later prompts keep those nodes and nearby context instead of replaying every prior page in full, and the branch/prune tree decides which earlier steps belong to the active plan at all. Memory gets a shape instead of a length: a dead search route stops eating prompt space once the agent has switched to a direct order lookup.

The ordering mattered more than I assumed. Selective history cut repeated actions and average steps, but it hurt some dense single-page and multi-page categories until the planning component landed (Section 5, Figure 5 and Tables 3–5). On its own, the history filter made things worse in places.

My prototype implements the same interface premise without copying the paper's code:

def align_observation(html: str) -> AlignmentResult:
    raw_parser = _RawObservationParser()
    raw_parser.feed(html)
    baseline = "\n".join(raw_parser.lines)
 
    element_parser = _ElementParser()
    element_parser.feed(html)
    aligned = _render_elements(element_parser.elements)
 
    return AlignmentResult(
        baseline_observation=baseline,
        aligned_observation=aligned,
        baseline_tokens=approximate_tokens(baseline),
        aligned_tokens=approximate_tokens(aligned),
        preprocessing_ms=elapsed_ms,
    )

The tests pin it down from three sides: a target such as button#return-order must survive compaction, hidden controls and scripts must not, and pivotal history from an abandoned branch must disappear. The evaluation separately requires 100% recall of the gold element IDs. The invariant I would steal from that excerpt is the pairing: every transformation reports what it removed alongside what it kept. What the excerpt does not contain is any of AgentOccam's model policy, any of WebArena, or any of the task-success result.

43.1% on 812 tasks, and what that still leaves failing

The fairest comparison in that block is 43.1% against 16.5%: same model family, same environment, two interfaces. Read one way it is a 26.6 percentage-point gain. Read the other way it is a 56.9% failure rate, and which of those two numbers you find more interesting probably depends on whether you are writing a paper or an on-call rota. "State of the art" and "ready to act on a customer's behalf" sit nowhere near each other.

Would Gemini have killed the result?

No, and I expected it to. My first suspicion on reading Table 2 was that the whole effect was one prompt quirk in one model family, because interface changes tuned against one model so often evaporate on the next.

Two models do not make model independence. They did make me drop the prompt-quirk theory, which I had been fairly attached to.

Which of the cuts earned the gain

The authors add components one at a time, and that section is what earned my attention. Reducing actions produces the first large gain. Disabling scrolling helps some sites and inflates observation size. Page simplification cuts context and usually improves success. Selective history removes repeated actions and sometimes throws away dense-page context worth keeping. Branch and prune recover performance across most site categories (Section 5, Figure 5).

Those ablations kill the simplest reading of the paper, which is that fewer tokens caused the whole thing. Several components contribute, and one of them is planning.

Simpler, and about 1.9 times more input per task

The cost story is where I am least confident, and where I think the common reading is wrong. My impression is that the community treats AgentOccam as a cost win. The paper's own tables do not support that. The plain agent averaged 2,210.2 observation tokens per step and 6.2 steps per task. AgentOccam averaged 2,930.9 tokens and 9.0 steps (Tables 4 and 5). Multiply those averages and you get a rough input-volume proxy of 13,703 versus 26,378 tokens per attempted task, about 1.9 times higher for AgentOccam. Treat that multiplication as my inference, not a provider bill: the paper reports the averages separately and does not report wall time, output tokens, or cost per successful task.

So is it cheaper? Per completed task, I suspect it is, since higher success spreads the spend over more finished work; per attempted task, the paper's own averages say no. My point is narrower. Nothing in this paper lets you say "simpler means cheaper" out loud without measuring your own workload first.

Once through 129 live questions

The real-web result rests on 129 WebVoyager questions with deterministic answers, where AgentOccam scored 54.3% against 51.9% for the authors' Agent-E replication (Appendix C, Table 9). Each task ran once. Open-ended tasks were dropped because their answers and the human judgements about them move, and GitHub tasks were dropped because anti-scraping behaviour caused timeouts and risked getting the host blocked.

Both exclusions were the right call, and both are a warning about how fast live-browser numbers rot. Emergence WebVoyager later audited the benchmark, identified 11 methodological shortcomings, and rebuilt it as 535 tasks with explicit success criteria and 95.9% inter-annotator agreement (Sections 1 and 4). WebArena Verified did the same for all 812 WebArena tasks, replaced brittle or subjective checks with deterministic evaluators, and reported that its baseline had suffered about 11% false negatives.

Walk out of the benchmark into enterprise work and the gap opens up. WorkArena++ contains 682 composite knowledge-work tasks. On the 98-task human curriculum, people achieved 93.9% while the GPT-4o agent achieved 2.1% (Section 4.4, Table 2). A cleaner interface can help with that, and it does not erase long-horizon planning, arithmetic, permission, or recovery failures.

Security is absent from the paper altogether, which I want to state plainly because the absence is easy to miss in a paper this tidy. WASP tested prompt injection in end-to-end web environments. Across its configurations, 17% to 86% of runs showed intermediate hijacking, while end-to-end attacker completion reached up to 16.7% (Section 4, Table 2). The authors attribute the low completion partly to "security by incompetence". A more capable agent removes that accidental protection, and AgentOccam is a recipe for a more capable agent.

Six papers on the desk, one whose claim could fail cleanly

I screened six serious candidates against claim support, baseline strength, causal evidence, production transfer, and testability. One person, criteria I chose; a reader with a different production problem would have shortlisted differently, and I would not defend this table as a survey.

AgentOccam won because its claim can fail cleanly. Hold the model and the task fixed, swap two interfaces, and measure success, token volume, steps and unsafe actions. Compare that with adopting an entire agent architecture and hoping the aggregate improves: one of those is an experiment and the other is a migration.

What I wrote down before running the compactor

My predictions, before the first run: somewhere between 35% and 50% observation reduction, with full gold-element recall. The history filter would be the biggest single win, since full-page replay repeats more text than anything else does. And if anything fired the stop threshold, it would be a missing checkbox label or a control nested three levels deep.

The repository holds eight hand-authored synthetic HTML tasks across e-commerce, inventory, support, expenses, human resources, developer tools, forums, and travel. Each fixture names the element IDs required to complete its instruction. The compactor keeps interactive controls and useful structure, drops scripts and hidden controls, deduplicates output, reduces the action list, and replays pivotal lines from the active branch. No website, model API, credential, or third-party package is involved.

git clone https://github.com/shravan1996/ship-the-paper-browser-agents.git
cd ship-the-paper-browser-agents
PYTHONPATH=src python3 -m unittest discover -s tests -v
PYTHONPATH=src python3 -m occam_browser.evaluate

The full implementation, fixtures, tests, and machine-readable result are in the prototype repository.

Every gold control survived, and the 46.0% stopped mattering

Three predictions, three hits, and the recall result surprised me not at all: hand-authored fixtures are the kindest possible page. The history filter was indeed the largest single cut at 56.9%, ahead of observation size at 46.0% and actions at 33.3%. Nothing tripped the stop threshold, not the checkbox labels and not the nested controls.

What did catch me out was my own reaction to the headline number. I went in expecting the size reduction to be the interesting output, and came out thinking it is the least interesting thing the experiment produced. A smaller prompt tells you nothing about whether the model will choose better with it. The paper measures that; I could not. And a smaller prompt tells you nothing about whether the action the model picks is one it should be allowed to take, which neither of us measured.

Where a tidy page turns against you

The GitLab case the authors chose to report

The paper's generic rules usually help. Its own ablation also notes a GitLab case where simplification hid an easier solution, and it surprised me that the authors reported it at all; most papers would have let that one go. It is the case I would worry about most, because production pages are full of custom widgets, unlabeled controls, virtualized lists, and text that only appears after you interact with something. So what do you do when the compactor drops a control? Do not average it away. A missing-control rate belongs in your release blockers, next to a failing test. Keep the raw trace, compare target recall workflow by workflow, and fall back to the uncompressed representation the moment the parser loses semantic coverage.

Two documents, and the scroll action you removed

Removing scroll or tab actions helped the benchmark, but I expect multi-document reconciliation needs both. One global allowlist will either be too broad for low-risk tasks or too narrow for legitimate ones. Define capabilities per workflow. Read-only search may receive click, type, back, and extract. Purchasing should require a separate proposal action that cannot commit the transaction.

Strip the noise and the injection gets louder

Compaction buys legibility, and there is a nasty corollary. Strip the surrounding noise from a forum post and a malicious instruction inside it becomes more prominent, not less. Treat every byte of page text as data. Keep user intent in a separate trusted channel. Validate each action against identity, destination, data classification and expected task state. WASP is the evidence that a defensive prompt on its own is no boundary at all.

The evaluator changes under the claim

Live pages drift, benchmark tasks expire, and evaluators are wrong more often than anyone budgets for. Store the environment version, initial state, model version, prompt, action schema, trace, network events and final state. Re-score old traces whenever an evaluator changes. WebArena Verified's audit is the proof that benchmark maintenance is part of the system, not tidying up after the science.

More steps per success, and a queue nobody staffed

AgentOccam bought its higher task success with more average steps and more observation tokens per step. A production test therefore has to report completed-task cost, p50 and p95 wall time, loop rate and human-review minutes, or it is a demo with a chart. Stop a run when the action budget expires or the agent repeats a state. A better success percentage can hide a queue nobody can staff.

The scorecard where the constraint outvotes the average

The average says testable. The binding constraint says not autonomous, and when the two disagree I let the constraint win, because raising task success without touching authority raises expected value and expected harm together, in the same release.

A worker that proposes, and a gateway that says no

Follow one request through the figure. Before the browser even starts, a policy gateway checks the user, the website, the task class, the data boundary, the action budget and the allowed capabilities. Only then does the worker compact the observation and ask the model for one aligned action.

Reads run inside the sandbox and pass through an outcome validator. Writes, sends, purchases, approvals and deletions become proposals instead of actions; anything irreversible stops at a proposed action and goes no further on the synchronous path. The reviewer sees the instruction, the target, the changed fields and the relevant trace before saying yes. Every state transition and network event is logged. Budget exhaustion, a repeated state, a policy mismatch or an unexpected domain all fail closed, which I would argue is the only setting worth having when the input channel is the open web.

1. 01

   Bound one read-only workflow

   Owner

   Application lead

   Artifact

   A task contract naming sites, inputs, outputs, and forbidden actions.

   Signal

   Every historical case fits the contract without exception handling.

2. 02

   Freeze the evaluation set

   Owner

   Applied science

   Artifact

   At least 200 versioned traces with deterministic outcome checks and target-element annotations.

   Signal

   Two reviewers agree on success and failure criteria for at least 95% of cases.

3. 03

   Compare interfaces with the same model

   Owner

   Applied science

   Artifact

   Paired runs for the verbose and aligned interfaces.

   Signal

   Task success does not fall, target recall stays at 100%, and completed-task cost improves or stays inside budget.

4. 04

   Add the authority boundary

   Owner

   Platform and security

   Artifact

   Per-workflow action capabilities, a destination allowlist, prompt-injection tests, and a kill switch.

   Signal

   Forbidden mutations are blocked before browser execution in every test.

5. 05

   Run read-only shadow traffic

   Owner

   Platform

   Artifact

   Replayable traces and a daily failure report.

   Signal

   p95 response time, loop rate, and evaluator disagreement stay below their limits for two weeks.

6. 06

   Decide, redesign, or stop

   Owner

   Product, security, and operations

   Artifact

   An evidence-backed decision record.

   Signal

   The workflow either stays read-only, moves to approval-gated writes, or is retired.

The workflow ships only when all of these hold:

• Paired evaluation shows no task-success regression and 100% recall of required controls.
• p95 wall time, model cost per completed task, and human-review time fit the workflow's value.
• Prompt-injection tests, domain escapes, forbidden actions, and repeated-state loops fail closed.
• Every proposed mutation includes a replayable trace and an explicit approver.

It comes back when any of these fires:

• Any unauthorized write, send, purchase, approval, deletion, or cross-tenant read.
• Required-control recall falls below 100% or task success drops more than 2 percentage points against the verbose interface.
• More than 5% of runs exceed the action budget or require manual recovery.
• Evaluator disagreement exceeds 5%, or live-site drift invalidates more than 2% of the frozen tasks.

And a human reviews every state-changing action, without exception. Read-only extraction can skip review only when the destination, fields, and final-state evaluator are deterministic and the trace is retained.

Wait on writes, build the interface anyway

AgentOccam passes the research gate comfortably. The same-model comparison and the component ablations together show that interface design can matter more than another layer of agent strategy, and I did not believe that going in. My prototype backs the narrow premise underneath: a useful browser representation can get materially smaller and still keep every marked control.

None of that adds up to production authority, and here the seam between the paper and my judgement matters. The paper does not measure prompt injection, permission boundaries, wall time, or the cost of operating against websites that change under you; my prototype does not call a model at all. WorkArena++ says composite enterprise work is nowhere near solved, and WASP says page content can redirect an agent before it finishes either the user's task or the attacker's. The Wait is mine, built on those three gaps.

The next step I would fund is narrow: a frozen-trace comparison, then a read-only shadow worker. The temptation at that point is to add writes so the pilot feels real to whoever approved the budget. Resist it. Writes come after the policy layer, the deterministic outcome checks, the approval flow and the kill criteria have all survived the same workflow.

I would change this verdict if:

1. the aligned interface matches or beats the verbose baseline on at least 200 representative traces while reducing completed-task cost;
2. a WASP-style security suite shows zero unauthorized actions and every mutation is blocked before explicit approval; and
3. a two-week read-only shadow trial stays within response-time, loop, drift, and evaluator-disagreement limits.

Before you shrink the next agent's interface

What would move me from here is a paired run on real traces: verbose interface against aligned interface, same model, with target recall reported per workflow instead of averaged. My prototype measured representation, not decisions, and I would rather see someone else's number than defend mine. If you have run it, especially on pages with custom widgets or content that renders only after interaction, tell me what recall did. I expect it fell somewhere, and I want to know where.

Acknowledgements

This article builds on AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents by Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala.

The prototype is an original implementation inspired by the paper's observation and action-space alignment. It does not copy AgentOccam source code. The authors' public implementation is available under Apache-2.0.

The evaluation fixtures are synthetic and were created for this article. No private data, live website content, or paid model output is included.

References

Primary research

• AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents. Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala, ICLR 2025

Supporting and contrary evidence

• WebArena: A Realistic Web Environment for Building Autonomous Agents. The self-hosted benchmark used by AgentOccam
• SteP: Stacked LLM Policies for Web Actions. A stronger specialized-policy baseline
• Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems. Hierarchical planning, DOM denoising, and change observation
• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. Live multimodal web-agent evaluation
• The BrowserGym Ecosystem for Web Agent Research. Unified browser-agent experimentation and benchmark infrastructure
• WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks. Enterprise-task transfer boundary
• WebArena Verified: Reliable Evaluation for Web Agents. Audited tasks and deterministic evaluation
• WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks. End-to-end prompt-injection evidence
• Emergence WebVoyager: Toward Consistent and Transparent Evaluation of (Web) Agents in the Wild. Live-web evaluation drift and methodology

Prototype

• Source, setup, tests, and evaluation
• Exact tested commit

Sources last checked: 2026-08-06