A smaller interface makes browser agents better. It doesn't make them safe

A browser agent that misreads a page can waste a task. A browser agent that reads a malicious page correctly can do something worse.

The distinction matters because the browser is two things at once: the agent's tool, and an untrusted input channel. The same page holding the button you want also holds advertisements, user comments, hidden controls, stale labels, and possibly a sentence written specifically to redirect your model. Better reasoning helps with the first problem. It can make the second one worse.

AgentOccam attacks the reasoning problem with an idea that costs almost nothing to try: before you add more models, more search, or more agent roles, change what the model sees and what it is allowed to say. The result is strong enough to be worth testing, so I tested the narrow version of it. My offline prototype cut an observation-token proxy by 46.0% and kept every marked control across eight synthetic workflows.

What this piece will not do: reproduce AgentOccam's WebArena numbers, call a model, or tell you whether a smaller interface makes an agent choose better. It tests one property of the representation and stops there.

I half expected the prototype to move my production verdict. It didn't, and that is the honest headline. A cleaner interface is worth building and it is not an authority boundary. I would run it on frozen traces, then in a read-only shadow worker. I would not let it purchase, send, delete, approve, or update a system of record without a separate policy decision and a human saying yes.

The short version

• The idea: treat the browser-to-model interface as a product surface, and remove the choices and context that do not help the task in front of it.
• Why it might work: models reason better over concise text and familiar actions than over verbose page structure and low-level mouse work.
• What I tested: a verbose HTML-like observation versus an AgentOccam-inspired compactor, smaller action vocabulary, and selective history on eight synthetic business workflows.
• What happened: mean observation size fell from 224.25 to 120.75 proxy tokens, exposed actions fell from 12 to 8, and all gold element IDs remained present.
• Where it breaks: a useful action can still be one the agent is not allowed to take, and a tidy observation can still carry a prompt injection. Neither is in the paper's scope.
• Decision: wait on autonomous write access; continue with frozen-trace evaluation and a read-only shadow trial.

Start with the problem

Take a support workflow you have probably automated before: find ticket T-390, check the complaint, assign it to Payments, save. What does a conventional browser agent get handed at each step? Some combination of a screenshot, a document object model, an accessibility tree, previous screenshots, past actions, tool instructions, and a large menu of commands. Out of all that, the model has to find the relevant control, remember the goal, choose valid syntax, and recover when the site changes under it.

More context feels safer, because nothing has been left out. In practice it gives the model more ways to be wrong. Duplicate labels compete for attention. Repeated navigation fills the prompt. Reversible actions like scrolling become somewhere to stall. And a full history keeps abandoned plans alive long after they stopped helping.

AgentOccam calls this an observation and action-space alignment problem. Here is the reframe worth keeping: the browser state is not the model's true world state. Somebody chose that representation, and somebody chose the action vocabulary too. If you built the agent, that somebody is you, and both are yours to redesign.

I would say the framing is the most reusable thing in the paper, more than any of its numbers. A browser agent is not a model plus Playwright. It is an information bottleneck and a capability surface, and if you never designed either of them on purpose, you inherited someone else's defaults.

How the method works

1. Remove actions that invite weak behavior

WebArena's baseline action space hands the model noop, hover, press, scroll, tab management, forward navigation and direct URL navigation. AgentOccam takes away the ones that were rarely useful, frequently misused, or that assumed embodied knowledge a text model does not reliably have. It folds some interactions into higher-level operations and loads the full page rather than asking the model to scroll for it.

It also adds workflow actions the baseline lacked: note, stop, go_home. Two more, branch and prune, let the model open a subplan or abandon one. Note what is absent here. Planning still comes from the same model, and there is no second planner service to run, pay for, or debug.

Capability design, then, and not yet security. Removing hover because it causes loops is a different act from blocking delete_account because policy forbids it, even though both shrink the same list. A production action set needs both kinds of reduction, and only one of them is in this paper.

2. Turn browser structure into readable text

Raw HTML and accessibility trees mix useful semantics with formatting tokens, repeated roles and layout scaffolding. AgentOccam merges elements that describe the same function, compresses lists and tables, and strips redundant text. The target is not the smallest prompt. It is the smallest prompt that still holds everything the next decision needs, which is a much harder thing to optimise for and much easier to get wrong.

Miss that distinction and you will prune on token count alone, which is the trap. A control that appears exactly once can be the only route to finishing the task. Any metric worth having pairs compression with task-relevant recall, and reports both.

3. Replay the active plan, not the entire past

The agent marks “pivotal” page nodes while choosing an action. Future prompts keep those nodes and nearby context instead of replaying each prior page in full. The branch/prune tree also decides which earlier steps belong to the active plan.

Memory gets a shape instead of a length. A dead search route stops eating prompt space once the agent has switched to a direct order lookup. Selective history cut repeated actions and average steps, though it hurt some dense single-page and multi-page categories until the planning component landed (Section 5, Figure 5 and Tables 3–5). That ordering matters: on its own, the history filter made things worse in places.

The prototype implements the same interface premise without copying the paper's code:

def align_observation(html: str) -> AlignmentResult:
    raw_parser = _RawObservationParser()
    raw_parser.feed(html)
    baseline = "\n".join(raw_parser.lines)
 
    element_parser = _ElementParser()
    element_parser.feed(html)
    aligned = _render_elements(element_parser.elements)
 
    return AlignmentResult(
        baseline_observation=baseline,
        aligned_observation=aligned,
        baseline_tokens=approximate_tokens(baseline),
        aligned_tokens=approximate_tokens(aligned),
        preprocessing_ms=elapsed_ms,
    )

Correctness check: the tests assert that a target such as button#return-order survives compaction, hidden controls and scripts do not, and pivotal history from an abandoned branch disappears. The evaluation separately requires 100% recall of the gold element IDs.

The invariant worth stealing from that excerpt is the pairing: every transformation reports what it removed alongside what it kept. It reproduces none of AgentOccam's model policy, none of WebArena, and none of the task-success result.

What the research actually shows

The strongest result

The fairest comparison is therefore 43.1% against 16.5%, same model family, same environment. A 26.6 percentage-point gain, and still a 56.9% failure rate. Which of those two numbers you find more interesting probably depends on whether you are writing a paper or an on-call rota. “State of the art” and “ready to act on a customer's behalf” are nowhere near the same threshold.

Two models is not model independence. It does make me less worried that the whole result is one prompt quirk in one model family, which was my first suspicion.

Does the mechanism earn the credit?

The authors add components one at a time, which is the part that earned my attention. Reducing actions produces the first large gain. Disabling scrolling helps some sites and inflates observation size. Page simplification cuts context and usually improves success. Selective history removes repeated actions and sometimes throws away dense-page context worth keeping. Branch and prune recover performance across most site categories (Section 5, Figure 5).

Useful causal evidence, and it kills the simplest reading of the paper, which is that fewer tokens caused the whole thing. Several components contribute and one of them is planning.

The cost story is mixed, and I am less confident about this part than about the rest. My impression is that the community reads AgentOccam as a cost win, and I do not think the paper supports that reading. The plain agent averaged 2,210.2 observation tokens per step and 6.2 steps per task. AgentOccam averaged 2,930.9 tokens and 9.0 steps (Tables 4 and 5). Multiplying those averages gives a rough input-volume proxy of 13,703 versus 26,378 tokens per attempted task, about 1.9 times higher for AgentOccam. This is an inference, not a provider bill: the paper reports averages separately and does not report wall time, output tokens, or cost per successful task.

Higher success can still win on cost per completed task, and it probably does. My point is narrower: nothing in this paper lets you say “simpler means cheaper” out loud without measuring your own workload first.

Where the evidence is thin

The real-web result rests on 129 WebVoyager questions with deterministic answers, where AgentOccam scored 54.3% against 51.9% for the authors' Agent-E replication (Appendix C, Table 9). Each task ran once. Open-ended tasks were dropped because their answers and the human judgements about them move, and GitHub tasks were dropped because anti-scraping behaviour caused timeouts and risked getting the host blocked.

Both exclusions are the right call, and both are a warning about how quickly live-browser numbers rot. Emergence WebVoyager later audited the benchmark, identified 11 methodological shortcomings, and rebuilt it as 535 tasks with explicit success criteria and 95.9% inter-annotator agreement (Sections 1 and 4). WebArena Verified similarly audited all 812 WebArena tasks, replaced brittle or subjective checks with deterministic evaluators, and reported that its baseline had suffered about 11% false negatives.

The gap widens sharply once you leave the benchmark and walk into enterprise work. WorkArena++ contains 682 composite knowledge-work tasks. On the 98-task human curriculum, people achieved 93.9% while the GPT-4o agent achieved 2.1% (Section 4.4, Table 2). A cleaner interface can help, but it does not erase long-horizon planning, arithmetic, permission, or recovery failures.

Security is not a weak part of this paper. It is an absent one. WASP tested prompt injection in end-to-end web environments. Across its configurations, 17% to 86% of runs showed intermediate hijacking, while end-to-end attacker completion reached up to 16.7% (Section 4, Table 2). Low completion was partly “security by incompetence.” A more capable agent can remove that accidental protection.

Why this paper earned the test

I screened six serious candidates against claim support, baseline strength, causal evidence, production transfer, and testability.

Six candidates, screened by one person against criteria I chose. I expect a different reader with a different production problem would have shortlisted differently, and I would not defend this table as a survey. AgentOccam won because its claim can fail cleanly. You can hold the model and the task fixed, swap two interfaces, and measure success, token volume, steps and unsafe actions. Compare that with adopting an entire agent architecture and hoping the aggregate improves: one of those is an experiment and the other is a migration.

Rebuilding the claim

Experimental setup

The repository contains eight hand-authored synthetic HTML tasks across e-commerce, inventory, support, expenses, human resources, developer tools, forums, and travel. Each fixture names the element IDs required to complete its instruction. The compactor keeps interactive controls and useful structure, drops scripts and hidden controls, deduplicates output, reduces the action list, and replays pivotal lines from the active branch.

No website, model API, credential, or third-party package is used.

git clone https://github.com/shravan1996/ship-the-paper-browser-agents.git
cd ship-the-paper-browser-agents
PYTHONPATH=src python3 -m unittest discover -s tests -v
PYTHONPATH=src python3 -m occam_browser.evaluate

The full implementation, fixtures, tests, and machine-readable result are in the prototype repository.

What I expected

Somewhere between 35% and 50% observation reduction, with full gold-element recall. I also expected the history filter to be the biggest single win, since full-page replay repeats more text than anything else does. If anything was going to fire the stop threshold, I assumed it would be a missing checkbox label or a control nested three levels deep.

The recall result did not surprise me. What did was how little the compression number moved my view of the paper: I went in expecting the size reduction to be the interesting output and came out thinking it is the least interesting thing the experiment produced, because nothing about a smaller prompt tells you the model will choose better with it.

What happened

What breaks outside the experiment

Compression can hide the only valid path

The paper's generic rules usually help. Its own ablation also notes a GitLab case where simplification hid an easier solution, and it surprised me that the authors reported it at all. That case is the one I would worry about, because production pages are full of custom widgets, unlabeled controls, virtualized lists, and text that only appears after you interact with something. So what should you do when the compactor drops a control? Not average it away. A missing-control rate belongs in your release blockers, next to a failing test. Keep the raw trace, compare target recall workflow by workflow, and fall back to the uncompressed representation the moment the parser loses semantic coverage.

A smaller action set can remove required work

Removing scroll or tab actions helped the benchmark, but I expect multi-document reconciliation needs both. One global allowlist will either be too broad for low-risk tasks or too narrow for legitimate ones. Define capabilities per workflow. Read-only search may receive click, type, back, and extract. Purchasing should require a separate proposal action that cannot commit the transaction.

The page can instruct the agent

Compaction buys you legibility, not trust, and there is a nasty corollary. Strip the surrounding noise from a forum post and a malicious instruction inside it becomes more prominent, not less. Treat every byte of page text as data. Keep user intent in a separate trusted channel. Validate each action against identity, destination, data classification and expected task state. WASP is the evidence that a defensive prompt on its own is not a boundary.

Evaluation changes underneath the claim

Live pages drift, benchmark tasks expire, and evaluators are wrong more often than anyone budgets for. Store the environment version, initial state, model version, prompt, action schema, trace, network events and final state. Re-score old traces whenever an evaluator changes. WebArena Verified's audit is the proof that benchmark maintenance is part of the system rather than tidying up after the science.

Success can cost more than failure

AgentOccam bought its higher task success with more average steps and more observation tokens per step. So a production test has to report completed-task cost, p50 and p95 wall time, loop rate and human-review minutes, or it is not a production test. Stop a run when the action budget expires or the agent repeats a state. A better success percentage can hide a queue nobody can staff.

The production scorecard

The average says testable. The binding constraint says not autonomous, and when those two disagree the constraint wins. Raising task success without touching authority raises expected value and expected harm together, in the same release.

A production shape I would test

Read it as one request. Anything irreversible stops at a proposed action and goes no further on the synchronous path. Before the browser even starts, a policy gateway checks the user, the website, the task class, the data boundary, the action budget and the allowed capabilities. Only then does the worker compact the observation and ask the model for one aligned action.

Reads run inside the sandbox and pass through an outcome validator. Writes, sends, purchases, approvals and deletions become proposals instead of actions. The reviewer sees the instruction, the target, the changed fields and the relevant trace before saying yes. Every state transition and network event is logged. Budget exhaustion, a repeated state, a policy mismatch or an unexpected domain all fail closed, which is the only setting worth having when the input channel is the open web.

From prototype to shadow traffic

1. 01

   Bound one read-only workflow

   Owner

   Application lead

   Artifact

   A task contract naming sites, inputs, outputs, and forbidden actions.

   Signal

   Every historical case fits the contract without exception handling.

2. 02

   Freeze the evaluation set

   Owner

   Applied science

   Artifact

   At least 200 versioned traces with deterministic outcome checks and target-element annotations.

   Signal

   Two reviewers agree on success and failure criteria for at least 95% of cases.

3. 03

   Compare interfaces with the same model

   Owner

   Applied science

   Artifact

   Paired runs for the verbose and aligned interfaces.

   Signal

   Task success does not fall, target recall stays at 100%, and completed-task cost improves or stays inside budget.

4. 04

   Add the authority boundary

   Owner

   Platform and security

   Artifact

   Per-workflow action capabilities, a destination allowlist, prompt-injection tests, and a kill switch.

   Signal

   Forbidden mutations are blocked before browser execution in every test.

5. 05

   Run read-only shadow traffic

   Owner

   Platform

   Artifact

   Replayable traces and a daily failure report.

   Signal

   p95 response time, loop rate, and evaluator disagreement stay below their limits for two weeks.

6. 06

   Decide, redesign, or stop

   Owner

   Product, security, and operations

   Artifact

   An evidence-backed decision record.

   Signal

   The workflow either stays read-only, moves to approval-gated writes, or is retired.

Ship gates

• Paired evaluation shows no task-success regression and 100% recall of required controls.
• p95 wall time, model cost per completed task, and human-review time fit the workflow's value.
• Prompt-injection tests, domain escapes, forbidden actions, and repeated-state loops fail closed.
• Every proposed mutation includes a replayable trace and an explicit approver.

Kill criteria

• Any unauthorized write, send, purchase, approval, deletion, or cross-tenant read.
• Required-control recall falls below 100% or task success drops more than 2 percentage points against the verbose interface.
• More than 5% of runs exceed the action budget or require manual recovery.
• Evaluator disagreement exceeds 5%, or live-site drift invalidates more than 2% of the frozen tasks.

Human review policy: review is mandatory for every state-changing action. Read-only extraction can skip review only when the destination, fields, and final-state evaluator are deterministic and the trace is retained.

My verdict

Wait on autonomous write-capable browser agents. Build the interface anyway.

AgentOccam passes the research gate comfortably. The same-model comparison and the component ablations together show that interface design can matter more than another layer of agent strategy, which is not what I expected going in. My prototype backs the narrow premise underneath that: a useful browser representation can get materially smaller and still keep every marked control.

I would say none of that adds up to production authority. The paper does not measure prompt injection, permission boundaries, wall time, or the cost of operating against websites that change under you. My prototype does not call a model at all. WorkArena++ says composite enterprise work is nowhere near solved, and WASP says page content can redirect an agent before it finishes either the user's task or the attacker's.

So the next step I would fund is narrow: a frozen-trace comparison, then a read-only shadow worker. The temptation at that point is to add writes so the pilot feels real to whoever approved the budget. Resist it. Writes come after the policy layer, the deterministic outcome checks, the approval flow and the kill criteria have all survived the same workflow, and not before.

I would change this verdict if:

1. the aligned interface matches or beats the verbose baseline on at least 200 representative traces while reducing completed-task cost;
2. a WASP-style security suite shows zero unauthorized actions and every mutation is blocked before explicit approval; and
3. a two-week read-only shadow trial stays within response-time, loop, drift, and evaluator-disagreement limits.

What to remember

What would move me here is a paired run on real traces: verbose interface against aligned interface, same model, with target recall reported per workflow rather than averaged. My prototype measured representation, not decisions, and I would rather see someone else's number than defend mine. If you have run it, especially on pages with custom widgets or content that renders after interaction, I want to hear what recall did.

Acknowledgements

This article builds on AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents by Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala.

The prototype is an original implementation inspired by the paper's observation and action-space alignment. It does not copy AgentOccam source code. The authors' public implementation is available under Apache-2.0.

The evaluation fixtures are synthetic and were created for this article. No private data, live website content, or paid model output is included.

References

Primary research

• AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents. Ke Yang, Yao Liu, Sapana Chaudhary, Rasool Fakoor, Pratik Chaudhari, George Karypis, and Huzefa Rangwala, ICLR 2025

Supporting and contrary evidence

• WebArena: A Realistic Web Environment for Building Autonomous Agents. The self-hosted benchmark used by AgentOccam
• SteP: Stacked LLM Policies for Web Actions. A stronger specialized-policy baseline
• Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems. Hierarchical planning, DOM denoising, and change observation
• WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models. Live multimodal web-agent evaluation
• The BrowserGym Ecosystem for Web Agent Research. Unified browser-agent experimentation and benchmark infrastructure
• WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks. Enterprise-task transfer boundary
• WebArena Verified: Reliable Evaluation for Web Agents. Audited tasks and deterministic evaluation
• WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks. End-to-end prompt-injection evidence
• Emergence WebVoyager: Toward Consistent and Transparent Evaluation of (Web) Agents in the Wild. Live-web evaluation drift and methodology

Prototype

• Source, setup, tests, and evaluation
• Exact tested commit

Sources last checked: 2026-08-06

A browser agent should earn a smaller interface first, and production authority last.