Branch six was never on the map

Every sensible policy in my first results table found the recorded best. Fixed parallel refine, 100%. The two-strike early-stop rule I had written down as the one that would kill a slowly improving branch, 100%. Greedy-best, the dumbest leaf picker in the set, which I had predicted would chase one high early leaf and never come back: 100%, after 80 serial rounds. I had pre-registered two misses. Neither came.

The paper is Dream-RSI, from Tong Zheng and colleagues. Its claim in one sentence: a finished discovery run, the tree of every generation, compile and kernel timing the agent produced, is an exact simulator for the search space that run reached, so you can score a different stopping rule, batch size or branching decision by walking the tree, without calling the agent again, and deploy only the winner. I rebuilt the walking and scoring part. On six synthetic history trees it selected the two-strike early-stop, which matched the recorded best at 44 attempts instead of 88, and the same rule carried to six held-out trees the selector had never seen. I did not rebuild the language-model loop that rewrites policy code, I did not reproduce the Gemini CLI runs, the Lasso solvers, the math constructions or KernelBench, and nothing here will tell you which discovery agent to buy. What I measured is selection over recorded trees, and what happens when the recorded trees are too narrow.

The miss I had been waiting for turned up in an ablation I nearly did not run. Freeze the width at four workspaces and the frozen policy missed the recorded best on four of those six trees, every time because the winner lived on a branch that initialization never opened. Replay handed it a confident score anyway. A score about the prefix it had seen. I had underweighted that limit going in, and the verdict rests on it.

If you are only here for the decision: the scorer is cheap and I would run it this month, in shadow, over discovery logs you already have, with the current policy in the candidate set so the pick can never be worse than production on those logs. The winner it picks is validated only on branches somebody already paid to open. Keep it off a live job until a coverage check fails closed and the next round has a kill switch, and keep the loop that rewrites policy code open until that code is public.

Ten workspaces, eleven steps, same as last week

Take a kernel-engineering job. You want a faster LayerNorm. The agent proposes an implementation, the evaluator times it against a reference, the score goes up or it doesn't, and you do this a few hundred times. The expensive object here is not any one proposal. It is the policy that decided which workspace to continue, how many to run at once, and when to stop.

Most teams I have watched keep that policy as a constant: N parallel workspaces, M refinement steps, maybe a note in the prompt that says "don't get stuck." Nobody changes it, because judging a change means running another long discovery job, and the feedback that would tell you the new policy is better arrives only after the job ends. The paper names this bottleneck in its introduction: meta-level feedback is delayed and expensive, the space of policies is large, and most of the ones you would try are bad.

The move everyone reaches for first is to stuff the last run into the prompt and ask the model to explore better. Figure 5 in the paper tries a version of that, on ConvDiv, and the guided runs do worse than the unguided ones under the same budget. I went in expecting a small penalty there. My read now is that it is the paper's warning shot: in a long-horizon search with many parallel threads, a strong opinion about where to look next shrinks the space you are willing to pay for.

Two terms carry the rest. A discovery tree is the record of one run: a root, one chain per workspace hanging off it, and at every node a filesystem snapshot, an artifact, diagnostics and a score. Replay is walking that tree under a different policy and revealing only nodes that are already on disk.

Look at the dashed node first. Once you have seen it, everything else in the figure is a loop you can implement beside an agent you already run.

A policy is a program that picks the next batch

Dream-RSI leaves the coding agent, the evaluator, and the execution interface fixed. The thing that changes is a small program that, given the tree so far, picks a batch of nodes to continue from. Eligible nodes are the root, which opens a new workspace, and the current leaves, which extend an existing one. A batch is at most W of those, matching the number of workers who can run a generate-and-evaluate step at once (Section 3). A thinner change than the name recursive self-improvement suggests, and I mean that as a compliment.

In the paper's first round, that program is a hand-written parallel refine: launch several independent workspaces and keep each one polishing its current candidate. Recursive Fixed Exploration, the controlled baseline, keeps that program for every later round. Dream-RSI does not.

The tree is recorded as if it will have to be replayed. Each non-root node has one parent and at most one recorded child, so the structure is a bundle of chains hanging off the root, one chain per workspace. Larger scores are better. That sounded like a limitation to me until I replayed it. Alternative policies do not get to invent siblings halfway down a chain. They get to choose which chains to open, how far to push each one, how to group the pushes into parallel batches, and when to stop. Online, attaching a child means calling the agent. Offline, it means revealing a node that is already on disk.

Replay starts from the root and reveals nodes under the same decision interface. Selecting the root returns the earliest-created child of the root that you have not yet seen, which opens one stored branch. Selecting a leaf returns that leaf's unique recorded child, or nothing. The policy is not allowed to peek at unrevealed scores. Appendix B.2 calls this prefix-only, and it is the constraint that stops the dreaming loop from cheating. After at most K2 rounds, or an empty batch, or a fully revealed tree, the walk gets a replay value. Here is the reveal rule as it stands in my copy:

def child_of(self, node_id: str) -> List[str]:
    if node_id == ROOT_ID:
        unopened = self._unopened_root_children()
        return [unopened[0]] if unopened else []
    child = self.tree.unique_child(node_id)
    if child is None or child in self.observed:
        return []
    return [child]

Three tests pin it down. test_root_opens_earliest_unopened_branch asserts that three successive root selections return b0, b1, then b2, and a fourth returns nothing. test_non_root_reveals_unique_child_only asserts that a leaf reveals its one child and then goes empty. test_replay_never_creates_nodes asserts that a full walk leaves the set of node ids unchanged. I wrote the third one after catching myself wondering whether a bug in the walker could quietly grow the tree; it can't, and now I know it can't.

Equation 1 then scores the walk as best revealed quality, minus a cost on the number of revealed attempts, plus a bonus for packing those attempts into fewer decision rounds. Because the currently deployed policy is in the candidate set, the selected version cannot be worse than what you already run, on this history. I want to be careful about the scope of that guarantee. It is local to the trees you replayed. It says nothing about the next online job.

A policy-development agent then edits the policy file, using replay feedback, for M revisions. I did not rebuild that agent. The dreaming step I did rebuild is the scoring and the selection.

One dataset carries the Lasso average

The 1.7× call reduction against the frozen policy is the comparison I trust, because the agent is held fixed. The 162× figure against SimpleTES is a different model, a different budget, and a different paper. I will admit that the 162× was the number that first made me open the paper, and I would now put it in a footnote.

The controlled baseline is the right one. Round 1 is identical by construction. Later rounds are where dreaming is allowed to change the policy, and Figure 3(b) is the picture of that divergence: the two curves share a start, then Dream-RSI sits down and to the left. Figure 6, on ConvDiv, shows the learned policy doing something I would actually want in production: it cuts evaluated attempts from 110 to 50 while scores are moving, then spends again when they stall, and those widenings line up with the next jumps.

Figure 5 is the ablation that isolates how history is used. Injecting directional guidance from prior trajectories into the prompt, on top of either paradigm, underperforms the unguided counterpart. History as a simulator beat history as advice, on that kernel.

I had read "competitive or improved" in the abstract as better everywhere. Wrong, and it is the view I changed while reading the table, before I ran anything. The RCV1 win did not surprise me. The five datasets that quietly went the other way did. An average that is almost entirely one dataset is still a result; I am not sure the sentence in the abstract is carrying it.

No seeds, no betas, no limitations section

There is no limitations section. Variance across seeds is not reported. beta1 and beta2 in Equation 1 are not given as numbers. Appendix B.2 scores revisions with a Pareto AUC minus a parallelism penalty and a separate patience knob also called beta, which is not Equation 1. I do not know which objective the actual policy-development agent optimized. Official code is "being prepared for release"; the GitHub repository has a paper PDF, a citation file, and no license.

On the math tasks (Table 1), Dream-RSI wins Sum-Difference against the frozen baseline and against SimpleTES, ties the crowded Circle Packing number at 2.635983, and loses Autocorrelation to SimpleTES (1.456375 against 1.453675, lower is better), while also sitting slightly behind its own frozen baseline (1.456001). KernelBench (Figure 4) is the cleanest efficiency story: 2.43× and 1.79× fewer generations to matched performance on VGG16 and LayerNorm, 2.09× and 1.44× higher scores at comparable budgets on ConvDiv and ConvMax.

Five self-improvement papers, one with a replay rule

I screened five serious candidates against claim support, baseline strength, causal evidence, production transfer, and testability. Recency is a filter, not a ranking; anything whose first public version is older than 22 March 2026 is out. EvoX (26 February 2026) and Hyperagents (19 March 2026) missed the window, so they did not reach the gate. AlphaEvolve and PACEvolve are older still.

The shortlist is a judgement call, and so is the pick. Dream-RSI won because it's the one paper in the window that turns a log you already have into a test you can run without another rollout. SwarmResearch is the one I would build next if I cared more about keeping search diverse than about scoring the controller offline. When official code lands, I expect it to change how hard the integration row feels far more than it changes the coverage problem.

What I wrote down before the first scoring run

Two predictions went into the repository next to those thresholds. Early-stop would save attempts and miss at least one late peak, because a two-strike rule looks like it would kill a branch that improves by 0.08 a step. Greedy-best would miss late peaks for a different reason: after initialization it would see a high early leaf and never come back. I would have bet on at least one of those misses showing up in the main table. I have already confessed the third expectation, that the 162× SimpleTES comparison was the number that earned the test.

The trees are synthetic and labelled so in the repository. Six history trees and six held-out trees, each with eight branches of eleven attempts, 88 recorded nodes, matching the shape of the paper's 10-workspace, 11-step first round at a smaller scale. Four families: a late peak on branch 0, a late peak on branch 6, an early peak whose best score is already at depth 0, and a plateau-then-jump kernel analogue. Recommended tags point at frequent, usually suboptimal mechanisms, so a guidance policy has something plausible to lock onto.

Five prefix-only policies, including the current one. No language model anywhere. Equation 1 uses beta1 = 0.02 and beta2 = 0.05, written down before the first scoring run; those numbers are mine, because the paper does not give its own. W = 8, shared initialization of eight workspaces. Apple M4 Pro, Python 3.14.6, no network.

git clone https://github.com/shravan1996/ship-the-paper-dream-rsi
cd ship-the-paper-dream-rsi
python3 -m unittest discover -s tests
python3 eval.py

The full implementation and the corpus generator are in the prototype repository. The excerpt above is the shortest path to the measured result.

Early-stop halved the attempts and missed nothing

Continue fired on the pre-registered rule: quality ratio 1.000, attempt ratio 0.500. The row I keep coming back to is semantic guidance, the arm that treats history as a recommended mechanism to lean into. It found the recorded best on one tree in six, the math family, where the winning score was already sitting on an initialized node.

Why did the two-strike rule keep the late peaks?

Because initialization had already opened the branch each peak lives on, and a late peak still climbs every step. Two strikes never accumulate on a branch that keeps improving by 0.08, so the rule spends its cuts on the chains that had already stalled.

Greedy is the one I got more wrong. With every branch root already scored, greedy eventually walks almost every chain, one leaf at a time, hits the recorded best, and burns 80 rounds and 87 attempts to do it. I did not expect a 100% hit rate from the dumbest leaf picker in the set. Where did it fail, then, if it hit every recorded best? On cost and serial rounds, and only there. Equation 1 rejects that arm through the cost and parallelism terms. Quality matching on its own would have waved it through.

I had been treating "always refine the current best leaf" as a way to get stuck. Full initialization already paid for a foot in every door, so getting stuck, in this setup, looks like 87 serial attempts and not a missed winner. I had assumed the miss and the waste were the same failure. They are two.

Then I froze the width at four

The miss I had pre-registered for the main table showed up in the coverage ablation instead. Freeze width at four, bootstrap four, and frozen parallel refine never sees branch 6. Late-peak families drop from a recorded best near 9.14 to whatever the first four branches can offer, around 7.8. Frozen-width fixed hit the recorded best on 2 of 6 history trees. Early-stop at the same width, reusing the workers it had freed, hit 6 of 6, because a freed worker goes back to the root and opens the next stored branch.

What I did not expect was how clean the miss would be. Every one of the four failures was a winner on a branch the frozen policy never opened, and replay still returned a number for that policy, with no flag, because from inside the prefix nothing was missing. I had gone in thinking of coverage as a caveat for the discussion section. I would now say it is the mechanism of the failure, and the reason the rest of this article leans the way it does.

Three ways a dreamed policy hurts you on a live job

The next job opens a street that is not on the map

Replay is exact over recorded nodes. The paper is explicit that a requested plan beyond the frozen trace is out of support and cannot earn replay reward (Appendix B.2). Production discovery is the opposite move: you run the new policy so that it will go somewhere the last tree did not. A high replay score is then evidence about yesterday. The warning sign is a winner whose best node sat on a branch the last job never opened, or a policy that shrinks width and then gets deployed into a task whose last winner was a late workspace. The control is to refuse promotion unless the selected policy still covers the branches that held the last K winners, and to keep the current policy as the live fallback.

Last week's best kernel in the prompt

Figure 5 is the paper's version of this failure. Mine is semantic_guidance, which only refines the recommended tag and stops when that tag is exhausted. It found the recorded best only on the math family, where the winning score was already sitting on an initialized node. Late peaks on an unrecommended mechanism were never reached. If your team currently dumps last week's best kernel into the prompt as "try more of this," you are running that arm. Keep history in the tree store, not in the instruction, until a replay score says a directional prior is worth the coverage you give up.

The cost term nobody published

I chose beta1 = 0.02 and beta2 = 0.05 because a one-point quality gap should be worth about fifty extra probes on my 0-10 scores. Move those knobs and greedy, or adaptive, can win the same trees. The paper never states the coefficients, and the appendix talks about a different formula. I suspect a published beta sweep would move the winner on these trees more than a new seed would. Freeze beta1 and beta2 in the gate, commit them next to the policy diff, and reject a winner that only appears at one setting.

Coverage is the row the average cannot fix

The average says wait, and I would still wait if it drifted up, because of the coverage row. You can decorate a prefix with a beautiful score and have no evidence at all about the part of the search that will matter on Monday. The scores are my judgement; the failure-modes 2 in particular is a call about how silent the miss is, and someone who runs discovery jobs with generous width might score it 3.

Dream overnight, gate before the next batch

The live path is the next batch job. Dreaming happens overnight, or at the end of a round, against trees that are already closed. If the gate fails, the current policy keeps the next job. If it passes, the winner runs once, with a cost ceiling and a quality floor copied from the last frozen round, and any coverage miss, meaning a best node on a branch the last tree never opened, rolls back. I would implement that rollback before I let anyone call this recursive self-improvement in a status meeting.

Shadow the scorer on one logged job

1. 01

   Pick one discovery job that already logs per-attempt scores

   Owner

   Applied science

   Artifact

   A single workflow: one kernel family, one solver, or one math construction, with a scalar evaluator

   Signal

   Last month's runs can be rebuilt as trees without new agent calls

2. 02

   Freeze a replay suite from those logs

   Owner

   Platform

   Artifact

   `evals/v0` of recorded trees, with branch ids and the then-deployed policy pinned

   Signal

   Replaying the pinned policy reproduces its attempt count and best score

3. 03

   Score a small candidate set that includes the current policy

   Owner

   Applied science

   Artifact

   A table of V, quality, attempts, and hit of the recorded best

   Signal

   Selection is never worse than current on this suite

4. 04

   Add the coverage check

   Owner

   Platform

   Artifact

   A gate that lists branches holding the last K winners and refuses a policy that cannot open them

   Signal

   The W = 4 miss on late-branch trees is a failing test, not a dashboard curiosity

5. 05

   Shadow the winner on the next recorded round

   Owner

   Applied science + platform

   Artifact

   A side-by-side of frozen policy versus dreamed policy on the newly closed tree

   Signal

   Quality within 2 percent of frozen, attempts at most 80 percent, for two consecutive rounds

6. 06

   One live round, or stop

   Owner

   Platform + risk

   Artifact

   A decision record with the policy diff, β values, coverage report, and the rollback that fired or did not

   Signal

   Kill if attempts exceed the last frozen round, if quality drops more than 2 percent, or if the best node sits on a new branch the gate did not require

Before a dreamed policy touches a live round, three things have to be true. Replaying the current policy on the frozen suite matches its logged attempt count and best score. The selected policy is no worse than current on that suite, with attempts down at least 20 percent. And every branch that held a winner in the last K rounds is openable by the selected policy.

Any one of the following kills it. A live round spends more discovery-agent calls than the last frozen round. Downstream quality drops more than 2 percent against the frozen policy on the same evaluator. Or the best node of a live round sits on a branch the previous tree never opened, and the gate did not require that branch.

A person reads the policy diff whenever width, maximum depth, or the cost term changes, and whenever the selected policy would be allowed to open fewer workspaces than the last winner needed. Replay tables and branch lists can shrink that review. They cannot skip it for a change that reallocates cluster time.

Wait for the live round, ship the scorer

Wait for live redeploy of a dreamed exploration policy. Ship the scorer in shadow on one logged discovery job.

The paper earned the test: a frozen-policy baseline with the same agent is the comparison I wanted, and the replay rule is specified well enough to implement without weights. My prototype fired Continue on the narrow claim. I am still waiting on the live path, because the failure I care about was never "the cheaper rule was worse on the trees I had." It is "the cheaper rule looked great on the trees I had, and the next job needed a branch those trees never grew."

Should you put the cheaper policy on the next live job? I think not, not yet. I would put the scorer on a kernel-engineering or solver-search log this month, with the current policy in the candidate set and a coverage check that fails closed. I would not let the winner spend the next GPU-hour until official code exists or a reimplementation has survived two shadowed rounds, and I would not describe that setup as recursive self-improvement. What you have is probably a cheaper stopping rule, selected on yesterday's map. My impression is that the live round is still the part that can hurt you.

I would change this verdict if:

1. Public code, or a faithful reimplementation, beats frozen exploration on the Lasso Flash comparison under pinned seeds, not only on an RCV1-weighted average.
2. A live round, gated on coverage, beats frozen width on a task whose winner sat on a branch the previous tree did not contain.
3. A published beta sweep, or one you run on your own logs, leaves the selected family of policies stable instead of flipping with the cost term.

What a recorded tree can and cannot tell you

If you have a discovery tree from a real kernel or solver run, I want to hear whether replay picked a cheaper rule, and whether the winner lived on a branch the first round never opened. I would especially like to hear from anyone whose jobs run at narrow width, because four workspaces is where my copy stopped seeing things, and I have no idea yet whether real trees are kinder than mine or worse.

Acknowledgements

This article builds on Dream-RSI: Recursive Self-Improvement through Evolving Worlds by Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, Xue Wang, Yue Zhuan, Wang-Cheng Kang, Renkai Xiang, Heng Huang, Xinwu Cheng, and Yunsong Guo.

The prototype reimplements the replay rule from Section 3 and Equation 1. It does not use official code, which the authors list as still being prepared. The project page is dream-rsi.com. The public repository at the time of writing is zhengkid/Dream-RSI, paper and citation only.

SimpleTES, SwarmResearch, MLEvolve, and MetaSkill-Evolve were opened as screening candidates. EvoX and Hyperagents were opened, then dropped for recency, and were not scored as if they were current.

References

Primary research

• Dream-RSI: Recursive Self-Improvement through Evolving Worlds. Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, Xue Wang, Yue Zhuan, Wang-Cheng Kang, Renkai Xiang, Heng Huang, Xinwu Cheng, and Yunsong Guo, 2026. First public version 14 September 2026.

Supporting and contrary evidence

• Structured Scaling of AI Discovery Across Diverse Scientific Domains (SimpleTES). Haotian Ye and colleagues, 2026. The 51,200-generation discovery loop Dream-RSI compares against; public code.
• SwarmResearch: Orchestrating Coding Agents for Open-Ended Discovery. Yuvraj Virk, Zack Edds, Chunqiu Steven Xia, and Lingming Zhang, 2026. History as shepherd context, not as a replay world.
• MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discovery. Shangheng Du and colleagues, 2026. Cross-branch memory as guidance.
• MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution. Zefeng Wang, Minxi Yan, Jinhe Bi, and colleagues, 2026. Recursive improvement of the improvement procedure, not of an exploration controller over discovery trees.
• EvoX: Meta-Evolution for Automated Discovery. Shu Liu and colleagues, 2026. Opened, then excluded: first public version 26 February 2026, outside the six-month window.

Prototype

• Source, setup, and evaluation
• Exact tested commit

Sources last checked: 2026-09-22