Research Notes › RSI · CLI
Notes on papers related to RSI — one card per paper.
Start here → Harness Engineering for Self-Improvement (Lilian Weng, Jul 2026). A survey framing RSI as a self-referential feedback loop and arguing the near-term path runs through harness engineering. It indexes many of the individual papers covered below (STOP, DGM, AlphaEvolve, ACE/MCE, Self-Harness, AHE, …) — the roadmap for this daily log.
Skill accumulation is not monotonic — past a critical pool size new skills degrade performance — and the damage is structurally irreversible, because a defective skill becomes reference material for everything distilled after it; admission therefore has to be gated before an artifact enters the next generation rather than rolled back afterwards.
Scoring every candidate on a fixed task set wastes rollouts on unrelated behaviour and lets aggregate scores hide specific regressions; HarnessLens verifies each candidate only on behaviour-relevant tasks behind an attributable-evidence gate, gaining 7.6–13.6% held-out performance on far less evaluation budget.
When each real evaluation is expensive, a fixed cheap surrogate breaks under search-induced distribution shift — so Janus co-evolves executable proxy evaluators alongside the programs being searched, and uses their predictions only to prioritise candidates rather than to decide.
Under a limited evaluation budget the decisive choice is made before generation — which archived candidate to reuse as parent and which operator to apply — so two surrogates score that pre-generation action directly instead of leaving its expected outcome to predefined rules.
Stratifies the evolution pool by baseline failure behaviour and splits tasks three ways — proposer-visible search, proposer-hidden selection, and held-out generalisation — reaching 20–35 points over the default harness after only 4–12 accepted changes, with gains that transfer across model families without re-evolution.
Audits the skill bank itself rather than the score it produces, and finds two failures an aggregate number cannot show: coalition pollution, where bank-level gains conceal skills that contribute negatively in combination, and cross-domain utility reversal, where a skill that helped at the source hurts after transfer.
Attacks overfitting to the evolution tasks head-on: constrain edits to task-agnostic changes that generalise beyond them, add the agent's own first-person report of how it used the harness to the trajectory-derived evidence, and optimise components separately so their edits stop interfering with each other.
Takes the opposite bet on generality — every task family keeps its own harness, hot-swapped through a fixed task-injection seam — and puts the shared part one level up, in an evolver that rewrites the harness and a meta-evolver that rewrites the evolver's own strategy code.
Isolates harness-evolving ability from base-model strength across Search, Office and General agent domains by keeping only tasks that auxiliary-task evolution shows are genuinely sensitive to framework changes, then splitting them by that sensitivity so cross-suite generalisation is measured rather than assumed.
Makes evaluation cost part of the measurement: the optimizer gets a seed harness, graded feedback and a fixed target-evaluation budget, and is scored by the normalised gain of its nominated candidate over the seed on a held-out partition it never sees.
Closes the loop in both directions — build harnesses for a fixed model, update the model from verified sibling trajectories, rebuild the harnesses as capability changes — on a substrate of typed ports, atoms and recipes that keeps each intervention's identity and provenance auditable across rounds.
States the feedback problem in harness evolution precisely — in long-horizon stochastic environments rewards are sparse, outcomes high-variance, and failures hard to attribute to concrete harness mechanisms — then answers it by bootstrapping from competent human trajectories rather than by reading the search itself more densely.
Three moves against the thousands-of-samples cost of code evolution — parent sampling that balances exploration against exploitation, code-novelty rejection sampling that discards a candidate before spending an evaluation on it, and a meta-scratchpad that carries reasoning between rounds.
Give a coding agent a base model, one H100 and ten hours, and it does the whole post-training loop unaided — badly: 23.2% vs 51.1% for the official instruct model, and some of that gap is closed by cheating.
A benchmark that grades a model not on solving the task but on writing the agent that solves it — with the anti-reward-hacking plumbing that makes such a score believable.
Turns harness engineering into an executable code search, driven by a coding agent that reasons over the full raw history of prior attempts.