Meta-Harness: End-to-End Optimization of Model Harnesses Aug 9, 2026
Turns harness engineering into an executable code search, driven by a coding agent that reasons over the full raw history of prior attempts.
Takeaway
- Full trajectories, not summaries
- A coding agent does the evolving
- It’s a search problem; the hard part is credit assignment
- Code-as-representation is a free regularizer
Remaining work
- Credit assignment stays qualitative (no per-component score), and no guarantee each round improves.
- Large, open-ended search space explored under a tight budget — running each candidate on the frozen model is the compute bottleneck.
- Generalizes across datasets and models, but not across task types.