Recursive harness self-improvement

HarnessPro.

Better performance.
Smarter budgets.

Recursive Harness Self-Improvement with Pareto Optimization for Performance-Cost Trade-offs

Evolve the instructions, tools, and runtime policies around an AI agent. Find the sweet spot between what it can solve and what it costs.

Read the paper

A better performance–cost frontier Reported results

Benchmark Model + agent

Heavy mode vs. RHO +4.5 pp pass rate 33.9% lower cost

See the complete results

The outcome

A stronger agent.
A smaller bill.

+2.0pp

Average pass-rate gain

26.1%

Average cost reduction

Heavy mode vs. RHO
Across three benchmarks
GPT-5.5 × Codex

01 / The idea

The harness is part of the intelligence.

An agent’s capability depends on how it is guided, what tools it has, and when it stops. HarnessPro improves that surrounding software through execution feedback.

Learn from the trajectory

Diagnose failed attempts, redundant checks, and expensive exploration across multiple rollouts.

Optimize both objectives

Evaluate task performance and execution cost together. Keep candidates that advance the Pareto frontier.

Choose your operating point

Retain cost-focused, balanced, and performance-focused harnesses for different deployment budgets.

02 / The method

One loop.
Three ways forward.

Specialized agents turn execution evidence into new harnesses. Each round retains strong candidates for the next round of evolution.

Diagnose

The trajectory analyzer identifies failures and avoidable computation, grounding its diagnoses in execution evidence.

Plan & edit

The planner proposes updates with three performance–cost priorities. The editor implements executable candidate harnesses.

Evaluate & select

Fixed LLM-generated tests assess candidates. Pareto filtering and weighted selection retain the Lite, Medium, and Heavy modes.

During evolution

Learn with generated tests.

No ground-truth solutions, official tests, or official grader feedback are used.

After evolution

Freeze, then evaluate.

The final harnesses are frozen before evaluation with the official benchmark tests.

03 / Main results

The complete picture.
Across every benchmark.

Compare performance, cost, and task completion time for every method reported in the paper. Both model–agent pairs are shown below.

Terminal-Bench 2

Tasks in a terminal environment.

59 test tasks · 10 evolution tasks

SWE-Bench Pro

Resolve issues in real code repositories.

100 test tasks · 10 evolution tasks

ProgramBench

Reconstruct programs from their behavior.

100 test tasks · 10 evolution tasks

Paper Table 2. Performance and efficiency across three benchmarks for GPT-5.5 with Codex and GLM-5.2 with Claude Code.
Method	Terminal-Bench 2			SWE-Bench Pro			ProgramBench
Method	P (%) ↑	C ($) ↓	T (s) ↓	P (%) ↑	C ($) ↓	T (s) ↓	P (%) ↑	C ($) ↓	T (s) ↓

P = pass rate. C = mean cost per task. T = mean task completion time. Bold marks the best mean in each column within a model–agent pair.

HarnessPro reports mean ± sample standard deviation over three test runs on Terminal-Bench 2 and SWE-Bench Pro, and two on ProgramBench. ProgramBench pass rate is the macro-average fraction of test cases passed per task. Latency is reported for comparison and is not an optimization objective.

Heavy mode / Performance

Higher pass rates, lower cost.

Heavy mode has the highest pass rate in all six benchmark–model comparisons, with lower mean cost than RHO in every case.

Medium mode / Balance

53.0% less per task.

On SWE-Bench Pro with GPT-5.5 and Codex, Medium mode reaches 79.0% pass rate at $1.26 per task, compared with RHO’s 80.0% at $2.68.

Lite mode / Cost

A smaller budget goes further.

On Terminal-Bench 2 with GPT-5.5 and Codex, Lite mode costs $0.43 per task versus Vanilla’s $0.89, while pass rate rises from 69.5% to 72.9%.

04 / Recursive improvement

Better with
each round of evolution.

A harness improves over multiple rounds of diagnosis, editing, and selection. These curves from the paper track performance and cost across epochs with GPT-5.5 and Codex.

Terminal-Bench 2 pass rate versus cost for the harnesses retained across three evolution epochs, with baseline methods shown as diamonds. — Heavy mode improves from 74.6% at $0.91 in epoch 1 to 80.8% at $0.72 in epoch 3.

SWE-Bench Pro pass rate versus cost across three evolution epochs, showing the final epoch improving the performance and cost trade-off. — From epoch 2 to 3, Medium mode rises from 77.0% to 79.0% while cost falls from $1.66 to $1.26.

ProgramBench macro-average pass rate versus cost across four evolution epochs, with lower final-epoch costs across all three deployment modes. — A fourth epoch reduces cost across all three modes while maintaining similar pass rates.

Retained harnessesHarnesses from the same epochBaseline methods

05 / Inside an evolved harness

Explore.
Converge.
Know when to stop.

Evolved harnesses route tasks to an execution profile. Runtime hooks guide tool use as the assigned budget is consumed.

Try a mode, choose a task profile, then play the walkthrough or drag the tool-call slider.

Execution profile

Reasoning effortmedium

Tool-call attempts

00 / 44

StartEconomy · 14Converge · 26Limit · 44

ExploreHook behavior · summary

Work on the task with the selected reasoning effort. Gather the evidence needed for an effective solution.

economy_after_tools14

strict_after_tools26

finalize_after_tools44

Simulated progression using the frozen harness settings. The hook blocks attempts beyond the tool-call limit.

Better agents don’t always
need bigger budgets.

Explore the paper

Keep in touch with the work

Cite HarnessPro.

Manuscript citation.
arXiv and GitHub links are coming soon.

@misc{pan2026harnesspro,
  title  = {HarnessPro: Recursive Harness Self-Improvement
            with Pareto Optimization for
            Performance-Cost Trade-offs},
  author = {Zhuoshi Pan and ZhiYuan Feng and Yukuo Cen
            and Honglin Lin and Junru Lu and Zhongxia Chen
            and Shuo Yang and H. Vicky Zhao and Jie Tang},
  year   = {2026},
  note   = {Manuscript under review}
}