Self-Evolving Harness on Multiple Tasks with the Agent as Its Own Optimizer

arXiv:2609.38372v1 Announce Type: new Abstract: A harness is the code around a language-model agent that organizes prompts, calls tools, manages context, and controls execution. As models grow stronger, recent work has begun to let agents improve their own harnesses, a line of work known as self-evolving harnesses. In most existing methods, a separate proposer running on a human-designed harness modifies the solver's harness, and a separate harness is evolved for each benchmark. Real-world tasks come from many domains, so both the evolution and the evaluation of a harness should cover a diverse range of tasks. We propose a framework close to recursive self-improvement: the same frozen model, on the same version of the harness, first solves tasks as the solver and then, as the proposer, reads the complete run records and directly edits the harness that runs it. Each evolution batch draws tasks from five benchmarks in different domains. To measure generalization, training and held-out tasks are strictly separated, and we additionally evaluate on five out-of-distribution benchmarks never used during evolution. We frame the evolution process as deep-learning training with two stages, multi-task pretraining and continual training. Starting from a 49-line seed harness, the harness obtained at the end of the first stage improves the average score by 4.48 points on the in-distribution benchmarks and by 12.64 points on the out-of-distribution benchmarks, surpassing Codex on the former and matching it on the latter. In the second stage, continued evolution on Claw-Eval, one of the ou

Improved performance on tasks, but uncertain long-term implications

The paper describes a significant improvement in AI agents' ability to self-optimize across multiple tasks. The results show substantial gains in performance, both in-distribution and out-of-distribution. This suggests a positive trend in AI's capability to adapt and improve autonomously.

SOURCE     arXiv cs.AI (tier 1)
   ↓
DOCUMENT   dd826902-70f2-41eb-8e9c-d2281eb2df0d
           https://arxiv.org/abs/2609.38372
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  3 runs, methodology 1.0.1
   ↓
SCORE      +24  (Favourable)
   ↓
CONFIDENCE 68%