Post
114
Does a successful harness edit make the next improvement easier?
Harness RSI studies reusable changes to agent tools, context, memory and execution control. The continual-RSI question is whether retained experience improves later proposals, evaluation efficiency and adaptation—not only the final task score.
Three distinctions matter:
• Retained vs. reset memory: does history help at the same search budget?
• Artifact vs. optimizer transfer: does an unchanged harness help another model, or must optimization restart?
• Held-out transfer vs. target adaptation: which target feedback influenced selection?
Track the evidence chain:
failure → modification → comparison → retained lesson → limits of reuse.
Keep regressions and search costs alongside gains. A better warm start alone does not establish faster subsequent improvement.
Updated November 4: 13 verified papers added; KARMA v2 rechecked. Dataset v0.4.1 contains 145 records across 113 works, 60 AI-assisted primary-source checks and 45 scoped signals. 85 inherited records and zero independent human adjudications remain explicit. Records are not independent effects. The original 95-record domain atlas stays frozen; these are literature observations, not our own experimental results.
The v0.4.1 correction patch fixes benchmark identities and evidence scope; counts are unchanged.
Dataset: lucas1026/harness-rsi-evidence
Paper: https://lucasxinyu.github.io/harness-rsi/paper/Harness_RSI_Preprint_2026-11-04_corrected.pdf
Corrected November 4 author reading version; not peer-reviewed; separate from posted Preprints.org v2.
Atlas: lucas1026/harness-rsi-atlas
Studies and counterexamples: https://huggingface.co/blog/lucas1026/beyond-model-weights-what-can-we-actually-improve
🎬 Attached: 90-second silent overview.
Which retained experience changed your next improvement decision?
#AIAgents #HarnessOptimization #RecursiveSelfImprovement
Harness RSI studies reusable changes to agent tools, context, memory and execution control. The continual-RSI question is whether retained experience improves later proposals, evaluation efficiency and adaptation—not only the final task score.
Three distinctions matter:
• Retained vs. reset memory: does history help at the same search budget?
• Artifact vs. optimizer transfer: does an unchanged harness help another model, or must optimization restart?
• Held-out transfer vs. target adaptation: which target feedback influenced selection?
Track the evidence chain:
failure → modification → comparison → retained lesson → limits of reuse.
Keep regressions and search costs alongside gains. A better warm start alone does not establish faster subsequent improvement.
Updated November 4: 13 verified papers added; KARMA v2 rechecked. Dataset v0.4.1 contains 145 records across 113 works, 60 AI-assisted primary-source checks and 45 scoped signals. 85 inherited records and zero independent human adjudications remain explicit. Records are not independent effects. The original 95-record domain atlas stays frozen; these are literature observations, not our own experimental results.
The v0.4.1 correction patch fixes benchmark identities and evidence scope; counts are unchanged.
Dataset: lucas1026/harness-rsi-evidence
Paper: https://lucasxinyu.github.io/harness-rsi/paper/Harness_RSI_Preprint_2026-11-04_corrected.pdf
Corrected November 4 author reading version; not peer-reviewed; separate from posted Preprints.org v2.
Atlas: lucas1026/harness-rsi-atlas
Studies and counterexamples: https://huggingface.co/blog/lucas1026/beyond-model-weights-what-can-we-actually-improve
🎬 Attached: 90-second silent overview.
Which retained experience changed your next improvement decision?
#AIAgents #HarnessOptimization #RecursiveSelfImprovement