Skip to content
Global sound
AI Post Transformers

AI Post Transformers

Self-Improvement Memory Enables Continual Recursive Self-Improvement

Show notes

This episode examines a November 2026 paper on continual recursive self-improvement, where an LLM-driven optimizer keeps improving an agent's harness (the code around a frozen model) after the benchmark or the underlying model changes. It starts from a striking result: a harness evolved with one model scored about five and a half points below the plain seed harness when moved to a different model. The cause is that the optimizer inherits the evolved code without the reasons behind each edit, much like a legacy codebase with no commit history. The proposed fix, Self-Improvement Memory, logs each edit's root cause, rationale, predicted outcome, observed outcome and provenance. It is compared against cold start, harness transfer and problem-only memory conditions. The discussion also relates the idea to Reflexion and to continual learning, and it notes that most of the evidence comes from single optimization runs, which is a reason to treat the headline gains cautiously.

Sources:
1. Self-Improvement Memory Enables Continual Recursive Self-Improvement — Xin Yu, Fei Xia, Jiaqi Pan, Youbao Tang, Chengzhi Liu, Adams Yu, Sicong Jiang, Yanfei Chen, Lingzhou Xue, Ting Yu, 2026
http://arxiv.org/abs/2611.18856
2. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents — Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, Jeff Clune, 2025
https://scholar.google.com/scholar?q=Darwin+G%C3%B6del+Machine%3A+Open-Ended+Evolution+of+Self-Improving+Agents
3. Automated Design of Agentic Systems (ADAS) — Shengran Hu, Cong Lu, Jeff Clune, 2024
https://scholar.google.com/scholar?q=Automated+Design+of+Agentic+Systems+%28ADAS%29
4. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation — Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Tauman Kalai, 2023
https://scholar.google.com/scholar?q=Self-Taught+Optimizer+%28STOP%29%3A+Recursively+Self-Improving+Code+Generation
5. A Comprehensive Survey of Continual Learning: Theory, Method and Application — Liyuan Wang, Xingxing Zhang, Hang Su, Jun Zhu, 2023
https://scholar.google.com/scholar?q=A+Comprehensive+Survey+of+Continual+Learning%3A+Theory%2C+Method+and+Application
6. Reflexion: Language Agents with Verbal Reinforcement Learning — Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao, 2023
https://scholar.google.com/scholar?q=Reflexion%3A+Language+Agents+with+Verbal+Reinforcement+Learning
7. Voyager: An Open-Ended Embodied Agent with Large Language Models — Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Linxi Fan, Anima Anandkumar and colleagues, 2023
https://scholar.google.com/scholar?q=Voyager%3A+An+Open-Ended+Embodied+Agent+with+Large+Language+Models
8. Agent Workflow Memory — Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, Graham Neubig, 2024
https://scholar.google.com/scholar?q=Agent+Workflow+Memory
9. A Survey on the Memory Mechanism of Large Language Model based Agents — Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen and colleagues…

Live, last hour: 12,115 songs on 5,250 stations in 121 countries

Discover the world through radio

More ways to discover (7)

Heard something good? Save it, or open it straight away in Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
Diva 91.6Continue listening