Gå til indhold
Global sound
Daily Paper Cast

Daily Paper Cast

What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?

· 24 min. · Episode 2481

Episodebeskrivelse

🤗 Upvotes: 23 | cs.LG, cs.CL

Authors:

Kejian Zhu, Zhuoran Jin, Shangqing Tu, Tianyi Men, Yupu Hao, Jiachun Li, Kang Liu, Jun Zhao

Title:

What Does Multi-Turn Reinforcement Learning Really Improve in LLM Agents?

Arxiv:

http://arxiv.org/abs/2611.06483v1

Abstract:

While multi-turn agentic reinforcement learning (RL) excels on long-horizon tasks, its performance gains remain highly sensitive to interaction constraints. Evaluating across sampling breadth $K$ and interaction depth $T$ via $Pass@(K,T)$, we show that trajectory-level RL advantages depend on sufficient turn budgets, diminishing significantly when budgets tighten. Besides, agentic RL does not uniformly elevate single-step decision quality; rather, its strength concentrates in error recovery and long-horizon resilience. To mitigate the deployment risks caused by this budget dependence, we provide a preliminary discussion on the benefits and limitations of process supervision signals and introduce BudgetMix RL as an alternative that varies interaction budgets during training. Experiments demonstrate that BudgetMix RL achieves robust performance across diverse execution settings. Project Page: https://rethinking-agentic-rl.github.io/

Website: share.transistor.fm

Live, seneste time: 10.073 sange på 4.198 stationer i 127 lande

Opdag verden gennem radioen

Flere måder at opdage på (9)

Hørt noget godt? Gem det, eller åbn det direkte i Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
ctuClubbingFortsæt med at lytte