Notas do episódio
TL;DR: Agents are not rewarded for keeping the agents they delegate to aligned. Proposal: RL environments where the helpers are deliberately misaligned, so the agent can't earn its reward without dealing with that.
The $300 bounty from my last post also covers this one (details at the end).
After the OpenAI/Hugging Face swarm incident, I wrote a think piece, Memes, not agents, are what will kill us.
In it I argued that alignment needs to work on three different levels. Compared to the original writeup, I would now reframe them a little:
• The model weights. What is under the control of the…
Site: lesswrong.com
