AI Extinction Is the Wrong Test

Last week, Evan Hubinger, who leads alignment science at Anthropic, said he believes artificial intelligence has a greater than 10 percent chance of killing every human within the next decade. His position makes the warning impossible to ignore: he works on the safety of Anthropic’s AI systems. His chief concern is that a future, more capable AI might escape human control.
Concern about that possibility has also reached Congress. Senator Bernie Sanders and Representative Greg Casar have announced a proposal to ban artificial superintelligence and pause advanced AI development while safety…