Notes d’épisode
The nonprofit AI-progress assessment organization, METR (Model Evaluation and Threat Research) released a graph recently that has become “an industrywide obsession,” according to the New York Times. But are there better ways to track AI progress? Here, the Derby Mill panel weighs in.
METR uses a task-completion time horizon to assess the progress of AI frontier systems. The “time horizon” is the duration that it would take a human expert to complete a set of diverse software development tasks.
METR’s 50% time horizon, then, is the “duration at which an agent is predicted to succeed” at a…
Site web: insights.intrepidgp.com

