Note de episod
This paper introduces a new metric, the "50%-task-completion time horizon," to quantify AI capabilities by relating AI performance on tasks to the typical time humans take to complete them. The study timed domain-expert humans on a diverse set of research and software engineering tasks (RE-Bench, HCAST, and a new suite called SWAA) and evaluated the performance of 13 frontier AI models (2019-2025) on these tasks. The key finding is that the 50% time horizon of frontier AI models has been doubling approximately every seven months since 2019, potentially accelerating in 2024. Extrapolation of…

