Salt la conținut
Global sound
AI Insiders

AI Insiders

Measuring AI Ability to Complete Long Tasks

· 16 min · Sezonul 1 Episodul 12

Note de episod

This paper introduces a new metric, the "50%-task-completion time horizon," to quantify AI capabilities by relating AI performance on tasks to the typical time humans take to complete them. The study timed domain-expert humans on a diverse set of research and software engineering tasks (RE-Bench, HCAST, and a new suite called SWAA) and evaluated the performance of 13 frontier AI models (2019-2025) on these tasks. The key finding is that the 50% time horizon of frontier AI models has been doubling approximately every seven months since 2019, potentially accelerating in 2024. Extrapolation of…

Live, ultima oră: 6.486 melodii pe 2.751 posturi din 120 țări

Descoperă lumea prin radio

Mai multe moduri de a descoperi (7)

Ai auzit ceva frumos? Salvează-l sau deschide-l direct în Spotify.

★ Artist of the week The fastest riser on the radio this week
Your song here? From € 9 per week →
181.FM - Christmas SwingContinuă să asculți