Naar de inhoud
Global sound
Eye on AI Weekly Research Watch

Eye on AI Weekly Research Watch

Do Large Language Models Know Colombian Law? A Reliability Benchmark for the Colombian Legal System

· 2 min · Seizoen 1

Beschrijving

LLMs are increasingly used in legal work, yet their reliability outside the United States is poorly documented. This expert-validated benchmark has 1,042 items across ten areas of law and three formats, and was used to test 15 models. Closed-question accuracy reaches 0.905, but free-text correctness never exceeds 0.45, and only about half of cited norms are correct. Models often sound responsive while being wrong, which is risky for non-experts.

Authors: Rubén Manrique, Michelle Castellanos, Jorge Morales, Juan David Gutiérrez, Antonio Barreto Rozo, Joaquín Vélez Navarro

Paper…

Website: share.transistor.fm

Live, afgelopen uur: 8.027 nummers op 2.965 zenders in 123 landen

Ontdek de wereld via de radio

Meer manieren om te ontdekken (7)

Iets moois gehoord? Bewaar het, of open het direct in Spotify.

★ Artiest van de week De snelste stijger op de radio deze week
Jouw nummer hier? Vanaf € 9 per week →
Nisville RadioVerder luisteren