Show notes
Researchers have raised a troubling issue regarding the evaluation of AI models. So-called arena-style leaderboards, where humans vote on which of two models performed better, are surprisingly defenseless against intentional manipulation.
In recent years, traditional static benchmarks have lost credibility due to training data contamination problems. Arena-style evaluation gained attention as an alternative, with the expectation that manipulation would be harder if actual users were voting.
However, this research reveals that manipulation is possible through three approaches: voting…
Website: tech.airadio.news

