OpenAI introduces Community Evals for transparent model benchmarking

AI Impact Summary

OpenAI is shifting away from relying on opaque, black-box leaderboards and towards a community-driven evaluation system. This change aims to address the misalignment between benchmark scores and real-world model performance, as well as inconsistencies in reported scores across various sources. By decentralizing evaluation reporting and allowing the community to openly submit and verify results, OpenAI hopes to foster greater transparency and collaboration in the field of large language model evaluation.