OpenAI introduces Arabic Instruction Following and updates AraGen benchmark

AI Impact Summary

OpenAI is introducing Arabic Instruction Following and updating AraGen, signaling a significant expansion of their Arabic language AI capabilities. The release of the AraGen-12-24 benchmark, incorporating Claude-3.5-Sonnet, represents a substantial investment in evaluating and refining Arabic LLMs. This update includes a revised dataset and prompt system, designed to better align with current model performance and address potential biases, as evidenced by the observed ranking shifts.