BigCodeArena: Executing AI-Generated Code for Evaluation

AI Impact Summary

BigCodeArena introduces a novel end-to-end evaluation method for AI-generated code by executing generated solutions and gathering human feedback. This approach addresses the limitations of traditional benchmarks that rely solely on static code comparison, offering a more realistic assessment of code quality and functionality. The platform's real-time execution, multi-language support, and interactive testing features provide valuable insights into model performance across diverse coding scenarios and programming languages, ultimately informing the selection of the best code generation models.