Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation

A new benchmarking framework, Xpertbench, has been proposed to evaluate the proficiency of large language models in complex, open-ended tasks. Current benchmarks have plateaued, and experts have struggled to design rubrics for evaluating these models. Xpertbench aims to bridge this gap by providing a rubric-based evaluation system that assesses models' ability to think critically and reason like experts. The framework includes a set of expert-designed rubrics and a large dataset of expert-generated tasks. Researchers can use Xpertbench to evaluate their models and compare their performance with others. This development has the potential to accelerate the development of more advanced language models and improve their real-world applications.