MLCommons

MLPerf is the benchmark family MLCommons creates and maintains to measure AI system performance and improvement over time using open, peer-reviewed benchmarking methods across model and system stacks.

MLPerf Endpoints is a benchmarking approach and suite focused on measuring deployed AI endpoint performance, including throughput and interactivity/time-to-first-token style measures under load, with results published through an endpoints visualization and reporting workflow.

AILuminate is a family of safety and security benchmarks that assesses generative AI systems across hazard categories and provides public results and grading criteria intended to guide development and purchasing and to support standards/policy discussions.

Croissant is a community-built metadata format for ML datasets that standardizes how dataset metadata and loading semantics are described to improve discoverability, governance, and usability across tools and platforms.

MedPerf is an open benchmarking platform designed for medical AI evaluation using federated evaluation principles so that medical data can remain with data providers while models are evaluated and results are approved for sharing.

MLCommons’ dataset working groups develop and release open datasets intended to improve evaluation coverage; for example, the People’s Speech dataset is described as a large open speech recognition dataset project within MLCommons dataset efforts.