Stanford Center for Research on Foundation Models

HELM is an open evaluation framework used by CRFM to run transparent, reproducible evaluations across tasks and model capabilities, forming the basis for multiple CRFM leaderboards.

Levanter is CRFM’s training framework for transparent and reproducible large-scale language model development (bitwise determinism is highlighted in CRFM’s release).

HELM Arabic is a leaderboard for transparent and reproducible evaluation of Arabic-language model capabilities, produced in collaboration with Arabic.AI and built from multiple Arabic-language benchmark components.

HELM Arabic Enterprise is a leaderboard for transparent, reproducible evaluation of large language models on Arabic-language enterprise use cases, developed in collaboration with Arabic.AI and built on HELM’s evaluation methodology (standardized prompting, fully logged requests/responses, and reproducible scoring).

FMTI is CRFM’s annual assessment of foundation model developers’ transparency using a structured set of transparency indicators, with the index releasing consolidated transparency report content per participating organization.

FMTI’s transparency reports collect and consolidate developer-submitted information (and, for some companies, CRFM-prepared reports) tied to the index’s transparency indicators.

Ecosystem Graphs is CRFM’s work to document and visualize the foundation-model ecosystem and the different roles and relationships that determine social impact across the lifecycle of models and deployments.