OSAIM
ModelsFamiliesLeaderboardsCompareBest for…TimelineBlog
Open Source AI Models

Databricks

DBRX

Databricks' 132B mixture-of-experts (12 of 16 experts active per token) built on the MegaBlocks stack. Released under the Databricks Open Model Licence.

Visit homepage ↗

History & context

Databricks released DBRX in March 2024 as the flagship model built on their Mosaic training infrastructure. 132B total parameters in a 16-expert MoE with 4 active per token (36B active). Trained on 12T tokens.

DBRX was near-frontier on release and remains a well-documented reference MoE — the Databricks blog and paper published extensive training-loss curves and architectural details that are still cited. It has been surpassed by Llama 3.3 70B, DeepSeek V3, and Qwen 2.5 72B on most benchmarks, but retains value for research and enterprise deployment inside the Databricks ecosystem.

Licence is the Databricks Open Model Licence — permits commercial use with an acceptable-use policy.

Flagship model

DBRX Instruct
132B

Databricks' 132B mixture-of-experts — 16 experts, 4 active per token (36B active params). Trained on 12T tokens on Mosaic infrastructure and released under the Databricks Open Model Licence. DBRX was best-in-class on release; now beaten by Llama 3.3 70B and Qwen 2.5 72B on most benchmarks, but retains value as a well-documented MoE reference.

Context
33K
License
dbrx-open
VRAM Q4
79.2 GB

1 model in this family

DBRX Instruct
132B

Databricks' 132B mixture-of-experts — 16 experts, 4 active per token (36B active params). Trained on 12T tokens on Mosaic infrastructure and released under the Databricks Open Model Licence. DBRX was best-in-class on release; now beaten by Llama 3.3 70B and Qwen 2.5 72B on most benchmarks, but retains value as a well-documented MoE reference.

Context
33K
License
dbrx-open
VRAM Q4
79.2 GB
Comparing DBRX against another family? Try the side-by-side comparator or browse all leaderboards.