Mixtral 8×7B Instruct
The mixture-of-experts release that introduced 8 experts of 7B each, 2 active per token. ~13B active parameters with 47B total, which makes per-token inference roughly as fast as a 13B dense model while approaching 70B dense quality. Apache 2.0 weights mean it's still a popular self-hosting choice. Memory footprint is the main constraint — the full 47B parameters must be loaded even though only a quarter are active per token.
- Parameters
- 46.7B
- Context length
- 33K
- Modality
- text
- Released
- 2023-12-11
- Tokenizer
- LlamaTokenizer
Memory & hardware
- VRAM (fp16)
- 93.4 GB
- VRAM (Q4)
- 28 GB
- Recommended
- A100 80GB or 2× RTX 4090
- Quantizations
- fp16, q8_0, q5_k_m, q4_k_m
Benchmarks
Hosted inference pricing
No provider we track publishes a per-token price for this model today. What each one used to offer is listed below.
Run it yourself
Drop-in commands for the three most common open-source inference paths. The Ollama tag is a best-effort match against the registry; verify the size variant before pulling.
ollama run mistral
vllm serve mistralai/Mixtral-8x7B-Instruct-v0.1
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mixtral-8x7B-Instruct-v0.1")
model = AutoModelForCausalLM.from_pretrained(
"mistralai/Mixtral-8x7B-Instruct-v0.1", device_map="auto", torch_dtype="auto"
)mistralai/Mixtral-8x7B-Instruct-v0.1 Related models
Same family or similar size — useful when shopping around.
Cohere's 35B model tuned for RAG and tool use. The open weights are released under CC-BY-NC (commercial use requires the Cohere API). Strong multilingual coverage and a fine-grained RAG-mode output format that makes downstream citation easier.
- Context
- 128K
- License
- mrl
- VRAM Q4
- 21 GB
Bilingual EN/中 34B chat model. Apache 2.0 licensed with strong Chinese-language performance and competitive English chat quality. Good default for bilingual production workloads.
- Context
- 33K
- License
- apache-2-0
- VRAM Q4
- 20.4 GB
Vision-language variant of Yi 34B. Image-text reasoning via an MLP adapter on a CLIP encoder. Useful for bilingual EN/中 multimodal workloads where the major Western vision-language models underperform on Chinese text in images.
- Context
- 4K
- License
- apache-2-0
- VRAM Q4
- 20.4 GB
32B sweet-spot Qwen 3, Apache 2.0. Reasoning-mode toggle inherited from smaller siblings; strong on math, code and agentic tool use. Fits on a single H100 in fp16 and on a 4090 at Q4.
- Context
- 33K
- License
- apache-2-0
- VRAM Q4
- 19.2 GB
32B sweet-spot model: strong reasoning, fits on one H100 in fp16, on a 4090 at Q4. The 32B size in particular hits a quality/cost knee — quality scales with parameters faster than cost up to ~32B, and slower afterwards. Favoured for production chat where 7B isn't sharp enough and where 70B+ would over-spec the hardware budget. Apache 2.0 licence.
- Context
- 128K
- License
- apache-2-0
- VRAM Q4
- 19.2 GB
Qwen's reasoning-focused 'thinking' model. Generates long chains-of-thought before answering, similar to OpenAI's o1 and DeepSeek R1 lineage. Optimised for math and competition-style problem solving. The Preview tag means Qwen is iterating quickly; later versions may obsolete this one. Useful today for math-heavy workloads where a slow, careful answer is preferred to a fast wrong one.
- Context
- 33K
- License
- apache-2-0
- VRAM Q4
- 19.2 GB