Llama 3.2 3B
Pocket-sized Llama 3 variant for edge deployment. Surprising chat quality after instruction tuning makes it competitive with much larger models from a previous generation. At Q4 it fits in ~2 GB of VRAM and runs on consumer GPUs and recent Apple Silicon. A strong default for on-device chat, summarisation, and structured extraction tasks where the workload doesn't need frontier reasoning quality.
- Parameters
- 3B
- Context length
- 128K
- Modality
- text
- Released
- 2024-09-25
Memory & hardware
- VRAM (fp16)
- 6 GB
- VRAM (Q4)
- 1.8 GB
- Recommended
- RTX 3060 12GB / iPhone 15 Pro+
- Quantizations
- fp16, q8_0, q4_k_m, gguf
License: Llama 3 Community License
- SPDX
- —
- Commercial use
- Yes
- Modification
- Yes
- Redistribution
- Yes
Benchmarks
Hosted inference pricing
No provider we track publishes a per-token price for this model today. What each one used to offer is listed below.
No longer listed
Providers that used to serve this model. We don't republish their old rates — the dates below are the provider's own.
- groqGroq shut down llama-3.2-3b-preview on 14 April 2025.Source ↗
Run it yourself
Drop-in commands for the three most common open-source inference paths. The Ollama tag is a best-effort match against the registry; verify the size variant before pulling.
ollama run llama3.2:3b
vllm serve meta-llama/Llama-3.2-3B
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B")
model = AutoModelForCausalLM.from_pretrained(
"meta-llama/Llama-3.2-3B", device_map="auto", torch_dtype="auto"
)meta-llama/Llama-3.2-3B Related models
Same family or similar size — useful when shopping around.
The original 7B RLHF chat model. Historically important — the first widely-adopted commercially-usable open-weights chat model. Still cited as a baseline in most 2024–25 papers.
- Context
- 4K
- License
- llama-2
- VRAM Q4
- 4.2 GB
The workhorse 8B instruction-tuned model. Excellent quality-to-cost ratio and the broadest ecosystem support of any open-weights model — every major inference engine, fine-tuning library, and quantization toolchain has a 3.1 8B preset. Fits in 24 GB of VRAM at fp16, ~6 GB at Q4. Strong default for production chat where 70B is overkill, for fine-tuning on a specialist task, and for any workload where you want a known-good baseline.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 4.8 GB
Llama 3's first vision-language model. Image understanding via a separately-trained ViT adapter bolted onto Llama 3 weights. Useful for OCR-adjacent workloads, document understanding, and image captioning at a permissive licence. The 11B size makes it cheap to host. Combined with the 128K text context, it handles long PDF-with-images workflows comfortably on a single 4090.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 6.6 GB
Microsoft's flagship small-model demonstration: GPT-3.5-class on academic benchmarks at <4B parameters. The 4K context-window variant is the lightest; a 128K variant ships separately. MIT licensed, well-suited to on-device assistants and structured-extraction workloads where compactness matters more than absolute quality.
- Context
- 4K
- License
- mit
- VRAM Q4
- 2.3 GB
The smallest Llama 3 release, designed for on-device inference on phones and laptops. The 1B model runs comfortably in <2 GB of RAM at Q4 quantization and is fast enough for real-time chat on a modern smartphone. Useful for edge inference, on-device assistants where round-tripping to a server is undesirable, and as a draft model for speculative decoding in front of a larger Llama 3 variant.
- Context
- 128K
- License
- llama-3
- VRAM Q4
- 0.6 GB
TII's latest dense 7B from December 2024. Strong scores on commonsense reasoning benchmarks. TII's Falcon licence permits royalty-free commercial use with attribution.
- Context
- 33K
- License
- falcon-2
- VRAM Q4
- 4.2 GB