Phi-3 Mini 4K Instruct

Microsoft's flagship small-model demonstration: GPT-3.5-class on academic benchmarks at <4B parameters. The 4K context-window variant is the lightest; a 128K variant ships separately. MIT licensed, well-suited to on-device assistants and structured-extraction workloads where compactness matters more than absolute quality.

Parameters: 3.8B
Context length: 4K
Modality: text
Released: 2024-04-23
Tokenizer: LlamaTokenizer

Memory & hardware

VRAM (fp16): 7.6 GB
VRAM (Q4): 2.3 GB
Recommended: RTX 3060 12GB or M1 Pro
Quantizations: fp16, q8_0, q4_k_m, gguf

License: MIT

SPDX: MIT
Commercial use: Yes
Modification: Yes
Redistribution: Yes

License detail →

Benchmarks

MMLUMMLU 5-shot

70.9

card ↗

HumanEvalHumanEval 0-shot

57.3

card ↗

2 of 2 confirmed against the model card on 2026-09-20. How we verify.

Hosted inference pricing

No provider we track publishes a per-token price for this model today. What each one used to offer is listed below.

No longer listed

Providers that used to serve this model. We don't republish their old rates — the dates below are the provider's own.

deepinfraAbsent from DeepInfra’s pricing page and model API when we checked on 20 September 2026.Source ↗

Run it yourself

Drop-in commands for the three most common open-source inference paths. The Ollama tag is a best-effort match against the registry; verify the size variant before pulling.

Run Phi-3 Mini 4K Instruct locally

Ollama (easiest)

ollama run phi3

Single-line install + run; uses the official Ollama registry tag for this family.

vLLM (production)

vllm serve microsoft/Phi-3-mini-4k-instruct

High-throughput hosted inference; one command to expose an OpenAI-compatible HTTP server.

Transformers (Python)

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct", device_map="auto", torch_dtype="auto"
)

Direct PyTorch usage. Pin a torch / cuda version that matches your GPU.

Hugging Face ID: microsoft/Phi-3-mini-4k-instruct

Related models

Same family or similar size — useful when shopping around.

Llama 3.2 3B

Pocket-sized Llama 3 variant for edge deployment. Surprising chat quality after instruction tuning makes it competitive with much larger models from a previous generation. At Q4 it fits in ~2 GB of VRAM and runs on consumer GPUs and recent Apple Silicon. A strong default for on-device chat, summarisation, and structured extraction tasks where the workload doesn't need frontier reasoning quality.

Context: 128K
License: llama-3
VRAM Q4: 1.8 GB

Llama 2 7B Chat

The original 7B RLHF chat model. Historically important — the first widely-adopted commercially-usable open-weights chat model. Still cited as a baseline in most 2024–25 papers.

Context: 4K
License: llama-2
VRAM Q4: 4.2 GB

Falcon 3 7B Instruct

TII's latest dense 7B from December 2024. Strong scores on commonsense reasoning benchmarks. TII's Falcon licence permits royalty-free commercial use with attribution.

Context: 33K
License: falcon-2
VRAM Q4: 4.2 GB

Falcon Mamba 7B

The first major open-weights state-space model. Linear-time decoding, no KV cache — memory usage stays flat as context grows, which makes it interesting for very long-context workloads. Falcon licence.

Context: 16K
License: falcon-2
VRAM Q4: 4.2 GB

Mistral 7B v0.3

The original Mistral 7B refresh with 32K context and extended vocabulary. Permissive Apache 2.0 weights and the first widely-deployed sliding-window-attention model. Still useful in 2026 for very-low-cost inference and as a baseline for fine-tuning experiments.

Context: 33K
License: apache-2-0
VRAM Q4: 4.2 GB

OLMo 2 7B

Fully-open 7B model: weights, training data and code all released under permissive licences. Useful as a reference for reproducibility research and for teams that need full transparency on training data provenance.

Context: 4K
License: apache-2-0
VRAM Q4: 4.2 GB