Loading model data...

AI Models

204 models tracked

Deep dive into hardware requirements, benchmark data, and recommended builds for open models. Each card links to a full guide with quantization targets and compatible GPUs.

deepseek-ai/DeepSeek-Coder-V2-Instruct-0724

236B params132GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-V2.5

236B params132GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-235B-A22B

235B params118GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

mistralai/Mixtral-8x22B-Instruct-v0.1

141B params79GB Q4 VRAM

Developed by mistralai. Includes VRAM targets and compatible GPUs.

Open model guide →

mistralai/Mistral-Large-Instruct-2411

123B params69GB Q4 VRAM

Developed by mistralai. Includes VRAM targets and compatible GPUs.

Open model guide →

openai/gpt-oss-120b

120B params60GB Q4 VRAM

Developed by openai. Includes VRAM targets and compatible GPUs.

Open model guide →

RedHatAI/Llama-3.2-90B-Vision-Instruct-FP8-dynamic

90B params45GB Q4 VRAM

Developed by RedHatAI. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Next-80B-A3B-Instruct-FP8

80B params40GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Next-80B-A3B-Instruct

80B params40GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Next-80B-A3B-Thinking-FP8

80B params40GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Next-80B-A3B-Thinking

80B params40GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-72B-Instruct

73B params41GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-Math-72B-Instruct

73B params41GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

AI-MO/Kimina-Prover-72B

72B params36GB Q4 VRAM

Developed by AI-MO. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-72B-Instruct

72B params36GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

nvidia/Llama-3.1-Nemotron-70B-Instruct-HF

71B params40GB Q4 VRAM

Developed by nvidia. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.1-70B-Instruct

71B params40GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.3-70B-Instruct

71B params40GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

NousResearch/Hermes-3-Llama-3.1-70B

71B params40GB Q4 VRAM

Developed by NousResearch. Includes VRAM targets and compatible GPUs.

Open model guide →

RedHatAI/Llama-3.3-70B-Instruct-FP8-dynamic

70B params35GB Q4 VRAM

Developed by RedHatAI. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.1-70B-Instruct

70B params35GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Meta-Llama-3-70B-Instruct

70B params35GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w4a16

70B params35GB Q4 VRAM

Developed by RedHatAI. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.3-70B-Instruct

70B params35GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

01-ai/Yi-1.5-34B-Chat

34B params20GB Q4 VRAM

Developed by 01-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

dphn/dolphin-2.9.1-yi-1.5-34b

34B params17GB Q4 VRAM

Developed by dphn. Includes VRAM targets and compatible GPUs.

Open model guide →

codellama/CodeLlama-34b-hf

34B params17GB Q4 VRAM

Developed by codellama. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/deepseek-coder-33b-instruct

33B params19GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-Coder-32B-Instruct

33B params19GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/QwQ-32B-Preview

33B params19GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-32B-Instruct

33B params19GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-32B

32B params16GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/DeepSeek-R1-Distill-Qwen-32B-bnb-4bit

32B params16GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-32B-Instruct

32B params16GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-R1-Distill-Qwen-32B

32B params16GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-32B

32B params16GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

baichuan-inc/Baichuan-M2-32B

32B params16GB Q4 VRAM

Developed by baichuan-inc. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Coder-30B-A3B-Instruct

30B params15GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-30B-A3B-Instruct-2507

30B params15GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-30B-A3B-Instruct-2507-FP8

30B params15GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-6bit

30B params15GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-5bit

30B params15GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-30B-A3B-Thinking-2507

30B params15GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-4bit

30B params15GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-8bit

30B params15GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-30B-A3B

30B params15GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

google/gemma-2-27b-it

27B params16GB Q4 VRAM

Developed by google. Includes VRAM targets and compatible GPUs.

Open model guide →

mistralai/Mistral-Small-Instruct-2409

22B params13GB Q4 VRAM

Developed by mistralai. Includes VRAM targets and compatible GPUs.

Open model guide →

openai/gpt-oss-20b

20B params10GB Q4 VRAM

Developed by openai. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/gpt-oss-20b-unsloth-bnb-4bit

20B params10GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

mlx-community/gpt-oss-20b-MXFP4-Q8

20B params10GB Q4 VRAM

Developed by mlx-community. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/gpt-oss-20b-BF16

20B params10GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-14B-Instruct

15B params9GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-14B

14B params7GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-14B-Instruct

14B params7GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-14B

14B params7GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

OpenPipe/Qwen3-14B-Instruct

14B params7GB Q4 VRAM

Developed by OpenPipe. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-14B-Base

14B params7GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/Phi-3-medium-128k-instruct

14B params8GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-2-13b-chat-hf

13B params7GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

ai-forever/ruGPT-3.5-13B

13B params7GB Q4 VRAM

Developed by ai-forever. Includes VRAM targets and compatible GPUs.

Open model guide →

google/gemma-2-9b-it

9B params6GB Q4 VRAM

Developed by google. Includes VRAM targets and compatible GPUs.

Open model guide →

nvidia/NVIDIA-Nemotron-Nano-9B-v2

9B params5GB Q4 VRAM

Developed by nvidia. Includes VRAM targets and compatible GPUs.

Open model guide →

NousResearch/Hermes-3-Llama-3.1-8B

8B params5GB Q4 VRAM

Developed by NousResearch. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.1-8B-Instruct

8B params5GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

GSAI-ML/LLaDA-8B-Instruct

8B params4GB Q4 VRAM

Developed by GSAI-ML. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.1-8B-Instruct

8B params4GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-8B

8B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Meta-Llama-3-8B

8B params4GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Meta-Llama-3-8B-Instruct

8B params4GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit

8B params4GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Embedding-8B

8B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-R1-Distill-Llama-8B

8B params4GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.1-8B

8B params4GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/Meta-Llama-3.1-8B-Instruct

8B params4GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-8B-Base

8B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-8B-FP8

8B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

IlyaGusev/saiga_llama3_8b

8B params4GB Q4 VRAM

Developed by IlyaGusev. Includes VRAM targets and compatible GPUs.

Open model guide →

swiss-ai/Apertus-8B-Instruct-2509

8B params4GB Q4 VRAM

Developed by swiss-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-Guard-3-8B

8B params4GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/DeepSeek-R1-0528-Qwen3-8B-MLX-4bit

8B params4GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/DeepSeek-R1-0528-Qwen3-8B-MLX-8bit

8B params4GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

NousResearch/Meta-Llama-3.1-8B-Instruct

8B params4GB Q4 VRAM

Developed by NousResearch. Includes VRAM targets and compatible GPUs.

Open model guide →

GSAI-ML/LLaDA-8B-Base

8B params4GB Q4 VRAM

Developed by GSAI-ML. Includes VRAM targets and compatible GPUs.

Open model guide →

ibm-granite/granite-3.3-8b-instruct

8B params4GB Q4 VRAM

Developed by ibm-granite. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-7B-Instruct

8B params5GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/DialoGPT-small

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-R1

7B params4GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

mistralai/Mistral-7B-v0.1

7B params4GB Q4 VRAM

Developed by mistralai. Includes VRAM targets and compatible GPUs.

Open model guide →

facebook/opt-125m

7B params4GB Q4 VRAM

Developed by facebook. Includes VRAM targets and compatible GPUs.

Open model guide →

HuggingFaceM4/tiny-random-LlamaForCausalLM

7B params4GB Q4 VRAM

Developed by HuggingFaceM4. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-V3.1

7B params4GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/phi-4

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

EleutherAI/gpt-neo-125m

7B params4GB Q4 VRAM

Developed by EleutherAI. Includes VRAM targets and compatible GPUs.

Open model guide →

MiniMaxAI/MiniMax-M2

7B params4GB Q4 VRAM

Developed by MiniMaxAI. Includes VRAM targets and compatible GPUs.

Open model guide →

openai-community/gpt2-xl

7B params4GB Q4 VRAM

Developed by openai-community. Includes VRAM targets and compatible GPUs.

Open model guide →

sshleifer/tiny-gpt2

7B params4GB Q4 VRAM

Developed by sshleifer. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/Phi-3-mini-128k-instruct

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-V3-0324

7B params4GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

huggyllama/llama-7b

7B params4GB Q4 VRAM

Developed by huggyllama. Includes VRAM targets and compatible GPUs.

Open model guide →

hmellor/tiny-random-LlamaForCausalLM

7B params4GB Q4 VRAM

Developed by hmellor. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct

7B params4GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

numind/NuExtract-1.5

7B params4GB Q4 VRAM

Developed by numind. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-Coder-7B-Instruct

7B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/DialoGPT-medium

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

openai-community/gpt2

7B params4GB Q4 VRAM

Developed by openai-community. Includes VRAM targets and compatible GPUs.

Open model guide →

zai-org/GLM-4.6-FP8

7B params4GB Q4 VRAM

Developed by zai-org. Includes VRAM targets and compatible GPUs.

Open model guide →

HuggingFaceH4/zephyr-7b-beta

7B params4GB Q4 VRAM

Developed by HuggingFaceH4. Includes VRAM targets and compatible GPUs.

Open model guide →

llamafactory/tiny-random-Llama-3

7B params4GB Q4 VRAM

Developed by llamafactory. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/Phi-4-mini-instruct

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

HuggingFaceTB/SmolLM2-135M

7B params4GB Q4 VRAM

Developed by HuggingFaceTB. Includes VRAM targets and compatible GPUs.

Open model guide →

trl-internal-testing/tiny-LlamaForCausalLM-3.2

7B params4GB Q4 VRAM

Developed by trl-internal-testing. Includes VRAM targets and compatible GPUs.

Open model guide →

liuhaotian/llava-v1.5-7b

7B params4GB Q4 VRAM

Developed by liuhaotian. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-2-7b-hf

7B params4GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

EleutherAI/pythia-70m-deduped

7B params4GB Q4 VRAM

Developed by EleutherAI. Includes VRAM targets and compatible GPUs.

Open model guide →

parler-tts/parler-tts-large-v1

7B params4GB Q4 VRAM

Developed by parler-tts. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

7B params4GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

google/gemma-3-270m-it

7B params4GB Q4 VRAM

Developed by google. Includes VRAM targets and compatible GPUs.

Open model guide →

skt/kogpt2-base-v2

7B params4GB Q4 VRAM

Developed by skt. Includes VRAM targets and compatible GPUs.

Open model guide →

ibm-granite/granite-docling-258M

7B params4GB Q4 VRAM

Developed by ibm-granite. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/phi-2

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

lmsys/vicuna-7b-v1.5

7B params4GB Q4 VRAM

Developed by lmsys. Includes VRAM targets and compatible GPUs.

Open model guide →

rednote-hilab/dots.ocr

7B params4GB Q4 VRAM

Developed by rednote-hilab. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-V3

7B params4GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-1.7B

7B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

dicta-il/dictalm2.0-instruct

7B params4GB Q4 VRAM

Developed by dicta-il. Includes VRAM targets and compatible GPUs.

Open model guide →

BSC-LT/salamandraTA-7b-instruct

7B params4GB Q4 VRAM

Developed by BSC-LT. Includes VRAM targets and compatible GPUs.

Open model guide →

openai-community/gpt2-large

7B params4GB Q4 VRAM

Developed by openai-community. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-1.7B-Base

7B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-7B-Instruct

7B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/Phi-3-mini-4k-instruct

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-7B

7B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

petals-team/StableBeluga2

7B params4GB Q4 VRAM

Developed by petals-team. Includes VRAM targets and compatible GPUs.

Open model guide →

vikhyatk/moondream2

7B params4GB Q4 VRAM

Developed by vikhyatk. Includes VRAM targets and compatible GPUs.

Open model guide →

distilbert/distilgpt2

7B params4GB Q4 VRAM

Developed by distilbert. Includes VRAM targets and compatible GPUs.

Open model guide →

mistralai/Mistral-7B-Instruct-v0.2

7B params4GB Q4 VRAM

Developed by mistralai. Includes VRAM targets and compatible GPUs.

Open model guide →

bigscience/bloomz-560m

7B params4GB Q4 VRAM

Developed by bigscience. Includes VRAM targets and compatible GPUs.

Open model guide →

zai-org/GLM-4.5-Air

7B params4GB Q4 VRAM

Developed by zai-org. Includes VRAM targets and compatible GPUs.

Open model guide →

mistralai/Mistral-7B-Instruct-v0.1

7B params4GB Q4 VRAM

Developed by mistralai. Includes VRAM targets and compatible GPUs.

Open model guide →

HuggingFaceTB/SmolLM-135M

7B params4GB Q4 VRAM

Developed by HuggingFaceTB. Includes VRAM targets and compatible GPUs.

Open model guide →

trl-internal-testing/tiny-random-LlamaForCausalLM

7B params4GB Q4 VRAM

Developed by trl-internal-testing. Includes VRAM targets and compatible GPUs.

Open model guide →

openai-community/gpt2-medium

7B params4GB Q4 VRAM

Developed by openai-community. Includes VRAM targets and compatible GPUs.

Open model guide →

trl-internal-testing/tiny-Qwen2ForCausalLM-2.5

7B params4GB Q4 VRAM

Developed by trl-internal-testing. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/Phi-4-multimodal-instruct

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

rinna/japanese-gpt-neox-small

7B params4GB Q4 VRAM

Developed by rinna. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/mistral-7b-v0.3-bnb-4bit

7B params4GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/Phi-3.5-vision-instruct

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-R1-0528

7B params4GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-2-7b-chat-hf

7B params4GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2-7B-Instruct

7B params4GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/Phi-3.5-mini-instruct

7B params4GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-0.6B-Base

6B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Reranker-0.6B

6B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Embedding-0.6B

6B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-0.6B

6B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2-0.5B

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-Math-1.5B

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-0.5B-Instruct

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-Coder-1.5B

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2-0.5B-Instruct

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Alibaba-NLP/gte-Qwen2-1.5B-instruct

5B params3GB Q4 VRAM

Developed by Alibaba-NLP. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2-1.5B-Instruct

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

5B params3GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

Gensyn/Qwen2.5-0.5B-Instruct

5B params3GB Q4 VRAM

Developed by Gensyn. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-0.5B

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/VibeVoice-1.5B

5B params3GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-1.5B

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-1.5B-Instruct

5B params3GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-4B-Thinking-2507

4B params2GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-4B-Thinking-2507-FP8

4B params2GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-4B-Base

4B params2GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/Qwen3-4B-Thinking-2507-MLX-4bit

4B params2GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/Qwen3-4B-Thinking-2507-MLX-8bit

4B params2GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

lmstudio-community/Qwen3-4B-Thinking-2507-MLX-6bit

4B params2GB Q4 VRAM

Developed by lmstudio-community. Includes VRAM targets and compatible GPUs.

Open model guide →

kaitchup/Phi-3-mini-4k-instruct-gptq-4bit

4B params2GB Q4 VRAM

Developed by kaitchup. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-4B-Instruct-2507

4B params2GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-4B

4B params2GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen3-Embedding-4B

4B params2GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

microsoft/Phi-3.5-mini-instruct

4B params3GB Q4 VRAM

Developed by microsoft. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.2-3B-Instruct

3B params2GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.2-3B

3B params2GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

deepseek-ai/deepseek-coder-1.3b-instruct

3B params2GB Q4 VRAM

Developed by deepseek-ai. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-3B

3B params2GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

context-labs/meta-llama-Llama-3.2-3B-Instruct-FP16

3B params2GB Q4 VRAM

Developed by context-labs. Includes VRAM targets and compatible GPUs.

Open model guide →

google-t5/t5-3b

3B params2GB Q4 VRAM

Developed by google-t5. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.2-3B-Instruct

3B params2GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

bigcode/starcoder2-3b

3B params2GB Q4 VRAM

Developed by bigcode. Includes VRAM targets and compatible GPUs.

Open model guide →

inference-net/Schematron-3B

3B params2GB Q4 VRAM

Developed by inference-net. Includes VRAM targets and compatible GPUs.

Open model guide →

Qwen/Qwen2.5-3B-Instruct

3B params2GB Q4 VRAM

Developed by Qwen. Includes VRAM targets and compatible GPUs.

Open model guide →

ibm-research/PowerMoE-3b

3B params2GB Q4 VRAM

Developed by ibm-research. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/Llama-3.2-3B-Instruct

3B params2GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

google/gemma-2b

2B params1GB Q4 VRAM

Developed by google. Includes VRAM targets and compatible GPUs.

Open model guide →

ibm-granite/granite-3.3-2b-instruct

2B params1GB Q4 VRAM

Developed by ibm-granite. Includes VRAM targets and compatible GPUs.

Open model guide →

LiquidAI/LFM2-1.2B

2B params1GB Q4 VRAM

Developed by LiquidAI. Includes VRAM targets and compatible GPUs.

Open model guide →

google/gemma-2-2b-it

2B params1GB Q4 VRAM

Developed by google. Includes VRAM targets and compatible GPUs.

Open model guide →

allenai/OLMo-2-0425-1B

1B params1GB Q4 VRAM

Developed by allenai. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-Guard-3-1B

1B params1GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/Llama-3.2-1B-Instruct

1B params1GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

unsloth/gemma-3-1b-it

1B params1GB Q4 VRAM

Developed by unsloth. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.2-1B

1B params1GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

apple/OpenELM-1_1B-Instruct

1B params1GB Q4 VRAM

Developed by apple. Includes VRAM targets and compatible GPUs.

Open model guide →

meta-llama/Llama-3.2-1B-Instruct

1B params1GB Q4 VRAM

Developed by meta-llama. Includes VRAM targets and compatible GPUs.

Open model guide →

TinyLlama/TinyLlama-1.1B-Chat-v1.0

1B params1GB Q4 VRAM

Developed by TinyLlama. Includes VRAM targets and compatible GPUs.

Open model guide →

google/gemma-3-1b-it

1B params1GB Q4 VRAM

Developed by google. Includes VRAM targets and compatible GPUs.

Open model guide →