Quick Answer: RTX 3090 offers 24GB VRAM and starts around $979.99. It delivers approximately 209 tokens/sec on deepseek-ai/DeepSeek-OCR. It typically draws 350W under load.
RTX 3090 still delivers strong results for large language models thanks to its 24GB VRAM. It is ideal for enthusiasts leveraging the Ampere generation for budget workstation builds.
Buy directly on Amazon with fast shipping and reliable customer service.
Rotate out primary variants whenever validation flags an issue.
💡 Not ready to buy? Try cloud GPUs first
Test RTX 3090 performance in the cloud before investing in hardware. Pay by the hour with no commitment.
| Model | Quantization | Tokens/sec | VRAM used |
|---|---|---|---|
| deepseek-ai/DeepSeek-OCR | Q4 | 209.15 tok/sEstimated Auto-generated benchmark | 2GB |
| unsloth/Llama-3.2-1B-Instruct | Q4 | 203.52 tok/sEstimated Auto-generated benchmark | 1GB |
| google/gemma-3-1b-it | Q4 | 201.55 tok/sEstimated Auto-generated benchmark | 1GB |
| nari-labs/Dia2-2B | Q4 | 200.81 tok/sEstimated Auto-generated benchmark | 2GB |
| allenai/OLMo-2-0425-1B | Q4 | 200.76 tok/sEstimated Auto-generated benchmark | 1GB |
| Qwen/Qwen2.5-3B | Q4 | 198.27 tok/sEstimated Auto-generated benchmark | 2GB |
| TinyLlama/TinyLlama-1.1B-Chat-v1.0 | Q4 | 197.60 tok/sEstimated Auto-generated benchmark | 1GB |
| tencent/HunyuanOCR | Q4 | 196.73 tok/sEstimated Auto-generated benchmark | 1GB |
| ibm-granite/granite-3.3-2b-instruct | Q4 | 196.55 tok/sEstimated Auto-generated benchmark | 1GB |
| deepseek-ai/deepseek-coder-1.3b-instruct | Q4 | 194.47 tok/sEstimated Auto-generated benchmark | 2GB |
| Qwen/Qwen2.5-3B-Instruct | Q4 | 193.79 tok/sEstimated Auto-generated benchmark | 2GB |
| ibm-research/PowerMoE-3b | Q4 | 193.59 tok/sEstimated Auto-generated benchmark | 2GB |
| meta-llama/Llama-3.2-1B-Instruct | Q4 | 191.64 tok/sEstimated Auto-generated benchmark | 1GB |
| meta-llama/Llama-3.2-3B | Q4 | 191.33 tok/sEstimated Auto-generated benchmark | 2GB |
| unsloth/gemma-3-1b-it | Q4 | 189.53 tok/sEstimated Auto-generated benchmark | 1GB |
| google/gemma-2-2b-it | Q4 | 186.96 tok/sEstimated Auto-generated benchmark | 1GB |
| apple/OpenELM-1_1B-Instruct | Q4 | 185.77 tok/sEstimated Auto-generated benchmark | 1GB |
| google/gemma-2b | Q4 | 185.33 tok/sEstimated Auto-generated benchmark | 1GB |
| google-bert/bert-base-uncased | Q4 | 184.49 tok/sEstimated Auto-generated benchmark | 1GB |
| WeiboAI/VibeThinker-1.5B | Q4 | 184.26 tok/sEstimated Auto-generated benchmark | 1GB |
| google-t5/t5-3b | Q4 | 183.09 tok/sEstimated Auto-generated benchmark | 2GB |
| LiquidAI/LFM2-1.2B | Q4 | 179.42 tok/sEstimated Auto-generated benchmark | 1GB |
| unsloth/Llama-3.2-3B-Instruct | Q4 | 178.80 tok/sEstimated Auto-generated benchmark | 2GB |
| inference-net/Schematron-3B | Q4 | 176.51 tok/sEstimated Auto-generated benchmark | 2GB |
| bigcode/starcoder2-3b | Q4 | 174.50 tok/sEstimated Auto-generated benchmark | 2GB |
| meta-llama/Llama-Guard-3-1B | Q4 | 173.15 tok/sEstimated Auto-generated benchmark | 1GB |
| meta-llama/Llama-3.2-1B | Q4 | 170.29 tok/sEstimated Auto-generated benchmark | 1GB |
| facebook/sam3 | Q4 | 169.50 tok/sEstimated Auto-generated benchmark | 1GB |
| HuggingFaceTB/SmolLM-135M | Q4 | 169.26 tok/sEstimated Auto-generated benchmark | 4GB |
| microsoft/Phi-4-multimodal-instruct | Q4 | 169.13 tok/sEstimated Auto-generated benchmark | 4GB |
| context-labs/meta-llama-Llama-3.2-3B-Instruct-FP16 | Q4 | 168.93 tok/sEstimated Auto-generated benchmark | 2GB |
| bigscience/bloomz-560m | Q4 | 168.41 tok/sEstimated Auto-generated benchmark | 4GB |
| microsoft/Phi-3-mini-4k-instruct | Q4 | 168.28 tok/sEstimated Auto-generated benchmark | 4GB |
| Qwen/Qwen2-7B-Instruct | Q4 | 168.09 tok/sEstimated Auto-generated benchmark | 4GB |
| openai-community/gpt2 | Q4 | 167.92 tok/sEstimated Auto-generated benchmark | 4GB |
| microsoft/Phi-3.5-mini-instruct | Q4 | 167.77 tok/sEstimated Auto-generated benchmark | 4GB |
| meta-llama/Meta-Llama-3-8B-Instruct | Q4 | 167.63 tok/sEstimated Auto-generated benchmark | 4GB |
| meta-llama/Llama-3.2-3B-Instruct | Q4 | 167.42 tok/sEstimated Auto-generated benchmark | 2GB |
| lmstudio-community/DeepSeek-R1-0528-Qwen3-8B-MLX-8bit | Q4 | 167.39 tok/sEstimated Auto-generated benchmark | 4GB |
| MiniMaxAI/MiniMax-M2 | Q4 | 167.28 tok/sEstimated Auto-generated benchmark | 4GB |
| lmstudio-community/DeepSeek-R1-0528-Qwen3-8B-MLX-4bit | Q4 | 167.22 tok/sEstimated Auto-generated benchmark | 4GB |
| Qwen/Qwen3-4B-Base | Q4 | 166.99 tok/sEstimated Auto-generated benchmark | 2GB |
| EleutherAI/gpt-neo-125m | Q4 | 166.69 tok/sEstimated Auto-generated benchmark | 4GB |
| google/embeddinggemma-300m | Q4 | 166.67 tok/sEstimated Auto-generated benchmark | 1GB |
| llamafactory/tiny-random-Llama-3 | Q4 | 166.46 tok/sEstimated Auto-generated benchmark | 4GB |
| microsoft/phi-2 | Q4 | 166.33 tok/sEstimated Auto-generated benchmark | 4GB |
| Tongyi-MAI/Z-Image-Turbo | Q4 | 166.28 tok/sEstimated Auto-generated benchmark | 4GB |
| Qwen/Qwen2.5-1.5B-Instruct | Q4 | 166.09 tok/sEstimated Auto-generated benchmark | 3GB |
| black-forest-labs/FLUX.1-dev | Q4 | 166.02 tok/sEstimated Auto-generated benchmark | 4GB |
| petals-team/StableBeluga2 | Q4 | 165.96 tok/sEstimated Auto-generated benchmark | 4GB |
Note: Performance estimates are calculated. Real results may vary. Methodology · Submit real data
| Model | Quantization | Verdict | Estimated speed | VRAM needed |
|---|---|---|---|---|
| vikhyatk/moondream2 | Q8 | Fits comfortably | 112.41 tok/sEstimated | 7GB (have 24GB) |
| vikhyatk/moondream2 | FP16 | Fits comfortably | 53.70 tok/sEstimated | 15GB (have 24GB) |
| Qwen/Qwen3-32B | Q4 | Fits comfortably | 58.82 tok/sEstimated | 16GB (have 24GB) |
| RedHatAI/Meta-Llama-3.1-70B-Instruct-quantized.w4a16 | FP16 | Not supported | 21.12 tok/sEstimated | 137GB (have 24GB) |
| OpenPipe/Qwen3-14B-Instruct | FP16 | Not supported | 48.10 tok/sEstimated | 29GB (have 24GB) |
| openai-community/gpt2-xl | Q4 | Fits comfortably | 142.13 tok/sEstimated | 4GB (have 24GB) |
| meta-llama/Llama-Guard-3-8B | Q4 | Fits comfortably | 157.06 tok/sEstimated | 4GB (have 24GB) |
| ibm-granite/granite-3.3-2b-instruct | FP16 | Fits comfortably | 74.36 tok/sEstimated | 4GB (have 24GB) |
| microsoft/VibeVoice-1.5B | Q4 | Fits comfortably | 145.91 tok/sEstimated | 3GB (have 24GB) |
| ibm-granite/granite-docling-258M | Q4 | Fits comfortably | 149.28 tok/sEstimated | 4GB (have 24GB) |
| Qwen/Qwen3-Next-80B-A3B-Thinking-FP8 | FP16 | Not supported | 12.63 tok/sEstimated | 156GB (have 24GB) |
| bigcode/starcoder2-3b | Q4 | Fits comfortably | 174.50 tok/sEstimated | 2GB (have 24GB) |
| bigcode/starcoder2-3b | Q8 | Fits comfortably | 120.28 tok/sEstimated | 3GB (have 24GB) |
| bigcode/starcoder2-3b | FP16 | Fits comfortably | 67.78 tok/sEstimated | 6GB (have 24GB) |
| unsloth/gemma-3-1b-it | Q4 | Fits comfortably | 189.53 tok/sEstimated | 1GB (have 24GB) |
| unsloth/gemma-3-1b-it | Q8 | Fits comfortably | 133.23 tok/sEstimated | 1GB (have 24GB) |
| lmsys/vicuna-7b-v1.5 | Q4 | Fits comfortably | 158.19 tok/sEstimated | 4GB (have 24GB) |
| lmsys/vicuna-7b-v1.5 | Q8 | Fits comfortably | 114.95 tok/sEstimated | 7GB (have 24GB) |
| lmsys/vicuna-7b-v1.5 | FP16 | Fits comfortably | 58.14 tok/sEstimated | 15GB (have 24GB) |
| Qwen/Qwen2.5-3B | Q4 | Fits comfortably | 198.27 tok/sEstimated | 2GB (have 24GB) |
| Qwen/Qwen2.5-3B | FP16 | Fits comfortably | 75.48 tok/sEstimated | 6GB (have 24GB) |
| nvidia/NVIDIA-Nemotron-Nano-9B-v2 | FP16 | Fits comfortably | 43.94 tok/sEstimated | 19GB (have 24GB) |
| NousResearch/Meta-Llama-3.1-8B-Instruct | Q4 | Fits comfortably | 141.35 tok/sEstimated | 4GB (have 24GB) |
| AI-MO/Kimina-Prover-72B | FP16 | Not supported | 11.75 tok/sEstimated | 141GB (have 24GB) |
| lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-4bit | Q4 | Fits comfortably | 82.53 tok/sEstimated | 15GB (have 24GB) |
| lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-8bit | Q8 | Not supported | 62.29 tok/sEstimated | 31GB (have 24GB) |
| lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-8bit | FP16 | Not supported | 32.86 tok/sEstimated | 61GB (have 24GB) |
| GSAI-ML/LLaDA-8B-Base | Q4 | Fits comfortably | 139.81 tok/sEstimated | 4GB (have 24GB) |
| baichuan-inc/Baichuan-M2-32B | Q4 | Fits comfortably | 50.12 tok/sEstimated | 16GB (have 24GB) |
| baichuan-inc/Baichuan-M2-32B | Q8 | Not supported | 40.01 tok/sEstimated | 33GB (have 24GB) |
| WeiboAI/VibeThinker-1.5B | Q4 | Fits comfortably | 184.26 tok/sEstimated | 1GB (have 24GB) |
| WeiboAI/VibeThinker-1.5B | Q8 | Fits comfortably | 135.30 tok/sEstimated | 2GB (have 24GB) |
| WeiboAI/VibeThinker-1.5B | FP16 | Fits comfortably | 71.67 tok/sEstimated | 4GB (have 24GB) |
| Gensyn/Qwen2.5-0.5B-Instruct | FP16 | Fits comfortably | 56.86 tok/sEstimated | 11GB (have 24GB) |
| Qwen/Qwen3-Reranker-0.6B | FP16 | Fits comfortably | 56.45 tok/sEstimated | 13GB (have 24GB) |
| Qwen/Qwen2.5-32B-Instruct | Q8 | Not supported | 37.82 tok/sEstimated | 33GB (have 24GB) |
| meta-llama/Llama-3.1-8B | Q4 | Fits comfortably | 162.11 tok/sEstimated | 4GB (have 24GB) |
| meta-llama/Llama-3.1-8B | Q8 | Fits comfortably | 98.74 tok/sEstimated | 9GB (have 24GB) |
| deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct | FP16 | Fits comfortably | 62.51 tok/sEstimated | 15GB (have 24GB) |
| numind/NuExtract-1.5 | Q4 | Fits comfortably | 153.01 tok/sEstimated | 4GB (have 24GB) |
| mistralai/Mistral-7B-v0.1 | FP16 | Fits comfortably | 53.27 tok/sEstimated | 15GB (have 24GB) |
| LiquidAI/LFM2-1.2B | Q4 | Fits comfortably | 179.42 tok/sEstimated | 1GB (have 24GB) |
| meta-llama/Llama-3.3-70B-Instruct | Q4 | Not supported | 30.02 tok/sEstimated | 34GB (have 24GB) |
| meta-llama/Llama-3.3-70B-Instruct | Q8 | Not supported | 23.43 tok/sEstimated | 69GB (have 24GB) |
| google-t5/t5-3b | Q4 | Fits comfortably | 183.09 tok/sEstimated | 2GB (have 24GB) |
| rednote-hilab/dots.ocr | Q4 | Fits comfortably | 146.45 tok/sEstimated | 4GB (have 24GB) |
| meta-llama/Llama-3.1-70B-Instruct | Q4 | Not supported | 55.03 tok/sEstimated | 34GB (have 24GB) |
| zai-org/GLM-4.5-Air | Q4 | Fits comfortably | 149.90 tok/sEstimated | 4GB (have 24GB) |
| zai-org/GLM-4.5-Air | Q8 | Fits comfortably | 106.82 tok/sEstimated | 7GB (have 24GB) |
| vikhyatk/moondream2 | Q4 | Fits comfortably | 139.82 tok/sEstimated | 4GB (have 24GB) |
Note: Performance estimates are calculated. Real results may vary. Methodology · Submit real data
Data-backed answers pulled from community benchmarks, manufacturer specs, and live pricing.
An Ampere builder reports ~18 tok/s on Llama 3 70B Q4 with a single 3090, and ~36 tok/s after adding tensor parallelism across four cards.
Source: Reddit – /r/LocalLLaMA (mqzh3yo)
Enthusiasts routinely see ~100 tokens/sec on Qwen 30B-A3B when tuned on a single RTX 3090, making it a budget-friendly coding workhorse.
Source: Reddit – /r/LocalLLaMA (mqs2r45)
Builders using x1 risers for dual 3080/3090 rigs measured no meaningful tokens/sec loss—the main penalty is slower model swaps, not slower inference.
Source: Reddit – /r/LocalLLaMA (mr10ib4)
RTX 3090 provides 24 GB GDDR6X, draws 350 W, and uses triple 8-pin PCIe power connectors. NVIDIA recommends a 750 W PSU.
Source: TechPowerUp – RTX 3090 Specs
Nov 2025 snapshot: Amazon at $1,099 (check current availability).
Source: Supabase price tracker snapshot – 2025-11-03
Explore how RTX 4090 stacks up for local inference workloads.
Explore how RTX 3080 stacks up for local inference workloads.
Explore how RTX 3070 stacks up for local inference workloads.
Explore how NVIDIA A6000 stacks up for local inference workloads.
Explore how RX 7900 XTX stacks up for local inference workloads.
Side-by-side VRAM, throughput, efficiency, and pricing benchmarks for both GPUs.
Side-by-side VRAM, throughput, efficiency, and pricing benchmarks for both GPUs.
Side-by-side VRAM, throughput, efficiency, and pricing benchmarks for both GPUs.