Can I Run It? Local AI Setup Generator
Tell us your GPU, what you want it for, and your OS, and we'll name the exact model and quant that fit, the command to run it, and roughly how fast it'll go. No guesswork, no sign-up, runs in your browser, and every result has a shareable link.
Qwen3 32B fits
Runs at Q4_K_M (~20 GB), using most of your 24 GB VRAM. Keep context moderate and close other GPU-heavy apps, and you're set.
- QuantQ4_K_M
- Weights~20 GB of weights
- Context128K context
- LicenseApache-2.0
Best all-round dense model for a 24GB card.
Rough estimate. Real speed depends on quant, context length, and your backend.
Copy-paste setup
- 1
Install Ollama Windows
winget install Ollama.OllamaOr download the Windows installer from ollama.com/download and run it.
- 2
Pull & run the model
ollama run qwen3:32bDownloads and runs Qwen3 32B at Q4_K_M. The first run pulls a few GB.
Prefer a GUI to a terminal? Install LM Studio from lmstudio.ai, search "Qwen3 32B", and grab the Q4_K_M build.
You came for roleplay, and this card can run her
You can assemble it yourself from the setup above, backend, GGUF, character card, sampler tuning, or you can skip to the finished version. Ember is an uncensored 18+ companion that installs in one step and runs entirely on this card: her own tuned brain, voice, and memory that survives a restart. Nothing leaves the machine. $29 paid once in crypto, no account, no card on file, yours forever.
See Ember →Want the companion without the setup?
Your card can handle the model above. If what you were really after is an uncensored companion rather than a stack to maintain, Ember is that, finished: one installer, her own tuned brain, voice and memory, all local. $29 paid once in crypto, yours forever. 18+.
See Ember →This machine is under the companion floor
A local companion that answers at conversation speed needs an NVIDIA card with 8 GB of VRAM. On CPU-only or a smaller card you can still run the model above, it will just answer slowly enough to break the illusion. The honest next step is the card, not an app.
Cheapest GPUs that clear the bar →How this pick is made
Everything runs in your browser from a fixed model table, with no tracking and no black box.
We take your card's memory, subtract a slice for the KV cache, runtime and desktop, and pick the highest-quality model in your goal whose Q4_K_M weights fit that budget. If Q4 won't fit, we drop to Q3_K_M; if there's spare room we flag that you can go up to Q5/Q6.
The tokens/sec band is a physical ballpark, roughly memory bandwidth divided by the model's active-weight size, not a measured benchmark. MoE models read only their active parameters per token, so they run faster than their total size suggests. Want the exact VRAM math for a specific model and context? Use the VRAM Calculator.
From “can I run it?” to running it
Two things decide whether a local model runs on your machine: whether the weights plus the KV cache fit in memory, and whether it's fast enough to be pleasant. This tool answers the first exactly and the second approximately. Pick the card you actually own, the job you actually have, and it names a specific model and quantization instead of a vague “it depends”.
The single biggest lever is quantization. A model at Q4_K_M is roughly a quarter the size of full precision for a barely-perceptible quality drop, which is why it's the default recommendation here. See the GGUF quantization cheat sheet for the full ladder. New to all of this? Start with how to run AI locally and the local AI hardware guide. Want an uncensored setup? The uncensored local AI guide covers the model choices, and the model database lets you filter every option by VRAM, license and use case.
If the verdict says your hardware is tight, the honest answer is a smaller model, a lower quant, or a bigger card, in that order. And if what you wanted all along was the companion rather than the stack, Ember is the finished version: an uncensored 18+ desktop app that runs on your own NVIDIA card with 8 GB of VRAM, $29 paid once, nothing sent to a server.
Local AI setup FAQ
What local LLM can I run on 8GB VRAM?
On 8GB of VRAM you can comfortably run an 8B model at Q4_K_M. Good picks are Llama 3.1 8B or Qwen3 8B for general chat, or a Dolphin/abliterated 8B for uncensored chat, using about 5GB for the weights and leaving room for context. 12B to 14B models fit only at a low quant or short context. This tool picks the largest sensible model for your exact card and goal.
Do I need a GPU to run a local AI?
No. Ollama and LM Studio both run on the CPU alone, and a 7B to 8B model at Q4 works with about 16GB of system RAM. The catch is speed: expect only a few tokens per second on CPU versus dozens on a GPU, which is fine for a one-off question and painful for a conversation. Companion-grade responsiveness needs a GPU: see the cheapest GPU for local AI for what actually clears the bar.
Is Ollama or LM Studio better for beginners?
For the command line, Ollama is simplest: install it, then `ollama run <model>` downloads and starts the model in one step. LM Studio is a graphical app where you browse and download GGUF models and chat in a window, no terminal needed. Beginners who dislike the command line usually prefer LM Studio; scripters and API users prefer Ollama. Both are free.
How many tokens per second will a local model do?
It depends mostly on memory bandwidth and model size: tokens/sec roughly equals your GPU's bandwidth divided by the model's active weight size. A 7B to 8B model on a modern GPU often runs 40 to 100 tok/s, a 70B model runs 10 to 20, and CPU-only runs in the low single digits. The band this tool shows is a ballpark, so real speed varies with quant, context and backend.
Can I run an uncensored AI model locally?
Yes. Abliterated or uncensored finetunes like Dolphin, Cydonia and similar strip the refusal layer and run in Ollama or LM Studio exactly like any other GGUF; you own the weights and nothing leaves your machine. If you want that as a finished product instead of a weekend of wiring, Ember is an 18+ desktop app that runs on your own NVIDIA card (8GB VRAM), paid once at $29.
Recommendations come from a curated table of real, currently-available models. Tokens/sec is a rough estimate, not a benchmark. We link to tools we make, no ads and no trackers.