If you’re standing up a local AI rig in 2026, you don’t really pick a model — you pick a family. The model you download this month gets a point release next month, a quantized re-bake the week after, and an abliterated variant before you’ve finished tuning your prompt. What stays constant is the family: its architecture lineage, its license, its tokenizer, its quirks, and how cleanly it takes to uncensoring. Bet on the right family and your whole workflow ages well. Bet wrong and you’re re-learning everything in 90 days.
This is the honest version of the comparison — written for someone who is going to actually ollama run the thing on their own GPU, not benchmark-chase a leaderboard. We’ll cover where each family genuinely leads, the license traps that bite self-hosters, and the question most “best LLM” listicles dodge: which family abliterates cleanest for uncensored local use.
The 2026 open-weight landscape at a glance
Six labs now ship competitive open-weight families, and the field has consolidated hard. As of mid-2026 the live contenders are Alibaba’s Qwen (3.5 / 3.6), Meta’s Llama 4, Mistral (Small 4 / the Nemo lineage), and Google’s Gemma (3 / 4) — with DeepSeek V4 and Zhipu’s GLM circling as strong outsiders. The big shift from a year ago: Mixture-of-Experts (MoE) went mainstream, so “parameter count” no longer maps to VRAM the way it used to. A 26B-total MoE model with ~4B active parameters runs closer to a 4–8B dense model in speed while punching well above its weight in quality.
Here’s the shape of it:
| Family | 2026 flagship form | Architecture | License | Best at |
|---|---|---|---|---|
| Qwen | 3.5 / 3.6, dense 0.6B→32B + huge MoE | Dense + MoE | Apache 2.0 | All-rounder, coding, multilingual |
| Llama | Llama 4 (Scout / Maverick) | MoE, server-scale | Llama Community (MAU cap, EU limits) | Long context, reasoning at scale |
| Mistral | Small 4, Nemo lineage | Dense | Apache 2.0 | Efficiency, creative writing, RP |
| Gemma | 3 (dense) / 4 (dense + MoE) | Dense + MoE | Gemma Terms (custom) | Edge / unified-memory, vision |
The practical headline: Qwen and Mistral give you Apache 2.0 freedom, Llama 4 is mostly too big to run at home, and Gemma is the unified-memory and Mac pick. Everything below is detail on that sentence. If you’re brand new to running any of this, start with how to run AI locally — your chosen quant level is what makes any of these families actually fit your card.
Qwen 3.5 / 3.6: strengths and quirks
Qwen is the family I’d hand someone who says “I just want one model that’s good at everything.” Alibaba ships the widest size ladder of any family — the dense line runs 0.6B / 1.7B / 4B / 8B / 14B / 32B, and the MoE line climbs much higher (Qwen3 already shipped a 235B-A22B MoE, with even larger ~400B-class MoE checkpoints reported for the 3.5/3.6 generation). So there is a real Qwen for an 8GB card and a real Qwen for a 4090. The dense models in the 7B–32B range are the sweet spot for local rigs.
Strengths: strong reasoning and coding, genuinely excellent multilingual coverage (it’s the family to beat for non-English chat), and a hybrid “thinking” mode in the 3.x line that you can toggle for harder problems. Per Alibaba’s model cards, the Qwen3 line is released under Apache 2.0 — the most permissive license in this entire comparison. That alone makes Qwen the default for anyone who cares about ownership.
Quirks: out of the box, official Qwen instruct models are politely cautious — they’ll refuse or moralize on a fair amount of edgy-but-legal territory. The good news (more on this below) is that the family abliterates extremely cleanly, so the community uncensored variants are abundant and lose very little capability. The thinking mode can also be chatty; for companion or roleplay use you usually want it off so the model doesn’t narrate its reasoning at you.
Hardware: a Qwen 14B at Q4_K_M lands around 9GB and fits comfortably in 12GB VRAM; the 7–8B class is the 8GB pick. Match the dense size to your card and the quant to your headroom, and you can keep the whole thing on-GPU.
Llama 3.3 / 4: where it leads
Llama built the modern open-weight ecosystem — most tooling, quant pipelines, and fine-tunes were born targeting Llama, and that gravitational pull still helps. But Llama 4 made a hard architectural turn toward MoE and large scale. The Llama 4 lineup (Scout, Maverick) starts above 100B total parameters, which means the flagship models are effectively server-class, not home-GPU class.
Where it leads: raw reasoning ceiling and context length — Llama 4 Scout’s headline context window is enormous (reported in the millions of tokens), which is a real differentiator for document-heavy and agentic work. For long-context retrieval and chatting with large document sets on capable hardware, it’s compelling.
The catch — license: the Llama Community License is not a true open-source license. It carries an acceptable-use policy, a monthly-active-user cap (the well-known ~700M MAU threshold) above which you must seek a separate Meta license, and use restrictions tied to the EU for some multimodal releases. For a solo self-hoster none of this is fatal, but it’s the reason license-purists reach for Qwen or Mistral first. If you’re considering Llama specifically because cloud models keep stonewalling you, the deeper issue is covered in why cloud AI censors you.
Reality check for home users: the locally-runnable Llamas in 2026 are mostly the previous-gen 8B/3.3-class dense models and community continuations, not the Llama 4 flagships. They’re solid, well-supported, and abliterate fine — just no longer the frontier.
Mistral / Nemo: the creative and efficient pick
Mistral is the family for people who value doing more with less and who write. The French lab’s calling card has always been efficiency — Mistral models historically punch above their parameter count — and permissive Apache 2.0 licensing on its open releases. The Nemo lineage (the ~12B class co-developed with NVIDIA) remains a beloved local workhorse, and Mistral Small 4 continues the tradition of a dense model that feels larger than it is.
Strengths: prose. Mistral and its many community fine-tunes have a less “corporate” default voice than Llama or Gemma, which is why so much of the roleplay and creative-writing scene is built on Mistral/Nemo derivatives. If your use case is creative writing or character chat, this family and its fine-tunes are over-represented for a reason. It’s also light: a 12B-class model at Q4_K_M is very friendly to 12GB cards and runnable on 8GB with a tighter quant.
Quirks: Mistral’s base instruct models are relatively lightly aligned to begin with — which is a feature here. They tend to be less preachy than peers, so even before any uncensoring they’re more cooperative for adult-but-legal content. The trade-off is that on the hardest reasoning and math tasks, a same-size Qwen often edges it out.
Gemma 3 / 4: MoE and unified-memory fit
Gemma is Google’s open-weight family, and in 2026 it’s the edge and unified-memory champion. Gemma covers the full spectrum from ~2B edge models up to ~31B workstation-class, and Gemma 4 added MoE variants (e.g. a ~26B-total model with ~4B active) alongside the dense ones.
Why it matters for hardware: Gemma’s small-and-mid models are unusually RAM-efficient, and the MoE variants give you big-model quality at small-model inference cost. This makes Gemma the standout pick for Apple Silicon and other unified-memory machines, where the GPU and CPU share one memory pool and a fat-but-sparse MoE model is ideal. On a Mac, you size by total unified memory rather than dedicated VRAM, which is exactly the kind of machine Gemma’s sparse MoE checkpoints were made for. Gemma also ships strong native vision in its larger models.
Quirks: Gemma is the most safety-tuned family here by default — Google aligns it heavily, so stock Gemma refuses the most. It also ships under Google’s custom Gemma Terms of Use, not a standard OSI license. That’s not a dealbreaker for personal local use, but read on, because Gemma’s heavy alignment is exactly why its abliterated variants are so interesting.
License gotchas per family
This is where a lot of people get burned, so plainly:
| Family | License | The gotcha |
|---|---|---|
| Qwen | Apache 2.0 | None meaningful — commercial-friendly, no user cap |
| Mistral (open releases) | Apache 2.0 | None on the Apache models; some premier models are API-only/commercial — check the specific release |
| Llama | Llama Community License | ~700M MAU cap, acceptable-use policy, EU restrictions on some multimodal weights |
| Gemma | Gemma Terms of Use | Custom (not OSI), prohibited-use policy attached |
Two rules of thumb. One: if you ever want to build something commercial or just want zero legal ambiguity, Qwen and Mistral’s Apache 2.0 releases are the cleanest. Two: the license governs the weights, not your private conversations — running any of these locally means your chats never leave your machine regardless of license. And always confirm the license on the specific checkpoint you download — community fine-tunes and abliterations inherit the base license, but re-bakers occasionally muddy it.
Which family abliterates cleanest for uncensored local use
This is the question that actually separates the families for a lot of MyLocalAI readers, and it inverts the “which refuses least out of the box” ranking. Abliteration is the orthogonalization technique that removes a model’s learned refusal direction from its residual stream — it’s not a jailbreak or a system-prompt trick, and done well it preserves almost all capability.
Here’s the honest field report from the 2026 abliteration scene:
- Qwen abliterates cleanest. The community has converged on Qwen as a favorite for uncensored builds — the dense 7B–32B models orthogonalize with minimal capability loss, and the supply of high-quality abliterated Qwen variants (often with companion-personality fine-tunes layered on top) is the deepest of any family. If you want one uncensored all-rounder, this is it.
- Gemma abliterates dramatically. Because stock Gemma is the most heavily aligned, removing the refusal direction produces the most striking before/after — the “Heretic”-style Gemma 4 abliterations are well-regarded and keep Gemma’s vision and efficiency intact. Great on unified memory.
- Mistral barely needs it. Mistral/Nemo is so lightly aligned that the abliterated versions are a modest step rather than a transformation — which is itself a reason the creative-writing crowd loves the base models.
- Llama abliterates fine but lags at the frontier. The home-runnable Llama 3.x abliterations (the 8B class especially) are mature and heavily pulled; the Llama 4 flagships are too big to be a practical local uncensored pick.
For exact model names, quant tags, and download pointers, the best uncensored local AI models roundup stays current. One non-negotiable caveat: abliterated models will not refuse genuinely harmful requests, so they’re for consenting-adult, lawful, private use — you own the responsibility the alignment used to carry.
Which to bet on by use case
Skip the leaderboard agonizing. Match the family to the job:
- One model for everything / coding / multilingual → Qwen. Apache 2.0, widest size range, cleanest uncensoring. The safest single bet in 2026.
- Creative writing & roleplay → Mistral / Nemo. Best prose voice, lightest alignment, huge fine-tune ecosystem.
- Mac / mini-PC / unified memory / vision → Gemma. RAM-efficient, MoE-fast, abliterates dramatically.
- Massive context on a real server → Llama 4. Only if the hardware (and the license terms) suit you.
- AI companion that’s private, uncensored, and yours → Qwen or Mistral, abliterated, run locally.
That last one is the use case this blog cares about most, and it’s worth being concrete. A companion model needs three things: it has to be uncensored enough to actually stay in character, light enough to run responsively on consumer VRAM, and fully local so your most personal conversations never touch someone else’s server. Qwen and Mistral abliterations hit all three — which is exactly the recipe Ember is built on: a sold-once, 100%-local AI companion that runs these open-weight families on your own machine through Ollama, with nothing logged to a cloud. If you’ve now decided which family to bet on and you want the companion experience without wiring it together yourself, that’s the path Ember was made for.