For years the rule for running big local models was simple and brutal: capacity equals VRAM, and VRAM is expensive. A 70B model meant two used 3090s and a space-heater build. AMD’s Ryzen AI Max+ 395, codename “Strix Halo”, breaks that rule in a 4-liter box that sips power. It pairs 16 Zen 5 cores with a big Radeon 8060S iGPU and up to 128GB of unified LPDDR5X memory, most of which you can hand to the GPU. That unlocks 70B and even 120B-class models on a machine that costs around $2,000 and stays quiet on your desk.
Here’s the honest position up front: Strix Halo is the capacity king and the speed compromise. It runs models that would need $4,000+ of NVIDIA glass, but it runs them slower. Whether that trade is genius or a trap depends entirely on which models you actually want to run. Let’s get specific.
What “unified memory” means for LLMs
On a normal PC, your GPU has its own dedicated VRAM (24GB on a 4090) that’s separate from system RAM. A model has to fit entirely in that VRAM to run fast; spill over and you fall off a performance cliff. Strix Halo has no separate VRAM. The CPU and GPU share one 256-bit LPDDR5X-8000 pool, and the iGPU can address most of it directly with no PCIe copy in the middle.
For LLM inference this is the whole ballgame. Model size is capped by the memory pool, not by a tiny VRAM budget. If you’ve read our explainer on RAM vs VRAM for local AI, Strix Halo is what happens when that distinction mostly dissolves. The catch is bandwidth: token generation speed is set by how fast you can stream the weights, and this pool moves at roughly 256 GB/s (measured closer to 215-240 GB/s), versus about 1,000 GB/s on a discrete 4090. Same capacity story, very different speed story.
How much of the 128GB is GPU-usable
You don’t get all 128GB for the model, the OS needs some. The allocatable share depends on your platform:
- Windows: set the iGPU’s dedicated allocation in BIOS/UEFI, typically up to ~96GB, leaving ~32GB for the system.
- Linux: the flexible path. With
amdgpuGTT tuning you can push the GPU’s reach much higher, people routinely reach ~110GB usable for a model, leaving enough for the OS.
A representative Linux kernel argument looks like this (verify current syntax for your kernel/driver):
amdgpu.gttsize=131072 ttm.pages_limit=33554432Either way, you have more usable AI memory than any single consumer GPU on the market. A 70B at Q4 (~40-45GB) leaves headroom for a big context window; see how much VRAM a 70B model really needs for the sizing math that now fits inside one box.
The speed reality: memory-bandwidth bound
This is where you need to be clear-eyed. Generation speed scales with bandwidth ÷ active model size. Because the pool is ~256 GB/s, big dense models are slow. Here’s the realistic picture (Q4-class quants, Linux + Vulkan, roughly, your mileage varies with quant, context, and backend):
| Model tier | Active params | Fits in pool? | Rough generation speed |
|---|---|---|---|
| 8B dense (Qwen3, Llama) | 8B | Easily | ~30-50 tok/s |
| 24B dense (Mistral Small 3.2) | 24B | Easily | ~12-18 tok/s |
| 27-32B dense (Qwen3 32B, Qwen3.6 27B, Gemma 4 31B) | 27-32B | Easily | ~8-12 tok/s |
| 70B dense (Llama 3.3 70B) | 70B | Yes | ~4-6 tok/s |
| 26-35B MoE (Qwen3-30B-A3B, Qwen3.6 35B-A3B, Gemma 4 26B-A4B) | ~3-4B | Easily | ~40-70+ tok/s |
| 120B MoE (gpt-oss-120b class) | ~5B | Yes | ~30-55 tok/s |
Read our guide on what tokens-per-second is actually usable to calibrate: ~5 tok/s on a 70B is fine for a reference query, painful for a long draft. And note the asterisk that reviews often bury, prompt processing (prefill) is Strix Halo’s weakest event. Feeding it a 20K-token document is much slower than on a discrete GPU, because prefill is compute-bound and the iGPU has far less raw compute than a 4090.
The contrarian insight: MoE flies, dense 70B crawls
Look at that table again. The 120B MoE model is 6-10x faster than the dense 70B, despite being bigger on disk. That’s not a typo, and it’s the single most important thing to understand before buying this machine.
Mixture-of-Experts models only activate a few billion parameters per token (gpt-oss-120b activates roughly 5B; Qwen3-30B-A3B about 3B). Generation speed depends on active params, not total size, but total size still has to fit in memory. Strix Halo’s giant pool is the perfect host: enormous capacity to hold the whole MoE, and the small active footprint sidesteps the bandwidth ceiling. Our MoE models on low VRAM explainer covers the mechanism. The practical takeaway: buy Strix Halo to run big MoE models, not to brute-force dense 70Bs. If your dream is fast dense-70B chat, a used-GPU rig may serve you better.
This is why the newest sparse releases are such a natural fit here: Qwen3.6’s 35B-A3B (~3B active) and Gemma 4’s 26B-A4B (~4B active) are exactly the kind of hold-it-all, fire-only-a-sliver models the big pool was made for, and both run at that top MoE speed tier. On the dense side, the pool can even hold a 128B open-weight model like Mistral Medium 3.5, but as a dense model it runs at 70B-class speeds or slower, so treat that as a capacity flex, not a fast daily driver.
ROCm, Vulkan, and the software reality
The hardware is ahead of the software, so set expectations. As of 2026:
- Vulkan (Mesa RADV) is often the most reliable and fastest llama.cpp backend for the Radeon 8060S (gfx1151), and it correctly addresses the full unified pool. Build with
-DGGML_VULKAN=ON. - ROCm 7 support has landed and keeps improving, and is the path you want if you need vLLM. It’s no longer the guaranteed-fastest option it once was on other AMD cards, check current benchmarks.
- Ollama works, but its bundled llama.cpp has at times lagged upstream Vulkan performance by a meaningful margin, if speed matters, a fresh upstream llama.cpp build can be noticeably faster.
Once configured, day-to-day use is normal:
ollama run gpt-oss:120b
ollama run qwen3:32bIf you’re new to AMD’s stack, our AMD GPU local LLM guide covers the ROCm-vs-Vulkan decision in more depth. Budget an afternoon for BIOS memory allocation and driver setup, this is not yet as plug-and-play as an NVIDIA box.
How it stacks up against the alternatives
| Option | Usable AI memory | Bandwidth | 70B dense | 120B MoE | Price ballpark |
|---|---|---|---|---|---|
| Strix Halo 395 (128GB) | ~96GB (Win) / ~110GB (Linux) | ~256 GB/s | ~4-6 tok/s | ~30-55 tok/s | ~$2,000 |
| Mac Studio (128GB, M-series) | ~100GB+ | ~400-820 GB/s | faster | faster | notably higher |
| RTX 4090 (24GB) | 24GB | ~1,000 GB/s | won’t fit (heavy offload) | won’t fit | ~$1,600 GPU only |
| 2× RTX 3090 (48GB) | 48GB | ~936 GB/s each | ~15-20 tok/s | too tight | ~$1,400 used + build |
The pattern is clear. A Mac Studio with comparable memory is the natural rival, same unified-memory idea, meaningfully more bandwidth, meaningfully more money. Discrete NVIDIA wins on raw speed per model but loses hard on capacity: no single consumer card holds a 70B, and multi-GPU builds are loud, hot, and power-hungry. Strix Halo’s pitch is maximum model size per dollar, in a silent small box, as long as you accept single-digit tok/s on the biggest dense models.
Should you buy one?
Buy Strix Halo (Framework Desktop, or a mini-PC like the GMKtec/HP/Asus units) if you want to run large MoE models, big-context workloads, or experiment across many model sizes without a GPU farm, and you value silence, low power, and a tiny footprint over raw throughput. Skip it if your daily driver is fast dense-70B chat or heavy long-document prefill; there, a discrete GPU still rules. For most people running mid-size models, a 24GB GPU box remains the sweet spot, Strix Halo is the specialist’s tool for going bigger than that on the cheap.
Running it privately, and the no-hardware detour
The best part: everything above runs 100% on your own machine, with nothing leaving your desk. On a 128GB Strix Halo box you have room for genuinely large models with long memory, fully offline, is local AI worth it makes the broader case for owning the stack.
And if the use case quietly driving this hardware shopping is an AI companion, one constraint is worth knowing before you buy: the packaged companions, Ember included, are CUDA-only, so a Strix Halo APU means assembling the stack yourself. Ember is local like the stack you’d build here, it just needs NVIDIA silicon to be local on. So if privacy-by-architecture is why you’re buying the box, this hardware still gets you there, you’ll assemble the companion layer yourself from the tools above.