Text Generation WebUI, almost everyone calls it oobabooga, after the developer’s username, is a browser-based control panel for running large language models on your own hardware. Where a tool like Ollama hides almost everything behind one command, oobabooga puts the whole machine on the dashboard: which backend loads the model, how each quantization is split between GPU and RAM, every sampler the model exposes, and a stack of extensions for voice, vision, and retrieval. If you’ve outgrown the “it just works” tools and want to actually steer local inference, this is the UI people point you to. This guide covers what it is, who should bother, how to install it without fighting Python, how to load GGUF and EXL2 models, the three working modes, extensions, and the honest trade-off versus simpler apps like Ollama.
What Text Generation WebUI actually is
At its core, oobabooga is a Gradio web interface that sits on top of several different inference backends. It doesn’t have its own way of running models, instead it wraps the big ones (llama.cpp, ExLlamaV2, Hugging Face Transformers, and others) behind a single UI and a consistent set of controls. You pick a model file, pick a loader, tune the parameters, and chat. Because it’s a web UI, it runs locally and you reach it in your browser at an address like http://127.0.0.1:7860, nothing leaves your machine.
A few things make it distinct from the simpler crowd:
- It speaks many model formats, not just one. GGUF, EXL2, plain Transformers weights, and several quantized formats all load through the same interface.
- It exposes the full parameter surface, temperature, top-p, top-k, repetition penalty, and modern anti-repetition samplers, instead of hiding everything behind defaults.
- It ships an OpenAI-compatible API, so other apps and scripts can talk to it as if it were OpenAI.
- It has a real extension system for text-to-speech, speech-to-text, vision, retrieval-augmented generation, and more.
Worth noting: the project has been actively rebranded and reorganized recently (the repository now also presents itself as a “desktop app for local LLMs” with vision and tool-calling), so some menu names and packaging details shift between versions. Treat the specifics below as the stable shape of the tool, and check the in-app labels against whatever build you download.
Who it’s for (and who should skip it)
Be honest with yourself here. Oobabooga rewards people who want control and tolerate complexity. It frustrates people who just want to talk to a model.
You’ll like it if you:
- Want to try the same model at different quantizations and loaders and feel the difference.
- Care about samplers, you’ve hit repetition loops and want DRY, top-a, or min-p to fix them.
- Run roleplay or writing workflows that need a notebook/raw-completion mode, not just turn-based chat.
- Want one local backend that can serve an OpenAI-compatible API to your other tools.
- Are curious about light fine-tuning (LoRA/QLoRA) without leaving the UI.
You should probably skip it if you:
- Just want a chat window and a model that works. Ollama or LM Studio will get you there in minutes with far less to learn, see Ollama vs LM Studio vs Jan for the easy-mode options.
- Are completely new to local AI. Start with the fundamentals in local AI for beginners, then come back when “loader” and “quantization” mean something to you.
Oobabooga is a power tool. The payoff is real, but so is the learning curve.
Installing oobabooga (the install that doesn’t fight you)
For years the reputation of “oobabooga setup” was painful, manual Python environments, CUDA mismatches, the works. That’s no longer the situation you have to accept. There are two sane paths today.
Option 1: the start scripts (one-click installers)
The maintained installer approach is a set of start scripts you download with the repository, typically named along the lines of start_windows.bat, start_linux.sh, start_macos.sh, and a WSL variant. You download the latest release, unzip it, and run the script for your OS. The script downloads a self-contained Miniconda, creates an isolated Python environment inside that folder, and installs everything for you. On first run it usually asks what hardware you have (NVIDIA, AMD, Apple Silicon, or CPU) so it grabs the right build.
The important part: because the environment lives in the folder, it doesn’t pollute your system Python and you can delete the whole directory to uninstall cleanly. After the first install, a matching update script pulls the latest code and refreshes dependencies.
Option 2: the portable build (easiest, llama.cpp only)
More recently the project added portable packages, a single zip per platform (on the order of a few hundred megabytes) that you unzip and run, with no installation step at all. These bundle a portable Python plus the llama.cpp server, so they’re optimized for GGUF models and launch fast out of the box. If your whole plan is “run GGUF files with a nice UI,” this is the lowest-friction route. The trade-off is that the portable build is narrower than the full installer, it’s built around llama.cpp rather than the full set of loaders.
Whichever path you take, when it finishes it prints a local URL (commonly http://127.0.0.1:7860). Open that in your browser and you’re in. If you’ve ever installed Ollama, think of this as the same idea with a graphical front and a lot more dials.
A practical note: download the right hardware variant. Grabbing a CPU build on an NVIDIA box is the most common reason a fresh install runs slowly or refuses to load a model.
Loading models: GGUF vs EXL2 (and where files go)
This is where oobabooga’s “many formats” nature shows up, and where newcomers get confused. The two formats most people use locally are GGUF and the ExLlama family (you’ll see EXL2 and the newer EXL3 quants), and they’re handled differently.
| GGUF (llama.cpp) | EXL2 / EXL3 (ExLlama) | |
|---|---|---|
| What it is | Single-file quantized format, CPU+GPU | GPU-first quantized format |
| Best when | You want to split between VRAM and system RAM, or run partly on CPU | The whole model fits in VRAM and you want speed |
| File layout | One .gguf file | A whole folder of files |
| Where it goes | Drop the file into models/ | Put the entire repo folder into models/ |
| Typical loader | llama.cpp | ExLlamaV3 (older builds: ExLlamaV2) |
The mechanics in the UI:
- Download a model. Either paste a Hugging Face repo path into the Model tab’s downloader, or place files into the
modelsdirectory yourself. Stick to trustworthy uploads, our guide to finding safe GGUF models on Hugging Face covers what to look for. - Pick the loader. GGUF files use the llama.cpp loader; ExLlama-format folders use ExLlamaV3 in current builds (older builds shipped ExLlamaV2 for EXL2). Oobabooga usually auto-detects a sensible default, but you can override it.
- Set the key knobs. For GGUF the big one is how many layers to offload to the GPU (often labeled something like
n-gpu-layers/ “GPU layers”), more layers in VRAM means faster generation until you run out of VRAM and spill into system RAM, where speed collapses. Set the context length here too. - Click Load.
If quantization names like Q4_K_M or “4.0bpw” are a blur, read the GGUF quantization cheat sheet first, picking the right quant for your VRAM is the difference between a snappy model and a crawling one. And if you’re matching a model to a specific card, the best local LLMs for 24GB VRAM breakdown shows how EXL2 quants and context budgets actually fit.
Beyond those two, oobabooga can also load full-precision Transformers weights and several other quantized formats. The exact roster of loaders has changed across versions, some older ones have been retired, so trust the loader dropdown in your build over any fixed list.
Chat, Notebook, and Instruct: the three ways to generate
One of oobabooga’s genuine strengths is that it doesn’t force everything into a chat bubble. There are effectively three working surfaces.
- Chat tab. The familiar turn-based conversation, with character cards, avatars, and saved chat histories. Inside it you choose a Mode:
- chat, pure conversational roleplay, driven by a character’s context rather than a rigid instruction template.
- instruct, uses the model’s instruction template (the format it was trained to follow) for assistant-style Q&A and tasks.
- chat-instruct, a clever hybrid that wraps the chat prompt inside a single instruction-following turn, which often makes instruct-tuned models behave better in a roleplay frame.
- Notebook tab. A free-form scratchpad. You type text and the model continues it, no roles, no turns, no template. This raw-completion mode is invaluable for creative writing, prompt experiments, and getting a model to do things turn-based chat won’t let it.
- Default tab. A simpler two-box prompt-and-output view for one-shot generation.
The matching Parameters tab is where you actually tune behavior: sampler settings live here, and an Instruction template sub-tab lets you set a custom system message and stopping strings used by the instruct modes. If a model keeps breaking character or rambling past where it should stop, this tab is usually the fix, not a different model.
Extensions: voice, vision, RAG, and an API
Out of the box oobabooga is text-in, text-out, but its extension system is where it stretches into a full local-AI workstation. A few of the built-in ones worth knowing:
- openai, exposes an OpenAI-compatible API so other apps, scripts, and front-ends can hit your local model as if it were the OpenAI endpoint. This is how you wire oobabooga into editors, agents, or a separate chat UI.
- coqui_tts / silero_tts, text-to-speech, so your model can talk back.
- whisper_stt, speech-to-text, so you can dictate your messages with a microphone.
- superboogav2, retrieval-augmented generation, letting the model pull from your own documents (PDF, DOCX, and similar).
- multimodal / vision, feed images to vision-capable models.
There’s also built-in support for light LoRA/QLoRA fine-tuning via a training tab, so you can nudge a model on your own data without a separate toolchain. Extensions are toggled in the UI (or enabled at launch), and there’s a community ecosystem beyond the bundled set.
That OpenAI-compatible API is the quietly important feature. It means oobabooga can be your backend while you use whatever front-end you prefer, much like people pair a separate UI with Ollama in a SillyTavern + Ollama setup. One engine, many faces.
Oobabooga vs Ollama: which should you run?
This is the comparison most people are really asking about, so let’s be direct.
| Text Generation WebUI (oobabooga) | Ollama | |
|---|---|---|
| Setup effort | Moderate, installer or portable build | Very low, one command |
| Model formats | GGUF, EXL2, Transformers, more | GGUF (via its own model registry) |
| Control over samplers | Extensive, exposed in UI | Minimal by default |
| Interface | Full graphical web UI | CLI first; UI via third-party apps |
| Extensions | TTS, STT, vision, RAG, training | None built in; you bolt on tools |
| API | OpenAI-compatible | Its own API (plus an OpenAI-compatible layer) |
| Best for | Tinkering, control, experimentation | Speed, simplicity, scripting, “just run it” |
Neither is “better” in the abstract, they’re built for different temperaments. Ollama wins when you want a model running in 60 seconds and you’ll talk to it through some other app; it’s clean, scriptable, and almost impossible to get wrong. Oobabooga wins when the how matters, when you want to A/B two quants, rescue a repetitive model with better samplers, run a raw notebook, or stack voice and RAG on top of a model you chose deliberately.
A reasonable path for many people: start on Ollama to learn the landscape, and move to oobabooga the first time you wish you could change something the simpler tool won’t let you touch.
Honest limitations
- It’s heavier. More moving parts means more that can break, a wrong hardware build, a loader mismatch, an extension dependency. Budget time for the first session.
- The UI is busy. Gradio is functional, not beautiful, and the sheer number of options is intimidating at first. That’s the cost of control.
- Specifics drift between versions. Loaders get added and removed, and the project has been reshaping its packaging. Don’t assume a year-old tutorial’s button names still match.
- It won’t make a small model smart. All the samplers in the world won’t fix a model that’s too quantized or too small for the task. Hardware and model choice still dominate the experience.
The bottom line
Text Generation WebUI is the most flexible local-LLM interface most people will ever need: one UI that loads GGUF, EXL2, and Transformers models, exposes every sampler, offers chat, notebook, and instruct workflows, ships voice/vision/RAG extensions, and serves an OpenAI-compatible API. The price is a steeper start than the one-command tools. If you want real control over local inference and you’ll spend an afternoon learning the dashboard, the modern installers make oobabooga genuinely approachable, and once it clicks, it’s hard to go back to a UI that hides the dials.