Continue Dev Local Setup: A Private Copilot Alternative With Ollama

Continue is an open-source coding assistant for VS Code and JetBrains that does the same job as GitHub Copilot, chat in a side panel, inline tab-autocomplete, edit-in-place, except you choose the model behind it. Point it at a model running on your own machine and you get a Copilot-style experience with no subscription, no API key, and no copy of your code leaving 127.0.0.1. This guide walks through a working continue dev local setup end to end via Ollama: installing the extension, pulling the right two models (a capable one for chat, a small fast one for autocomplete), and writing the config.yaml that ties them together. It also gives you a straight answer on where this still loses to cloud Copilot, so you know what you’re trading.

The setup is genuinely simple once you understand one thing: Continue uses two different models for two different jobs, and they have very different requirements. Get that distinction right and everything else falls into place.

Why run Continue against a local model

Copilot and its cloud cousins are convenient, but every keystroke of context, the file you’re editing, the surrounding code, your prompt, is sent to a remote service to generate a suggestion. For an open-source side project, fine. For a private repo, a client’s codebase under NDA, or anything you’d rather not hand to a third party’s telemetry, it’s a real exposure. A local model collapses that to zero: the completion request never leaves your machine.

The second reason is cost and permanence. Continue itself is free and open-source, and a local model costs only electricity. There’s no monthly seat, no per-token meter, and nothing that can rate-limit you mid-session or change its terms. If you want the full cost-and-control comparison, we lay it out in local AI vs cloud AI. The honest trade-offs, raw capability and speed, are covered plainly at the end of this guide.

Step 1: Get Ollama running and pull two models

Continue doesn’t run models itself; it talks to a model server. The easiest local server is Ollama, which exposes an API on 127.0.0.1:11434 and handles downloads, quantization, and GPU offload for you. If you haven’t installed it, start with our how to install Ollama walkthrough, it’s a single installer on macOS, Linux, and Windows. (Ollama is the simplest on-ramp, but anything that speaks its API works; the Continue config is similar in spirit for other local backends.)

Now pull two models, because Continue’s chat and autocomplete have different needs:

# A capable coding model for chat / edit
ollama pull qwen2.5-coder:7b

# A small, fast code model for tab-autocomplete
ollama pull qwen2.5-coder:1.5b

Why two? Chat wants a smart model that reasons well over multiple turns, a few seconds of latency is acceptable because you’re reading the answer. Autocomplete is the opposite: it fires constantly as you type, so it must respond in a fraction of a second or you’ll have moved on before the suggestion lands. A big model is too slow for that loop; a tiny one is perfect. We’ll size both properly below, the model tags above are sensible starting points, not the only valid choices.

If you’re picking a chat coder in mid-2026, the newer families are worth a look. Qwen3-Coder is the current generation of the coder line and, usefully, ships fill-in-the-middle support in every size. Its qwen3-coder:30b is a 30B-A3B mixture-of-experts (only ~3B active per token, so it stays fast) that fits a 24GB card at 4-bit and handles agentic, repo-scale work with a 256K context; the larger qwen3-coder-next (80B total, ~3B active) is another sparse option if you have the memory. Poolside’s Laguna XS.2, a 33B-total / 3B-active open MoE under Apache 2.0, is a newer agentic-coding alternative that runs on modest hardware. Any of these can slot into the chat role in the config below in place of qwen2.5-coder:7b; the small qwen2.5-coder:1.5b is still a fine, fast pick for the autocomplete role, since the newest coder families start at sizes too large for a tiny always-on completion model.

Step 2: Install the Continue extension

Continue ships as an extension for both major editor families:

• VS Code, open the Extensions panel (Ctrl/Cmd+Shift+X), search for Continue, and install it. A Continue icon appears in the sidebar.
• JetBrains (IntelliJ, PyCharm, GoLand, etc.), open Settings → Plugins, search the Marketplace for Continue, and install.

After installing, you’ll see Continue’s chat panel in the sidebar. A fresh install may nudge you toward hosted models or an onboarding flow, ignore that. We’re going to wire it straight to Ollama with a config file, which is the part that matters and the part that’s changed recently.

Step 3: The config file, config.yaml, not config.json

This is the step where older tutorials will lead you astray, so read carefully. Continue migrated its configuration from config.json to config.yaml. The YAML format is now the preferred one, and if both files exist, Continue loads config.yaml. Plenty of guides still floating around show the old JSON models / tabAutocompleteModel structure, that approach is legacy. Use YAML.

The config lives in Continue’s global directory:

• macOS / Linux: ~/.continue/config.yaml
• Windows: %USERPROFILE%\.continue\config.yaml

The single biggest conceptual change in the YAML format is roles. Instead of separate top-level fields for the chat model and the autocomplete model, every model is now listed once under a unified models: array, and you tag each one with the roles it should fill, chat, autocomplete, edit, apply, embed, rerank, and so on. One model can hold several roles; a model with no matching role simply won’t be used for that job.

Here’s a minimal working config that wires both of the models we pulled:

name: Local Assistant
version: 0.0.1
schema: v1
models:
  - name: Qwen2.5 Coder 7B (chat)
    provider: ollama
    model: qwen2.5-coder:7b
    apiBase: http://localhost:11434
    roles:
      - chat
      - edit
      - apply

  - name: Qwen2.5 Coder 1.5B (autocomplete)
    provider: ollama
    model: qwen2.5-coder:1.5b
    apiBase: http://localhost:11434
    roles:
      - autocomplete

A few things to note about that structure. The provider: ollama line is what tells Continue to talk to your local Ollama server. The model: value is the exact tag you pulled, copy it verbatim from ollama list. The apiBase defaults to http://localhost:11434, so you can usually omit it for a standard local install; include it if your Ollama runs on a different host or port. Save the file and Continue picks up the change, the chat panel’s model dropdown should now show your local chat model, and autocomplete starts firing as you type.

The exact schema keys evolve, so if a field doesn’t behave as expected, check the current config.yaml reference in Continue’s docs rather than assuming a key from an old blog post. That’s a general rule for fast-moving tools, not a knock on Continue.

Why autocomplete needs a code model specifically

Here’s a trap worth flagging. You might be tempted to use one good general chat model for everything. Don’t, for autocomplete. Inline completion relies on fill-in-the-middle (FIM), the model is given the code before and after your cursor and asked to fill the gap. Generic chat models (a plain Llama, a Mistral instruct) generally don’t understand FIM tokens and will produce garbage or nothing for tab-completion.

Use a code-specialized model that ships FIM support. The common choices are families like Qwen2.5-Coder (and the newer Qwen3-Coder, which keeps FIM support across the line), StarCoder2, and DeepSeek-Coder, in their small sizes (roughly the 1.5B-3B range) so they stay fast. This is why the config above uses a separate tiny coder model for the autocomplete role and a larger one for chat, they’re solving different problems and FIM support is non-negotiable on the autocomplete side.

If you want to tune the completion behavior, Continue exposes an autocompleteOptions block on a model with the autocomplete role. The available knobs have included things like debounceDelay (milliseconds to wait before firing), maxPromptTokens (how much surrounding code to send), and modelTimeout. Treat the exact key names as docs-checkable rather than memorized, but the idea is that you can throttle how aggressively completion triggers if it feels noisy:

  - name: Qwen2.5 Coder 1.5B (autocomplete)
    provider: ollama
    model: qwen2.5-coder:1.5b
    roles:
      - autocomplete
    autocompleteOptions:
      debounceDelay: 300
      maxPromptTokens: 1024

Don’t forget the chat model’s context window

One Ollama-specific gotcha bites Continue users the same way it bites every local-coding setup: the context window. Ollama has historically defaulted to a small context (on the order of a couple thousand tokens). For tab-autocomplete that’s mostly fine, the prompt is small. But for chat, where you’re asking the model to reason over a file or a pasted error plus several turns of conversation, a tiny window means the model silently forgets the earlier context. It looks like the model is dim; it’s really just half-blind.

You raise this on the Ollama side, not in Continue. The cleanest lever is the server-side default when you launch Ollama:

OLLAMA_CONTEXT_LENGTH=16384 ollama serve

The caveat to state plainly: context isn’t free. A bigger window consumes more VRAM on top of the model weights, and pushing it too high can spill into system RAM and tank your speed or trigger an out-of-memory error. Pick a window your card can actually back. If you want the memory math and how to find your ceiling without guesswork, our deep dive on how to increase the Ollama context window walks through it.

Sizing the models to your hardware

The autocomplete model should stay small and fast on essentially any GPU, that’s the whole point of it. The chat model is where your VRAM budget actually matters, because that’s the one doing real reasoning. A rough, honest mapping:

Your VRAM	Realistic chat model class	Autocomplete model	What chat feels like
8GB	~7B coder	~1.5B coder	Single-file help, explanations, small functions. Tight on context.
12-16GB	14B-class coder	~1.5-3B coder	Reliable single-file refactors and tests; comfortable headroom.
24GB	32B-class coder	~1.5-3B coder	The first tier where chat feels like a genuine pair, with room for context.

These are model classes, not a leaderboard, treat them as a starting point and test on a real task from your own repo. For the full breakdown of which coding model fits which card, including quantization trade-offs, see best local coding model by VRAM tier; if you’re on a 24GB card and want the most out of it, best local LLM for 24GB VRAM covers the sweet-spot setups.

A blunt note on quantization: heavily compressed models produce noticeably worse code than the same model at a higher precision. If your chat answers keep coming back subtly wrong or your edits are malformed, the model may be too aggressively quantized for the job, not too small. Try a less-compressed variant before giving up on a model.

Speed: what “usable” actually feels like

Two speed regimes matter here, and they’re different. Autocomplete has to be effectively instant, if the suggestion doesn’t appear in well under a second, you’ve already typed past it. That’s exactly why it runs on a tiny model; a small coder on any modern GPU clears that bar comfortably. If your completions feel laggy, your autocomplete model is too big, not too small.

Chat is the part you’ll judge by tokens per second as the answer streams in. For interactive back-and-forth, anything in the rough ballpark of 20-40 tokens/second feels fine, comparable to reading speed. Drop much below ~10 tok/s and a long explanation becomes a coffee-break wait. The variables are model size, quantization, and how full the context window is. We get into what counts as comfortable in tokens per second: what’s actually usable. Treat any specific number as a band, not a promise, your hardware and quantization decide where you land.

Honest expectations vs cloud Copilot

You deserve a straight answer rather than a sales pitch. A local Continue setup is genuinely good at the daily 80%: tab-completing the line you’re obviously about to write, explaining an error, scaffolding a function, writing a unit test, translating a snippet between languages, doing a scoped single-file refactor. For that work it’s fast enough, free per token, and completely private.

Where local still loses to cloud Copilot:

• Raw completion quality on the hardest lines. Copilot’s models are large and heavily tuned for FIM; a 1.5-3B local model is good, not state-of-the-art. Expect slightly more “almost right” suggestions you’ll tweak.
• Whole-repo and many-file reasoning. The biggest frontier models hold more of a large codebase in their head and reason across it better. A “understand this sprawling legacy system and re-architect it” task still favors cloud.
• Polish and ecosystem features. Cloud assistants ship agentic, multi-step features that lean on huge context windows and infrastructure no single consumer GPU can match today.

The honest framing: a local Continue setup isn’t a strictly-better replacement for cloud Copilot, it’s a different trade. You give up some ceiling on the hardest tasks and some completion polish, and in exchange you get privacy by construction, zero marginal cost, and an assistant that never rate-limits you or changes its terms mid-project. For a developer who simply doesn’t want their codebase leaving their machine, that trade is usually worth making, and nothing stops you from keeping a cloud key configured in Continue for the rare task that genuinely needs it.

Quick reference

The whole local setup, start to finish:

# 1. pull two models in Ollama
ollama pull qwen2.5-coder:7b      # chat / edit
ollama pull qwen2.5-coder:1.5b    # fast autocomplete

# 2. (optional) raise chat context on the Ollama server
OLLAMA_CONTEXT_LENGTH=16384 ollama serve

Then install the Continue extension in VS Code or your JetBrains IDE, and drop a config.yaml in ~/.continue/ listing both models under models:, the big one tagged with the chat/edit roles and the small one with the autocomplete role. Get those pieces right, the YAML format, the roles split, a FIM-capable code model for autocomplete, a chat model your VRAM can hold, and a sane context window, and you have a private, offline Copilot alternative that never sends a line of your code to anyone. Start with models your hardware holds comfortably, prove them on a real task from your own repo, and scale up only when you hit a wall.