If you are a lawyer asking whether you can use AI without breaching confidentiality, you are already asking the right question — and most of your colleagues aren’t. The honest answer is that the location of the model matters more than the brand on the box. A general-purpose cloud chatbot sends every prompt you type to a third party’s servers, where it is processed, often logged, and sometimes used to improve the product. A local AI runs entirely on a machine you control, so the privileged document never leaves your office network. This guide walks through the ethics calculus, what “local” actually guarantees, which models do real document work, and a private stack you can stand up in an afternoon.
The ethics problem: cloud AI vs. attorney-client privilege
Your duty of confidentiality (ABA Model Rule 1.6, and its state analogues) is not the same as evidentiary privilege, but both are at risk the moment client information leaves your custody. Rule 1.6(c) requires you to “make reasonable efforts to prevent the inadvertent or unauthorized disclosure of” client information. Rule 1.1’s competence duty now widely includes a technology-competence comment — you are expected to understand the tools you use.
Pasting a deposition transcript or a draft settlement agreement into a consumer chatbot is a disclosure to a third party. Whether it waives privilege is a fact-specific legal question courts are still working out, but you don’t want to be the test case. The risk isn’t hypothetical paranoia about the AI company reading your files; it’s the ordinary plumbing of cloud software:
- Server-side storage. Any cloud LLM necessarily receives and processes your text on its own infrastructure. That’s how the architecture works — the model isn’t on your laptop.
- Training and retention. Consumer tiers of major chatbots have historically defaulted to using conversations to improve models unless you opt out. OpenAI, for example, documents distinct retention and training behavior between its consumer and enterprise/API products in its published policies — which is exactly why the defaults matter.
- Subpoena and breach exposure. Data sitting on a vendor’s servers can be subpoenaed from the vendor, exposed in a vendor breach, or accessed by vendor staff under their terms — none of which is within your control.
This is the same structural problem we cover in why cloud AI censors you: once your data is on someone else’s computer, their rules — and their risk surface — apply to it.
What ‘local’ guarantees that a vendor DPA can’t
Law firms love a Data Processing Agreement. A DPA is a contract: the vendor promises not to train on your data, to encrypt it, to delete it on schedule. Contracts are valuable, but they are a promise about behavior, enforced after the fact, in court, after you’ve already discovered the breach.
Local AI is different in kind. It is an architectural guarantee, not a contractual one. When the model weights live on your workstation and inference happens on your own GPU, there is no network request to inspect, no retention setting to misconfigure, no sub-processor to audit. The confidential document is read off your local disk into your local RAM and never touches a wire.
| Cloud AI + DPA | Local AI | |
|---|---|---|
| Data leaves your network | Yes | No |
| Protection mechanism | Contractual promise | Physical/architectural |
| Breach exposure | Vendor’s servers + yours | Yours only |
| Subpoena-able from third party | Yes | No |
| Verifiable by you | Trust + audits | tcpdump shows zero egress |
You can literally prove it: run the model with your network monitor open and watch nothing leave on port 443. The loopback API that tools like Ollama expose lives at 127.0.0.1:11434 — that’s your own machine talking to itself. For the broader principle, see our AI data privacy guide, which lays out the trust-boundary thinking in plain terms.
Realistic use cases (with the human-in-the-loop caveat)
Local models are not as raw-capable as the frontier cloud systems, but for structured, document-grounded work they are genuinely useful today:
- Summarizing discovery. Feed a 60-page deposition and ask for a timeline, a list of admissions, or every reference to a specific date. A mid-size local model handles this well when the document is supplied as context rather than recalled from memory.
- First-draft drafting. Demand letters, routine correspondence, clause skeletons, deposition outlines. The model produces a scaffold; you supply the judgment.
- Contract review. “List every indemnification provision,” “flag auto-renewal clauses,” “compare these two NDAs and tell me where they diverge.” This is pattern-matching over text the model can see — a strength.
The non-negotiable rule: the AI is a junior associate who never gets the final word. Every output is reviewed by a licensed attorney before it touches a client matter. LLMs hallucinate citations — the sanctioned-lawyer headlines from fabricated case law all came from people who skipped review. A model that runs on your laptop hallucinates exactly as readily as one in the cloud; local solves confidentiality, not accuracy. Verify every cite against an authoritative source, always.
Model picks: Qwen 14B/32B-class for document work — and where they fall short
For legal document work on a single workstation, the Qwen 14B and 32B-class models are the current sweet spot. They have strong instruction-following, long-context handling, and competent reasoning, and they’re freely available as open weights. You’d pull one with a single command:
ollama run qwen3:32bSizing rules of thumb (real numbers, no hand-waving):
| Model class | Quantized footprint (Q4_K_M) | Practical VRAM | Good for |
|---|---|---|---|
| 14B | ~9 GB | 12–16 GB GPU | Summaries, drafting, fast turnaround |
| 32B | ~20 GB | 24 GB GPU (e.g. RTX 3090/4090) | Heavier review, longer documents |
The Q4_K_M tag is a quantization level — it compresses the weights so the model fits in consumer VRAM with minimal quality loss. If you’re choosing hardware, a used 24 GB card is the comfortable floor for the 32B class; see our qwen3-32b review and how to run AI locally for the full setup path.
Where they fall short, honestly: local 14B–32B models trail the best frontier cloud models on the hardest multi-step legal reasoning, novel-issue analysis, and very long (100k+ token) documents. They are summarizers and drafters, not oracles. If a matter turns on subtle reasoning, the local model gives you a first pass and you do the thinking. Anyone who tells you a 32B model on a desktop matches a top-tier frontier system on complex legal analysis is overselling.
A simple private stack: a local model + a document front-end
You don’t need to be an engineer. The minimum viable private stack is two pieces:
- The engine — Ollama. One-line install, runs the model, exposes a local-only API:
curl -fsSL https://ollama.com/install.sh | sh ollama pull qwen3:32b - A document front-end. A chat UI that lets you upload PDFs and ask questions against them. Pair it with retrieval (RAG) so the model answers from your documents rather than its training data — this is how you cut hallucination and keep answers grounded.
We have full walkthroughs for exactly this: chat with documents locally for the upload-and-ask workflow, and build a local RAG with Ollama for the grounded-retrieval version. Both keep every byte on your hardware. If you’re coming from a consumer chatbot and want a like-for-like swap, the private ChatGPT alternative with no data sharing writeup shows the equivalent local UI.
A reasonable firm setup: one workstation with a 24 GB GPU acting as the “AI server,” Ollama bound to localhost (or to your LAN behind the firewall, never the open internet), and the document UI on top. No subscription, no per-seat metering, no data egress.
The no-GPU option for solo practitioners
Local is the gold standard, but it has a real prerequisite: hardware. A solo practitioner on a three-year-old laptop with no discrete GPU can’t comfortably run a 32B model, and buying a 24 GB card plus standing up a server is a project, not an afternoon.
If the choice is “use a consumer chatbot that trains on inputs” versus “use nothing,” there’s a middle path: a hosted instance that does not sell or train on your data. The privacy gap between a no-train, no-log hosted service and a free consumer chatbot is enormous — the architecture is still cloud, but the data practices are the inverse. For a solo who needs something working today without buying a GPU, that’s a defensible interim step while you decide whether to invest in a local rig. The key questions to demand answers to in writing: Do you train on my inputs? Do you retain logs, and for how long? Who can access them? If the answers aren’t “no, no, and no one,” keep looking.
Adjacent professions: therapists and doctors face the same calculus
This isn’t a lawyers-only problem. The structure is identical for anyone bound by a confidentiality duty over third-party data:
- Therapists and counselors owe duties under state licensing rules and, where applicable, HIPAA. Session notes are some of the most sensitive text that exists.
- Physicians and clinicians handle Protected Health Information under HIPAA, where a cloud vendor processing PHI typically must be a Business Associate under a signed BAA — the medical cousin of the DPA, with the same “it’s a promise, not a wall” limitation.
For every one of these professions, the same conclusion holds: the only way to guarantee the data never leaves is to keep the compute local. A model running on the practice’s own machine sidesteps the entire Business-Associate question because there is no associate.
What not to do: limits and disclaimers every professional should keep
Local AI removes the confidentiality risk. It does not remove your professional judgment, and it adds a few obligations of its own:
- Never skip human review. Local doesn’t mean accurate. Verify every citation, figure, and legal conclusion against authoritative sources.
- Don’t expose the API to the internet. Keep Ollama on
127.0.0.1or behind your firewall. A local model reachable from the open web is no longer private. - Encrypt the disk. The documents and the model are now on your drive — full-disk encryption and physical security become your responsibility.
- Check your engagement letters and jurisdiction rules. Some clients or matters may require explicit disclosure of AI use; some courts now require certification. Know your local rules.
- This article is not legal or ethics advice. It’s a technical guide to keeping data off the cloud. Confirm your obligations with your bar association and your firm’s ethics counsel before deploying anything to live matters.
The bottom line: for confidential client work, architecture beats contracts. Running the model yourself is the only approach where confidentiality is a property of the system rather than a promise from a vendor.
If you have the hardware, go fully local with the stack above. If you’re a solo without a GPU and need a privacy-respecting option that works today — one that does not sell or train on what you type — a hosted, no-data instance is the pragmatic starting point while you weigh building your own rig.