Is Your Self-Hosted AI Actually Private? Audit Every Data Path
Tech Guru"I installed a model on my PC" is a useful start, not a privacy assessment. A local AI setup may include a desktop client, model server, search tool, image-description service, document index, browser integration, reverse proxy, and backup job. Any one of them can change where prompts, files, or outputs travel. The question is not whether the model file lives on your disk. The question is whether the complete task stays within boundaries you understand and can verify.
Quick Answer
Map the request from client to model and through every optional tool, then test each path separately. Confirm the exact local model ID, listen addresses, sidecar settings, outbound connections, logs, and backups. Block egress in a controlled test and verify that ordinary inference still works while web search and cloud models fail clearly. Do not call the system "offline" on the strength of a localhost URL or a single empty packet capture; account login, updates, plugins, and cached routes may behave differently.
Define What You Mean by Private
Before changing settings, pick the claim you need to support:
| Claim | What it requires | What it does not promise |
|---|---|---|
| Local inference | A downloaded model does the inference on hardware you control | No app telemetry, no hosted tool calls, no local logging |
| No prompt egress | Prompts and attached data do not reach an external endpoint during tested workflows | Nothing ever leaves on updates, sign-in, diagnostics, or other workflows |
| Offline operation | The tested workflow succeeds with external network access blocked | Secure local storage or protection from a compromised local client |
| Confidential team service | Authentication, network segmentation, retention, and backups match your policy | The model will obey every instruction or avoid prompt injection |
An apparently identical chat can take different paths depending on whether it uses text, an image, web search, a tool, a document, or a fallback model. For example, OpenCodex documents optional sidecars for web search and vision, including hosted backends. Ollama distinguishes local models from cloud models and offers a setting to disable its cloud features. LM Studio says downloaded models and local chat can operate offline, but enabling its API on the LAN changes who can send requests. Those are distinct product behaviors, not a single "self-hosted" switch.
Draw the Real Request Path
Record each component in a simple ledger before the test:
| Step | Process and endpoint | Data it can see | Owner and evidence |
|---|---|---|---|
| 1. Client | Desktop app, browser UI, extension, or script | Prompt, selected files, account identity | App version, settings, process connections |
| 2. Router/proxy | OpenCodex, reverse proxy, API gateway, or none | Prompt, headers, model choice, optional logs | Route table, provider list, request log |
| 3. Model server | Ollama or LM Studio on a loopback or LAN address | Prompt, tool definitions, retrieved context | Bind address, model ID, server logs |
| 4. Optional tools | Search, vision, RAG, MCP, external APIs | Queries, images, documents, tool results | Tool inventory and permission policy |
| 5. Persistence | Chat history, vector index, log store, backups | Prompts, outputs, embeddings, attachments | Retention, encryption, backup destination |
Do not assume a 127.0.0.1 client URL means every later hop is local. A proxy on loopback can call a remote API. Conversely, a LAN-hosted model on another machine can keep inference inside your network while still requiring strong authentication and firewall policy. If any row is unknown, your privacy claim is still unverified.
Step 1: Inventory the Effective Configuration
Do this on a test host or during an approved maintenance window. Save a redacted inventory outside the public article or code repository: operating system, client version, model-server version, model ID/digest, proxy version, active plugins, account state, listen addresses, and outbound firewall policy. Do not paste token-bearing configuration into a chat prompt.
For Ollama, these are safe read-only checks on macOS, Linux, or Windows where the CLI is installed:
ollama --version
ollama list
ollama ps
ollama list shows what is downloaded; ollama ps shows what is loaded right now. Neither command proves that the client selected the intended model. Cross-check the client and proxy request logs. A model tag ending in :cloud is a reason to stop and verify whether inference is hosted; do not infer location from the vendor name alone.
Ollama's FAQ says the server binds to 127.0.0.1:11434 by default and that OLLAMA_HOST changes the bind address. On macOS or Linux, lsof -nP -iTCP -sTCP:LISTEN can show listeners. On Linux, ss -ltnp is another option. On Windows PowerShell, use Get-NetTCPConnection -State Listen and inspect the owning process. Look specifically at model, proxy, and web UI ports, not just port 11434. A listener on 0.0.0.0 or :: is reachable on more interfaces than loopback unless other controls block it.
If you use LM Studio, check Serve on Local Network, Require Authentication, and any per-request MCP/tool permissions in its Server Settings. LM Studio documents that its API does not require authentication by default, and recommends authentication when serving beyond localhost. If you use OpenCodex, list every configured provider and combo/fallback route; its hosted-provider support means the selected destination matters more than the dashboard's presence on localhost.
Step 2: Remove Accidental Cloud Paths
Do not delete accounts or break production integrations as a first move. Create a test profile with one local model and no hosted fallback. Disable or remove optional web-search and vision sidecars for the test. A text-only local model should fail an image task or explicitly say it cannot process it, rather than silently passing the image to a cloud describer.
Ollama documents a local-only option. In its current FAQ, setting disable_ollama_cloud to true in ~/.ollama/server.json, or using OLLAMA_NO_CLOUD=1, disables cloud models and web search after restart. If you use a managed service, set the environment variable in that service's configuration and verify the effective state in logs. A setting in one shell does not necessarily reach a background process started elsewhere.
{
"disable_ollama_cloud": true
}
This controls Ollama's own cloud features; it does not disable remote calls made by another client, proxy, browser extension, MCP server, or update service. Do not use it as a substitute for inventory and network policy. Recheck the current FAQ before applying the setting because names and supported behavior can change.
Step 3: Run a Positive and Negative Test
Use a distinct, non-sensitive canary phrase such as TG-LOCAL-CANARY-2026-09-19 in a text-only question. It is an audit marker, not a secret. Record the exact time, model, client, and request ID if available. Ask a question that needs no web access. Confirm the selected model in the client/proxy and watch the local model process. Then repeat with external egress blocked for that host or test VLAN. The local inference path should still answer; a web-search request and a known cloud-model request should fail in a visible, documented way.
For a controlled network trace, capture metadata, not raw prompts, where possible: firewall connection logs, DNS logs, and per-process connection observations. A raw packet capture can include personal data and credentials; keep it private, time-bound, and access-controlled. A short sample capture is useful for finding a destination but cannot prove the absence of every future route. DNS-over-HTTPS, cached connections, IPv6, QUIC, container networking, and service workers can hide or change the path you thought you were watching.
The acceptance record should answer:
- Which process opened each external connection during the canary task?
- Which domains and addresses were contacted, and why?
- Did the expected local model load and produce the answer?
- Did a network block produce a clear failure for hosted features, or a silent fallback?
- Did any prompt, image, filename, retrieved document, or canary marker appear in a remote service log you control?
Do not expose application passwords, bearer tokens, full prompts, or raw packet contents when sharing audit screenshots. A useful sanitized record identifies the app version, selected provider, target endpoint, test timestamp, and redactions. This article describes the audit method; it does not report a completed TechGeeks egress test.
Step 4: Audit Data at Rest
Network traffic is only half the privacy story. Check where the client stores conversation history, what the proxy records, whether a RAG index keeps extracted text or embeddings, whether application logs include prompts, and where backups go. A locally stored prompt may be copied to a cloud backup without ever leaving during the model request. Set retention by data class, not by whichever default the installer chose.
For a team service, separate ordinary inference from model management, tool execution, and administrative settings. Back up configuration and necessary data, but protect backup encryption keys separately. Test restore with dummy content before claiming the backup is useful. Document how to revoke API tokens, disable a tool, stop new requests, and remove a compromised document from both its source store and any index. Deleting one chat row may not delete logs, embeddings, or snapshots.
Common Failure Patterns
- "The model is on my PC, so the app is offline." The app or proxy can still use a hosted account, sidecar, web search, update service, or remote MCP tool.
- "Nothing showed up in one packet capture." The feature may not have been exercised, a connection may have been cached, or you watched the wrong interface.
- "Only my LAN can reach it." A LAN is not one trust level. Guests, IoT devices, and compromised endpoints may reach a wide bind unless firewall and authentication rules prevent it.
- "No prompts are logged." Check client history, proxy logs, reverse proxy access logs, container logs, and backup destinations separately.
- "The model did not execute the malicious instruction." That is one observation, not proof that prompt injection or tool misuse is impossible. Keep tool permissions narrow.
What This Evidence Would and Would Not Prove
A successful egress-blocked text test would show that that exact workflow can complete without an external inference call at that time. It would not prove that sign-in, updates, image prompts, web search, voice, document import, other models, or future versions behave the same way. A firewall deny log would establish an attempted connection to a destination; it would not reveal whether prompt contents were transmitted inside encrypted traffic before the deny. A local server log would show the server received a request, not necessarily that it was the only recipient.
This article's featured image is editorial artwork, not a photograph of a tested environment or a configuration screenshot. To substantiate an observed result in your own setup, preserve a sanitized endpoint inventory, one positive and one negative route test, and a recorded restore step.
Related TechGeeks Resources
- AI Workflow Notes: Start Here for the broader local AI reading path.
- Local AI Homelab: Ollama and Open WebUI for the surrounding model-server build and hardware choices.
- How to Audit Your Exposed Services for a wider service-inventory method.
