Model Routing: Different Models for Different Jobs

The short answer: start with manual model selection, measure each candidate on your real tasks, and automate only stable routes with an obvious override and fallback. Model family names are examples, not permanent winners.

This guide is for an Ollama and Open WebUI operator who has several local models available and wants predictable choices without wasting memory or sending sensitive prompts to the wrong provider. The successful outcome is a small routing policy you can explain, test, override, and roll back.

That idea is called model routing.

Model routing means sending each prompt to the model that fits the job.

Think of it like tools in a garage:

• You do not use a sledgehammer to tighten a tiny screw.
• You do not use a tiny screwdriver to break concrete.
• You pick the tool that matches the task.

A suitable model can be faster, lighter on the GPU, or more reliable for a defined task. You must measure that fit; a label such as "coder" or "reasoning" is not evidence by itself.

Prerequisites and Decision Criteria

• Record the exact Ollama model tag or digest, quantization, context setting, and installed Ollama/Open WebUI versions.
• Choose representative prompts and a human-checkable scoring rubric for each task class.
• Set a latency and memory budget for cold starts, warm responses, and concurrent users.
• Define which data classes must remain local and which providers, tools, and logs each route may use.
• Keep one known-good default model and a manual model picker available during rollout.

Product names and interfaces in this article were checked on July 15, 2026. Model catalogs move quickly: reopen the official model cards, licenses, Open WebUI extension docs, and Ollama library entry before publishing or changing a production route.

Start with Manual Routes

Interactive Model Routing Flow

Click each step to see what it means in a beginner-friendly local AI setup.

1Classify Task2Pick Model3Route Request4Monitor Cost5Improve

Select a step

Choose a box above to view details.

ChatGemma

CodeQwen Coder

ReasonNemotron

SummarySmall fast model

If you are new, start with manual routing:

1. Open Open WebUI.
2. Pick the model from the model dropdown.
3. Ask the question.
4. Switch models when the task changes.

That is already model routing. You do not need automation on day one.

Later, you can create named Open WebUI workspace-model presets or write a Pipe Function that presents custom routing logic as a model. That Pipe is server-side Python code and must be reviewed and trusted like any other privileged plugin.

Why Not Use One Big Model for Everything?

You can, but it wastes resources.

Big models are useful, but they usually need more:

• VRAM.
• RAM.
• Disk space.
• Electricity.
• Waiting time.

For a simple "summarize this paragraph" job, a smaller model may answer quickly and well. For "debug this weird Python error across five files," a coding model is probably better. For "help me design a migration plan," a reasoning model may be worth the wait.

The goal is not to collect the most models. The goal is to build a useful local AI toolbox.

A Simple Routing Table

Use this as a candidate matrix to test, not a claim that one family wins every task:

Job	Candidate type	Example family to evaluate	Acceptance focus
Everyday chat	General instruction model	Gemma or another local general model	Correctness, tone, latency, and refusal behavior on normal prompts
Code help	Code-focused model	Qwen Coder or another code model	Tests pass; no invented APIs, unsafe commands, or missing edge cases
Planning	Reasoning-focused model	Nemotron or another reasoning model	Assumptions, constraints, rollback, and trade-offs are explicit
Fast summaries	Small instruction model	A small Gemma, Qwen, Phi, or similar model	Key facts retained with no unsupported additions
High-impact synthesis	Best validated model that fits the policy	Your tested local candidate	Quality gain justifies latency and memory; human review remains required

Your exact model names will depend on what Ollama and your hardware support. The pattern matters more than the exact tag.

Candidate Route 1: Gemma for Normal Chat

Gemma is one general-model family worth evaluating for everyday chat. Keep it only if the specific release and quantization pass your prompt set on your hardware.

Good chat prompts include:

• "Explain this like I am new."
• "Help me write a polite email."
• "What does this error message probably mean?"
• "Give me dinner ideas from these ingredients."
• "Quiz me on networking basics."

The Gemma family is built for general generation tasks such as question answering, summarization, and reasoning. That makes it a good default assistant in a local AI setup.

A Conservative Chat Preset

In Open WebUI, name the model something obvious:

Chat - Gemma

Give it a simple system prompt:

You are a helpful local AI assistant. Explain clearly, ask follow-up questions only when needed, and keep answers beginner-friendly.

This gives non-technical users a safe default. They do not need to know about parameters, context windows, or VRAM.

Candidate Route 2: Qwen Coder for Code

Qwen Coder is one code-focused family worth evaluating when the task involves programming. Run generated code in a disposable test environment and compare it with at least one general-model baseline.

Good coding prompts include:

• "Explain this function."
• "Find the bug in this snippet."
• "Write a small Bash script."
• "Convert this Python code to JavaScript."
• "Review this error log and suggest the likely cause."
• "Help me understand this Docker Compose file."

Qwen Coder models are designed for code generation, code reasoning, and code fixing, so they are sensible coding candidates. That design intent is not a result on your repository; compare a coder model with your general model using the same tasks and tests.

Protect Secrets and Repositories

Never paste secrets into a coding model.

Before you paste code, remove:

• API keys.
• Passwords.
• SSH private keys.
• .env values.
• Customer data.
• Private company code you are not allowed to upload or process.

Local is better for privacy than a random cloud chat, but your local AI stack can still store chat history, logs, and uploaded files.

A Code Preset That Limits Risk

Name the model:

Code - Qwen Coder

Use a system prompt like:

You are a careful coding assistant. Explain changes in beginner-friendly language. Prefer small, testable examples. Warn when a command could delete data or expose secrets.

Candidate Route 3: A Reasoning Model for Planning

A reasoning-focused model such as a suitable Nemotron release may be a candidate for planning and trade-offs. Longer visible reasoning is not proof of a better plan, so score assumptions, dependencies, failure modes, and recovery steps directly.

Good planning prompts include:

• "Plan my homelab backup strategy."
• "Compare these two network layouts."
• "Break this project into beginner steps."
• "What can go wrong with this setup?"
• "Help me migrate from one server to another."

NVIDIA's Nemotron family is aimed at agentic and reasoning-heavy AI use cases. In a homelab, that makes it a useful choice for planning, analysis, and complex decision support if your hardware can run the model you choose.

Keep Planning Separate from Execution

A reasoning model may produce a better plan, but you should still review it.

Use it to think through:

• Order of operations.
• Risks.
• Dependencies.
• Rollback steps.
• Questions you forgot to ask.

Do not blindly run commands from any model. Read them first.

A Planning Preset for Review

Name the model:

Plan - Nemotron

Use a system prompt like:

You are a planning assistant. Break work into safe steps, call out risks, and include a rollback idea when changes affect servers, networks, files, or accounts.

Candidate Route 4: A Small Model for Summaries

A small model may reduce latency and memory use, but confirm it preserves names, numbers, qualifications, and negative findings from your source text.

Use a small summary model for:

• Chat title generation.
• Short document summaries.
• Meeting-note cleanup.
• "What did we decide?" summaries.
• Compressing old chat history.
• Turning long text into bullet points.

A small model may not be the best deep thinker, but it does not need to be. If the job is "make this shorter," speed matters.

A Low-Latency Summary Preset

Name the model:

Summarize - Small

Use a system prompt like:

You summarize text clearly. Preserve names, dates, decisions, and action items. Do not invent details that are not in the source.

This is a great place to use a smaller Gemma, Qwen, Phi, or similar model that runs quickly on your hardware.

Candidate Route 5: A Validated Model for Hard Tasks

Do not route high-impact work to a model merely because it is the largest one that loads. Use the candidate that performs best on the defined task while meeting privacy, latency, and recovery requirements.

Hard tasks include:

• Reviewing a long plan.
• Combining several documents.
• Debugging a messy issue.
• Writing a careful final draft.
• Comparing multiple options.
• Handling vague prompts where the model needs more judgment.

The large model is not your default hammer. It is your "bring this out when the easy route is not enough" option.

Reserve Deep Work for Hard Cases

Name it:

Deep Work - Large

Use a system prompt like:

You handle difficult tasks carefully. State assumptions, ask for missing critical information, and prefer accuracy over speed.

Manual Routing in Open WebUI

Manual routing is the easiest way to start.

Create or pull a few models in Ollama, then select the model you want in Open WebUI before you ask the question.

Your model list might look like:

Chat - Gemma
Code - Qwen Coder
Plan - Nemotron
Summarize - Small
Deep Work - Large

This naming style helps beginners. Nobody has to remember which tag is for coding or which model is bigger.

Open WebUI Workspace-Model Presets

Open WebUI's Models workspace lets an operator create a preset over a base model with a friendly name, system prompt, parameters, knowledge, tools, and access controls. A preset is a wrapper, not a second independent copy of the model. Current documentation says a user needs access to both the workspace model and its underlying base model.

That means one base model can become a friendly preset.

Example:

Base model: qwen3-coder:30b
Friendly name: Code - Qwen Coder
Purpose: coding help
Access: users who need coding support
System prompt: careful coding assistant

The qwen3-coder:30b tag is a concrete current example, not a default recommendation. Ollama listed that artifact at about 19 GB when checked July 15, 2026, before runtime context and other allocations; confirm the current tag, license, total memory need, and task score before choosing it.

This matters when multiple people use the same AI server. You can make the interface easier and reduce mistakes.

Automatic Routing in Open WebUI

Once manual routing feels comfortable, you can automate some of it.

Current Open WebUI documentation describes Pipe Functions as server-side extensions that register as selectable models and can proxy or route requests. A Pipe can implement model selection, fallbacks, timeouts, and policy checks, but Open WebUI does not turn a keyword table into a trustworthy router automatically.

Security boundary: Functions execute arbitrary Python with the Open WebUI backend's access. Review the full source, pin dependencies, restrict outbound access and credentials, and test with a normal account before enabling a community Function. A malicious or buggy router can expose prompts, files, environment variables, and provider keys.

In beginner terms:

User asks a question
        |
        v
Router checks the prompt
        |
        +--> code words? send to Qwen Coder
        |
        +--> planning words? send to Nemotron
        |
        +--> summarize request? send to small model
        |
        +--> otherwise send to Gemma

Start with Auditable Routing Rules

A keyword router can be a transparent diagnostic baseline:

If the prompt includes...	Route to...
code, bug, function, traceback, Dockerfile, script	Code - Qwen Coder
plan, architecture, migration, tradeoffs, risk	Plan - Nemotron
summarize, shorten, title, bullet points	Summarize - Small
hard mode, review deeply, compare all options	Deep Work - Large
Anything else	Chat - Gemma

This table is intentionally brittle: it demonstrates why you need an override, a confidence threshold, an unknown route, a timeout, and a tested fallback. Users should be able to see which rule fired and manually choose a model when the router guesses wrong.

Do Not Auto-Route Sensitive Data to Cloud Models

This series focuses on local AI, but many tools can mix local and cloud providers.

If your Open WebUI setup includes cloud models, label them clearly.

Use names like:

Cloud - Large Reasoning
Local - Gemma Chat
Local - Qwen Coder

That way users know when data may leave the homelab.

For private documents, code, family data, school records, client files, or anything under an agreement, prefer local-only routing unless you have permission and understand the risk.

Watch Out for "Smart" Routing Mistakes

Automatic routing can make bad choices.

Examples:

• A user asks a normal question that includes the word "scripture," and the router sees "script" and sends it to the coding model.
• A user asks for a "plan for dinner," and the router sends it to the heavy reasoning model.
• A private code prompt gets routed to a cloud coding model because the names were unclear.
• A tiny summary goes to the largest model and wastes time.

Keep the first router simple. Add logging for which model was selected, but do not log private prompt text unless you really need it.

Hardware Reality Check

Your router is only useful if the models actually run well.

Before creating a complicated setup, test:

• How long each model takes to answer.
• How much VRAM it uses.
• Whether it fits alongside other running services.
• Whether multiple users can use it at the same time.
• Whether the answers are good enough for the job.

If a model takes five minutes to answer a normal question, users will stop using it. In many homelabs, a smaller fast model is more useful than a giant model that technically runs but feels stuck.

Evidence and Routing Test Method

This article is documentation-backed and supplies a test method; TechGeeks did not run the named model families head-to-head for this draft. Vendor model cards describe intended capabilities, while the independent RouteLLM research supports the broader idea that routing trades model capability against cost. Neither establishes the best route for your prompts or quantizations.

1. Collect at least 20 representative prompts per route, including ambiguous, multilingual, long-context, sensitive, and out-of-scope cases.
2. Write the expected route before running the router. Keep a separate rubric for answer quality, factual support, code tests, and harmful or unauthorized behavior.
3. Run every candidate with fixed model tags, context, temperature, and prompt templates. Record cold and warm latency, memory, failures, and output quality.
4. Test negative cases: a prompt containing misleading keywords, an unavailable model, a timeout, an overloaded Ollama server, and a cloud route forbidden by data policy.
5. Accept automation only when it meets your routing-accuracy threshold, never violates the data boundary in the test set, falls back predictably, and preserves manual override.

Beginner Model Routing Checklist

Use this to build your first setup:

• [ ] Pick one default chat model, such as Gemma.
• [ ] Pick one coding model, such as Qwen Coder.
• [ ] Pick one planning or reasoning model, such as Nemotron if your hardware supports it.
• [ ] Pick one small model for summaries.
• [ ] Pick one larger model for hard tasks.
• [ ] Give each model a friendly name in Open WebUI.
• [ ] Add a short system prompt for each role.
• [ ] Test each model with three real prompts.
• [ ] Write down which model is best for which job.
• [ ] Start with manual model selection.
• [ ] Add automatic routing only after users understand the model names.
• [ ] Label cloud models clearly if you use any.
• [ ] Do not route private files or code to cloud models by accident.

Example Starter Prompts

Use these to test your routing setup.

Chat - Gemma

Explain what a reverse proxy does for a beginner running a home server.

Expected result: clear beginner explanation.

Code - Qwen Coder

Explain this Bash command and tell me if it could delete files:
find . -name "*.tmp" -delete

Expected result: command explanation plus safety warning.

Plan - Nemotron

Plan a safe weekend project to move Open WebUI from one mini PC to another. Include backups, testing, and rollback steps.

Expected result: step-by-step migration plan.

Summarize - Small

Summarize this in five bullets and preserve any action items:
[paste notes here]

Expected result: short, structured summary.

Deep Work - Large

Review this full homelab AI design and identify security, reliability, and usability problems:
[paste design here]

Expected result: deeper review with trade-offs.

A Practical First Router

If you want to build automatic routing later, start with this logic:

1. If user manually selected a model, respect that.
2. If policy requires local-only handling, remove every cloud candidate.
3. If the prompt matches a tested code route, use the validated code model.
4. If it matches a tested planning route, use the validated planning model.
5. If it is within the tested summary limits, use the validated small model.
6. If classification is uncertain, use the known-good default or ask the user.
7. On timeout or load failure, report the fallback and preserve manual override.

The first rule is important. Do not make automation fight the user.

Prefer Predictable Routes

A good router does not need to feel fancy. It should do three things:

• Pick a reasonable model.
• Make the choice obvious.
• Let the user override it.

Clear model names make route choice and failure diagnosis visible. Start with manual selection, record what each candidate does well, and automate only decisions that pass the routing test set.

Risk, Recovery, and Change Control

Export the prior Open WebUI workspace-model and Function configuration, preserve the old Pipe source and dependency versions, and keep the default model selectable before deploying a router. Roll out to one user group, compare route logs without retaining unnecessary prompt text, and disable the Function if misroutes, timeouts, or privacy violations cross your acceptance threshold.

A router failure should fail closed for restricted providers and tools. It should never interpret "model unavailable" as permission to send a local-only prompt to a cloud service. For high-impact code, legal, medical, financial, or infrastructure decisions, model selection does not remove the need for qualified human review and external validation.

What This Evidence Does Not Prove

• A vendor model card does not prove that a particular quantization is best for your language, codebase, prompt template, or hardware.
• A correct route does not prove the selected model's answer is correct, secure, or legally usable.
• Better results on public benchmarks do not establish better results on private operational tasks or resistance to prompt injection.
• The RouteLLM research demonstrates routing methods under its evaluated models, data, and cost assumptions; it does not validate this article's simple keyword rules.
• A local route label does not prove local processing unless every provider, tool, retrieval service, log, and fallback path is also local.

Related TechGeeks Reading

• Implementing AgenticOps Safely: Human Approval, Audit Trails, and Rollback
• Linux and Homelab Notes: Start Here

References

• Open WebUI: Models Workspace
• Open WebUI: Functions
• Open WebUI: Pipe Functions
• Open WebUI: Plugin Security Warning
• Ong et al.: RouteLLM
• Google: Gemma Models Overview
• Qwen: Qwen3-Coder
• Ollama Library: Current Qwen3-Coder Tags
• NVIDIA: Nemotron-H Reasoning Family
• NIST AI 600-1: Generative AI Profile