Local AI Models Explained: Gemma, Llama, Qwen, Nemotron, Phi, DeepSeek, and More

Start Here: The Plain-English Answer

The short answer: choose the smallest current model that can complete your real task at the context length you need, then compare exact checkpoints with the same prompts and acceptance criteria. A family name such as Gemma, Llama, Qwen, or Nemotron does not tell you the parameter count, quantization, license, memory requirement, tool support, or result quality by itself.

If you are new to local AI, the hardest part is not the install command. The hardest part is understanding what all the model names mean.

You will see names like:

gemma3:1b
gemma4:e2b
llama3.2:3b
qwen3:4b
deepseek-r1:8b
nemotron
mistral
granite
nomic-embed-text
bge-m3

At first, that looks like a parts catalog from another planet. It is not as bad as it looks.

The beginner question is this:

Why do some local AI models run on my laptop, while others need an expensive GPU, and why do they behave differently?

That is what this guide answers.

The short version:

Beginner Question	Plain-English Answer
What is a model?	The AI brain. It is the file or set of files that generates answers.
What is Ollama?	A local model runner. It loads the model and serves it on your computer.
What is Open WebUI?	A browser-based chat front end, similar in feel to ChatGPT.
What is Codex-style work?	A coding-agent workflow where an assistant can inspect files, suggest edits, run commands, and show diffs.
What is Gemma?	Google's open model family. Great for learning local AI because there are small, efficient versions.
What is Llama?	Meta's open-weight model family. Huge ecosystem and lots of tutorials.
What is Qwen?	Alibaba's model family. Strong for coding, math, multilingual use, and small-to-mid-size local models.
What is Nemotron?	NVIDIA's model family focused on reasoning, retrieval, safety, and agent workflows. More advanced homelab territory.
What is Phi?	Microsoft's small-model family. Good when you care about lightweight models and low latency.
What is DeepSeek?	A model family known for reasoning and coding. Run smaller distills locally, not the full giant models.
What is an embedding model?	A search helper for documents. It does not chat like a normal assistant.

If you remember only one rule, remember this:

Pick the smallest model that does the job well enough, in a format your tools support, at a speed you can tolerate, under a license you can use, with answers you can verify.

That one sentence saves beginners from most local AI mistakes.

Current Scope: July 15, 2026

This is a family chooser, not a permanent leaderboard. The model catalog changes faster than home hardware: Gemma 4 and Qwen3.5 are current families, Llama 4's headline models are far beyond a normal beginner PC, and NVIDIA continues to add specialized Nemotron variants. The exact model card, license, input modalities, context limit, tag, quantization, and runtime support control what you can actually run. Recheck those items on publication day and before using any model in a product or business workflow.

Runtime support changes too. Ollama 0.30 broadened its default GGUF/Vulkan path across more AMD, Intel, and NVIDIA devices, and its Apple silicon work includes MLX. That means model-family reputation alone cannot answer “will this run well on my PC?” Load the exact build, use ollama ps to inspect processor and context, and test your task on your hardware.

The Restaurant vs Home Kitchen Analogy

Cloud AI is like eating at a restaurant.

You do not own the kitchen. You do not maintain the ovens. You do not buy the commercial fridge. You order the food, and someone else handles the hard parts.

ChatGPT, Claude, Gemini, Perplexity, and hosted model APIs feel like that. They are easy to use because the servers, GPUs, scaling, updates, and model routing are handled somewhere else.

Local AI is like cooking at home.

You get more control. You can keep more private data in your house. You can experiment. You can change the tools. But you also need the right kitchen equipment.

That equipment is your:

Home Kitchen Item	Local AI Equivalent
Counter space	RAM and VRAM
Stove or oven	CPU and GPU
Recipe	Prompt
Ingredients	Documents, code, notes, images, audio, logs
Cookware	Ollama, llama.cpp, LM Studio, vLLM, Open WebUI
Cook	The model generating the answer

A tiny kitchen can still make a sandwich. It should not try to run a steakhouse.

That is the local AI hardware lesson in one sentence.

What Local AI Is, And What It Is Not

A local AI model is an AI model that runs on your own computer, workstation, or server.

That could be:

Setup	Example
Laptop	Ollama running a small Gemma or Llama model
Mini PC	Open WebUI plus a small model for casual chat
Desktop GPU	8B to 32B class models, depending on VRAM
Homelab server	Open WebUI, Ollama, RAG, Paperless-ngx integration, and model routing
Workstation	Larger models, coding agents, long context, multiple users

Local AI can be useful for:

• Learning how models work
• Private note summarization
• Document search
• Basic writing help
• Coding help
• Home lab automation planning
• Log review
• Small personal assistants
• Testing model behavior before buying hardware

Local AI is not automatically:

• Smarter than ChatGPT
• Safer than cloud AI
• Fully private in every app
• Able to browse the web by itself
• A replacement for professional advice
• Free from hallucinations
• Easy to scale to many users

Important privacy note:

Local does not automatically mean private forever.

If your app uses telemetry, cloud sync, web search, remote plugins, hosted embeddings, or cloud APIs, some data may still leave your computer. Privacy depends on the whole stack, not only the model.

The Local AI Stack: Model, Runner, UI, Agent, And Tools

Beginners often mix up the layers. This causes a lot of frustration.

A model is not the same as a web UI. A web UI is not the same as an agent. An agent is not the same as web search.

Here is the stack:

Layer	What It Means	Common Examples
Model	The AI brain that generates text, reads images, or creates embeddings	Gemma, Llama, Qwen, Phi, DeepSeek, Nemotron, Mistral, Granite
Runner	The program that loads and runs the model	Ollama, llama.cpp, LM Studio, vLLM, SGLang, TensorRT-LLM
Web UI	The browser interface you use to chat	Open WebUI, AnythingLLM, LM Studio
Agent or harness	The workflow layer that decides when to use tools	Codex CLI, Open WebUI tools, LangChain, LlamaIndex, Qwen-Agent
Tool	A specific ability outside the model	Web search, document lookup, shell commands, APIs, calculator, weather service
Knowledge base	A searchable collection of your documents	Open WebUI Knowledge, Chroma, Qdrant, Paperless-ngx exports, vector databases

This distinction matters.

If your model cannot answer "What is the weather in Plano right now?", that does not mean the model is broken. It usually means the tool layer is not configured with a live weather or search source.

If your model cannot read your Paperless-ngx documents, that does not mean Gemma is broken. It means you have not built the retrieval path that finds those documents and sends relevant chunks into the prompt.

If Codex-style work feels weak with a tiny local model, that does not mean Codex-style workflows are useless. It means the model may not be strong enough for open-ended code changes, or the task needs to be scoped more tightly.

Interactive Diagram: What Happens When You Ask A Local AI Question?

The diagram below is intentionally simple. Click each layer and think of it as a local AI checklist.

Interactive Flow: Your Question To The Local Model

Select a stage to see what it does.

1. BrowserYou type the question
2. Open WebUIChatGPT-like interface
3. ToolsSearch, docs, APIs
4. OllamaModel runner
5. ModelAI brain
6. HardwareCPU, RAM, GPU, VRAM

Select a stage

Click one of the boxes above to see the beginner explanation.

The Big Beginner Terms

LLM

LLM means Large Language Model.

That is the kind of model that writes text, answers questions, summarizes notes, explains code, and chats.

Examples:

• Gemma
• Llama
• Qwen
• Phi
• Mistral
• DeepSeek
• Nemotron
• Granite

Not every AI model is an LLM. Some models create images, transcribe audio, detect objects, or create embeddings for search.

Parameter Count

You will see model sizes like 1B, 3B, 4B, 7B, 8B, 12B, 14B, 27B, 31B, 70B, 120B, or 671B.

The "B" means billion parameters.

A parameter is a learned number inside the model. You can think of parameters like knobs the model learned to tune during training.

More parameters can mean:

• Better writing
• Better reasoning
• Better instruction following
• More knowledge
• Better coding
• Better multilingual behavior

But more parameters also usually mean:

• More memory needed
• More GPU VRAM needed
• Slower answers
• More heat
• More power use
• More waiting

Beginner translation:

Bigger can be better, but bigger is not automatically better for your PC.

A small model that responds quickly can be more useful than a large model that barely runs.

Quantization

Quantization is model compression.

Analogy:

Quantization is like saving a high-resolution photo as a smaller JPEG.

The file gets smaller. It loads faster. It uses less space. But if you compress it too much, you lose fine detail.

Common quantization labels:

Label	Beginner Meaning
BF16 / FP16	High quality, high memory use
Q8	Large, usually strong quality
Q6	Good quality, still fairly large
Q5	Balanced quality and size
Q4	Common home AI sweet spot
Q3 / Q2	Smaller, but answers may degrade

For a first local AI setup, Q4 is often the practical starting point.

If the model gives poor answers, do not immediately blame the model family. You may be using a heavily compressed version.

RAM And VRAM

RAM is your computer's main memory.

VRAM is the memory on your graphics card.

For local AI, VRAM is especially important because GPUs are much faster than CPUs for many model workloads.

Analogy:

RAM and VRAM are desk space.

If the model fits on the desk, work moves smoothly. If it does not fit, the computer has to shuffle papers around. That gets slow.

Common beginner trap:

Storage space is not the same as memory.

A model file might be 7GB on disk, but running it can require more memory because of runtime overhead, context, images, cache, and the app itself.

CPU, GPU, And Offload

The CPU is the main processor.

The GPU is the graphics card.

CPU-only local AI can work, especially for small models. It can also feel slow.

GPU local AI usually feels much better if the model fits in VRAM.

Offloading means part of the model runs on the GPU and part runs in regular system RAM or CPU. It can let larger models run, but it often reduces speed.

When you check Ollama, you may see:

ollama ps

Look for the processor column. If it says 100% GPU, the model is fully on the GPU. If it says something like 54% GPU / 46% CPU, the model is split.

That split is not always bad, but it explains why a model may feel slower than expected.

Context Window

The context window is how much text the model can consider at once.

Analogy:

The context window is the model's whiteboard.

If the whiteboard is small, old information gets erased as new information comes in. If the whiteboard is large, the model can keep more of the conversation, documents, or code in view.

But larger context uses more memory.

Beginner warning:

A 128K or 256K context window on a model page does not mean your home PC can comfortably use that full context.

Long context increases memory use because the model has to track more tokens while answering.

Token

A token is a chunk of text.

Sometimes one token is a word. Sometimes it is part of a word. Sometimes it is punctuation.

For beginners, it is fine to think:

More tokens = more text.
More text = more memory and more time.

Tokens Per Second

Tokens per second is how fast the model generates an answer.

Analogy:

Tokens per second is like words per minute.

A model can be smart but painfully slow. For daily use, speed matters.

If a model takes 90 seconds to answer every simple question, you probably will not use it much.

Base Model vs Instruct Model

A base model is like raw autocomplete.

An instruct model is trained to follow instructions.

Analogy:

A base model is like someone who has read a lot but has not been trained to be helpful. An instruct model has customer-service training.

Beginners usually want instruct, chat, or it models.

Examples of beginner-friendly naming clues:

instruct
chat
it

The it in some model names means instruction tuned.

Dense Model vs MoE Model

Dense models use most of the model for every answer.

MoE means Mixture of Experts. An MoE model has many internal expert sections, but only some are active for each token.

Beginner translation:

MoE can be efficient, but it is not magic.

The full model still needs to be stored and served. You still need enough hardware, especially for large MoE models.

Open-Weight vs Open Source

Open-weight means you can download the model weights.

Open source usually means the license grants broad rights to inspect, modify, and redistribute software.

These are not always the same.

Beginner warning:

Downloadable does not mean free for every use.

Before using a model in a business, client project, public service, or product, read the license.

RAG

RAG means Retrieval-Augmented Generation.

Plain English:

RAG is a setup where the AI searches your documents first, then answers using the relevant pieces.

RAG is how you build:

• Chat with PDFs
• Chat with scanned documents
• Chat with Paperless-ngx
• Chat with company manuals
• Chat with notes
• Chat with logs

RAG normally uses:

Component	What It Does
OCR	Turns scanned pages into text
Chunking	Splits documents into smaller pieces
Embedding model	Turns text chunks into searchable vectors
Vector database	Stores and searches those vectors
Chat model	Writes the answer
Citations or source links	Help you verify where the answer came from

The chat model is only one part of the system.

For Paperless-ngx, OCR quality and retrieval quality matter just as much as model choice.

Quick Beginner Recommendations

If you are brand new, use this table first.

Goal	Good First Choice	Why
I just want to learn local AI	Gemma 3 1B, Gemma 4 E2B, Llama 3.2 1B/3B, Qwen small models, Phi mini-class models	Small enough to experiment with
I want a ChatGPT-like web page	Open WebUI plus Ollama plus a small Gemma or Llama model	Easier to understand the stack
I want useful daily chat	4B to 8B class model if your hardware can handle it	Better answers without huge hardware
I want coding help	Qwen Coder, DeepSeek distilled, Codestral/Devstral, Granite Code, or a strong Qwen model	Coding-tuned models usually help more
I want document chat	Chat model plus embedding model plus RAG	The model needs a document lookup system
I want scanned document chat	Paperless-ngx plus OCR plus RAG	Scans must become searchable text first
I want vision or screenshots	A vision-language model, not a text-only model	Text-only models cannot see images
I want agents	Qwen, Nemotron, Llama, Mistral, Gemma, or DeepSeek with a proper harness	Tools and workflow matter as much as the model
I want to share a GPU with Plex/Tdarr	Smaller model, modest context, schedule heavy jobs	AI and media workloads can compete

For a home lab, the practical rule is:

Start small, prove the workflow, then move up one model size at a time.

Interactive Model And Hardware Chooser

This widget is a rough guide. It is not a benchmark. It is meant to teach the relationship between model size, task, and hardware.

Interactive Picker: What Should I Try?

Choose your main goal, model size, and GPU memory. The recommendation will update below.

Main goal
Learn local AIDaily chat assistantCoding helpDocuments and RAGImages or screenshotsAgent workflows

Model size you are considering
1B to 3B4B to 8B12B to 14B26B to 32B70B or larger

Your GPU VRAM
No dedicated GPU4GB8GB12GB16GB24GB32GB+

Recommendation

Start with a 1B to 3B instruction model such as Gemma, Llama, Qwen, or Phi. Use Ollama and Open WebUI after the command line test works.

Hardware Reality Check

Here is the beginner hardware map. These are rough categories, not guarantees.

Hardware Level	Practical Beginner Target	What It Feels Like
No dedicated GPU, 8GB RAM	1B to 3B text models	Good for learning, but slow
No dedicated GPU, 16GB to 32GB RAM	1B to 8B quantized models	Usable for experiments
4GB GPU	1B to 4B text models	Good test lab, tight daily driver
8GB GPU	4B to 8B, some 7B models	Useful but watch VRAM
12GB GPU	7B to 14B Q4 models	Nice beginner desktop tier
16GB GPU	8B to 14B comfortably, some larger models	Good for coding and RAG
24GB GPU	14B to 32B class models	Strong one-GPU home AI tier
32GB+ GPU	31B class models and larger experiments	Serious workstation territory
48GB+ GPU or multi-GPU	70B, large MoE, multiple users	Advanced lab/server territory

Memory math is not exact because model architecture, quantization, context length, KV cache, runtime, and multimodal input all change the requirement.

Still, this table gives you a sane starting point.

4GB VRAM Reality

4GB VRAM can run local AI. It is just not where you should expect big models.

Good fits:

• Gemma 3 1B
• Llama 3.2 1B or 3B
• Small Qwen models
• Small Phi models
• Small embedding models
• Short prompts
• Learning Open WebUI and Ollama

Poor fits:

• 12B+ text models
• 7B+ vision models
• 31B class models
• 70B models
• Long-context document chat
• Heavy coding agents
• Multiple users

If you have 4GB VRAM, the goal is to learn the workflow, not win a benchmark.

31B Class Reality

A 31B model can be very useful, but it is not a beginner GPU target.

For a good 31B-class local experience, plan around:

Component	Sensible Starting Point
GPU VRAM	32GB+ preferred for a cleaner fit
System RAM	64GB minimum, 128GB nicer
Storage	Fast NVMe recommended
CPU	Modern multi-core CPU
Cooling	Real airflow, not cramped
Power	Enough PSU headroom
Workloads	Avoid fighting heavy transcodes while loading models

Can you force a 31B model to run with less? Sometimes. It may offload to CPU/RAM. It may be slow. It may not stay fully in GPU memory. It may be frustrating.

For beginners, a fast 8B or 14B model often feels better than a struggling 31B model.

Plex, Tdarr, Jellyfin, Frigate, And GPU Sharing

If your AI server also runs Plex or Tdarr, be practical.

Media workloads may use:

• NVENC for video encoding
• NVDEC for video decoding
• GPU memory
• PCIe bandwidth
• CPU
• Disk IO
• Cooling capacity

Ollama may use:

• GPU compute
• GPU memory
• CPU during prompt processing
• System RAM if the model does not fit fully in VRAM
• Disk when pulling or loading models

Beginner rule:

Do not load the biggest model on the same GPU that is actively doing heavy transcodes.

Use smaller models, schedule heavy batches, and check:

nvidia-smi
ollama ps
docker stats

If the system gets sluggish, stop the AI model before a big media job:

ollama stop <model-name>

Model Families Explained

Now let us walk through the major model families a beginner is likely to see.

This is not a permanent ranking. The model world changes quickly. This is a practical map as of June 21, 2026.

Gemma: Google's Beginner-Friendly Local AI Family

Gemma is Google's open model family. It is one of the best families for a beginner to understand because there are small models, newer multimodal models, and broad local tooling support.

Good for:

• Learning local AI
• Open WebUI plus Ollama
• Lightweight chat
• Summarization
• Reasoning experiments
• Coding experiments
• Multimodal testing on newer Gemma 4 models
• A first local assistant

Why beginners should care:

Gemma gives you a clear path:

Tiny learning model -> small daily model -> larger serious model

That makes it easier to grow without changing the whole stack.

Gemma 3

Gemma 3 is still useful for beginner testing, especially if you want something small.

Common beginner example:

ollama run gemma3:1b

This is not a ChatGPT replacement. It is a great first model for understanding:

• Ollama
• Open WebUI model selection
• GPU loading
• CPU fallback
• Basic prompting
• Local privacy tradeoffs

Gemma 4

Gemma 4 is more capable and more complex.

Google's Gemma 4 model card describes Gemma 4 as a multimodal family that can handle text and image input, with audio support on E2B, E4B, and 12B models. It lists five sizes:

Gemma 4 Size	Beginner Translation
E2B	Small effective-parameter model, good to try first if supported
E4B	Stronger small model, often a better daily target than E2B
12B	Mid-size model, needs more memory
26B A4B	MoE model with about 4B active parameters, but still a larger model to host
31B	Large dense model, serious local hardware target

Ollama currently lists Gemma 4 tags such as:

Ollama Tag	Listed Size	Listed Context	Listed Input
gemma4:e2b	7.2GB	128K	Text, Image
gemma4:e4b	9.6GB	128K	Text, Image
gemma4:12b	7.6GB	256K	Text, Image
gemma4:26b	18GB	256K	Text, Image
gemma4:31b	20GB	256K	Text, Image
gemma4:e2b-it-qat	4.3GB	128K	Text, Image

Important warning:

The listed file size is not the full runtime memory promise.

Context length, images, KV cache, runtime overhead, and other GPU workloads still matter.

Beginner recommendation:

Your Situation	Try
Very small test box	gemma3:1b
4GB VRAM test lab	gemma3:1b, then maybe a small Gemma 4 QAT/Q4 tag if supported
8GB to 12GB VRAM	Gemma 4 E2B or E4B class tags
16GB to 24GB VRAM	Gemma 4 E4B, 12B, or possibly 26B depending on context
32GB+ VRAM	Gemma 4 31B becomes more realistic

What Gemma is not:

Gemma does not automatically give your local assistant web search, your documents, or Codex-style file-editing powers. Those come from Open WebUI tools, RAG, Codex CLI, or another harness.

Llama: The Big Community Default

Llama is Meta's open-weight model family. It has a huge community, broad tool support, and lots of tutorials.

Good for:

• General local chat
• First installs
• Learning prompts
• RAG tutorials
• App compatibility
• Community fine-tunes
• Experimenting across many model sizes

Why beginners should care:

If a local AI tool supports only a few model families, Llama is often one of them.

That makes Llama a safe learning path.

Llama 3.2 And Llama 3.x

For beginners, the smaller Llama 3.x models are often more useful than the giant flagship models.

Good beginner examples:

ollama run llama3.2:1b
ollama run llama3.2:3b

These are realistic on modest hardware.

Llama 4

Llama 4 is much more advanced.

Meta introduced Llama 4 Scout and Llama 4 Maverick as open-weight, natively multimodal, Mixture-of-Experts models. Scout and Maverick are exciting, but the important beginner note is this:

"Fits on a single H100" does not mean "fits nicely on my home desktop GPU."

An NVIDIA H100 is data center hardware. It is not the same class as a used gaming GPU.

Beginner recommendation:

Your Situation	Llama Path
Weak hardware	Llama 3.2 1B or 3B
Normal desktop	Llama 3.1/3.x 8B class models
Serious GPU	Larger Llama 3.x models
Advanced lab	Evaluate Llama 4 only after hardware review

Licensing note:

Llama is open-weight, but you still need to review Meta's license if you use it for business, products, or public services.

Qwen: Strong For Coding, Math, Multilingual, And Agents

Qwen is Alibaba's broad open-weight model family. It has become a favorite in local AI communities because it offers many sizes and tends to perform well for coding, math, multilingual use, and tool/agent workflows.

Good for:

• Coding help
• Math and reasoning
• Multilingual use
• Small-to-mid-size local chat
• Agent experiments
• Tool calling
• Strong local benchmarks

Why beginners should care:

Qwen gives you many sizes. That means you can choose a model that fits your machine instead of jumping straight to something huge.

Current-family note: Qwen3.5 was announced on February 15, 2026 as a multimodal family using a hybrid architecture. It is the current family to evaluate when the exact checkpoint and runtime support your task. The Qwen3 size map below remains useful because those smaller dense and MoE checkpoints are widely encountered in local tooling. Do not assume a newer family name makes every checkpoint smaller, faster, or better for your prompt.

Qwen3 includes dense models such as:

Qwen3 Dense Size	Beginner Meaning
0.6B	Tiny experiment model
1.7B	Small local model
4B	Very interesting beginner size
8B	Strong local daily tier
14B	Good larger home GPU tier
32B	Serious single-GPU/workstation tier

Qwen3 also includes MoE models such as:

Qwen3 MoE Size	Beginner Meaning
30B-A3B	30B total, about 3B active, still not tiny to host
235B-A22B	Very large, not a beginner local target

Qwen3 also introduced thinking and non-thinking modes.

Beginner translation:

• Thinking mode can spend more tokens reasoning before answering.
• Non-thinking mode is faster for simple questions.

This is useful, but it also means a Qwen model may feel slower if it is spending time reasoning.

Beginner recommendations:

Goal	Try
Learn local AI	Qwen 0.6B to 4B class
Daily chat	Qwen 4B or 8B
Coding help	Qwen Coder or Qwen coding-tuned variants
Reasoning	Qwen thinking variants
Multilingual	Qwen is worth testing early

Beginner warning:

Qwen model names can get busy:

base
instruct
coder
vl
embedding
reranker
moe
thinking
non-thinking

Choose an instruct/chat model unless you know you need another type.

Nemotron: NVIDIA's Agent And Reasoning-Oriented Family

Nemotron is NVIDIA's model family. It is especially interesting for homelabs that already use NVIDIA GPUs and want to explore agent workflows, retrieval, safety, long context, and production-style deployment.

Good for:

• NVIDIA-heavy systems
• Agentic workflows
• Reasoning
• RAG and retrieval
• Safety experiments
• Enterprise-style deployments
• Testing vLLM, SGLang, TensorRT-LLM, and NVIDIA tooling

Why beginners should care:

Nemotron helps show where local AI is going:

chatbot -> document assistant -> tool-using agent -> workflow system

But it may not be the easiest first model for a beginner.

NVIDIA's Nemotron 3 materials reference Nano, Super, and Ultra tiers. The smaller Nano-style models are the only ones a beginner should even think about first. Super and Ultra models are serious infrastructure territory.

Beginner recommendation:

Your Situation	Nemotron Advice
Brand new to local AI	Start with Gemma, Llama, Qwen, or Phi first
NVIDIA GPU homelab	Watch Nemotron Nano-class models
Building agents	Study Nemotron, but do not skip workflow safety
4GB VRAM	Do not start with Nemotron
24GB+ VRAM	Experiment carefully with smaller Nemotron options

Key warning:

Agentic AI is not just a smarter chat model.

An agent needs:

• Tools
• Permissions
• Logs
• Guardrails
• Human review
• Rollback
• Clear task boundaries

The model is one ingredient.

Phi: Microsoft's Small And Efficient Model Family

Phi is Microsoft's small language model family. It is useful for beginners because it focuses on lightweight deployment, low latency, and efficiency.

Good for:

• Small local assistants
• Low-resource machines
• Education
• Reasoning experiments
• Lightweight chat
• Edge or near-edge use cases
• Some multimodal/audio experiments, depending on the exact model

Why beginners should care:

Phi models can be surprisingly useful for their size.

That does not mean they beat every larger model. It means they are good candidates when your hardware is limited.

Beginner recommendation:

Goal	Phi Fit
Weak hardware	Strong fit
Fast simple assistant	Strong fit
Broad general knowledge	Good, but compare against Gemma/Llama/Qwen
Heavy coding agent	Not usually the first pick
Huge document chat	Use RAG and test carefully

Use Phi when you care about responsiveness and efficiency.

DeepSeek: Reasoning And Coding, But Use Distills Locally

DeepSeek is widely known for reasoning and coding performance.

The beginner trap is thinking the full DeepSeek models are normal home PC models.

They are not.

DeepSeek-V3, for example, is a very large MoE model. The public DeepSeek-V3 repository describes 671B total parameters with 37B activated for each token. That is not where a beginner starts.

For local AI, beginners usually use distilled DeepSeek models.

Distilled model means:

A smaller model trained to imitate or learn from a larger model's behavior.

Good for:

• Reasoning practice
• Math
• Coding explanations
• Debugging help
• Comparing reasoning styles

Beginner recommendations:

Your Hardware	DeepSeek Path
Weak hardware	Skip for now or try very small distills
8GB to 12GB VRAM	7B/8B distills may be realistic
16GB to 24GB VRAM	14B/32B distills become interesting
Serious workstation	Larger distills and advanced serving
Full DeepSeek V3/R1 class	Not beginner local hardware

Reasoning models can be slower because they may generate extra reasoning tokens before answering.

That can be useful for hard problems, but annoying for simple chat.

Mistral, Ministral, Codestral, Devstral, And Pixtral

Mistral is a European AI company with several model lines. For beginners, the names can be confusing.

You may see:

Name	Beginner Meaning
Mistral	General model family
Ministral	Smaller efficient models
Mixtral	Older MoE line
Codestral	Coding-focused models
Devstral	Software engineering/coding-agent models
Magistral	Reasoning-focused models
Pixtral	Vision/multimodal line
Voxtral	Audio-related line

Good for:

• General local models
• Coding
• Efficient small models
• Business experiments
• Multimodal and specialist workflows, depending on model

Mistral's current model docs list families such as Mistral Medium 3.5, Mistral Small 4, Mistral Large 3, Ministral 3 sizes, Devstral 2, Voxtral, OCR, and other specialized models.

Beginner recommendation:

Goal	Mistral Path
First local chat	Small Mistral/Ministral model
Coding	Codestral or Devstral-style models
Vision	Pixtral/vision-capable model, if supported
Audio	Voxtral-style model, if needed
Business use	Review exact license and deployment terms

Mistral can be excellent, but beginners should avoid getting lost in the names. Start with a small instruct model.

Granite: IBM's Enterprise-Friendly Model Family

Granite is IBM's model family. It is often less hyped than Llama or Qwen, but it matters for business-style local AI.

Good for:

• Enterprise workflows
• Tool calling
• Instruction following
• RAG
• Structured output
• Code and business tasks
• Workflows where reliability matters more than flash

IBM's Granite page describes Granite 4.1 as focused on instruction following, tool calling, predictable latency, stable token usage, and efficient business workloads.

Beginner translation:

Granite is worth testing when you want a practical work assistant, not just a flashy chat demo.

Beginner recommendation:

Goal	Granite Fit
Business document assistant	Strong candidate
Tool calling	Worth testing
Coding workflows	Check Granite Code options
Creative casual chat	Compare with Gemma, Llama, Qwen, Mistral
Weak hardware	Use smaller Granite options only

Embedding Models: The Search Models Beginners Forget

Embedding models are not normal chat models.

They turn text into numbers so your system can search by meaning.

Analogy:

An embedding model makes a smart card catalog for your documents.

If you ask:

What does my furnace manual say about error code 33?

The embedding model helps find the document chunks about furnace error code 33.

Then the chat model writes the answer.

Common local embedding model examples:

Embedding Model	Why It Matters
embeddinggemma	Google's embedding model option in Ollama docs
qwen3-embedding	Qwen embedding family
nomic-embed-text	Popular local embedding model
bge-m3	Multilingual and versatile retrieval model
all-minilm	Small and lightweight

Important rule:

Do not mix embedding models in the same index unless you rebuild the index.

If you create your document database with nomic-embed-text, keep using that same embedding model for queries. If you switch to another embedding model, rebuild the vector database.

Vision Models: When The AI Needs To See

Text-only models cannot inspect images.

If you want the AI to understand:

• Screenshots
• Scanned forms
• Pictures
• Charts
• Tables
• Diagrams
• Whiteboards
• UI errors

you need a vision-language model or another vision-specific model.

Examples of model families with vision or multimodal options:

Family	Vision/Multimodal Notes
Gemma 4	Text and image input, with audio on selected sizes
Llama 4	Natively multimodal open-weight models
Qwen-VL / Qwen vision lines	Strong for documents, OCR-style tasks, charts, and image Q&A
Phi multimodal	Smaller multimodal options
Mistral/Pixtral	Mistral's multimodal line
Florence-2	Lightweight vision tasks, not a general chat model

Vision models usually need more memory than text-only models because the system also processes image data.

Beginner warning:

A model that can read text does not automatically mean it can read an image of text.

For scanned documents, Paperless-ngx OCR plus RAG may be better than asking a vision model to read every page from scratch.

Coding Models: Why "Smart Chat" Is Not Always Best For Code

A good coding model is trained or tuned for code.

It may be better at:

• Understanding file structure
• Explaining errors
• Writing functions
• Refactoring
• Following stack traces
• Producing patches
• Working with tests
• Understanding common frameworks

Coding-focused families and lines include:

Coding Choice	Beginner Note
Qwen Coder	Strong local coding choice
DeepSeek distilled/coder variants	Good for reasoning and code explanations
Codestral / Devstral	Mistral coding-oriented lines
Granite Code	IBM code-oriented options
CodeGemma / Gemma coding-capable models	Useful in Google's ecosystem
Llama coding fine-tunes	Many community options

But a coding model alone is not the whole workflow.

For Codex-style work, you also need:

• A Git repo
• Clear task scope
• File inspection
• Diffs
• Tests
• Permission boundaries
• Human review

This is why "a model" and "Codex-style work" are not the same thing.

Codex-Style Work With Local Models

Codex-style work means a coding agent can operate inside a software project. It may inspect files, propose changes, run commands, and summarize diffs.

OpenAI's Codex CLI supports a local open source provider mode with --oss. The current Codex documentation describes Ollama and LM Studio as examples of local providers, and the oss_provider setting can default OSS mode to ollama.

Example:

codex --oss -c 'oss_provider="ollama"' -m gemma4:e2b

Or in a Codex config file:

oss_provider = "ollama"

Beginner reality check:

Using a local model with Codex-style tooling does not make the local model as capable as OpenAI-hosted Codex models.

The harness can still be powerful, but the local model controls the quality of reasoning and edits.

Good local-model Codex tasks:

• "Summarize this repo."
• "Find the main entry points."
• "Explain this error."
• "Add comments to this one file."
• "Write a small helper function."
• "Review this diff for obvious issues."

Poor beginner tasks for a tiny local model:

• "Rewrite this whole application."
• "Migrate the entire stack."
• "Fix all tests across the repo."
• "Design a secure production architecture from scratch."
• "Make autonomous changes without review."

Safety note:

Codex-style tools can run commands and edit files. Use sandboxing, approvals, Git, and review. For beginners, keep tasks scoped and inspect diffs before accepting changes.

Web Search: Why Models Do Not "Just Know" They Need The Internet

A model is not a browser.

The model can generate text from what it learned during training and what you put in the prompt.

It cannot automatically:

• Search Google
• Check current weather
• Read a website
• Query your firewall
• Search Paperless-ngx
• Look up a package release
• Inspect your server logs

unless the app or agent gives it a tool.

Think of it this way:

Thing	Who Provides It?
General language ability	Model
Current weather	Weather/search tool
Your documents	RAG/document tool
Your server logs	File/log access tool
Shell commands	Agent harness
Web browsing	Search/browser tool
Safe permissions	App, runner, sandbox, admin configuration

If Open WebUI web search fails, the model may say it cannot find sources. That is usually a search configuration issue, not a Gemma issue.

If you do not want Google Programmable Search Engine, look at alternatives supported by your WebUI or tool layer, such as SearXNG, Brave Search, DuckDuckGo-style providers, Tavily, SerpAPI, Bing, or another self-hosted/search API option. The exact choices depend on Open WebUI version and enabled integrations.

Documents, Paperless-ngx, And Local Knowledge

For scanned documents, local AI needs a pipeline.

The pipeline usually looks like this:

Scanner -> Paperless-ngx -> OCR text -> Export/API -> Embeddings -> Vector database -> Chat model -> Answer with sources

Paperless-ngx is excellent for:

• Scanned bills
• Receipts
• Manuals
• Contracts
• Records
• PDF organization
• OCR
• Tags
• Correspondents
• Document metadata

The AI side is usually better at:

• Summarizing
• Answering questions
• Finding patterns
• Explaining documents
• Comparing snippets
• Drafting follow-up questions

Important separation:

Paperless-ngx is your document filing cabinet. RAG is the AI search path into that cabinet.

If Paperless-ngx runs on another system, your AI system can access it by:

• Paperless-ngx API
• Read-only export folder
• Scheduled document export
• Shared network storage
• Database-backed integration, if you know what you are doing

For beginners, the safest pattern is:

1. Let Paperless-ngx handle scanning and OCR.
2. Export text or PDFs to a read-only location.
3. Let the AI/RAG system index that copy.
4. Keep Paperless-ngx as the source of truth.
5. Show source links in AI answers.

Do not start by letting an AI agent modify your document archive.

How To Decode A Model Name

Model names often include clues.

Example:

qwen3:4b

Plain English:

Part	Meaning
qwen3	Model family and generation
4b	Rough size, 4 billion parameters

Example:

gemma4:e2b-it-qat

Plain English:

Part	Meaning
gemma4	Gemma generation 4
e2b	Effective 2B class Gemma 4 model
it	Instruction tuned
qat	Quantization-aware training or quantized-friendly tag

Example:

deepseek-r1:8b

Plain English:

Part	Meaning
deepseek-r1	DeepSeek reasoning model line
8b	Smaller distilled/local-friendly size

Common naming words:

Name Part	Beginner Meaning
base	Raw model, not usually best for chat
instruct	Tuned to follow instructions
chat	Tuned for conversation
it	Instruction tuned
coder	Tuned for code
vl	Vision-language
embed	Embedding/search model
reranker	Improves search result ranking
moe	Mixture of Experts
distill	Smaller model trained from larger model behavior
q4, q5, q8	Quantization level

If you do not know what to choose, pick an instruct/chat model.

Practical Model Family Comparison

Family	Beginner Friendliness	Strengths	Watch-Outs	Good First Target
Gemma	High	Efficient, good small sizes, Open WebUI/Ollama friendly, multimodal in Gemma 4	Newest tags may need newer tools; larger sizes still need real hardware	Gemma 3 1B, Gemma 4 E2B/E4B
Llama	Very high	Huge ecosystem, lots of tutorials, broad compatibility	License review for business; Llama 4 is not casual GPU territory	Llama 3.2 1B/3B, 8B class
Qwen	High	Coding, math, multilingual, agent/tool use, many sizes	Many variants can confuse beginners	Qwen 4B/8B, Qwen Coder
Phi	High	Small, efficient, low-latency friendly	May feel narrower than larger general models	Phi mini/small class
DeepSeek	Medium	Reasoning, math, coding	Full models are huge; use distills locally	DeepSeek R1 distill 7B/8B/14B
Mistral	High	Efficient models, coding lines, business/dev ecosystem	Many names and license/deployment categories	Ministral/Mistral small, Codestral/Devstral for code
Granite	Medium	Enterprise, RAG, tool calling, structured workflows	Less flashy for casual chat	Granite small/code options
Nemotron	Medium	NVIDIA ecosystem, agents, reasoning, safety, RAG	Advanced hardware/tooling territory	Nano-class only after basics
Embedding models	High for RAG	Document search	They do not chat	embeddinggemma, nomic-embed-text, bge-m3
Vision models	Medium	Images, screenshots, OCR-like workflows	More memory and app support needed	Small VLMs first

Which Model Should A Beginner Choose First?

Use these scenarios.

Scenario 1: "I Have An Ordinary Laptop"

Start with:

ollama run gemma3:1b

Then try:

ollama run llama3.2:1b
ollama run llama3.2:3b

What you are learning:

• How to install Ollama
• How to run a local model
• How slow or fast CPU inference feels
• How model size changes answer quality
• What local AI can and cannot do

Do not start with:

• 31B models
• 70B models
• Llama 4
• Full DeepSeek
• Large Nemotron
• Long-context RAG

Scenario 2: "I Have A 4GB GPU"

Start with:

• 1B to 4B models
• Q4 quantized models
• Shorter context
• Text-only tests first

Good model families:

• Gemma
• Llama small
• Qwen small
• Phi small

Use 4GB VRAM to learn. Do not treat it as a serious 31B machine.

Scenario 3: "I Want A ChatGPT-Like Web UI"

Use:

• Ollama
• Open WebUI
• One small instruct model

Beginner path:

ollama run gemma3:1b

Then connect Open WebUI to Ollama.

Only after that works, try:

ollama pull gemma4:e2b

The goal is to make the stack understandable:

Browser -> Open WebUI -> Ollama -> Model -> CPU/GPU

Scenario 4: "I Want To Search My Documents"

Do not focus only on the chat model.

You need:

• Good OCR
• Clean text extraction
• Chunking
• Embedding model
• Vector database
• Chat model
• Source links

Good starter embedding models:

ollama pull embeddinggemma
ollama pull nomic-embed-text
ollama pull bge-m3

Use a chat model such as Gemma, Llama, Qwen, Mistral, or Granite to answer after retrieval.

Scenario 5: "I Want Coding Help"

Start with a coding model if possible.

Good families to test:

• Qwen Coder
• DeepSeek distills
• Codestral/Devstral
• Granite Code
• Llama code fine-tunes
• Gemma coding-capable models

Do not ask a tiny model to own a full production refactor.

Good prompt:

Read this one file and explain what it does. Do not edit anything yet.

Better beginner workflow:

1. Ask for explanation.
2. Ask for a plan.
3. Ask for a small patch.
4. Review the diff.
5. Run tests.
6. Commit only after review.

Scenario 6: "I Want Agents"

Start with the tool layer, not just the model.

An agent needs:

• A model
• A runner
• A tool system
• Permissions
• Memory or state, if needed
• Logs
• Human review
• Limits

Good agent-oriented model families:

• Qwen
• Nemotron
• Llama
• Mistral
• Gemma
• DeepSeek

But the model is only one part.

If an agent can run commands, delete files, call APIs, or change configs, treat it like a junior admin with a terminal. Scope the job and review the work.

Common Beginner Misconceptions

Misconception	Reality
Bigger always means better	Bigger may be slower, hotter, and unusable on your hardware
Local means totally private	Apps, tools, search, telemetry, and cloud connectors can still send data out
Open-weight means open source	Not always. Read the license
A model knows when to search	Search is a tool, not a built-in instinct
More context means more intelligence	More context means more text in view, not automatically better reasoning
A 5GB file needs only 5GB VRAM	Runtime memory can be higher
One model should do everything	Use different models for chat, code, embeddings, and vision
A confident answer is a correct answer	Models can hallucinate confidently
Quantized models are bad	Q4/Q5 models are often the local AI sweet spot
Benchmarks decide everything	Your hardware and prompts matter more

Hallucinations And Trust

A hallucination is when the model gives false or made-up information.

Analogy:

A model is like a very confident intern. It may be useful, but you still check important work.

Local models can hallucinate:

• Commands
• Citations
• Dates
• Package names
• Legal statements
• Medical advice
• Financial advice
• Security guidance
• Hardware compatibility claims

Smaller models may hallucinate more, but larger models can hallucinate too.

Safer use:

Summarize this official document I provide and list the exact sections you used.

Riskier use:

Tell me the correct legal answer and do not hedge.

For important topics, verify with primary sources.

Licenses And Business Use

Before using any local model in business, check the license.

Questions to ask:

Question	Why It Matters
Can I use this commercially?	Some models restrict business use
Can I redistribute it?	Public apps and packaged products may have limits
Do I need attribution?	Some licenses require notices
Are there usage restrictions?	Some models prohibit specific use cases
Is it open-weight or truly open source?	These are not the same
Does my client allow this model family?	Compliance matters

For home learning, licensing is often simpler. For business, do not skip this step.

A Noob-Friendly Testing Plan

Use this plan when trying a new model.

Step 1: Start With One Tiny Model

ollama run gemma3:1b

Ask:

Explain what a local AI model is in five sentences.

Step 2: Check What Is Loaded

ollama ps

Look at:

• Model name
• Processor
• Context
• Size
• Until/keep-alive behavior

Step 3: Watch GPU Use

nvidia-smi

Look at:

• GPU memory used
• GPU utilization
• Other processes
• Plex/Tdarr/ffmpeg workloads

Step 4: Ask Your Real Questions

Do not test only cute prompts.

Use your actual use cases:

• "Summarize this log."
• "Explain this Python error."
• "Draft a polite email."
• "Compare these two config options."
• "Help me understand this invoice."
• "Explain this Docker compose file."

Step 5: Move Up One Size

Try one larger model.

Example:

ollama pull gemma4:e2b

Then test the same questions.

Ask yourself:

Question	Why It Matters
Is the answer better?	Quality
Is it slower?	Daily usability
Is CPU higher?	Possible offload
Is VRAM full?	Hardware limit
Does Plex/Tdarr suffer?	Shared box impact
Do I trust the answer more?	Practical value

Step 6: Keep Notes

Track:

Model	Task	Speed	Quality	GPU Use	Notes
Gemma 3 1B	Basic chat	Fast	Simple	Low	Good first test
Gemma 4 E2B	Writing	Medium	Better	Higher	Watch VRAM
Qwen 4B	Code explanation	Medium	Good	Moderate	Good next test

This is better than guessing.

Evidence and Testing Method

The family descriptions, published model sizes, context claims, licenses, and runtime features in this guide are documentation-backed. TechGeeks did not independently run every family or reproduce vendor benchmark scores. Vendor benchmarks can help identify a candidate, but they do not establish performance on your hardware, at your quantization and context, with your prompt set.

Build a small evaluation set from work you can legally use: five representative prompts, one adversarial or ambiguous prompt, and one task with a known correct answer. Record the immutable model digest, quantization, runtime and version, prompt template, sampling settings, context, CPU/GPU and memory, and processor split. After one warm-up, run each case at least three times and capture time to first token, output tokens per second, peak RAM/VRAM, failures, and a task-specific quality result such as tests passed, facts correctly cited, or required fields present. MLPerf Client shows why latency and throughput need a defined scenario; Aider's benchmark notes show why coding results depend on harness, editing format, and test conditions.

For RAG, score retrieval separately from answer generation: whether the needed passage appeared in the top results, whether the answer cited it, and whether unsupported claims were rejected. For tools or agents, require denied-action tests, human approval for consequential changes, and an audit record. A chat answer that looks sensible is not an end-to-end retrieval or tool-use result.

Risk, Recovery, and Use Boundaries

Download weights and containers only from traceable publishers, review model and dataset licenses for the intended personal or commercial use, and keep model files out of normal document backups unless their size and provenance are intentional. Treat prompts, retrieved documents, chat history, plugins, and agent credentials as separate data stores with separate access controls. “Runs locally” does not remove malware, browser, backup, or insider risk.

Keep a known-good small model and runtime version before changing drivers, quantization, or serving software. If a new model exhausts memory or destabilizes a shared server, stop it, restore the previous runtime/model digest, reduce context or concurrency, and rerun the baseline prompts. Agent tools need least-privilege credentials, explicit approval for destructive or external actions, logs, and a tested rollback path. Generated commands should be reviewed in a sandbox before they touch important systems.

What This Evidence Does Not Prove

• A larger parameter count does not prove better answers for your task.
• An MoE model's active-parameter count does not equal its storage or total memory requirement.
• A maximum context claim does not prove reliable recall across that entire window or practical memory use locally.
• A vendor or community benchmark does not prove the same ranking under another quantization, prompt template, runtime, language, or hardware path.
• An open-weight release does not automatically grant every use right or disclose all training data.
• Local inference does not prove privacy if the UI, tools, plugins, telemetry, or cloud fallbacks still have egress.
• A model that writes plausible code does not prove the code is correct, secure, licensed appropriately, or safe to execute.

Related TechGeeks Reading

• AI Workflow Notes: Start Here
• NVIDIA vs Intel GPUs for Local AI
• Implementing AgenticOps Safely
• Remote Access Without Opening Router Ports

Beginner Glossary

Term	Plain-English Meaning
AI model	The trained AI brain
LLM	Large Language Model, usually for text
Parameter	A learned internal number in the model
Token	A chunk of text
Prompt	What you type into the model
Inference	Running the model to generate an answer
Fine-tuning	Extra training for a specific task
Quantization	Compressing a model to use less memory
Context window	How much text the model can consider at once
Embedding	A numeric representation used for search
RAG	Search documents first, then answer using them
VRAM	GPU memory
RAM	System memory
Model weights	The downloaded learned data of the model
Base model	Raw model, not usually ideal for chat
Instruct model	Model tuned to follow instructions
Chat model	Model tuned for conversation
Multimodal model	Model that handles more than one input type, such as text and images
Vision model	Model that can process images
Runner	Software that loads and runs the model
Web UI	Browser interface for chatting
Agent	Workflow that can use tools and make decisions
Tool	External ability such as search or an API call
OCR	Turning scanned images into text
Vector database	Search database for embeddings
Offload	Splitting model work between GPU and CPU/RAM
KV cache	Runtime memory used to track context while generating

Bottom Line

For a beginner, the winning path is not "download the biggest model."

The winning path is:

1. Start with a tiny model.
2. Learn the stack.
3. Watch RAM, VRAM, CPU, and GPU.
4. Test your real tasks.
5. Move up one size at a time.
6. Add RAG for documents.
7. Add tools for search.
8. Use coding agents carefully.
9. Verify important answers.
10. Check licenses before business use.

Gemma is a great starting family because it scales from small learning models to more serious Gemma 4 options. Llama is the community default. Qwen is excellent to test for coding and multilingual work. Phi is useful when hardware is limited. DeepSeek is interesting for reasoning, but use distills locally. Mistral is broad and practical. Granite is worth testing for business workflows. Nemotron is exciting for NVIDIA agent labs, but not the easiest first stop.

Start small. Get something working. Then make it better.

References

• Google AI for Developers, Gemma 4 model card
• Google AI for Developers, Gemma releases
• Ollama, Gemma 4 model tags
• Ollama, Model library
• Ollama Docs, Embeddings
• Ollama Blog, Embedding models
• Open WebUI, Documentation
• Meta AI, The Llama 4 herd
• Llama, Official model site
• Qwen, Qwen3: Think Deeper, Act Faster
• Qwen, Qwen3.5 announcement
• Qwen, Hugging Face organization
• NVIDIA Developer, Nemotron AI Models
• NVIDIA Developer, Nemotron 3 Nano Omni
• Microsoft Azure, Phi open models
• DeepSeek, DeepSeek-V3 GitHub repository
• Mistral AI, Models overview
• Mistral AI, Open-model licensing
• IBM, Granite models
• Ollama, GGUF, Vulkan, and model support update
• MLCommons, MLPerf Client benchmark methodology
• Aider, Benchmark notes and limitations
• OpenAI Developers, Codex CLI
• OpenAI Developers, Codex advanced configuration
• OpenAI Developers, Codex sandboxing