Home AI Hardware Levels: A Beginner's Guide to the PC You Need

If you are building your first home AI setup, the first confusing question is usually about hardware.

Interactive Hardware Tier Selector

Pick what you want to do. The selector shows the hardware level that usually fits that goal.

LearnExisting PCQuiet UIMini PCUseful Chat8-12GB GPUCoding/RAG16GB GPUSerious Lab24GB GPU31B Class32GB GPULab Server48GB+ GPUOccasional HugeHosted

Select a goal

Choose a box above to see the matching hardware tier.

Do you need a new PC? Can you use an old desktop? Is 8GB of VRAM enough? Why do people keep talking about RTX 3090s, RTX 4090s, and workstation GPUs? And what happens if the same box also runs Plex, Jellyfin, Frigate, or Tdarr?

This is the hardware decision guide in the TechGeeks home AI series. Use it before an Ollama install guide, Open WebUI setup, model-picking post, or RAG/document workflow build. It helps you decide what class of PC makes sense before you spend money.

You do not need to understand every acronym before you start. The goal is not to chase the biggest system possible. The goal is to pick the level that matches what you actually want to do.

Start Here: The Fast Answer

If you only read one section, start here.

If you want to...	Start here
I just want to learn local AI	Use your current PC first
I want a private ChatGPT-style web page at home	Use Ollama plus Open WebUI on a small server or desktop
I want local AI to feel useful every day	Use a desktop with a 12GB to 16GB NVIDIA GPU
I want coding help and document chat	Aim for a 16GB to 24GB GPU
I want a serious one-GPU AI homelab	Aim for a 24GB GPU
I want to run 31B-class models	Start looking at 32GB+ VRAM
I want large models, many users, or lots of headroom	Look at 48GB+ workstation hardware or hosted inference

The beginner mistake is buying hardware before you know what you want to run. If you are unsure, start one level lower than your first instinct suggests. A week of real usage will tell you whether you need more speed, more VRAM, more system RAM, or a separate box.

Current Runtime Context: July 15, 2026

The table above is a capacity-planning shortcut, not a benchmark result. Exact fit depends on the model build, quantization, context length, KV-cache format, runtime, operating system, and other workloads. Ollama chooses a default context from available VRAM and documents 4K below 24 GiB, 32K from 24 GiB to 48 GiB, and 256K at 48 GiB or more; increasing context can materially increase memory use. After loading a model, use ollama ps to confirm its actual processor split and context instead of assuming the GPU owns the whole workload.

NVIDIA remains the conservative compatibility choice for many local-AI tools, but it is no longer the only practical path. Ollama 0.30 enabled its GGUF/Vulkan engine by default across more AMD, Intel, and NVIDIA hardware, while Ollama's MLX work targets Apple silicon. Those announcements establish supported execution paths, not equal speed or feature parity on every device. An advertised NPU TOPS number is also irrelevant unless the exact runtime and model actually use that NPU. Recheck the Ollama hardware matrix, model tag, and driver requirements on publication day and again before buying.

Plain English: VRAM Is the First Bottleneck

Interactive Model-to-Hardware Recommender

Select the model classes and workloads you want to run. The result gives a practical starter hardware tier.

Tiny 1B-3B chat models Small 7B-9B models Medium 12B-14B models 26B-class models 31B-class models Coding assistant workloads RAG / document chat AI plus Plex/Jellyfin/Tdarr/Frigate Multiple users

No selection yet

Choose one or more model/workload boxes above.

VRAM Ladder: What Each Tier Feels Like

More VRAM usually means larger models, more context, or more headroom for other services.

Integrated

Learning only

8GB

Starter AI

12GB

Better small models

16GB

Comfortable home AI

24GB

Serious homelab

32GB

31B-class headroom

48GB+

Lab/server tier

Reminder: VRAM does not automatically guarantee speed or quality. Quantization, context length, and background workloads still matter.

For home AI, the most important spec is usually GPU VRAM, which means the memory on the graphics card. On a normal desktop with a separate NVIDIA or AMD card, VRAM is separate from normal system RAM. Some systems, such as Apple Silicon Macs and some integrated-GPU setups, use shared or unified memory instead, so read their memory specs a little differently.

System RAM, CPU, and storage matter, but if you want local AI models to feel good, VRAM is usually the first bottleneck because the model has to fit in memory before it can run well.

Think of VRAM like desk space. If the model is a big paper map, VRAM is the desk. A bigger desk lets you open a bigger map and keep more notes beside it. If the desk is too small, you can still try to work, but everything gets slower and more awkward.

VRAM is not a guarantee that every model in a named size class will run well. Quantization, context length, concurrent users, and background GPU work all change memory use. Still, VRAM is the easiest hardware number for beginners to use when comparing systems.

Think of it this way:

Part	Plain-English role
GPU VRAM	Decides which models fit and how much context you can use comfortably
GPU compute	Decides how fast tokens generate when the model is running on the GPU
System RAM	Normal computer memory; helps when some model work spills over from the GPU to the CPU and keeps the server stable
CPU	Handles the operating system, Docker containers, background services, and model work that falls back to the CPU
NVMe storage	Fast SSD storage for models, containers, databases, documents, and indexes
PSU/cooling	Power supply and cooling; keeps the system stable under long AI and media workloads

A token is a small chunk of text that the model reads or writes. Context is the amount of text the model can keep in mind at once. When you see model sizes like 3B, 8B, or 31B, the B means billion parameters, which is a rough measure of model size. An agent is an AI workflow that takes multiple steps or uses tools instead of just answering one chat message. Bigger models can be more capable, but they need more VRAM and are not always the right first step.

If you are just starting, do not buy hardware yet. Install Ollama and try a small model first. That will teach you more than a spec sheet will.

Beginner Vocabulary

These are the terms you will see over and over when setting up home AI.

Term	Plain-English meaning
Model	The AI brain you download and run, such as Gemma, Llama, or Qwen
Ollama	The local service that downloads models and runs them on your computer
Open WebUI	A browser-based chat interface for Ollama and other AI providers
VRAM	Memory on the graphics card; the biggest hardware limit for local AI
RAM	Normal computer memory used by the operating system and apps
Context	How much text the AI can pay attention to in one request
Quantization	A smaller version of a model that uses less memory, usually with some tradeoff
RAG	A setup where the AI searches your documents before answering
Agent	An AI workflow that can take multiple steps or use tools
Hosted inference	Running the model on someone else's GPU server instead of your own PC

Hardware Level 0: Existing PC

This is the "try it before spending money" level. It is the best place to learn the tools before buying parts.

In normal language: this is the "I want to see what local AI is about" setup. It may be slow, but it teaches you the workflow with almost no risk.

Typical specs:

Component	Typical range
CPU	Any reasonably modern Intel/AMD desktop or laptop CPU
RAM	8GB to 16GB
GPU	Integrated graphics or older GPU
Storage	100GB+ free space

Integrated graphics means the GPU is built into the computer instead of being a separate graphics card. It usually shares system memory and is much slower for AI than a dedicated NVIDIA GPU.

Good for:

• Learning Ollama
• Trying Open WebUI
• Running very small models
• Testing prompts, which are the instructions you give the model
• Learning what local AI feels like

Not good for:

• Large models
• Fast coding assistants
• Long-context agents
• Multiple users
• Running AI while also doing heavy media transcoding

Suggested models:

ollama pull llama3.2:1b
ollama pull llama3.2:3b

These are small-model examples for learning the workflow. They are not meant to represent the best possible local AI experience.

What to expect:

Small models may run, but they may be slow. If there is no real GPU acceleration, meaning the graphics card is not doing the AI work, the CPU will do most of the work. On an 8GB RAM system, start with 1B or 3B text models. Gemma 4 E2B can be useful on more capable older PCs, but its current Ollama tag is larger than the name suggests, so treat it as a step up from the smallest starter models.

Hardware Level 1: Mini PC or Low-Power Server

This level is for people who want a quiet, always-on AI test box. Think of it as a small server that can host the web interface and light workloads.

In normal language: this can be a good little control center for Open WebUI, but it usually is not the machine you buy for big models.

Typical specs:

Component	Recommended target
CPU	Modern Intel Core, AMD Ryzen, or similar mini PC CPU
RAM	16GB to 32GB
GPU	Integrated GPU
Storage	1TB NVMe preferred

Good for:

• Open WebUI frontend
• Lightweight local chat
• Document experiments using small models
• Running a central Ollama server for tiny models
• Learning Docker and reverse proxy basics

Not good for:

• Big coding models
• 12B+ models at a good speed
• 31B-class models
• Serious multi-user AI

Recommended use:

Use the mini PC as the web UI and services box, not necessarily as the heavy model box. Open WebUI is the browser interface you use to chat with models. Ollama is the service that downloads and runs those models. Later in the series, this is the kind of machine that can point Open WebUI to a bigger GPU desktop or a hosted provider, which is a cloud service that runs models somewhere else.

Hardware Level 2: Starter GPU Desktop, 8GB to 12GB VRAM

This is the first level where local AI starts to feel useful.

In normal language: this is the first real local AI desktop tier. It is good for learning and daily light use, but you should stay with smaller models.

Typical specs:

Component	Recommended target
CPU	6-core or 8-core desktop CPU
RAM	32GB
GPU	NVIDIA GPU with 8GB to 12GB of VRAM
Storage	1TB to 2TB NVMe
PSU	Quality 550W to 750W depending on the GPU

Good for:

• Gemma 4 E2B, with Gemma 4 E4B better suited to 12GB cards and modest context
• Llama 3B/8B-class models
• Small coding models
• Open WebUI with one or two users
• Learning RAG with small document sets

RAG means retrieval-augmented generation. In plain English, the system searches your documents first and then gives the relevant pieces to the AI model so it can answer with that extra context.

Not good for:

• 26B/31B-class models
• Long-context agents
• Heavy coding agents
• AI and heavy Tdarr jobs at the same time

Beginner model choices:

ollama pull llama3.2:3b
ollama pull gemma4:e2b
ollama pull qwen2.5-coder:7b

Practical advice:

An 8GB GPU can be useful, but it is tight. Use smaller models, lower context, and check ollama ps to confirm whether the model is fully on the GPU or split between the CPU and GPU. A 12GB GPU gives you much more breathing room for small and medium models, including trying gemma4:e4b, especially if you want longer prompts or a web UI running at the same time.

Model examples age quickly. If a newer coding model is available in Ollama by the time you read this, use the same hardware logic rather than treating the exact tag above as permanent. Smaller current coding models are the starter tier; larger coding tags such as 30B-class models belong on higher-VRAM systems.

Hardware Level 3: Comfortable Home AI Box, 16GB VRAM

This is the sweet spot for many homelab users. It is still a normal desktop-class build, but with enough memory and GPU capacity to feel practical.

In normal language: this is where local AI starts feeling less like a science project and more like a useful tool.

Typical specs:

Component	Recommended target
CPU	Ryzen 7 / Core i7 class or better
RAM	64GB
GPU	16GB VRAM
Storage	2TB NVMe
PSU	Quality 750W to 850W

Good for:

• Better small/medium models
• Coding assistants with scoped tasks, such as editing one project or solving a specific bug
• Open WebUI for a family or small team
• RAG with a reasonable document set
• Running AI while the server also does light background work

Not ideal for:

• 31B models without compromises
• Many users at once
• Long autonomous agents
• Heavy media transcoding at the same time

What this feels like:

This is the level where local AI stops feeling like a toy. You still need realistic expectations, but you can run useful models and switch between chat, coding, and document work.

Hardware Level 4: Serious Single-GPU Homelab, 24GB VRAM

This is where local AI gets much more capable, especially if you want one powerful GPU instead of a rack of lab hardware.

In normal language: this is the "I am serious about this" level for one GPU.

Typical specs:

Component	Recommended target
CPU	Ryzen 9 / Core i9 / high-end Core Ultra
RAM	64GB to 128GB
GPU	24GB VRAM
Storage	2TB to 4TB NVMe
PSU	Quality 850W to 1000W

Common GPU examples:

GPU class	Why people like it
RTX 3090 24GB	Used-market value and lots of VRAM, but check card condition, power draw, and cooling
RTX 4090 24GB	Strong performance and 24GB VRAM, but expensive
Used workstation cards	Sometimes good VRAM-per-dollar value, but check power, drivers, and cooling

NVIDIA lists both the RTX 3090 and RTX 4090 as 24GB cards. The RTX 4090 is much faster, but it does not give more model-size headroom than a 24GB RTX 3090.

Good for:

• 12B to 14B models comfortably
• Some 20B-ish quantized models
• Some 26B/31B-class models with limits
• Local coding assistants
• Better RAG and document workflows
• Open WebUI as a serious household or lab service

Important limit:

24GB VRAM is strong, but it is not infinite. It can run many larger quantized models, and some 31B-class tags may fit with low enough context and overhead. Quantized means the model is stored in a smaller format so it uses less memory. Long context, multiple users, and background GPU workloads can still push 24GB over the edge.

Hardware Level 5: 31B-Capable Workstation, 32GB VRAM

This is the target if you want to run 31B-class local models more comfortably. A 31B-class model is much larger than the 7B and 8B models most beginners start with.

In normal language: bigger local models become realistic here, but context length, model format, and other workloads still matter.

Typical specs:

Component	Recommended target
CPU	Ryzen 9, Core Ultra 9, Core i9, or Threadripper
RAM	128GB preferred
GPU	32GB VRAM preferred
Storage	2TB to 4TB NVMe
PSU	1000W to 1200W
Cooling	High-airflow case, serious CPU/GPU cooling

Common GPU example:

• RTX 5090 with 32GB VRAM

NVIDIA lists the RTX 5090 with 32GB of GDDR7 memory. GDDR7 is the type of memory on the graphics card. That capacity makes it a more comfortable consumer-class option for 31B local work than a 24GB card.

For example, Ollama currently lists the local gemma4:31b tag at about 20GB before KV cache and runtime overhead. The point of 32GB VRAM is the extra headroom, not a guarantee that every 31B long-context workflow will fit.

Before buying at this tier, verify the exact card specs, case clearance, power connectors, PSU requirements, and Linux driver support. The GPU may be the headline part, but the platform around it still matters.

Good for:

• 31B-class quantized models
• Larger context than smaller cards
• Better local coding-agent experiments
• More serious RAG and planning workflows
• Running Open WebUI as a real home AI service

Still not magic:

Even 32GB VRAM can be limited by context length, quantization, multiple users, and background GPU work. A 31B model may fit, but fitting is only the first step. It does not mean every long-context workflow will be fast.

Hardware Level 6: Workstation or Small Lab Server, 48GB+ VRAM

This level is for people who are serious about local AI and want room to grow. It is closer to workstation or lab-server territory than a normal gaming PC.

In normal language: this is no longer a casual PC upgrade. This is workstation or lab hardware.

Typical specs:

Component	Recommended target
CPU	Threadripper, Threadripper Pro, Xeon, or EPYC-class platform
RAM	128GB to 256GB+
GPU	48GB single GPU, or multiple GPUs for supported split/parallel workloads
Storage	4TB+ NVMe plus backup storage
PSU	1200W+ depending on the GPUs
Networking	2.5GbE, 10GbE, or better for lab use

Common GPU examples:

• NVIDIA RTX 6000 Ada with 48GB ECC VRAM
• NVIDIA RTX PRO 6000 Blackwell-class workstation cards with much larger VRAM budgets, if your budget and platform fit that tier

NVIDIA lists the RTX 6000 Ada with 48GB of ECC GDDR6 memory. ECC means error-correcting memory, which is more common in workstation hardware. That kind of card is expensive, but it gives much more headroom for local AI than typical gaming GPUs.

Do not assume two smaller GPUs automatically behave like one larger VRAM pool. Model splitting depends on the inference stack, the model, and the performance tradeoffs you are willing to accept.

This tier is usually not beginner hardware. Noise, heat, power, rack space, replacement parts, and platform support matter as much as the model size you want to run.

Good for:

• Larger models
• Larger context
• Agent workflows
• Multi-user Open WebUI
• Local development and testing
• More serious RAG datasets

This is overkill for:

• Basic chat
• Learning Ollama
• One user asking simple questions
• A first homelab AI experiment

Hardware Level 7: Use Hosted Inference Instead

Sometimes the right hardware level is "do not buy the hardware." Hosted inference means using someone else's GPU server through a web service or API instead of running every model at home.

In normal language: sometimes it is cheaper to rent the big model only when you need it instead of buying a large GPU that sits idle most of the week.

Hosted inference makes sense when:

• You only use big models occasionally
• You need frontier-size or cloud-only model families
• You need very long context without buying a 48GB+ card
• You need high reliability
• You do not want GPU heat, power draw, and noise
• You are still learning what you actually need

Ollama cloud models are one hybrid option: local tools can keep using the Ollama interface while supported cloud models run remotely instead of running on your GPU.

The best home setup is often hybrid:

Job	Run locally?
Private notes and small chat	Yes
Simple coding help	Yes, with the right model
Heavy reasoning once in a while	Maybe hosted
Huge agents or long context	Often hosted
Sensitive internal docs	Prefer local or private infrastructure

What About Context Length?

Context Length Mini Chart

More VRAM can allow larger default context windows, but actual loaded settings still need to be checked.

Under 24GB VRAM

4K

24GB to under 48GB

32K

48GB+ VRAM

256K

Caution: these defaults are not buying guarantees. Use ollama ps after loading a model to confirm context and processor placement.

Context length is how much text the model can pay attention to at once. More context lets the model consider more of a document, chat, or codebase in one request.

Long context is useful for:

• Large documents
• Coding agents
• Research tasks
• Long chat histories
• Multi-step workflows

But long context costs memory. In the table below, 4K means about 4,000 tokens, 32K means about 32,000 tokens, and 256K means about 256,000 tokens. Ollama's current context length page lists default context levels based on available VRAM:

VRAM	Ollama default context
Under 24GB	4K context
24GB to under 48GB	32K context
48GB or more	256K context

That is one reason the 24GB, 32GB, and 48GB tiers matter so much, but model size, quantization, KV cache, and other GPU workloads still matter.

Ollama's docs are not perfectly consistent here: the context length page now lists the VRAM-based defaults above, while the FAQ still says the default context window is 4096 tokens when explaining how to override it. Do not assume the number in a table is what your model actually loaded with. Use ollama ps to check processor placement and context after the model is running. Cloud models are set to their maximum context length by default.

What If I Also Run Plex, Jellyfin, Frigate, or Tdarr?

Media + AI Contention Flow

Plex / JellyfinUsually uses NVENC/NVDEC for transcodes, plus VRAM.

TdarrBatch ffmpeg jobs can run for hours and overlap with AI.

FrigateVideo analysis may use GPU, iGPU, Coral, or other accelerators.

OllamaNeeds VRAM and CUDA/Tensor compute when models run on GPU.

Shared LimitsVRAM, heat, power, driver scheduling, and copy engines.

FixesSchedule jobs, pause Tdarr, use smaller models, add a second GPU, or split boxes.

Beginner rule: do not plan a GPU around one workload. Plan around what runs at the same time.

Media services change the hardware decision because "GPU usage" is not one single thing.

On NVIDIA systems, there are four resources to think about:

Resource	Plain-English role
NVDEC	Fixed-function video decoding hardware
NVENC	Fixed-function video encoding hardware
CUDA/Tensor compute	General GPU math used by local AI inference, TensorRT, and some filters
VRAM	Shared GPU memory used by models, transcodes, frame buffers, and GPU apps

This distinction matters. Hardware transcoding in Plex and Jellyfin mostly uses the video encode/decode engines, not the same CUDA/Tensor compute that runs an LLM. That is why one or two light transcodes may coexist with a small model better than people expect.

But it is not free. All of those jobs still share VRAM, driver scheduling, PCIe bandwidth, power limits, heat, and sometimes CUDA for scaling, tone mapping, filters, or AI features. A model that barely fits in VRAM by itself may fail, slow down, or get pushed partly to the CPU when a media job starts.

Common media workloads:

Workload	What it usually uses	Coexistence risk
Plex transcode	Video decode/encode engine, CPU fallback for unsupported paths	Moderate: easy to underestimate during 4K or HEVC/HDR work
Jellyfin transcode	FFmpeg with Intel Quick Sync, NVIDIA NVDEC/NVENC, VA-API, AMF, VideoToolbox, or CPU	Moderate to high, especially with tone mapping or subtitle burn-in
Tdarr batch transcode	Long-running FFmpeg or HandBrake workers, CPU or GPU	High if GPU workers run while you are using AI
Frigate video decode	FFmpeg hardware decode when configured	Usually manageable, but many cameras add up
Frigate object detection	CPU, Coral Edge TPU, Hailo, OpenVINO, ONNX, TensorRT, or other accelerators	High if it uses the same NVIDIA GPU as Ollama
Ollama/Open WebUI	VRAM plus CUDA/Tensor compute when models are on the GPU	High sensitivity to available VRAM

Also remember that Plex hardware transcoding usually requires Plex Pass, while Jellyfin hardware acceleration is free but more sensitive to setup.

Beginner rule:

If a GPU has 8GB to 12GB of VRAM, treat AI and media transcoding as separate jobs. Do not plan on running a model that barely fits while Plex, Jellyfin, Frigate, or Tdarr also use the same GPU.

Better beginner setup:

Hardware layout	Why it helps
Intel iGPU for Plex/Jellyfin, NVIDIA GPU for AI	Quick Sync handles media while the NVIDIA card keeps VRAM for models
Intel iGPU or Coral/Hailo for Frigate, NVIDIA GPU for AI	Camera decode/detection stays off the model GPU
Tdarr on a schedule	Batch transcodes run overnight instead of during interactive AI use
Separate media box and AI box	Cleanest option when both workloads matter every day
Larger single GPU	Works, but only if you leave VRAM headroom and avoid overlapping heavy jobs

Scheduling advice:

• Prefer Direct Play in Plex/Jellyfin whenever possible. Every avoided transcode is free capacity.
• Limit Tdarr GPU workers. Start with one GPU worker, then increase only if AI and streams stay stable.
• Run Tdarr during quiet hours, not during the evening streaming window or coding-agent sessions.
• Pause Tdarr before loading a large model or long-context coding agent.
• If Ollama keeps a large model loaded, unload it before a big media batch with ollama stop <model>.
• For Frigate, separate video decoding from object detection in your mental model. Hardware decode reduces CPU use, while GPU object detection competes more directly with AI workloads.
• In Docker, pass GPU or iGPU devices only to the containers that need them. Containers do not automatically reserve separate VRAM for each app.
• Use nvidia-smi, Plex/Jellyfin dashboards, Frigate logs, and Tdarr worker status to see what is actually using the GPU.

VRAM headroom matters more than the spec sheet suggests:

GPU size	Practical coexistence guidance
8GB	Pick one heavy job at a time. AI and media together are mostly a learning setup.
12GB	Small models can coexist with light media use, but avoid 4K transcodes and Tdarr GPU batches.
16GB	Reasonable for light coexistence if models are not maxing out VRAM.
24GB	Good single-box target, but large models and 4K/HDR transcodes still need scheduling.
32GB+	Much more comfortable, but not a substitute for monitoring and worker limits.
Separate GPU/box	Best experience if media and AI are both important services.

The practical takeaway: do not buy a GPU only by asking, "Can it transcode?" or "Can it run this model?" Ask whether it can handle your model, your context length, your live streams, your cameras, and your batch jobs at the same time.

Recommended Starting Points by Budget Mindset

These are not exact shopping lists. They are target levels. Prices and used-GPU values change quickly, so buy the class of hardware that fits your workload instead of treating any single card in this article as a permanent best deal.

Build type	Hardware target
"I just want to learn"	Existing PC, 16GB RAM if possible, small models
"I want a quiet AI web UI"	Mini PC, 32GB RAM, Open WebUI, small models
"I want useful local chat"	Desktop, 32GB RAM, 12GB GPU
"I want coding and RAG"	Desktop, 64GB RAM, 16GB GPU
"I want serious local AI"	Desktop, 64GB to 128GB RAM, 24GB GPU
"I want 31B-class models"	Workstation, 128GB RAM, 32GB GPU
"I want lab/server headroom"	Workstation/server, 128GB to 256GB RAM, 48GB+ GPU

Recommended Upgrade Path

Beginner Upgrade Path

Upgrade after you know what is slow or too small in your actual workload.

1Use Current PC

Install Ollama, try small models, and learn the workflow.

2Add Web UI

Run Open WebUI and decide whether the service should stay on one box.

3Move to 12-16GB GPU

Get useful local chat, coding help, and small RAG projects.

4Upgrade to 24GB

Build a serious single-GPU homelab with better model headroom.

5Target 32GB+

Only jump here when you know you need 31B-class models or larger context.

6Split or Host

Use a second box/GPU or hosted inference for rare heavy jobs.

For most home users:

1. Start with your current PC.
2. Install Ollama and Open WebUI.
3. Try small models.
4. Decide if you need speed, size, or both.
5. Upgrade to a 12GB or 16GB GPU if you are learning.
6. Upgrade to 24GB if you are serious.
7. Build around 32GB+ only if you know you need larger models.

The mistake is buying a huge system before you know your actual workload. Try the small setup first, then let your real use case tell you where the bottleneck is.

Three Beginner Mistakes to Avoid

Before buying parts, watch out for these common traps.

Mistake	Why it hurts	Better approach
Buying a GPU only because it is fast in games	Gaming speed and local AI memory needs are not the same thing	Check VRAM first, then performance
Assuming two GPUs automatically combine VRAM	Most local AI setups do not treat two smaller cards like one big card	Prefer one larger VRAM card unless you know your software supports splitting
Running large AI models while Tdarr, Plex, Jellyfin, or Frigate uses the same GPU	Media jobs can take VRAM, heat, power, and GPU scheduling headroom	Schedule heavy jobs or split media and AI across different hardware

The simple rule: buy for the workload that runs at the same time, not just the one headline task you care about most.

Evidence and Testing Method

The sizing guidance in this article is documentation-backed planning advice. TechGeeks did not independently benchmark every GPU, CPU, Apple silicon generation, model, quantization, or shared-media combination named here. Vendor specifications establish memory capacity and supported features; they do not establish the speed, acoustics, power draw, or answer quality you will see.

Before buying, test the same three to five real prompts on hardware you already own or can return. Record the exact CPU/GPU and memory, operating system, driver, Ollama version, immutable model digest, quantization, configured context, and whether Plex, Jellyfin, Frigate, or Tdarr is active. Run each prompt at least three times after one warm-up and capture time to first token, output tokens per second, peak RAM and VRAM, wall power if a meter is available, fan/noise behavior, and any media-stream interruption. MLPerf Client is a useful independent example of why client-AI comparisons report latency and throughput by defined hardware and scenario rather than by model name alone.

Set acceptance criteria before shopping: the smallest useful model must fit without destabilizing the host, first-token delay and generation speed must be tolerable for your task, concurrent services must stay within their latency/error budget, and sustained heat and power must be acceptable. If the current machine passes, the evidence says to defer the purchase.

Purchase, Risk, and Recovery Boundaries

Do not make a shared home server the only copy of important data while experimenting. Back up Ollama/Open WebUI configuration and user data, keep the previous GPU and known-good driver available until the new path passes, and change one variable at a time. Check chassis clearance, connector type, power-supply capacity and cable guidance, circuit load, cooling, used-card return terms, and workstation/server compatibility before purchase. For a used GPU, test memory stability and sustained load inside the return window; a screenshot showing the correct VRAM is not a burn-in result.

If a driver or runtime update breaks inference, roll back to the recorded driver/runtime combination, restore the previous model tag or digest, and confirm that the baseline prompt set works before reintroducing media workloads. Hosted inference is a valid fallback for occasional large jobs, but it changes privacy, data-processing, account, and recurring-cost boundaries. Model licenses and the rights to documents used for RAG still apply whether the hardware is local or rented.

What This Evidence Does Not Prove

• A model fitting in VRAM does not prove it will meet your latency or quality target.
• A vendor's maximum context does not prove that context is practical at your chosen quantization or that the model uses distant context reliably.
• Vulkan or MLX support does not prove CUDA-equivalent speed, identical features, or support for every GPU generation.
• Fast tokens per second does not prove factual accuracy, safe tool use, or good results on your documents.
• A successful idle test does not prove stability while video transcodes, camera analysis, backups, and other users compete for the same host.
• The tier table does not prove a specific product is a good value; prices, warranties, thermals, and used-market condition change.

Related TechGeeks Reading

• AI Workflow Notes: Start Here
• NVIDIA vs Intel GPUs for Local AI
• Old Gaming PC vs Mini PC for a 24/7 Server
• Homelab Electricity Cost Calculator

Bottom Line

For home AI, hardware levels look like this:

Level	Best description
Existing PC	Learn the basics
Mini PC	Low-power web UI and small models
8GB to 12GB GPU	Starter local AI
16GB GPU	Comfortable home AI
24GB GPU	Serious single-GPU homelab
32GB GPU	31B-capable workstation with quantization
48GB+ GPU	Lab/server-class local AI
Hosted inference	When local hardware is not worth buying

If you are new, start small. If you are serious, buy VRAM. If you are running AI next to media services, plan for headroom.

For the rest of the TechGeeks home AI series, this post is the hardware map: use it to choose the box, then move on to software setup, model selection, Open WebUI, RAG, and safe remote access.

References

• Ollama context length documentation: https://docs.ollama.com/context-length
• Ollama 0.30 GGUF and Vulkan update: https://ollama.com/blog/improved-performance-and-model-support-with-gguf
• Ollama MLX performance on Apple silicon: https://ollama.com/blog/mlx-performance
• Ollama FAQ and GPU memory visibility: https://docs.ollama.com/faq
• Ollama cloud documentation: https://docs.ollama.com/cloud
• Ollama Gemma 4 model library: https://ollama.com/library/gemma4
• Ollama Llama 3.2 model library: https://ollama.com/library/llama3.2
• Ollama Qwen2.5-Coder model library: https://ollama.com/library/qwen2.5-coder
• NVIDIA RTX 3090 / 3090 Ti official specs: https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/
• NVIDIA RTX 5090 official specs: https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/
• NVIDIA RTX 4090 official specs: https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/
• NVIDIA RTX 6000 Ada official specs: https://www.nvidia.com/en-us/products/workstations/rtx-6000/
• NVIDIA Video Codec SDK: https://developer.nvidia.com/video-codec-sdk
• Plex hardware-accelerated streaming: https://support.plex.tv/articles/115002178853-using-hardware-accelerated-streaming/
• Jellyfin hardware acceleration: https://jellyfin.org/docs/general/post-install/transcoding/hardware-acceleration/
• Jellyfin NVIDIA hardware acceleration: https://jellyfin.org/docs/general/post-install/transcoding/hardware-acceleration/nvidia/
• Frigate video decoding hardware acceleration: https://docs.frigate.video/configuration/hardware_acceleration_video/
• Frigate object detectors: https://docs.frigate.video/configuration/object_detectors/
• Tdarr FAQ: https://docs.tdarr.io/docs/faq/
• Tdarr automations: https://docs.tdarr.io/docs/plugins/flow-plugins/automations/
• MLCommons MLPerf Client benchmark methodology: https://mlcommons.org/benchmarks/client/