Ollama — the Docker of local models

If llama.cpp is the engine, Ollama is the product: one install, then models become as easy as containers. The Docker analogy isn't loose — it's the literal design:

Docker	Ollama
`docker pull nginx`	`ollama pull llama3.1:8b`
Docker Hub	ollama.com registry
Dockerfile	Modelfile (`FROM`, `SYSTEM`, `PARAMETER`)
`docker run`	`ollama run`

What it silently handles — each item a scar you now recognize: quant selection (default pulls are typically Q4_K_M, Topic 29's sweet spot; tags like :8b-instruct-q8_0 pick others), chat templates (Topic 15's silent killer, defused), model loading/keep-alive, Metal/CUDA detection. And it runs a local server on port 11434 speaking the OpenAI dialect, which means this works against everything you'll ever build:

from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
r = client.chat.completions.create(model="llama3.1:8b",
        messages=[{"role": "user", "content": "hello"}])

Same three lines, different base_url, and your app switches between local Ollama, a vLLM cluster, or a commercial API. You used Ollama in Topics 29 and 34; now you know its whole shape. Its honest limit: it's single-user-oriented — some parallelism, but not built for real concurrent load. That's the next topic's job.

Summary

Friendly llama.cpp wrapper with registry, Modelfiles, and an OpenAI-compatible local server; the default way individuals run local models.

Mental model

Docker, for models.

Mistakes to avoid

Serving an actual user base from it (wrong tool — vLLM exists), and forgetting that ollama run model on an 8 GB machine with a 14 GB model will offload and crawl — the Topic 31 budget math still governs everything.

Exercise

Write a Modelfile that wraps a base model with the persona from your Lesson 6 fine-tune's system prompt (FROM llama3.1:8b, a SYSTEM block, PARAMETER temperature 0.3), then ollama create and compare its behavior against your actual fine-tuned GGUF. You're empirically re-testing the Topic 12 ladder: how far does prompting alone get you?