Phase 6 · Modern LLM engineering · Inference
Running open models locally
Download, quantize and run a model on your own machine.
Open this lesson Create a free account
Why it matters
Open-weight models (Llama, Qwen, Gemma, Mistral, DeepSeek and others) can be downloaded and run on your own hardware, with your data never leaving it. For some tasks a laptop-sized model is good enough, free per token, private and available offline. Knowing what fits where, and what the licenses allow, turns "can we run this ourselves?" from a guess into a five-minute calculation.
What you'll be able to do
- Estimate whether a model fits on a machine, and how fast it will run, from its size and quantization
- Run an open model locally with Ollama and call it from Python
- Read the parts of a model card and license that matter for business use
- Decide when a local open model beats a hosted API, and when it doesn't
Before this lesson
Next in this phase: Benchmarks and their limits