Glossary · Phase 6 · Modern LLM engineering
Llama.cpp
A fast open-source program for running quantized models on ordinary computers, including laptops without a big GPU.
Learn it properly
Running open models locally · Download, quantize and run a model on your own machine. About 30 minutes.
Other terms from the same lesson
- Hugging Face Hub: The main website for sharing open models and datasets, each with a model card.
- Ollama: A free app for downloading and running open models on your own computer with one command.
- unified memory: Memory shared by the CPU and GPU, as on Apple Silicon Macs, so a large model can use most of the computer's memory.