Framework & Tool
Ollama
Command-line + REST tool that makes running local LLMs as simple as `ollama run llama3`.
Definition
Ollama wraps llama.cpp with a friendly CLI, an OpenAI-compatible HTTP API, and a model registry. It handles downloads, quantisation choice and prompt templating, making local self-hosting of dozens of open models trivial on macOS, Linux and Windows.
Common use cases
- Local inference
- Developer tooling
- Air-gapped apps