Framework & Tool
llama.cpp
Pure-C++ inference of LLMs on CPU, GPU and Apple Silicon — the foundation of local AI.
Definition
llama.cpp by Georgi Gerganov pioneered fast quantised inference of Llama-family models on commodity hardware. It introduced GGUF format and Metal/CUDA/Vulkan backends, and underlies Ollama, LM Studio, GPT4All and many other consumer apps.
Common use cases
- Local inference
- Edge devices
- Privacy-preserving apps