RailwailRailwail
Framework & Tool

llama.cpp

Pure-C++ inference of LLMs on CPU, GPU and Apple Silicon — the foundation of local AI.

Definition

llama.cpp by Georgi Gerganov pioneered fast quantised inference of Llama-family models on commodity hardware. It introduced GGUF format and Metal/CUDA/Vulkan backends, and underlies Ollama, LM Studio, GPT4All and many other consumer apps.

Common use cases

  • Local inference
  • Edge devices
  • Privacy-preserving apps

Related terms

Ollama
Quantization
INT4
Back to glossary
    llama.cpp — AI Glossary | Railwail