Skip to content
Orbit Docs

GGUF

Published Updated

Also called GPT-Generated Unified Format

What is GGUF?

A highly optimized file format specifically designed for storing and loading quantized large language models rapidly on consumer hardware.

Created by the developer of llama.cpp, GGUF replaces older formats to provide a single file that contains both the model weights and all necessary metadata (like prompt templates). It is designed to be mapped directly into memory (mmap), allowing massive models to load and execute incredibly fast on CPUs and Apple Silicon.

How it relates to Orbit

Ollama and LM Studio both run GGUF model files, and Orbit can route a complexity tier to either. See Complexity Tiers & Routing for details.

When you would use it

You specifically look for GGUF files when you intend to run a model locally on consumer hardware, particularly if you are using tools based on llama.cpp.

Common operations

  • Downloading a single .gguf file to run a model in LM Studio.
  • Distributing a highly compressed version of a model on Hugging Face.

Related terms

Where this is taught