Skip to content
Orbit Docs

Running Your First Local Model

Published Updated

← Learning Atlas · Path 9 of 28 · Run your first AI model on your own computer: learn Ollama, llama.cpp, GGUF files and quantization, plus when vLLM and Hugging Face come into play.

This is the Learning Atlas from Orbit. Get your invite

Local models and inferenceHostinginfrastructuredatabases

  1. Step 1: Ollama

    Why this matters here

    Ollama is the easiest entry point — run it once and you have a local model responding to prompts without understanding the engine beneath.

    In short

    A lightweight, user-friendly tool that allows developers to easily download, run, and manage large language models locally on their own hardware.

    Full definition of Ollama →

    Next: llama.cpp ↓

  2. Step 2: llama.cpp

    Why this matters here

    llama.cpp explains the engine Ollama is built on, revealing why local inference works on ordinary laptop hardware.

    In short

    A highly optimized, bare-metal C/C++ library designed to run AI model inference efficiently on consumer hardware, particularly MacBooks and standard CPUs.

    Full definition of llama.cpp →

    Next: GGUF ↓

  3. Step 3: GGUF

    Why this matters here

    GGUF is the file format you'll download — knowing it helps you choose the right quantization level for your machine.

    In short

    A highly optimized file format specifically designed for storing and loading quantized large language models rapidly on consumer hardware.

    Full definition of GGUF →

    Next: Quantization ↓

  4. Step 4: Quantization

    Why this matters here

    Quantization explains why a 70B model can fit on a laptop. It's the most important concept for anyone choosing between local model variants.

    In short

    A technique used to compress the massive size of an AI model by reducing the precision of its mathematical weights, making it faster and cheaper to run.

    Full definition of Quantization →

    Next: vLLM ↓

  5. Step 5: vLLM

    Why this matters here

    vLLM is the production step up — once local testing works, vLLM shows how to serve the same model to multiple users efficiently.

    In short

    A high-throughput, open-source library for serving LLMs in production, famous for pioneering PagedAttention to efficiently manage memory during text generation.

    Full definition of vLLM →

    Next: Hugging Face ↓

  6. Step 6: Hugging Face

    Why this matters here

    Hugging Face is where you find model files, compare variants, and read community benchmarks before committing to a local deployment.

    In short

    The central collaborative hub for the machine learning community, hosting hundreds of thousands of open-weight models, datasets, and AI applications.

    Full definition of Hugging Face →

    Continue with Mobile app release tracks →