Skip to content
Orbit Docs

Running Your First Local Model

← Learning Atlas · Path 9 of 22

6 ordered steps, from Ollama to Hugging Face.

Local models and inferenceHostinginfrastructuredatabases

  1. Step 1: Ollama

    Ollama is the easiest entry point — run it once and you have a local model responding to prompts without understanding the engine beneath.

  2. Step 2: llama.cpp

    llama.cpp explains the engine Ollama is built on, revealing why local inference works on ordinary laptop hardware.

  3. Step 3: GGUF

    GGUF is the file format you'll download — knowing it helps you choose the right quantization level for your machine.

  4. Step 4: Quantization

    Quantization explains why a 70B model can fit on a laptop. It's the most important concept for anyone choosing between local model variants.

  5. Step 5: vLLM

    vLLM is the production step up — once local testing works, vLLM shows how to serve the same model to multiple users efficiently.

  6. Step 6: Hugging Face

    Hugging Face is where you find model files, compare variants, and read community benchmarks before committing to a local deployment.