# Run a model fully offline

Point the embedded engine at a folder of GGUF files and run a model in-process — no server, no key, no network.

You'll set up the **embedded** provider so a model runs entirely on your device
from local files — no API key, no base URL, no network call.

## What you need

- A GGUF model file (or several) already downloaded to the device. BYLM never
  ships a model; you bring the file.
- Enough device memory and storage for the model you pick.
- On Android, the files in a folder you can grant the app access to, and a
  64-bit Android: the embedded engine doesn't run on 32-bit Android.

## Steps

1. Open **Providers** and add a provider of kind **Llama.cpp on device**.
   It has no base URL and no API key.

![The embedded provider's form — a models-folder field and a compute backend (Vulkan), with no base URL or key](/images/provider_embedded.png)

2. Set the **Models directory** to the folder holding your `.gguf` files. On
   desktop, browse to the folder. On Android, grant a folder through the system
   file picker. The folder is scanned and each model file becomes selectable.

3. Optionally choose the **Compute backend** — _Auto_ lets the engine decide, or
   set CPU or Vulkan explicitly (ROCm as well, on Linux). It's saved per device
   and applied the next time a model loads.

4. Optionally cap **Decode threads** for CPU generation; leave it empty for an
   automatic count. Then tap **Save** — nothing on the form is saved until you
   do.

5. Create an agent, choose this provider, and pick one of the scanned models.
   Start a conversation — generation runs in-process, on your hardware, with
   nothing leaving the device.

## Make it yours

- **Keep an embedded agent beside a hosted one.** Because you choose the provider
  per agent, the same app can hold a private local agent and a frontier hosted
  agent side by side.
- **Load a vision projector only when it's needed.** A vision model's projector
  takes memory of its own; the provider's **Vision projector** setting can load
  it at the first image instead of with the model — see
  [On-device models](/docs/features/embedded-models#when-the-vision-projector-loads).
- **Swap backends to trade speed for compatibility** — Vulkan (or ROCm on Linux)
  to use the GPU, CPU as the safe fallback.

More in [Use Cases](/docs/use-cases): [Build a simple agent](/docs/use-cases/simple-agent) · [Give an agent a tool — fetch a URL](/docs/use-cases/fetch-tool)

---

[Bring Your LM](/) · [Docs](/docs/features) · [Marketplace](/marketplace)

[Articles](/articles) · [Releases](/releases) · [Bundle Format](/marketplace/bundle-format) · [Marketplace Terms](/terms) · [App Privacy](/privacy) · [Website Privacy](/website-privacy) · [Open Source](/open-source) · [llms.txt](/llms.txt)