# Providers — bring your own LLM

The kinds of model backend you can point an agent at, what each one needs from you, and how much it can tell you about a model.

Status: Shipped

Bring Your LM never ships a model. A **provider** is the backend that actually
runs one — you bring it, and every agent you build runs on a provider you
configured. They range from a hosted API you pay for to a model file that runs
entirely on your device.

## Why it matters

The provider is the one choice that decides where your prompts go and who pays
for the compute. A hosted provider is the least setup; a local server or the
embedded engine keeps everything on hardware you control. Because you pick per
agent, one app can hold a cheap local agent and a frontier hosted one side by
side.

## The providers

Six provider kinds, grouped by where the model runs. Each links to its own
section below with the exact fields it needs and the notes specific to it.

**Hosted APIs** — a service someone else runs; you supply a base URL and an API
key, and the app warns you before saving that prompts will be shared with that
third party:

- **[OpenAI Chat Completions](#openai-chat-completions)** — any endpoint that
  speaks the OpenAI Chat Completions API. The catch-all shape most hosted providers and
  self-hosted gateways expose.
- **[OpenRouter Chat Completions](#openrouter-chat-completions)** — one account that fans out to many upstream
  models, with a rich model catalog and per-turn cost in money.
- **[Claude Chat Completions](#claude-chat-completions)** — Claude models over
  Anthropic's OpenAI-compatible Chat Completions endpoint.

**Local servers** — a server you run yourself, on your machine or your network.
You supply its address; no key, no per-token cost, and your prompts never leave
your network:

- **[Ollama](#ollama)** — points at your Ollama server.
- **[Llama.cpp server](#llama-cpp-server)** — points at a llama.cpp server you
  host, with a listing speed-versus-detail trade-off worth knowing about.

**Embedded** — no server at all:

- **[Llama.cpp on device](#llama-cpp-on-device)** — the inference
  engine built into the app, running a GGUF model in-process on the device with
  no base URL, no key, and no network.

![The Providers screen, where each backend is configured](/images/providers.png)

## Feature support at a glance

What each provider needs from you, where your prompts travel, and how much the
app can learn about a model it serves. The columns are explained under
[What the app learns about a model](#what-the-app-learns-about-a-model).

| Provider | What it needs | Where prompts go | Model capabilities | Reasoning detection | Cost in money |
|---|---|---|---|---|---|
| [OpenAI Chat Completions](#openai-chat-completions) | Base URL + API key | Third-party service | None (bare list) | Manual opt-in | No |
| [OpenRouter Chat Completions](#openrouter-chat-completions) | API key | OpenRouter + upstream | Reliable (rich catalog) | Reliable | Yes |
| [Claude Chat Completions](#claude-chat-completions) | API key | Anthropic | Reliable (rich catalog) | Reliable | No |
| [Ollama](#ollama) | Server address | Your machine or network | Reliable | Best guess | No |
| [Llama.cpp server](#llama-cpp-server) | Server address | Your machine or network | Only in slow per-model mode | Best guess | No |
| [Llama.cpp on device](#llama-cpp-on-device) | Model folder | Stays on the device | From the GGUF header | Best guess | No |

## How it works

Add a provider on the **Providers** screen: name it, choose its kind, and give it
what that kind needs — a base URL and key for the hosted APIs, an address for a
local server, or a model folder for the embedded engine. When you create an
agent you pick one of your providers and then a model it serves, so the same
provider backs as many agents as you like.

The address has to be an `http` or `https` URL with its host written out; the
**Base URL** field says so and Save waits until it is. A provider that sends an
API key to a public `http` address — unencrypted, readable by anyone on the
network path — shows an _Unencrypted connection_ warning and asks you to **Save
anyway**; `https`, `localhost` and private-network addresses save as usual. A
change to a provider's address, authentication or headers applies to the next
request in every conversation, and leaving the form with unsaved edits asks
first.

The addresses and options you enter are saved in the app's local database on your
device. An API key never lands there: it is a saved secret in your device's OS
keystore, and the database only records which secret a provider uses — see
[Saved secrets](#saved-secrets). A secret's value is read back out only into a form
you opened, and never sent anywhere but the providers and tools that use it. The embedded engine's model-folder
location is also kept device-local and off the sync wire, since a path on one
device means nothing on another.

### Star the providers you reach for

Providers are offered in one order everywhere — on the **Providers** screen and
in the picker you get when choosing an agent's provider: starred first, then
newest to oldest. Tap the star on a provider's row to pin it, and it heads both
lists. The star travels with the provider when your devices sync, so the order
is the same on each of them. Each row leads with a mark for where its models
actually run — out over the wire, on a server you run yourself, or inside this
app — and the pencil past the star opens that provider's settings directly.

### Give a provider its own icon

Two OpenAI-compatible endpoints look alike in a list, because the mark on each
row says where the models run rather than which service this is. Set an icon in
the provider's own form to tell them apart — on the **Providers** screen and in
the picker you get when choosing an agent's provider: one of a set of glyphs, an emoji, or
artwork from the web. **Find automatically** searches the address the provider
already points at and shows every mark that host publishes, plus the ones on the
domain it belongs to, so you can pick the logo you recognise — the same picker
the tool icons use, down to the **Use icon** button that applies what you chose
and the **Default** tab that puts the where-it-runs mark back. Only the address travels between
your devices; each one fetches the artwork itself. A provider with no icon of its
own keeps the mark for where it runs.

An agent with the **Discover providers** tools can set one too — a glyph, an
emoji, a link, or the mark the provider's own host publishes. An icon reaches
nothing a request is made of, so it saves without asking; an agent changing
anything that is asks you first, as
[Let an agent set up a provider](#let-an-agent-set-up-a-provider) describes.

### Star the models you actually use

A hosted provider can serve hundreds of models, and you probably reach for a
handful. Tap the star next to a model — on the provider's model list or in the
picker you get when choosing an agent's model — and it moves to the top of both,
above everything the search and filters leave in place. Each provider keeps its
own starred list, and the list travels with the provider when your devices sync.
Starring a model that a later refresh no longer returns costs you nothing: it
simply stops appearing.

### Copy a provider you already have

To set up a second provider much like one you configured, use **Copy** on the
provider's page. The create form opens pre-filled with a duplicate — same kind,
address, authentication settings, custom headers and starred models — with the
name suffixed "(copy)". Nothing is written until you save, so you can point the
copy at a different address before it exists. That makes the common variations
cheap: the same account behind two base URLs, or one gateway configured twice
with different headers.

The copy uses the same saved secret as the original rather than a second copy of
it, so nothing is asked when you start it. Choosing or creating another secret in
the copy's selector leaves the original's alone — see
[Saved secrets](#saved-secrets).

**Export** beside Copy saves a remote provider's definition, without its API
key, as a `.bylm.json` file for another install to import — see
[Sharing](/docs/features/sharing).

The embedded engine offers no Copy. Its models folder and the models you
assemble from it are tied to the provider they were set up under and stay on the
device, so a duplicate would arrive with nothing in it.

### Saved secrets

Every API key you give the app is a saved secret that providers and fetch tools
use, listed under **Settings › Secrets**. The **API key** field on the provider
and fetch tool forms shows which secret it uses — its name, what else uses it,
and whether this device holds it, with a way to ask your other devices when it
does not. Tapping it opens a dialog listing your saved secrets, the ones already
sent to the provider's host first and the rest below with a warning mark:

- **Select** uses the picked secret. Before a secret reaches a new host the app
  asks: "Send this secret somewhere new?", naming where it goes now and the new
  host. _Cancel_ is the default.
- **Create new** takes a name and a value and saves the secret at once.
- **Modify** takes a new value for a listed secret, after naming every provider
  and tool it changes for, and saves it at once.

Choosing a secret is part of the form and is saved with **Save**; creating or
modifying one is saved in the dialog. The clear button on the field only stops
the provider using the secret — the secret stays saved. A secret you create and
then leave unused is marked **Not used** on the Secrets page, whose **Unused**
filter lists those secrets and **Delete unused secrets** deletes them together.

- **Deleting a provider keeps its secret**, listed on the Secrets page until you
  delete it there, since other providers may use it and an API key usually can't
  be fetched again from the service that issued it. The delete confirmation names
  the agents and tools that use the provider and lists the settings and
  conversations that do, since all of them stop working with it.

The Secrets page lists each secret with the hosts it is sent to and whether this
device holds it — never its value. Each secret opens its own page, which lists
every provider and fetch tool using it, each a link to that provider or tool, and
where to get the secret when you noted it. Its menu holds every action on the
secret: share it with another device, ask your other devices for it, and:

- **Edit** — change its name, a hint on where to get it, and a link to the page
  where one is made. The hint and link are shown wherever you are asked for the
  secret's value, and go with it when you export it. The link opens in your
  browser; the app never requests it.
- **Rotate** — replace the value everywhere it is used. You choose which of your
  verified devices get the new value now (the ones this device sent it to before
  are ticked), and whether the old value keeps working until each device has the
  new one or stops working at once. A device you leave out sees that its copy is
  outdated and can ask for the new one.
- **Revoke** — stop every device sending the secret as soon as it syncs. Nothing
  is deleted; if it leaked, revoke it with the service that issued it too.
- **Delete** — remove the secret, unlinking everything that used it.

**A secret goes only where this device approved it.** Each device keeps its own
list of the hosts it sends a secret to, shown under _Sent from this device to_ on
the secret's page. Typing or saving the secret on a provider or fetch tool form,
picking it with **Use a saved secret**, and receiving it from another device all
approve where its providers and tools point at that moment. If an address changes
some other way — edited on another device and synced here, rewritten by an import
or by an agent — the request stops before it is sent, naming the secret and the
host, and the secret's page lists that host with an **Approve** button. Rotating
the secret from its page keeps the list as it is, so a host an address moved to
still waits for that button. A host can be removed from the list there too. A
fetch tool whose host comes from the call is approved as a whole.

When two saved secrets hold the same value on this device, the Secrets page marks
them, and each one's page offers to merge the others into it, so a later rotation
reaches everything that uses either.

### Constants in the address and headers

A provider's base URL and custom header values can place a constant from
**Settings › Constants** with `{constants.<name>}` — an account id in the path, an
Azure resource name as the host, an organization in an `OpenAI-Project` header:

```text
https://api.cloudflare.com/client/v4/accounts/{constants.account}/ai/v1
https://{constants.resource}.openai.azure.com/openai/v1
```

Each placeholder gets a picker under **Placeholders** on the provider's form. A
provider belongs to no workspace, so it links global constants; a workspace that
overrides one of them sends its own text whenever a conversation there uses the
provider, while the provider's page and the global settings use the global text. A
constant in the host must fill whole labels on its own, and one that is missing or
does not fit stops the provider's requests with a message naming it.

A custom header can carry a saved secret too, as `{secrets.<name>}` — an
organization token beside the key, say `OpenAI-Organization: {secrets.org}`. Each
such slot gets a field under **Saved secrets** on the provider's form, the secret
is filled in only when a request leaves, and a request whose secret is missing
stops and names the slot. A secret goes in a header only: every request adds its
own path to the base URL, so none can sit there.

The secret is approved for the host the global constants give when you save it on
the form. A host an override gives is approved the first time a request there is
stopped, from the secret's page — see [Saved secrets](#saved-secrets).

### Request metadata in headers

A custom header value can place values the app fills for each request, such as
the conversation it belongs to. Some providers match a prompt cache better when
every request of one conversation names it:

```text
X-Session-Id: {meta.conversation.id}
```

The names are `{meta.conversation.id}`, `{meta.agent.id}`, `{meta.agent.name}`,
`{meta.workspace.id}`, `{meta.workspace.name}`, `{meta.app.name}`,
`{meta.app.shortName}` and `{meta.app.version}`. Every request a conversation makes
fills them: its replies, its tool calls' filters, its title and summary, and the
summary that compacts it. An agent called from a conversation fills them with its
own conversation and its own name, not the caller's. A request made outside a
conversation, such as listing models or testing the
connection, fills only the app's three. The Default workspace has no id. A name is
percent-encoded outside plain ASCII, so `Café` goes out as `Caf%C3%A9`. A name
outside this list is refused on save, and none of them can go in the base URL.

They also let you send your own `User-Agent` in place of the one the app writes:

```text
User-Agent: {meta.app.shortName}/{meta.app.version} ({meta.agent.name})
```

### A new provider needs its API key up front

A provider set to authenticate with an API key can't be saved without a secret
chosen: Save stays disabled and the field says so. The alternative is on the same form
— switch **Authentication** to _No authentication_ for an endpoint that genuinely
needs no credential. This applies to providers you add from scratch, because a
brand-new provider has no API key to fall back on anywhere. A copy arrives with
the original's secret already in place.

Editing a provider you already have works differently: there you can save other
changes with no secret chosen, and a secret that lives on another device and
hasn't arrived yet does not stop you either.

### Let an agent set up a provider

An agent given the `providers-save` tool can add a provider or change one you
have. Every save that decides where requests go or what reaches that host — a
new provider, a different host, a different kind, the authentication, the header
the key travels in, or the custom headers — waits for you to approve it first.
The prompt names the host, says when the secret you saved will be sent to a new
one, warns when it would cross the network unencrypted, lists each of those
fields as it was and as it would be, and counts what already uses the provider:
agents, fetch tools, titles and compaction settings, and conversations, which
all follow the change together. A different path on the same host asks too, since
one host can serve many accounts, and so does a rename, since agents and dialogs
know a provider by its name. Only a star saves without asking.

An agent cannot set up the embedded engine, which lives in this device's own
settings, and cannot put a credential in a custom header — the API key belongs
in a saved secret, not in the provider's synced settings. A header that needs one
writes `{secrets.<name>}` instead, and `providers-link_secret` with that `slot`
points it at a saved secret.

A provider it creates is saved without an API key, which the form would not
allow: an agent never holds a secret to give. With `secrets-save` it asks you to
create one — you type the value into a prompt, and the agent learns only the
secret's name — and `providers-link_secret` points the provider at it, or at a
secret you already have. Linking asks you first, naming the provider's host,
unless the secret is already sent there. `secrets-list` lists the saved secrets
by name, with the hosts they are sent to and what uses them, never their values;
`secrets-save` can also have you type a secret's new value, and `secrets-delete`
deletes a secret once you approve. No agent tool revokes a secret.

An agent with `providers-link_constant` attached can link a provider's
`{constants.…}` slot to a global constant; the dialog shows the text it places and
the host requests go to. `providers-unlink_constant` clears a slot without asking.
A provider serves every workspace, so it links only global constants, and a
workspace that needs its own account gives the global constant its own text there.

### When a provider rejects the request

If listing a provider's models fails, the message names the HTTP status the
provider answered with, so an outage reads differently from a refusal. A 401 or
403 — the provider turning the request down — is called out as an
authentication problem and comes with the things that fix it: whether its secret
is stored on this device, a way to ask your other devices for it, and a link into
the provider's form. If the provider is set to _No authentication_ and the
endpoint asks for a credential anyway, it says that too, since the fix is the
authentication setting rather than a missing secret.

## What the app learns about a model

When you pick a model, the app tries to learn four things about it: its
**modalities** (does it take images, not just text?), whether it supports
**tools**, whether it supports **reasoning/thinking**, and whether it supports
**structured output** — holding its reply to a JSON schema, which a model
filter's answer schema needs. The model list marks each and filters by them, and
the picker opens with its text-input and text-output filters on, so embedding
and image-only models stay out of the way until you switch those off. How
reliably it knows depends entirely on the provider kind, because each one
exposes model information differently — that is the "Model capabilities" and
"Reasoning detection" split in the table above, and the per-provider sections
spell out each case.

### Reasoning detection is a best guess on local backends

For **Ollama**, **Llama.cpp server**, and **Llama.cpp on device**, the app has only a
limited ability to detect which reasoning/thinking settings a particular model
file supports. There's no universal capability field for it, so the app leans
on weak signals — a capability string, marker text inside the model's chat
template, or a server launch flag — none of which is a reliable per-model answer.

The honest consequence: **the app can report a model as not supporting
thinking/reasoning even when the underlying weights actually do.** When that
happens the reasoning control is simply hidden for that model. When the app is
merely unsure (rather than wrongly certain), it instead shows a manual opt-in
checkbox, so you can turn reasoning on yourself if you know the model accepts it.

### Only OpenRouter reports cost in money

Every provider reports token counts for a turn. **Only OpenRouter also reports
what the turn actually cost in money**, because its API returns the spend
directly; the app surfaces that as a "Cost" figure. For every other kind you see
token usage but no monetary total.

## Provider details

### OpenAI Chat Completions

Any endpoint that speaks the OpenAI Chat Completions API — the catch-all, since
most hosted providers and self-hosted gateways expose this shape. Defaults to
OpenAI's own URL. OpenAI's newer Responses API is not used.

- **Needs** — a base URL and an API key. The app warns before saving that
  prompts will be shared with that third party.
- **Model info** — the endpoint returns a bare model list with no capability
  data at all, so the app can't tell a model's modalities, tools, or reasoning.
  You opt into reasoning by hand when you know the model accepts it.
- **Cost** — token usage only, no monetary total.

### OpenRouter Chat Completions

One account that fans out to many upstream models, reached over OpenRouter's
Chat Completions endpoint rather than its Responses API. The model list comes
from your account.

- **Needs** — an API key.
- **Model info** — OpenRouter ships a rich model catalog, so modalities, tool
  support, and reasoning are known up front and reliably, and a model supports
  structured output when OpenRouter lists it for that model.
- **Cost** — the only kind that reports what a turn cost in money, surfaced as a
  "Cost" figure (see [Only OpenRouter reports cost in money](#only-openrouter-reports-cost-in-money)).

### Claude Chat Completions

Claude models over Anthropic's OpenAI-compatible Chat Completions endpoint, rather
than Anthropic's native Messages API.

- **Needs** — an API key.
- **Model info** — a rich model catalog like OpenRouter's, so modalities, tool
  support, and reasoning are known up front and reliably.
- **Cost** — token usage only, no monetary total.

### Ollama

Points at your Ollama server, on your machine or your network. No key, no
per-token cost, and your prompts never leave your network.

- **Needs** — the server address.
- **Model info** — Ollama reports each model's capabilities cheaply (vision,
  tools, thinking, and whether it chats or only embeds), so it is reliable. An
  embedding model shows embeddings as its output, and the model picker leaves it
  out unless you switch off its text filter. Every model that chats
  supports structured output, which Ollama enforces with a grammar. Reasoning
  remains a best guess (see
  [Reasoning detection is a best guess on local backends](#reasoning-detection-is-a-best-guess-on-local-backends)).
- **Cost** — token usage only, no monetary total.

### Llama.cpp server {#llama-cpp-server}

The **Llama.cpp server** provider points at a llama.cpp server you host. No key,
no per-token cost, and your prompts never leave your network.

- **Needs** — the server address.
- **Model info** — the app knows a model's capabilities only in the server's slow
  per-model mode; see the listing trade-off below. In that mode every model
  supports structured output, which the server enforces with a grammar.
- **Cost** — token usage only, no monetary total.

A llama.cpp server can describe its models two ways, and the app lets you choose
which — it's a genuine speed-versus-detail trade-off:

- **Fast (the default).** The app just asks the server for its model list. This
  is quick, but the bare list carries almost no metadata: the app won't know a
  model's context length, tool support, modalities, or reasoning capability.
- **Detailed (opt-in).** The app additionally asks the server to describe each
  model. To answer, the server has to **load every model into memory one at a
  time and re-read its GGUF metadata**, which is how it recovers real
  capabilities — but it can be very slow when you have several models, and it
  runs them one after another because the server serves one at a time.

Turn the detailed mode on only when you want the app to know your models'
capabilities and can wait for the server to load each one; leave it off for a
snappy list when you already know what you're picking.

### Llama.cpp on device {#llama-cpp-on-device}

The **Llama.cpp on device** provider is the inference engine built into
the app. It runs a GGUF model in-process, on your device, with no server at all —
no base URL, no API key, and no network. You
choose the hardware backend it runs on (CPU, Vulkan, and so on). Its form has
just where the files are and how to run them; for assembling models (base plus a
multimodal projector), vision, and speculative decoding, see
[On-device models](/docs/features/embedded-models):

![The embedded provider's form — a models-folder field and a compute backend, with no base URL or key](/images/provider_embedded.png)

- **Models directory** — the folder holding your `.gguf` files. On desktop it's
  a path you browse to; on Android you grant a folder through the system file
  picker. The folder is scanned and each model file in it becomes selectable
  when you build an agent. This setting stays on the device only.
- **Compute backend** — the hardware the model runs on. Auto lets the engine
  choose; you can also pin **CPU** or **Vulkan** (plus **ROCm** on Linux). It's
  saved per device and applied the next time a model loads.
- **Decode threads** — how many CPU threads to use while generating. Leave it
  empty for an automatic count.
- **Vision projector** — whether a vision model's projector loads with the
  model or only once a conversation shows it an image, the default. Loading it
  later saves memory but makes that turn wait; see
  [On-device models](/docs/features/embedded-models#when-the-vision-projector-loads).
- **Model info** — read straight off the GGUF file, so everything is inferred
  from the file's header rather than declared. Every model supports structured
  output, which the engine enforces with a grammar.
- **Cost** — token usage only, no monetary total.

Everything on this form is saved by **Save**, not as you type, and leaving with
changes asks first. Changing the folder, threads or context size leaves the
settings you made for each model on this device alone.

#### Your model files can live anywhere — and stay shared

This is the part that sets BYLM apart from most apps built on llama.cpp. Those
apps typically make you download a model into the app's own storage, where no
other app can reach it — so a multi-gigabyte file you already have has to be
copied in again, and it's locked to that one app.

BYLM doesn't. **A GGUF file can sit anywhere on the device** — downloaded by a
browser or a separate model-manager app, parked in your Downloads folder, shared
across several apps — and you simply point BYLM's models folder at it and pick
it. Nothing is copied into app-private storage; the same file keeps serving every
app that uses it.

This works because our llama.cpp libraries are patched to fit the Android
security model. Instead of demanding a real filesystem path (which Android's
scoped storage blocks for files outside app-private space), the engine loads the
model from the file handle Android hands back through the folder grant you
approved. That one change is what lets you keep your models where you want them,
out in the open, instead of trapped inside the app.

## What it doesn't do yet

The app is distributed for Android — from Google Play or as a
[sideloadable APK](/#android) — and for Linux desktop as an
[AppImage](/#linux), one plain and one carrying the ROCm backend for AMD cards.
There is no macOS or Windows build.

More in [Features](/docs/features): [On-device models — run fully offline](/docs/features/embedded-models) · [Agents — the thing you build](/docs/features/agents) · [Tools — giving an agent more than the model knows](/docs/features/tools) · [Workspaces — keep your work in scopes](/docs/features/workspaces) · [Conversations — how a turn runs](/docs/features/conversations) · [Attachments — images and files in a message](/docs/features/attachments) · [Context management — what the model still sees](/docs/features/context-management) · [Background turns — replies that keep running](/docs/features/background-turns) · [Local-first & private by design](/docs/features/local-first) · [Sync — your data across your devices](/docs/features/sync) · [Sharing — move agents, tools and providers as a file](/docs/features/sharing) · [Marketplace — find and publish shared agents, tools and providers](/docs/features/marketplace) · [Running a Couchbase Edge Server](/docs/features/couchbase-edge-server) · [Spell checking — catch typos while you write](/docs/features/spell-check)

---

[Bring Your LM](/) · [Docs](/docs/features) · [Marketplace](/marketplace)

[Articles](/articles) · [Releases](/releases) · [Bundle Format](/marketplace/bundle-format) · [Marketplace Terms](/terms) · [App Privacy](/privacy) · [Website Privacy](/website-privacy) · [Open Source](/open-source) · [llms.txt](/llms.txt)