Ollama’s Two Control Planes: Local Compute vs. API Destination

Ollama configuration involves two independent decisions:

1. Local compute selection: Which documented hardware may a local Ollama server use?
2. API location: Does an application call a local server or Ollama Cloud?

Server-side environment variables answer the first question. The application’s base URL answers the second. This distinction matters when debugging an integration: a correct GPU setting cannot repair a client pointed at the wrong endpoint, while changing an API URL does not select a particular local GPU.

There is also a hybrid path. A signed-in local Ollama server can provide access to cloud models, so a localhost endpoint alone does not establish where model computation occurs. Ollama API introduction

This is a documentation-based explanation. No hardware, API, or performance testing was performed.

Responsibility map

The compute controls below cover the documented NVIDIA CUDA and AMD ROCm paths discussed here. They are not a complete inventory of accelerator backends: Ollama’s hardware documentation also describes Vulkan-specific GPU selection. Ollama GPU documentation

Selecting compute for a local server

Compute-selection variables belong to the environment of the Ollama server process. They limit which documented devices that process may use.

NVIDIA CUDA

On a system with multiple NVIDIA GPUs, CUDA_VISIBLE_DEVICES limits Ollama to a comma-separated subset:

CUDA_VISIBLE_DEVICES=<GPU-UUID>

Numeric IDs are allowed, but the documentation warns that their ordering may vary and says UUIDs are more reliable. The GPU UUIDs can be found with nvidia-smi -L. To ignore NVIDIA GPUs and force CPU use through this control, set an invalid GPU ID such as -1:

CUDA_VISIBLE_DEVICES=-1

Those behaviors—including the UUID preference and invalid-ID convention—are documented on Ollama’s GPU page. Ollama GPU documentation

AMD ROCm

For multiple AMD GPUs using the documented ROCm path, ROCR_VISIBLE_DEVICES limits Ollama to a comma-separated subset:

ROCR_VISIBLE_DEVICES=<GPU-UUID>

Ollama likewise recommends using a UUID when available instead of relying on a numeric value. An invalid ID such as -1 tells Ollama to ignore the GPUs and force CPU use:

ROCR_VISIBLE_DEVICES=-1

The documentation says rocminfo can show the available devices. These instructions cover the ROCm control specifically; they do not imply that every AMD GPU, driver, operating system, or container arrangement will work. Ollama GPU documentation

Choosing the API destination

The calling application chooses between the local server and direct cloud access through its base URL. For the native Ollama API, the documented choices are:

Local:  http://localhost:11434/api
Cloud:  https://ollama.com/api

A native chat request consequently uses one of these endpoints:

http://localhost:11434/api/chat
https://ollama.com/api/chat

Direct cloud requests require an API key, while local requests do not. Direct cloud access also does not require an Ollama installation. Ollama API introduction Ollama quickstart

Authentication is therefore another clue about the chosen route, but it is not a hardware selector. An API key on a direct cloud request says nothing about the devices visible to a separate local server.

Compatibility clients follow the same local-or-cloud split

Ollama publishes different base URLs for its supported client interfaces.

For OpenAI-compatible clients:

Local:  http://localhost:11434/v1
Cloud:  https://ollama.com/v1

For Anthropic clients:

Local:  http://localhost:11434
Cloud:  https://ollama.com

The Anthropic client appends /v1/messages to its base URL. This explains why the forms differ: the native Ollama API starts from /api, the OpenAI-compatible base includes /v1, and the Anthropic client starts from the host-level URL. Ollama API introduction

Ollama also says its OpenAI and Anthropic compatibility interfaces each cover a subset of the original API. Choosing a compatible endpoint should not be treated as a promise of identical feature coverage. Ollama quickstart

Windows shows where each setting belongs

After a standard Windows installation, Ollama runs in the background and serves its API at http://localhost:11434. Ollama for Windows

That arrangement puts configuration in two places:

• Device-visibility variables must apply to the background Ollama process.
• The integrating application must use the intended local or cloud base URL.

This description applies to the standard Windows application. It does not cover every standalone CLI, service, container, or custom-bind configuration.

A compact diagnostic sequence follows from that division. First, inspect the client’s base URL. If it points to https://ollama.com, investigate the direct cloud request rather than the local server’s GPU environment. If it points to http://localhost:11434, confirm that the local server is the intended destination, then inspect the environment of that server process for the relevant CUDA or ROCm visibility variable.

localhost does not prove local computation

Ollama documents a hybrid option in which cloud models are accessed through a signed-in local server. In that arrangement, the application contacts localhost, but the selected model is a cloud model. Ollama API introduction

Record these three choices separately when reviewing a configuration:

• Endpoint: local server or direct cloud API
• Model path: local model or cloud model
• Covered local compute control: NVIDIA CUDA visibility, AMD ROCm visibility, or forced CPU use

These documented controls do not establish performance, memory requirements, general fallback behavior, network isolation, feature parity, or universal hardware compatibility. They answer narrower questions: where the application sends its request and which covered devices a local Ollama server may see.

Sources and further reading

• https://docs.ollama.com/api/introduction
• https://docs.ollama.com/gpu
• https://docs.ollama.com/quickstart
• https://docs.ollama.com/windows