Choosing the Right Ollama Interface for Images, Decisions, and Embeddings

Choosing an Ollama interface starts with the data your application needs back. A photo describer needs readable prose, a screenshot router may need a bounded label, and a local semantic index needs numeric vectors.

The three documented paths below have distinct request and response contracts. This is a documentation-based comparison; no hands-on testing was performed. It does not compare model quality, latency, reliability, or accuracy.

The three paths at a glance

The short decision rule is: use vision chat for prose about an image, SystemOne for bounded answers, and /api/embed for vectors derived from text.

A single project might use more than one path without treating them as interchangeable. For example, an image-organizing application could request a readable description through vision chat, ask a fixed category question through SystemOne, and embed a textual caption for later use. Each call would still have its own documented input and output contract.

Vision chat: return prose about an image

Ollama’s vision example places a prompt and image together inside a user message sent to /api/chat. The complete request shape includes the model and a messages array:

{
  "model": "gemma4",
  "messages": [
    {
      "role": "user",
      "content": "What is in this image?",
      "images": ["<base64-encoded image>"]
    }
  ],
  "stream": false
}

In the documented cURL walkthrough, a cat photograph is downloaded, base64-encoded, and submitted with that question using the gemma4 model (Ollama vision documentation).

The Python example shows the application-facing response:

response = chat(
  model='gemma4',
  messages=[
    {
      'role': 'user',
      'content': 'What is in this image? Be concise.',
      'images': [path],
    }
  ],
)

print(response.message.content)

Application code reads the generated description from response.message.content. The Python example supplies the image by path, while comments on the same page also identify base64-encoded data and raw bytes as alternatives (Ollama vision documentation).

Among these three interfaces, this contract most directly matches an application that needs conversational or descriptive text about an image. The example documents this request shape with gemma4; it does not establish vision support across every model.

SystemOne decisions: return constrained answers

Ollama’s decision interface sends state and one or more named questions to /v1/systemone, then returns corresponding structured entries under answers (Ollama decision documentation).

The documentation defines three question types:

• choice accepts 2–26 named options with descriptions and returns the most likely option plus each option’s probability.
• noul represents a false-or-true question and returns the probability of true from 0 to 1.
• score accepts 2–10 descriptions ordered from lowest to highest and returns a probability-weighted average of their zero-based levels.

Those limits and result formats come from Ollama’s question-type table (Ollama decision documentation).

One example asks which label fits a support ticket:

{
  "model": "nimble",
  "state": "Our checkout has returned 500 errors since 9am.",
  "questions": {
    "label": {
      "type": "choice",
      "instructions": "Which label fits this ticket?",
      "criteria": {
        "billing": "Payments and refunds",
        "bug": "Software errors",
        "account": "Login and account access"
      }
    }
  }
}

The documented response places the selected label in answers.label.choice, alongside probabilities for the options and a confidence field (Ollama decision documentation).

A screenshot example combines textual state, an images array containing a base64-encoded image, and two questions. A noul question asks whether the image contains Ollama, while a choice question asks which application appears. The documented noul answer has this structure:

{
  "has_ollama": {
    "type": "noul",
    "noul": 0.959
  }
}

That value is documentation output, not an independently measured result. The example uses clef-flash and establishes one documented combination of state, an image, and multiple questions—not image support for every decision model (Ollama decision documentation).

SystemOne most directly matches an application whose next step expects a named option, a true-probability value, or an ordered score. Its numeric fields are part of the response contract, but the documentation examples do not demonstrate calibration, determinism, or correctness.

Embeddings: return vectors from text

The /api/embed endpoint accepts either one text string or an array of strings. It returns embeddings as number[][], an array of numeric vectors (Ollama embed API reference).

A single input uses a string:

{
  "model": "embeddinggemma",
  "input": "Why is the sky blue?"
}

A batch places several strings in the same input array:

{
  "model": "embeddinggemma",
  "input": [
    "First sentence",
    "Second sentence",
    "Third sentence"
  ]
}

Ollama documents both forms. Its client examples inspect the first vector’s length for a single input and the number of returned vectors for a batch (Ollama embedding documentation).

This contract directly matches an application that needs vectors from one or more text inputs. The supplied references document text inputs only, so they do not establish image embeddings or prescribe a vector-search design.

The endpoint’s truncate option defaults to true. Inputs exceeding the selected model’s context window are truncated by default; setting truncate to false makes an overlong input return an error (Ollama embed API reference). The reference does not specify one universal context-window size.

Three implementation checks

Before wiring an interface into a project, verify the field the next component will consume:

1. For vision chat, confirm that it expects free-form message.content.
2. For SystemOne, select the required question type and read its named entry under answers.
3. For embeddings, handle the nested vectors in embeddings and define a policy for overlong text.

The essential distinction is the returned application data: prose about an image, bounded answers, or vectors from text. Model support, output quality, latency, calibration, and acceptable error rates require separate evaluation in the intended deployment.

Sources and further reading

• https://docs.ollama.com/capabilities/vision
• https://docs.ollama.com/capabilities/decision
• https://docs.ollama.com/api/embed
• https://docs.ollama.com/capabilities/embeddings