What Actually Goes to the Model

To better understand how Ollama works, take the following conversation as an example:

>>> /set system "You are terse."
Set system message.

>>> What is 2+2?
4.

>>> And times 3?

Let's use the API to recreate this conversation and better understand what goes on under the hood.

The CLI sends the following message:

# Test this command to understand how the CLI works
curl -s http://localhost:11434/api/chat -d '{
  "model": "'"$MODEL"'",
  "stream": false,
  "messages": [
    {"role": "system", "content": "You are terse."},
    {"role": "user", "content": "What is 2+2?"}
  ]
}' | jq .

In the next turn, the CLI sends the following message:

# Test this command too
curl -s http://localhost:11434/api/chat -d '{
  "model": "'"$MODEL"'",
  "stream": false,
  "messages": [
    {"role": "system", "content": "You are terse."},
    {"role": "user", "content": "What is 2+2?"},
    {"role": "assistant", "content": "4."},
    {"role": "user", "content": "And times 3?"}
  ]
}' | jq .

Inside Ollama, the server takes that array, runs it through the model's chat template (the Jinja-like template baked into the GGUF or Modelfile), and produces a single flat prompt string. That string, plus the next-token generation, is all the model sees.

You can see the template for any model with:

# Set the model
MODEL=granite3.3:2b

# Show the template
ollama show $MODEL --template

For granite3.3:2b, the relevant chunk is the following:

...
{{- /* Standard Messages */}}
{{- range $index, $_ := .Messages }}
{{- if (and
    (ne .Role "system")
    (or (lt (len .Role) 7) (ne (slice .Role 0 7) "control"))
    (or (lt (len .Role) 8) (ne (slice .Role 0 8) "document"))
)}}
<|start_of_role|>
{{- if eq .Role "tool" }}tool_response
{{- else }}{{ .Role }}
{{- end }}<|end_of_role|>
{{- if .Content }}{{ .Content }}
{{- else if .ToolCalls }}<|tool_call|>
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}
{{- end }}
{{- if eq (len (slice $.Messages $index)) 1 }}

Local AI Engineering with Ollama

Run, understand, customize, fine-tune, and build agentic apps on your own hardware

Enroll now to unlock all content and receive all future updates for free.

Unlock now $26.99 Learn More

Previous Next