DHIVA LABS · LEARN

Appearance

Beginner guide ​

In about fifteen minutes you set up DL Prism with two free local models, write a rule that sends coding questions to a coding model, check the decision, and send a real request. No cloud account or API key is needed.

What you need ​

  • The prism gateway installed. See Install.
  • Ollama, a free program that runs AI models on your own computer. Install it from its website.
  • About 10 GB of free disk space for the two models, and a terminal.

New to APIs? What is an API explains the ideas used here.

Download two local models ​

  1. Download a general model and a coding model with Ollama:

    bash
    ollama pull llama3.1:8b
ollama pull qwen2.5-coder:7b

  2. Check that Ollama lists them:

    bash
    ollama list

Ollama serves these models at http://127.0.0.1:11434 and speaks the same OpenAI API that Prism uses.

Write your config file ​

Prism reads a YAML file named prism.yaml. It looks first for the file given with -c, then for prism.yaml in the current folder, then for ~/.config/prism/prism.yaml.

  1. Create the folder:

    bash
    mkdir -p ~/.config/prism

  2. Create ~/.config/prism/prism.yaml in a text editor with this content:

    yaml
    server:
  listen: 127.0.0.1:8130

providers:
  - name: ollama
    type: openai
    base_url: http://127.0.0.1:11434/v1
    local: true
    models:
      - id: llama3.1:8b
        aliases: [llama3]
        context_window: 128000
        capabilities: [reasoning]
      - id: qwen2.5-coder:7b
        aliases: [qwen-coder]
        context_window: 32768
        capabilities: [code]

routing:
  default: llama3
  rules:
    - if: {task: code}
      then: qwen-coder

What each part means:

  • providers lists where models live. Ollama is an openai-type provider, because it speaks the OpenAI API. local: true marks it as running on your machine.
  • models lists the models Prism may use. id is the name the provider knows; aliases are short names you can use instead.
  • routing.default is the model used when no rule matches.
  • routing.rules are checked top to bottom, and the first match wins. This one says: if the request is about code, use qwen-coder.

Check the model list ​

bash
prism models
text
MODEL                        PROVIDER     LOCAL  CONTEXT   COST $/MTOK (in/out)   ALIASES
llama3.1:8b                  ollama       yes    128000    0.00 / 0.00            llama3
qwen2.5-coder:7b             ollama       yes    32768     0.00 / 0.00            qwen-coder

Both models are there, marked local, with no cost. If you see an error instead, Prism names the problem in the file, for example a missing id.

Dry-run your first route ​

prism explain shows which model a prompt would go to, without sending anything to a model:

bash
prism explain "Write a Python function that removes duplicates"
json
{
  "chain": [
    {
      "local": true,
      "model": "qwen2.5-coder:7b",
      "provider": "ollama"
    }
  ],
  "est_input_cost_usd": 0,
  "mode": "auto",
  "profile": {
    "task": "code",
    "language": "python",
    "vision": false,
    "est_tokens": 12
  },
  "reason": "rule 1 matched (task=code)"
}

How to read it:

FieldMeaning
modeauto: Prism chose the model. Asking for a model by name gives manual
profileWhat Prism worked out about the request: the task kind, the programming language, whether it has images, and a rough token count
reasonWhy this model was chosen. Here, your first rule matched
chainThe model that will answer, followed by any fallbacks
est_input_cost_usdAn estimate of the input cost. Local models cost nothing

Now try a question that is not about code:

bash
prism explain "Explain how DNS works"

This time task is general, no rule matches, and the reason is no rule matched; using routing.default. The chain holds llama3.1:8b.

Prism works out the task with quick text checks, not with an AI model, so explain is instant.

Start the gateway ​

bash
prism serve
text
prism 0.2.0 listening on http://127.0.0.1:8130 (2 models, online)
point any OpenAI-compatible client at base_url http://127.0.0.1:8130/v1

Leave this terminal open. The gateway runs until you press Ctrl + C.

Send a real request ​

Open a second terminal and send a chat request. -i shows the response headers as well as the answer:

bash
curl -i http://127.0.0.1:8130/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "Write a Python function that removes duplicates"}]}'

The body is a normal OpenAI chat completion with the model's answer. The first answer can take a while, because Ollama loads the model into memory. Among the headers you find Prism's explanation:

HeaderValue
X-Prism-Modeauto
X-Prism-Providerollama
X-Prism-Modelqwen2.5-coder:7b
X-Prism-Reasonrule 1 matched (task=code)

To pick a model yourself, put its name or alias in model, for example "model": "llama3". The mode is then manual.

Open the dashboard ​

With the gateway running, open http://127.0.0.1:8130 in a browser. The Playground lets you chat and select Explain route to see the same decision, Models lists your models and Routing shows your rules. See The web dashboard.

What you learned ​

  • prism.yaml lists providers, models and routing rules.
  • prism models checks the config; prism explain dry-runs a decision.
  • prism serve starts one OpenAI-compatible endpoint at http://127.0.0.1:8130/v1.
  • "model": "auto" lets your rules choose; a model name chooses directly.
  • The X-Prism-* headers say who answered and why.

Next steps ​