LLM Providers

Specrun supports multiple LLM providers out of the box. Switch providers by changing a few lines in your spec — no code changes needed.

OpenAI

llm:
  type: OpenAiConfig
  model: gpt-4o

Set your API key via environment variable:

export OPENAI_API_KEY=sk-...

Or reference it in the spec:

llm:
  type: OpenAiConfig
  model: gpt-4o
  api_key: $OPENAI_API_KEY

OpenAI-Compatible

For any endpoint that implements the OpenAI API format:

llm:
  type: OpenAiCompatibleConfig
  model: my-model
  base_url: https://my-endpoint.com/v1
  api_key: $MY_API_KEY

vLLM

For self-hosted models via vLLM:

llm:
  type: VllmConfig
  model: meta-llama/Llama-3.1-8B
  base_url: http://localhost:8000

Ollama

For local models via Ollama:

llm:
  type: OllamaConfig
  model: llama3.1

Ollama defaults to http://localhost:11434. To use a different host:

llm:
  type: OllamaConfig
  model: llama3.1
  base_url: http://my-server:11434

Environment Variables

API keys can reference environment variables using the $ prefix:

api_key: $MY_API_KEY

Specrun resolves these at runtime from your shell environment.