Provider Selection

Two LLM providers. One dispatcher.

Providers

Provider	Use	Source
Anthropic	Default. Haiku 4.5 for intent/chat/fan-out, Sonnet 4.6 for the tool loop.	functions/pivotAiAgent/providers/anthropic.ts
OpenRouter	Fallback / enterprise routing to GPT, Llama, Mistral, etc. via the OpenAI-compatible API.	functions/pivotAiAgent/providers/openrouter.ts

Dispatcher

Single entry point: functions/pivotAiAgent/providers/index.ts.

export async function callLlm(req: LlmRequest): Promise<LlmResponse> {
  switch (req.provider) {
    case 'anthropic':  return callAnthropicProvider(req)
    case 'openrouter': return callOpenRouterProvider(req)
    default: throw new Error(`Unknown LLM provider: ${req.provider}`)
  }
}

Provider is set per-request in LlmRequest.provider. Resolution happens in ai-config.ts (per-company or per-env override). When no override is set, defaults to Anthropic.

Model routing inside Anthropic

Phase	Model	Why
Intent classification	Haiku 4.5	Fast, cheap, small task.
Chat path response	Haiku 4.5	Fast streaming.
Tool loop (data path)	Sonnet 4.6	Needs the better reasoning for tool selection / multi-hop queries.
Fan-out: answer, widgets, suggestions	Haiku 4.5 × 3 (parallel)	The hard work was the tool loop; these are templated transformations of structured data.

Anthropic API key rotation

functions/pivotAiAgent/providers/anthropic.ts rotates between multiple Anthropic keys (per-user profile) to distribute rate-limit pressure across keys. Configured via env / Secret Manager.

Fallback behavior

Today, the dispatcher does not automatically fall back from Anthropic to OpenRouter mid-request. A failed Anthropic call surfaces as an error to the user. See Reference: TODOs for the open item to add per-request fallback retry.

Caching

Anthropic's prompt cache is used for the long system prompts (tool definitions, persona, company memory). Cache reads/writes are recorded separately in LlmUsage so cost accounting reflects the actual cache-hit price. See Token & Cost Tracking.