# Providers (/docs/providers) A provider is what the runtime invokes to talk to a model. `@pleach/core` ships two — `AiSdkProvider` (Vercel AI SDK) and `AnthropicSdkProvider` (Anthropic's official SDK) — plus one interface (`AgentProvider`) you implement to wrap any other SDK. The underlying SDK packages are optional peer deps. The runtime imports them dynamically at provider construction, so you only install the ones you use. See [Model resolution matrix](/docs/model-resolution-matrix) for how `(family × callClass)` resolves to a concrete model id, and [Tools](/docs/tools) for how `ProviderToolDef` is built. This page is the reference for the routing cluster's transport layer. The three concepts that decide *what* fires through which provider — `CallClass`, `Seam`, `family-lock` — are framed in [Family-lock → the routing cluster](/docs/family-lock#the-routing-cluster). `ProviderFamily` is a closed union of seven: `anthropic | openai | google | deepseek | moonshot | mistral | xai`. Each family locks the tokenizer, prompt-cache key, tool-call dialect, and refusal pattern — see [Family-lock](/docs/family-lock). The transport is one of four: `native | openrouter | byok-native | byok-openrouter`, locked at session start and never silently mutated. ## Same shape, different provider [#same-shape-different-provider] The fastest swap path is `AiSdkProvider` + OpenRouter. The model identifier is a `/` string — change it, change providers. Everything else on the page stays identical. ```typescript // Anthropic import { createOpenRouter } from "@openrouter/ai-sdk-provider"; import { AiSdkProvider, SessionRuntime } from "@pleach/core"; const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! }); const runtime = new SessionRuntime({ provider: new AiSdkProvider({ model: openrouter("anthropic/claude-sonnet-4-5"), }), storage: supabaseAdapter, userId: "user_123", }); ``` ```typescript // OpenAI — same file, change one string const runtime = new SessionRuntime({ provider: new AiSdkProvider({ model: openrouter("openai/gpt-4o"), }), storage: supabaseAdapter, userId: "user_123", }); ``` ```typescript // Google Gemini — same file, change one string const runtime = new SessionRuntime({ provider: new AiSdkProvider({ model: openrouter("google/gemini-2.5-flash"), }), storage: supabaseAdapter, userId: "user_123", }); ``` One `OPENROUTER_API_KEY`, seven families (`anthropic | openai | google | deepseek | moonshot | mistral | xai`), identical request shape. The runtime's `family-lock` reads the `/` prefix and locks the tokenizer, prompt-cache key, and tool-call dialect for the session — no silent cross-family fallback. Want direct vendor SDKs instead? Drop the OpenRouter wrapper for `@ai-sdk/anthropic`, `@ai-sdk/openai`, `@ai-sdk/google` etc. — covered below. ```typescript import { AiSdkProvider, AnthropicSdkProvider } from "@pleach/core"; import type { AgentProvider, AgentExecutionConfig, ProviderMessage, ProviderToolDef, ProviderCapabilities, ProviderStreamEvent, TokenUsage, } from "@pleach/core/providers"; ``` ## `AiSdkProvider` (Vercel AI SDK) [#aisdkprovider-vercel-ai-sdk] Wraps `streamText` from `ai@6.x`. Right pick when you want unified provider switching, the AI SDK's tool dialect, and the ecosystem of community providers. The recommended default — pair with [OpenRouter](https://openrouter.ai) to reach any family from one key. ```typescript import { createOpenRouter } from "@openrouter/ai-sdk-provider"; import { AiSdkProvider, SessionRuntime } from "@pleach/core"; const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! }); const runtime = new SessionRuntime({ provider: new AiSdkProvider({ model: openrouter("anthropic/claude-sonnet-4-5"), maxSteps: 5, }), storage: supabaseAdapter, userId: "user_123", }); ``` Drop the OpenRouter wrapper for a direct provider package any time — `@ai-sdk/anthropic`, `@ai-sdk/openai`, `@ai-sdk/google`, etc.: ```typescript import { anthropic } from "@ai-sdk/anthropic"; const runtime = new SessionRuntime({ provider: new AiSdkProvider({ model: anthropic("claude-sonnet-4-5"), maxSteps: 5, }), storage: supabaseAdapter, userId: "user_123", }); ``` ### Config [#config] | Field | Type | Default | Purpose | | ---------- | ---------------------- | -------- | ---------------------------------------------- | | `model` | AI SDK `LanguageModel` | required | Any AI SDK model factory output | | `maxSteps` | `number` | `1` | Maps to AI SDK v6's `stopWhen: stepCountIs(N)` | Install the `ai` package plus a provider package — OpenRouter is the default; switch to `@ai-sdk/anthropic` / `@ai-sdk/openai` / `@ai-sdk/google` for direct vendor access: ```bash npm install ai @openrouter/ai-sdk-provider # or @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, etc. ``` ## `AnthropicSdkProvider` [#anthropicsdkprovider] Wraps `@anthropic-ai/sdk` directly. Right pick when you want native Anthropic features (prompt caching, extended thinking, tool-use beta flags) without going through a unified wrapper. ```typescript import { AnthropicSdkProvider, SessionRuntime } from "@pleach/core"; const runtime = new SessionRuntime({ provider: new AnthropicSdkProvider({ apiKey: process.env.ANTHROPIC_API_KEY!, model: "claude-sonnet-4-5", maxTokens: 4096, }), storage: supabaseAdapter, userId: "user_123", }); ``` ### Config [#config-1] | Field | Type | Default | Purpose | | ----------- | -------- | ------------------------------ | ----------------------------------------- | | `apiKey` | `string` | required | Anthropic API key | | `model` | `string` | `"claude-sonnet-4-5-20250514"` | Model id | | `maxTokens` | `number` | `4096` | Output token cap per call | | `baseURL` | `string` | – | Override for proxies / regional endpoints | ```bash npm install @anthropic-ai/sdk ``` ## The `AgentProvider` interface [#the-agentprovider-interface] Implement this to wrap any other SDK — OpenAI directly, a custom gateway, a local Ollama instance, anything that streams. ```typescript interface AgentProvider { execute(config: AgentExecutionConfig): AsyncIterable; abort(): void; readonly capabilities: ProviderCapabilities; } ``` ### `AgentExecutionConfig` [#agentexecutionconfig] What the runtime hands you per call. | Field | Type | Purpose | | ---------------- | -------------------------- | ---------------------------------------------------------- | | `messages` | `ProviderMessage[]` | Normalized conversation history | | `tools` | `ProviderToolDef[]?` | Tools available this call | | `systemPrompt` | `string?` | Composed system prompt (post-`composeBudgetedPrompt`) | | `model` | `string?` | Specific model id when the matrix resolved one | | `temperature` | `number?` | Sampling temperature | | `maxTokens` | `number?` | Output token cap | | `abortSignal` | `AbortSignal?` | User pressed stop / turn aborted | | `providerConfig` | `Record?` | Provider-specific pass-through (cache headers, beta flags) | ### `ProviderMessage` [#providermessage] The normalized message shape your provider converts to its native format. ```typescript interface ProviderMessage { id?: string; role: "user" | "assistant" | "system" | "tool"; content: unknown; tool_calls?: Array<{ id: string; name: string; arguments: unknown }>; tool_call_id?: string; name?: string; } ``` ### `ProviderStreamEvent` [#providerstreamevent] A tagged union. The runtime adapts these into the public `StreamEvent` consumers see. ```typescript type ProviderStreamEvent = | { type: "message.start"; messageId: string; role: string } | { type: "message.delta"; delta: string } | { type: "message.complete"; messageId: string; usage?: TokenUsage } | { type: "thinking.delta"; delta: string } | { type: "thinking.complete" } | { type: "tool.started"; toolCallId: string; toolName: string; arguments?: unknown } | { type: "tool.completed"; toolCallId: string; toolName: string; result?: unknown } | { type: "tool.failed"; toolCallId: string; toolName: string; error: string } | { type: "error"; error: string; code?: string } | { type: "done" }; ``` `TokenUsage` carries `inputTokens`, `outputTokens`, `totalTokens`, `cacheReadTokens`, `cacheWriteTokens` — all optional, all surfaced on `message.complete`. ### `ProviderCapabilities` [#providercapabilities] Feature detection so the runtime knows what to ask for. ```typescript interface ProviderCapabilities { streaming: boolean; toolCalling: boolean; thinking: boolean; structuredOutput: boolean; maxContextTokens: number; parallelToolCalls: boolean; } ``` The runtime reads `capabilities` to decide what to ask for. If `toolCalling` is `false`, tool-using sessions won't dispatch tools through this provider; if `thinking` is `false`, thinking-delta events are dropped. ### Minimal custom provider sketch [#minimal-custom-provider-sketch] ```typescript // lib/providers/myProvider.ts import type { AgentProvider, AgentExecutionConfig, ProviderStreamEvent } from "@pleach/core"; export class MyProvider implements AgentProvider { readonly capabilities = { streaming: true, toolCalling: true, thinking: false, structuredOutput: false, maxContextTokens: 128_000, parallelToolCalls: false, }; private controller: AbortController | null = null; async *execute(config: AgentExecutionConfig): AsyncIterable { this.controller = new AbortController(); const signal = this.controller.signal; config.abortSignal?.addEventListener("abort", () => this.controller?.abort()); const stream = await callMyApi(config, { signal }); for await (const chunk of stream) { yield translateChunk(chunk); } } abort() { this.controller?.abort(); } } ``` Drop it onto the runtime: ```typescript const runtime = new SessionRuntime({ provider: new MyProvider(), storage: supabaseAdapter, userId: "user_123", }); ``` ## Family-strict cascade with `pickNextInFamily` [#family-strict-cascade-with-picknextinfamily] Provider failure inside the graph cascade walks the locked family ladder rather than silently widening cross-family. The primitive is `pickNextInFamily(family, currentModel, triedModels)` — it returns the next rung in the same family, or `null` when every rung is exhausted. ```typescript // `pickNextInFamily` is host-supplied — the harness ships no model // registry, so you author the in-family ladder walk and register it via // the `contributeFamilyPivot` plugin hook (below). Its signature: declare function pickNextInFamily( family: string, currentModel: string, triedModels: Set, ): { modelId: string; callClass: string } | null; const next = pickNextInFamily( "anthropic", "claude-opus-4-7", new Set(["claude-opus-4-7"]), ); // → { modelId: "claude-sonnet-4-7", callClass: "synthesize", ... } | null ``` The cascade pivot in `defaultAgentGraph.ts` (its `providerErrorCascadeAttempted` and `synthesisRecoveryAttempted` blocks) walks `pickNextInFamily` in-family. When every rung exhausts, it emits `[UXParity:family-exhausted]`, calls `setFamilyExhaustedState({...})`, and returns `shouldContinue: false`. Hosts surface a `FamilyExhaustedToast` (or wire `contributeFamilyExhaustedSurface` — see below) and the user explicitly picks another family. **Do not silently widen cross-family.** That was the prod regression shape from `canvas2-prompt7-audit-2026-05-18`: a cross- family fallback inside the cascade doubled cost on synthesize failures and broke the family-lock invariants downstream tools depend on. `null` from `pickNextInFamily` is the explicit termination signal — surface it, don't paper over it. Non-matrix-resolvable models (BYOK / `modal-llm` / unrecognized slugs) preserve legacy cross-family fallback inside the `family === null` branch. ## Reasoning-only completion recovery [#reasoning-only-completion-recovery] A reasoning / chain-of-thought model (deepseek-v4, o1-class, gemini-thinking) can finish a stream **cleanly** having emitted reasoning but **zero user-facing text and zero tool calls** — or hang until the seam watchdog force-unwinds it. Treated naively both look like an outage: the turn cascades to another provider, doubling cost, and can ship an empty answer. The correct behavior is one same-model **content-elicitation retry** ("write your final answer directly") before declaring the model unavailable. The seam layer owns this recovery so it fires on the graph path, gated by an opt-in DI on `SessionRuntimeConfig`. Unset, the seam is byte-identical to today — no recovery arm: ```typescript import type { ReasoningRecoveryStrategy } from "@pleach/core/types/strategies"; const reasoningRecovery: ReasoningRecoveryStrategy = { // Authoritative: is this a reasoning model eligible for recovery? classifier: (modelId) => modelId.startsWith("deepseek-v4"), // The re-ask. A STATIC string, or a FUNCTION of the empty turn's // own (bounded) reasoning for a CORRECTIVE directive — higher // recovery reliability than a cold generic re-ask. elicitationPrompt: (ctx) => ctx.reasoningText ? `You reasoned:\n${ctx.reasoningText}\n\nNow write the final answer for the user directly.` : "Now write your final answer for the user directly. Do not include your reasoning.", }; const runtime = new SessionRuntime({ /* … */, reasoningRecovery }); ``` The strategy shape mirrors `ProviderDegradationStatsResolver` — the host owns every domain value (the model set, the elicitation string) so no reasoning-model list lives in core. The recovery fires at most once per invoke, on **both** the clean-empty return path and the watchdog-unwind throw path, and only cascades if the retry is also content-free. It emits `[UXParity:reasoning-only-recovery:*]`. `@pleach/core/providers` also publishes the shared failure taxonomy — `FailureCategory` (`AUTH` / `BILLING` / `RATE_LIMIT` / `MODEL_UNAVAILABLE` / `TOOL_HALLUCINATION` / …), `FailureClassification`, `BillingDisposition`, `ModelUnavailableReason` — plus `classifyEmptyStream` (the precondition classifier the seam recovery keys on). Cross-SKU consumers (gateway, replay, observe) share the vocabulary; the provider-specific string→category matchers stay host-injected. ## Plugin contribution hooks for retry, continuation, and family pivot [#plugin-contribution-hooks-for-retry-continuation-and-family-pivot] Four optional `HarnessPlugin` contribution hooks lift the retry loop's domain knowledge out of the host. Each is opt-in; an unset hook returns `null` and the runtime falls back to its default behavior. | Hook | Returns | Collector | What it owns | | ---------------------------------- | ------------------------------------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `contributeRetryPolicy` | `RetryPolicyContribution` | `runtime.plugins.collectRetryPolicy()` | Retry behavior on transient failures — discovery-only tool set, workflow-narration nudge patterns, nudge template | | `contributeContinuationPolicy` | `ContinuationPolicy` | `runtime.plugins.collectContinuationPolicy()` | When a tool result needs continuation vs termination — fetch-tool name set, stuck-discovery namespace prefixes, sandbox-clause regex | | `contributeFamilyPivot` | `FamilyPivotContribution` | `runtime.plugins.collectFamilyPivot()` | Classify a model into a family, then pick the next in-family rung — `deriveFamilyFromModelId` + `pickNextInFamily` paired | | `contributeFamilyExhaustedSurface` | `FamilyExhaustedSurfaceContribution` | `runtime.plugins.collectFamilyExhaustedSurface()` | UI surface (toast, banner, modal) when the family ladder exhausts — receives `family`, `triedModels`, `chatId`, optional `messageId` | The four hooks split the responsibility cleanly: `contributeRetryPolicy` * `contributeContinuationPolicy` carry DATA (tool name sets, regex patterns, templates); `contributeFamilyPivot` carries OPERATIONS (the harness has no model registry — both classification and in-family pick are host-supplied); `contributeFamilyExhaustedSurface` carries UI strategy (the harness has no view layer). Sketch: ```typescript import type { HarnessPlugin } from "@pleach/core"; import type { ProviderFamily } from "@pleach/core/modelfamily"; // Host-supplied — your own model-registry helpers. The harness has no // model registry, so you author these and hand them to the runtime // through `contributeFamilyPivot`. declare function deriveFamilyFromModelId(modelId: string): ProviderFamily | null; declare function pickNextInFamily( family: ProviderFamily, current: string, tried: Set, ): { modelId: string } | null; const myPlugin: HarnessPlugin = { name: "my-host", contributeFamilyPivot() { return { deriveFamilyFromModelId, pickNextInFamily: (family, current, tried) => pickNextInFamily(family, current, new Set(tried))?.modelId ?? null, }; }, contributeFamilyExhaustedSurface() { return { surfaceFamilyExhausted: ({ family, triedModels, chatId, messageId }) => { myToastBus.emit({ kind: "family-exhausted", family, attempted: Array.from(triedModels), chatId, messageId, }); }, }; }, }; ``` BYOK credential resolution is a sibling path: the `createPleachRuntime` factory accepts a `byokResolver` callback (per-session credential lookup) that the `TurnOrchestrator` reads via `SessionRuntime.getByokResolver()`. Any host with its own secrets store implements the same shape. ## Provider cascade with audit attribution [#provider-cascade-with-audit-attribution] Compose a primary plus fallback by wrapping two providers in a third. The cascade records why it fell through so the audit row joins back to a cause, not just a model id. ```typescript class CascadeProvider implements AgentProvider { constructor(private primary: AgentProvider, private fallback: AgentProvider) {} readonly capabilities = this.primary.capabilities; async *execute(config: AgentExecutionConfig): AsyncIterable { try { yield* this.primary.execute(config); } catch (err) { yield { type: "error", error: String(err), code: "5001" }; yield { type: "message.delta", delta: "[fallback engaged]" }; yield* this.fallback.execute({ ...config, providerConfig: { ...config.providerConfig, cascadeReason: "primary_5001" }, }); } } abort() { this.primary.abort(); this.fallback.abort(); } } ``` ## BYOK provider from session-scoped credentials [#byok-provider-from-session-scoped-credentials] A multi-tenant deployment carries the customer key on the session, not the process env. Resolve it at construction time from the same `userId` / `organizationId` you pass to the runtime. ```typescript async function providerForOrg(organizationId: string): Promise { const apiKey = await loadOrgKey(organizationId); // your secrets store return new AnthropicSdkProvider({ apiKey, model: "claude-sonnet-4-5", maxTokens: 4096, }); } const runtime = new SessionRuntime({ provider: await providerForOrg("org-acme"), storage: supabaseAdapter, userId: "user-7", }); ``` ## Provider vs `orchestratorConfig` [#provider-vs-orchestratorconfig] `SessionRuntimeConfig` accepts both `provider` (the `AgentProvider`-shaped surface above) and `orchestratorConfig` (a richer config object used by the legacy orchestrator path). Use `provider` for new code — it's the public substrate API. `orchestratorConfig` exists so hosts mid-migration don't break; new consumers should ignore it. The class behind `orchestratorConfig` was renamed: `OrchestratorClient` → [`TurnOrchestrator`](/docs/orchestrator-client). `OrchestratorClient` remains exported from `@pleach/core` as a deprecated alias for the `@pleach/core@1.x` migration window and is removed at `2.0.0`. Existing imports keep working; new code should reach for `TurnOrchestrator` (or, better, the `provider` surface above). ## Where to go next [#where-to-go-next]