pleach
Build

Providers

Wire an LLM provider into the runtime — Vercel AI SDK, Anthropic SDK, or your own — through the `AgentProvider` interface.

A provider is what the runtime invokes to talk to a model. @pleach/core ships two — AiSdkProvider (Vercel AI SDK) and AnthropicSdkProvider (Anthropic's official SDK) — plus one interface (AgentProvider) you implement to wrap any other SDK.

The underlying SDK packages are optional peer deps. The runtime imports them dynamically at provider construction, so you only install the ones you use. See Model resolution matrix for how (family × callClass) resolves to a concrete model id, and Tools for how ProviderToolDef is built.

This page is the reference for the routing cluster's transport layer. The three concepts that decide what fires through which provider — CallClass, Seam, family-lock — are framed in Family-lock → the routing cluster.

ProviderFamily is a closed union of seven: anthropic | openai | google | deepseek | moonshot | mistral | xai. Each family locks the tokenizer, prompt-cache key, tool-call dialect, and refusal pattern — see Family-lock. The transport is one of four: native | openrouter | byok-native | byok-openrouter, locked at session start and never silently mutated.

Same shape, different provider

The fastest swap path is AiSdkProvider + OpenRouter. The model identifier is a <family>/<model> string — change it, change providers. Everything else on the page stays identical.

// Anthropic
import { createOpenRouter } from "@openrouter/ai-sdk-provider";
import { AiSdkProvider, SessionRuntime } from "@pleach/core";

const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! });

const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model: openrouter("anthropic/claude-sonnet-4-5"),
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});
// OpenAI — same file, change one string
const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model: openrouter("openai/gpt-4o"),
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});
// Google Gemini — same file, change one string
const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model: openrouter("google/gemini-2.5-flash"),
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});

One OPENROUTER_API_KEY, seven families (anthropic | openai | google | deepseek | moonshot | mistral | xai), identical request shape. The runtime's family-lock reads the <family>/ prefix and locks the tokenizer, prompt-cache key, and tool-call dialect for the session — no silent cross-family fallback. Want direct vendor SDKs instead? Drop the OpenRouter wrapper for @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google etc. — covered below.

import { AiSdkProvider, AnthropicSdkProvider } from "@pleach/core";
import type {
  AgentProvider,
  AgentExecutionConfig,
  ProviderMessage,
  ProviderToolDef,
  ProviderCapabilities,
  ProviderStreamEvent,
  TokenUsage,
} from "@pleach/core/providers";

AiSdkProvider (Vercel AI SDK)

Wraps streamText from ai@6.x. Right pick when you want unified provider switching, the AI SDK's tool dialect, and the ecosystem of community providers. The recommended default — pair with OpenRouter to reach any family from one key.

import { createOpenRouter } from "@openrouter/ai-sdk-provider";
import { AiSdkProvider, SessionRuntime } from "@pleach/core";

const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! });

const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model:    openrouter("anthropic/claude-sonnet-4-5"),
    maxSteps: 5,
  }),
  storage:  supabaseAdapter,
  userId:   "user_123",
});

Drop the OpenRouter wrapper for a direct provider package any time — @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, etc.:

import { anthropic } from "@ai-sdk/anthropic";

const runtime = new SessionRuntime({
  provider: new AiSdkProvider({
    model: anthropic("claude-sonnet-4-5"),
    maxSteps: 5,
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});

Config

FieldTypeDefaultPurpose
modelAI SDK LanguageModelrequiredAny AI SDK model factory output
maxStepsnumber1Maps to AI SDK v6's stopWhen: stepCountIs(N)

Install the ai package plus a provider package — OpenRouter is the default; switch to @ai-sdk/anthropic / @ai-sdk/openai / @ai-sdk/google for direct vendor access:

npm install ai @openrouter/ai-sdk-provider   # or @ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, etc.

AnthropicSdkProvider

Wraps @anthropic-ai/sdk directly. Right pick when you want native Anthropic features (prompt caching, extended thinking, tool-use beta flags) without going through a unified wrapper.

import { AnthropicSdkProvider, SessionRuntime } from "@pleach/core";

const runtime = new SessionRuntime({
  provider: new AnthropicSdkProvider({
    apiKey:    process.env.ANTHROPIC_API_KEY!,
    model:     "claude-sonnet-4-5",
    maxTokens: 4096,
  }),
  storage: supabaseAdapter,
  userId:  "user_123",
});

Config

FieldTypeDefaultPurpose
apiKeystringrequiredAnthropic API key
modelstring"claude-sonnet-4-5-20250514"Model id
maxTokensnumber4096Output token cap per call
baseURLstringOverride for proxies / regional endpoints
npm install @anthropic-ai/sdk

The AgentProvider interface

Implement this to wrap any other SDK — OpenAI directly, a custom gateway, a local Ollama instance, anything that streams.

interface AgentProvider {
  execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent>;
  abort(): void;
  readonly capabilities: ProviderCapabilities;
}

AgentExecutionConfig

What the runtime hands you per call.

FieldTypePurpose
messagesProviderMessage[]Normalized conversation history
toolsProviderToolDef[]?Tools available this call
systemPromptstring?Composed system prompt (post-composeBudgetedPrompt)
modelstring?Specific model id when the matrix resolved one
temperaturenumber?Sampling temperature
maxTokensnumber?Output token cap
abortSignalAbortSignal?User pressed stop / turn aborted
providerConfigRecord<string, unknown>?Provider-specific pass-through (cache headers, beta flags)

ProviderMessage

The normalized message shape your provider converts to its native format.

interface ProviderMessage {
  id?: string;
  role: "user" | "assistant" | "system" | "tool";
  content: unknown;
  tool_calls?: Array<{ id: string; name: string; arguments: unknown }>;
  tool_call_id?: string;
  name?: string;
}

ProviderStreamEvent

A tagged union. The runtime adapts these into the public StreamEvent consumers see.

type ProviderStreamEvent =
  | { type: "message.start";    messageId: string; role: string }
  | { type: "message.delta";    delta: string }
  | { type: "message.complete"; messageId: string; usage?: TokenUsage }
  | { type: "thinking.delta";   delta: string }
  | { type: "thinking.complete" }
  | { type: "tool.started";     toolCallId: string; toolName: string; arguments?: unknown }
  | { type: "tool.completed";   toolCallId: string; toolName: string; result?: unknown }
  | { type: "tool.failed";      toolCallId: string; toolName: string; error: string }
  | { type: "error";            error: string; code?: string }
  | { type: "done" };

TokenUsage carries inputTokens, outputTokens, totalTokens, cacheReadTokens, cacheWriteTokens — all optional, all surfaced on message.complete.

ProviderCapabilities

Feature detection so the runtime knows what to ask for.

interface ProviderCapabilities {
  streaming:         boolean;
  toolCalling:       boolean;
  thinking:          boolean;
  structuredOutput:  boolean;
  maxContextTokens:  number;
  parallelToolCalls: boolean;
}

The runtime reads capabilities to decide what to ask for. If toolCalling is false, tool-using sessions won't dispatch tools through this provider; if thinking is false, thinking-delta events are dropped.

Minimal custom provider sketch

// lib/providers/myProvider.ts
import type { AgentProvider, AgentExecutionConfig, ProviderStreamEvent } from "@pleach/core";

export class MyProvider implements AgentProvider {
  readonly capabilities = {
    streaming:         true,
    toolCalling:       true,
    thinking:          false,
    structuredOutput:  false,
    maxContextTokens:  128_000,
    parallelToolCalls: false,
  };

  private controller: AbortController | null = null;

  async *execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent> {
    this.controller = new AbortController();
    const signal = this.controller.signal;
    config.abortSignal?.addEventListener("abort", () => this.controller?.abort());

    const stream = await callMyApi(config, { signal });
    for await (const chunk of stream) {
      yield translateChunk(chunk);
    }
  }

  abort() {
    this.controller?.abort();
  }
}

Drop it onto the runtime:

const runtime = new SessionRuntime({
  provider: new MyProvider(),
  storage:  supabaseAdapter,
  userId:   "user_123",
});

Family-strict cascade with pickNextInFamily

Provider failure inside the graph cascade walks the locked family ladder rather than silently widening cross-family. The primitive is pickNextInFamily(family, currentModel, triedModels) — it returns the next rung in the same family, or null when every rung is exhausted.

// `pickNextInFamily` is host-supplied — the harness ships no model
// registry, so you author the in-family ladder walk and register it via
// the `contributeFamilyPivot` plugin hook (below). Its signature:
declare function pickNextInFamily(
  family: string,
  currentModel: string,
  triedModels: Set<string>,
): { modelId: string; callClass: string } | null;

const next = pickNextInFamily(
  "anthropic",
  "claude-opus-4-7",
  new Set(["claude-opus-4-7"]),
);
// → { modelId: "claude-sonnet-4-7", callClass: "synthesize", ... } | null

The cascade pivot in defaultAgentGraph.ts (its providerErrorCascadeAttempted and synthesisRecoveryAttempted blocks) walks pickNextInFamily in-family. When every rung exhausts, it emits [UXParity:family-exhausted], calls setFamilyExhaustedState({...}), and returns shouldContinue: false. Hosts surface a FamilyExhaustedToast (or wire contributeFamilyExhaustedSurface — see below) and the user explicitly picks another family.

Do not silently widen cross-family. That was the prod regression shape from canvas2-prompt7-audit-2026-05-18: a cross- family fallback inside the cascade doubled cost on synthesize failures and broke the family-lock invariants downstream tools depend on. null from pickNextInFamily is the explicit termination signal — surface it, don't paper over it.

Non-matrix-resolvable models (BYOK / modal-llm / unrecognized slugs) preserve legacy cross-family fallback inside the family === null branch.

Reasoning-only completion recovery

A reasoning / chain-of-thought model (deepseek-v4, o1-class, gemini-thinking) can finish a stream cleanly having emitted reasoning but zero user-facing text and zero tool calls — or hang until the seam watchdog force-unwinds it. Treated naively both look like an outage: the turn cascades to another provider, doubling cost, and can ship an empty answer. The correct behavior is one same-model content-elicitation retry ("write your final answer directly") before declaring the model unavailable.

The seam layer owns this recovery so it fires on the graph path, gated by an opt-in DI on SessionRuntimeConfig. Unset, the seam is byte-identical to today — no recovery arm:

import type { ReasoningRecoveryStrategy } from "@pleach/core/types/strategies";

const reasoningRecovery: ReasoningRecoveryStrategy = {
  // Authoritative: is this a reasoning model eligible for recovery?
  classifier: (modelId) => modelId.startsWith("deepseek-v4"),
  // The re-ask. A STATIC string, or a FUNCTION of the empty turn's
  // own (bounded) reasoning for a CORRECTIVE directive — higher
  // recovery reliability than a cold generic re-ask.
  elicitationPrompt: (ctx) =>
    ctx.reasoningText
      ? `You reasoned:\n${ctx.reasoningText}\n\nNow write the final answer for the user directly.`
      : "Now write your final answer for the user directly. Do not include your reasoning.",
};

const runtime = new SessionRuntime({ /* … */, reasoningRecovery });

The strategy shape mirrors ProviderDegradationStatsResolver — the host owns every domain value (the model set, the elicitation string) so no reasoning-model list lives in core. The recovery fires at most once per invoke, on both the clean-empty return path and the watchdog-unwind throw path, and only cascades if the retry is also content-free. It emits [UXParity:reasoning-only-recovery:*].

@pleach/core/providers also publishes the shared failure taxonomy — FailureCategory (AUTH / BILLING / RATE_LIMIT / MODEL_UNAVAILABLE / TOOL_HALLUCINATION / …), FailureClassification, BillingDisposition, ModelUnavailableReason — plus classifyEmptyStream (the precondition classifier the seam recovery keys on). Cross-SKU consumers (gateway, replay, observe) share the vocabulary; the provider-specific string→category matchers stay host-injected.

Plugin contribution hooks for retry, continuation, and family pivot

Four optional HarnessPlugin contribution hooks lift the retry loop's domain knowledge out of the host. Each is opt-in; an unset hook returns null and the runtime falls back to its default behavior.

HookReturnsCollectorWhat it owns
contributeRetryPolicyRetryPolicyContributionruntime.plugins.collectRetryPolicy()Retry behavior on transient failures — discovery-only tool set, workflow-narration nudge patterns, nudge template
contributeContinuationPolicyContinuationPolicyruntime.plugins.collectContinuationPolicy()When a tool result needs continuation vs termination — fetch-tool name set, stuck-discovery namespace prefixes, sandbox-clause regex
contributeFamilyPivotFamilyPivotContributionruntime.plugins.collectFamilyPivot()Classify a model into a family, then pick the next in-family rung — deriveFamilyFromModelId + pickNextInFamily paired
contributeFamilyExhaustedSurfaceFamilyExhaustedSurfaceContributionruntime.plugins.collectFamilyExhaustedSurface()UI surface (toast, banner, modal) when the family ladder exhausts — receives family, triedModels, chatId, optional messageId

The four hooks split the responsibility cleanly: contributeRetryPolicy

  • contributeContinuationPolicy carry DATA (tool name sets, regex patterns, templates); contributeFamilyPivot carries OPERATIONS (the harness has no model registry — both classification and in-family pick are host-supplied); contributeFamilyExhaustedSurface carries UI strategy (the harness has no view layer). Sketch:
import type { HarnessPlugin } from "@pleach/core";
import type { ProviderFamily } from "@pleach/core/modelfamily";

// Host-supplied — your own model-registry helpers. The harness has no
// model registry, so you author these and hand them to the runtime
// through `contributeFamilyPivot`.
declare function deriveFamilyFromModelId(modelId: string): ProviderFamily | null;
declare function pickNextInFamily(
  family: ProviderFamily,
  current: string,
  tried: Set<string>,
): { modelId: string } | null;

const myPlugin: HarnessPlugin = {
  name: "my-host",

  contributeFamilyPivot() {
    return {
      deriveFamilyFromModelId,
      pickNextInFamily: (family, current, tried) =>
        pickNextInFamily(family, current, new Set(tried))?.modelId ?? null,
    };
  },

  contributeFamilyExhaustedSurface() {
    return {
      surfaceFamilyExhausted: ({ family, triedModels, chatId, messageId }) => {
        myToastBus.emit({
          kind: "family-exhausted",
          family,
          attempted: Array.from(triedModels),
          chatId,
          messageId,
        });
      },
    };
  },
};

BYOK credential resolution is a sibling path: the createPleachRuntime factory accepts a byokResolver callback (per-session credential lookup) that the TurnOrchestrator reads via SessionRuntime.getByokResolver(). Any host with its own secrets store implements the same shape.

Provider cascade with audit attribution

Compose a primary plus fallback by wrapping two providers in a third. The cascade records why it fell through so the audit row joins back to a cause, not just a model id.

class CascadeProvider implements AgentProvider {
  constructor(private primary: AgentProvider, private fallback: AgentProvider) {}
  readonly capabilities = this.primary.capabilities;

  async *execute(config: AgentExecutionConfig): AsyncIterable<ProviderStreamEvent> {
    try {
      yield* this.primary.execute(config);
    } catch (err) {
      yield { type: "error", error: String(err), code: "5001" };
      yield { type: "message.delta", delta: "[fallback engaged]" };
      yield* this.fallback.execute({
        ...config,
        providerConfig: { ...config.providerConfig, cascadeReason: "primary_5001" },
      });
    }
  }

  abort() { this.primary.abort(); this.fallback.abort(); }
}

BYOK provider from session-scoped credentials

A multi-tenant deployment carries the customer key on the session, not the process env. Resolve it at construction time from the same userId / organizationId you pass to the runtime.

async function providerForOrg(organizationId: string): Promise<AgentProvider> {
  const apiKey = await loadOrgKey(organizationId);   // your secrets store
  return new AnthropicSdkProvider({
    apiKey,
    model:     "claude-sonnet-4-5",
    maxTokens: 4096,
  });
}

const runtime = new SessionRuntime({
  provider: await providerForOrg("org-acme"),
  storage:  supabaseAdapter,
  userId:   "user-7",
});

Provider vs orchestratorConfig

SessionRuntimeConfig accepts both provider (the AgentProvider-shaped surface above) and orchestratorConfig (a richer config object used by the legacy orchestrator path). Use provider for new code — it's the public substrate API. orchestratorConfig exists so hosts mid-migration don't break; new consumers should ignore it.

The class behind orchestratorConfig was renamed: OrchestratorClientTurnOrchestrator. OrchestratorClient remains exported from @pleach/core as a deprecated alias for the @pleach/core@1.x migration window and is removed at 2.0.0. Existing imports keep working; new code should reach for TurnOrchestrator (or, better, the provider surface above).

Where to go next

On this page