# @pleach/gateway (/docs/gateway) The gate in the hedge — every call walks through here, in family, on budget, attributable to one tenant. `@pleach/gateway` is the multi-tenant routing SKU layered over the `@pleach/core` model-family substrate. One client per tenant accepts a `(family, callClass, model, prompt)` call, picks a transport, walks the in-family cascade on transient failure, emits one cost event per successful call, and never silently widens across families. The substrate's in-family cascade behavior — what locks at session start and how `pickNextInFamily` walks the column — lives in [Family-locked routing](/docs/family-lock). The [model resolution matrix](/docs/model-resolution-matrix) defines the `(family × callClass)` cells the cascade walks. This page is the multi-tenant routing client layered on top: per-tenant scoping, operator allowlist, BYOK fingerprinting, and the per-call cost event. The package is intentionally thin. The routing math, family matrix, walk order, and BYOK transport resolution all live in [`@pleach/core`'s model-family module](/docs/model-resolution-matrix); gateway re-exports `CallClass`, `ProviderFamily`, and `RoutingDecision` so consumers don't need to import from both places, but it doesn't reimplement any of them. ## Install [#install] ```bash npm install @pleach/gateway ``` ```bash pnpm add @pleach/gateway ``` ```bash yarn add @pleach/gateway ``` ```bash bun add @pleach/gateway ``` ```typescript import { GatewayClient, asTenantId, fingerprintByokKey, GatewayFamilyDeniedError, GatewayFamilyExhaustedError, GatewayTransportMissingError, type CostEvent, type GatewayResponse, type GatewayTransport, } from "@pleach/gateway"; ``` The package re-exports `CallClass`, `ProviderFamily`, and `RoutingDecision` from `@pleach/core` as well, so a consumer that talks only to the gateway doesn't need a second import line for the substrate types. ## `GatewayClient` [#gatewayclient] One client per tenant is the canonical pattern. `tenantId` is required at construction; the constructor throws if it's missing or empty. Every cost event the client emits carries that tenant id, and the operator-supplied `allowedFamilies` allowlist is scoped to the client instance. ```typescript const gateway = new GatewayClient({ tenantId: asTenantId("acme-corp"), transports: new Map([ ["anthropic", anthropicTransport], ["openai", openaiTransport], ]), allowedFamilies: new Set(["anthropic", "openai"]), costEventEmitter: { emit(event) { eventLog.append(event); }, }, }); const response = await gateway.route({ family: "anthropic", callClass: "synthesize", model: "claude-sonnet-4-6", prompt: "Summarize the meeting notes …", }); ``` `asTenantId(raw)` is the helper that turns a plain string into the branded `TenantId` the constructor expects. It validates the input is non-empty so the silent-isolation case (an unset env var interpolated as `""`) becomes a load-bearing throw at init rather than a months-later billing incident. ### Constructor options [#constructor-options] | Option | Required | What it does | | ------------------ | ----------- | ---------------------------------------------------------------------------------------------------------------------------- | | `tenantId` | yes | Branded `TenantId` — every cost event the client emits carries it. | | `transports` | effectively | `Map`. Omitted → every `route()` call throws `GatewayTransportMissingError`. | | `allowedFamilies` | no | `Set`. Calls with a family outside the set throw `GatewayFamilyDeniedError` before any transport invocation. | | `byokKey` | no | Client-scoped BYOK key. Header-only; never persisted. Per-call `route({ byokKey })` overrides it. | | `costEventEmitter` | no | Sink for `CostEvent`. Defaults to a no-op so tests and dry-runs don't NPE; production must wire a real sink. | ### `route()` [#route] Returns a `GatewayResponse` on success. The response carries the final `modelInvoked`, the `usage` token counts, a `cost` block with the family and rate snapshot, the full `routingDecision`, the `cascadeWalks` history (empty when the first rung succeeded), and a `familyExhausted` boolean. ### `close()` [#close] Marks the client closed; subsequent `route()` calls reject. Phase A has no underlying resources to release — transports are caller-supplied — so `close()` is purely a state flag today. ## The transport seam [#the-transport-seam] `GatewayClient` doesn't ship concrete HTTP clients. It consumes a narrow `GatewayTransport` interface and a `Map` the host supplies at construction. ```typescript interface GatewayTransport { readonly kind: | "openrouter" | "native-anthropic" | "native-openai" | "native-google" | "bedrock" | "azure" | "vertex"; invoke(req: GatewayTransportRequest): Promise; } ``` The contract is intentionally minimal — the host wraps its existing provider client (OpenRouter HTTP, the Anthropic SDK, an in-house Bedrock adapter) in a `GatewayTransport` and hands the map to the constructor. Gateway never opens a socket of its own. A transport map keyed by `family` is the Phase A shape. Region pins and `(family, callClass)` transport arbitration are future work. ## Per-call cost event [#per-call-cost-event] Every successful `route()` emits one event to the configured sink. The cadence is per-call, not batched: accuracy and reproducibility for compliance attestation outweigh the batching savings. A sink that wants its own batching can implement it internally; the gateway's contract with consumers is one event per call. ```typescript interface CostEvent { readonly type: "domain.gateway.cost.recorded"; readonly tenantId: TenantId; readonly family: ProviderFamily; readonly callClass: CallClass; readonly modelInvoked: string; readonly costUsd: number; readonly promptTokens: number; readonly completionTokens: number; readonly byokActive: boolean; readonly byokKeyHash?: string; readonly routingDecision: RoutingDecision; readonly timestamp: string; } ``` The full `routingDecision` shape ships inside the payload so a downstream consumer — typically `@pleach/compliance`'s attestation runtime — has the provenance to attest the call without a round-trip to the audit ledger. `raw_provider_cost_usd` and `markup_pct` live inside the decision separately from the marked `costUsd`, so a consumer that needs the pre-markup figure has it directly. `response.cost.usd` is `raw_provider_cost × (1 + markupPct)`. The 20% flat markup is sourced from `MODEL_FAMILY_MATRIX`; the gateway doesn't compute it independently. ## BYOK is header-only [#byok-is-header-only] A BYOK key passed to the constructor or to an individual `route()` call is forwarded to the transport via the request payload and never written anywhere else. The gateway computes a 16-character sha256 fingerprint of the key for three purposes: * The cache fingerprint key, so the same BYOK gets cache reuse without ever comparing raw keys. * The `byok_key_id` field on the routing decision for audit attribution. * The `byokKeyHash` field on the emitted cost event. ```typescript import { fingerprintByokKey, isSameByokKey } from "@pleach/gateway"; fingerprintByokKey("sk-ant-api03-abc…"); // → "8f3a2b1c4d5e6f78" isSameByokKey(a, b); // → true iff fingerprintByokKey(a) === fingerprintByokKey(b) ``` The 16-character hex slice is opaque — enough entropy to de-dup within a tenant, not enough to reverse. `brandedFingerprintByokKey` returns the same string under a branded `ByokKeyFingerprint` type for code that wants to distinguish gateway hashes from arbitrary hex strings at the type layer. The caller is responsible for never logging the raw key. Once it's fingerprinted, gateway forgets the plain value. ## Family-strict cascade on 503 [#family-strict-cascade-on-503] On a transient transport failure (status 502 / 503 / 504, rate-limit 429, timeout, abort), `GatewayClient` walks the in-family rung ladder via `pickNextInFamily(family, currentModel, attempted)` from the substrate. The walk follows the same order the runtime uses — `synthesize → reasoning → utility → converse` — skipping any rung already tried this call. Each attempt produces a `CascadeWalk` entry on the response. When every rung fails, the gateway throws `GatewayFamilyExhaustedError` with the attempted-model list attached. It does **not** widen across families. That's the gateway-side mirror of the runtime-side [Family-Strict Cascade Pivot](/docs/model-resolution-matrix#cross-family-fallback-policy): cross-family is a consumer decision, made explicitly, not a silent behavior of the routing layer. ```typescript try { const response = await gateway.route({ family: "anthropic", … }); } catch (err) { if (err instanceof GatewayFamilyExhaustedError) { // err.family — "anthropic" // err.attemptedModels — ["claude-sonnet-4-6", "claude-haiku-4-5", …] // Consumer chooses: widen to "openai", surface to user, or fail. } } ``` The graceful `familyExhausted: true` flag on the response is reserved for a future shape where a transport reports a degraded-but-non-failed rung; Phase A throws on hard exhaustion to keep the error contract honest. The cascade is bounded at 16 steps so a pathological transport can't infinite-loop the gateway. ## `allowedFamilies` governance [#allowedfamilies-governance] `allowedFamilies` is the operator-facing governance hook. When set, `route()` checks the requested family against the allowlist **before any transport invocation** and throws `GatewayFamilyDeniedError` on denial. The throw is synchronous-within-the-promise — denied calls never touch the wire, never count against rate budgets, and never emit a cost event. ```typescript const gateway = new GatewayClient({ tenantId: asTenantId("acme-corp"), transports: transportMap, allowedFamilies: new Set(["anthropic", "openai"]), }); await gateway.route({ family: "deepseek", … }); // → throws GatewayFamilyDeniedError // err.family — "deepseek" // err.allowedFamilies — Set { "anthropic", "openai" } ``` Use it to enforce per-tenant family policy ("acme-corp is on anthropic-only"), to block a family during a rollback, or to gate a preview family behind an explicit operator opt-in. The mechanism is structural — a tenant on the allowlist cannot reach a family that's not on it, regardless of what the model id string says. ## Errors [#errors] Three error classes, three failure modes. All thrown via promise rejection from `route()`; the constructor's `TypeError` for invalid `tenantId` is the fourth. | Error | When | Thrown before transport? | | ------------------------------ | --------------------------------------------------- | ------------------------------------- | | `GatewayFamilyDeniedError` | `family` is not in `allowedFamilies`. | Yes — governance check is first. | | `GatewayTransportMissingError` | No transport configured for the resolved family. | Yes — configuration check is second. | | `GatewayFamilyExhaustedError` | Cascade walked every in-family rung and all failed. | No — at least one rung was attempted. | `GatewayFamilyDeniedError` and `GatewayTransportMissingError` are configuration failures and never reach the wire. `GatewayFamilyExhaustedError` is a cascade outcome and carries the attempted-model list so the consumer can decide what (if anything) to widen to. ## Tenant scoping [#tenant-scoping] `tenantId` is required at construction and stamped on every emitted `CostEvent`. The gateway's tenant-scope contract aligns with the substrate's [tenant facet](/docs/tenant-facet) and the [multi-tenant deployment](/docs/multi-tenant) pattern — same field, same partition key. The gateway's cost numbers come from the transport's reported `usage`, not from reading the core audit ledger; the gateway never queries the ledger. A consumer that wants both surfaces partitioned the same way (`GROUP BY tenantId`) gets that because both stamp the same opaque `tenantId`, not because the gateway joins through the ledger. The `tenantId` field is opaque the same way the audit row's is. A `GatewayClient` per end customer is the SaaS pattern; a `GatewayClient` per cost center is the internal-Enterprise pattern (one Anthropic Workspace or OpenAI Project sits behind the gateway, and the cost event partitions spend across teams). The cost-event sink reads the same `GROUP BY tenantId` either way. One `GatewayClient` instance per tenant is the canonical usage pattern. Per-tenant API keys (rotating credentials scoped to the tenant, separate from BYOK) are deferred to a later phase. ## Phase A status [#phase-a-status] What ships in the Phase A cut: * `GatewayClient` with `route()` and `close()`. * Per-call `domain.gateway.cost.recorded` event emission with full `RoutingDecision` payload. * BYOK fingerprinting (`fingerprintByokKey`, `isSameByokKey`, `brandedFingerprintByokKey`). * Family-strict cascade-on-503 via `pickNextInFamily`. * Operator `allowedFamilies` allowlist governance. * Three typed error classes. * Re-exports of `CallClass`, `ProviderFamily`, and `RoutingDecision` from `@pleach/core`. * `createGatewayRuntime()` contract + `routeChatCompletion()` body. `routeChatCompletion()` validates the input shape (`tenantId` / `family` / `messages`) and the `allowedProviders` governance hook, resolves the family default provider, and returns a structurally-valid `RouteChatCompletionOutput`. The response is stub-shape — the `content` field carries a placeholder stub value, `usage` is zero, and no transport invocation occurs. The shape is lossless against the steady-state output so slice 3+ drops in real transports without consumer churn. `routeEmbedding()`, `getProviderHealth()`, and `getTrafficStats()` continue to throw the slice-1 sentinel. What's **not** in Phase A — lands in subsequent passes: * **Concrete HTTP transports.** Phase A consumes a caller-supplied `GatewayTransport` map for `GatewayClient`, and `routeChatCompletion()` returns a stub-shape response without invoking any wire transport. Production hosts wire a minimal HTTP wrapper around OpenRouter or their existing native provider client. A bundled OpenRouter + native-Anthropic / native-OpenAI / native-Google adapter set, plus the Bedrock / Azure / Vertex region-pin routing inside `routeChatCompletion`, arrives in subsequent Phase B slices. * **OTEL `llm.invocation` span emission.** The C7 telemetry rung that records every gateway call as an OTEL span lands once `runtime.otel` is consumable from `@pleach/core`'s exported surface. Until then, callers thread their own span context if they need it. * **Failover policy primitives.** Region pins, native-vs-openrouter transport arbitration, and per-call failover policy objects are v1.x work. Phase A's cascade is family-strict and walk-order-strict; there's no policy knob between them. ## Related SKUs [#related-skus] * [`@pleach/core`](/docs/core) — the family-locked matrix the gateway routes against. `pickNextInFamily`, `MODEL_FAMILY_MATRIX`, and the `CallClass` / `ProviderFamily` / `RoutingDecision` types all live there; gateway re-exports them for convenience. * [`@pleach/compliance`](/docs/compliance) — downstream consumer of `domain.gateway.cost.recorded` events. Reads the full `RoutingDecision` payload for attestation provenance without a round-trip to the audit ledger. * [`@pleach/observe`](/docs/observability) — wires the gateway's cost events into OpenTelemetry / Datadog / Honeycomb spans. For the full SKU map see [Which SKU do I need?](/docs/which-sku). ## Where to go next [#where-to-go-next]