@pleach/gateway

The gate in the hedge — every call walks through here, in family, on budget, attributable to one tenant. @pleach/gateway is the multi-tenant routing SKU layered over the @pleach/core model-family substrate. One client per tenant accepts a (family, callClass, model, prompt) call, picks a transport, walks the in-family cascade on transient failure, emits one cost event per successful call, and never silently widens across families.

The substrate's in-family cascade behavior — what locks at session start and how pickNextInFamily walks the column — lives in Family-locked routing. The model resolution matrix defines the (family × callClass) cells the cascade walks. This page is the multi-tenant routing client layered on top: per-tenant scoping, operator allowlist, BYOK fingerprinting, and the per-call cost event.

The package is intentionally thin. The routing math, family matrix, walk order, and BYOK transport resolution all live in @pleach/core's model-family module; gateway re-exports CallClass, ProviderFamily, and RoutingDecision so consumers don't need to import from both places, but it doesn't reimplement any of them.

Install

npm install @pleach/gateway

pnpm add @pleach/gateway

yarn add @pleach/gateway

bun add @pleach/gateway

import {
  GatewayClient,
  asTenantId,
  fingerprintByokKey,
  GatewayFamilyDeniedError,
  GatewayFamilyExhaustedError,
  GatewayTransportMissingError,
  type CostEvent,
  type GatewayResponse,
  type GatewayTransport,
} from "@pleach/gateway";

The package re-exports CallClass, ProviderFamily, and RoutingDecision from @pleach/core as well, so a consumer that talks only to the gateway doesn't need a second import line for the substrate types.

GatewayClient

One client per tenant is the canonical pattern. tenantId is required at construction; the constructor throws if it's missing or empty. Every cost event the client emits carries that tenant id, and the operator-supplied allowedFamilies allowlist is scoped to the client instance.

const gateway = new GatewayClient({
  tenantId: asTenantId("acme-corp"),
  transports: new Map([
    ["anthropic", anthropicTransport],
    ["openai",    openaiTransport],
  ]),
  allowedFamilies: new Set(["anthropic", "openai"]),
  costEventEmitter: {
    emit(event) {
      eventLog.append(event);
    },
  },
});

const response = await gateway.route({
  family:    "anthropic",
  callClass: "synthesize",
  model:     "claude-sonnet-4-6",
  prompt:    "Summarize the meeting notes …",
});

asTenantId(raw) is the helper that turns a plain string into the branded TenantId the constructor expects. It validates the input is non-empty so the silent-isolation case (an unset env var interpolated as "") becomes a load-bearing throw at init rather than a months-later billing incident.

Constructor options

route()

Returns a GatewayResponse on success. The response carries the final modelInvoked, the usage token counts, a cost block with the family and rate snapshot, the full routingDecision, the cascadeWalks history (empty when the first rung succeeded), and a familyExhausted boolean.

close()

Marks the client closed; subsequent route() calls reject. Phase A has no underlying resources to release — transports are caller-supplied — so close() is purely a state flag today.

The transport seam

GatewayClient doesn't ship concrete HTTP clients. It consumes a narrow GatewayTransport interface and a Map<family, transport> the host supplies at construction.

interface GatewayTransport {
  readonly kind:
    | "openrouter"
    | "native-anthropic"
    | "native-openai"
    | "native-google"
    | "bedrock"
    | "azure"
    | "vertex";
  invoke(req: GatewayTransportRequest): Promise<GatewayTransportResponse>;
}

The contract is intentionally minimal — the host wraps its existing provider client (OpenRouter HTTP, the Anthropic SDK, an in-house Bedrock adapter) in a GatewayTransport and hands the map to the constructor. Gateway never opens a socket of its own.

A transport map keyed by family is the Phase A shape. Region pins and (family, callClass) transport arbitration are future work.

Per-call cost event

Every successful route() emits one event to the configured sink. The cadence is per-call, not batched: accuracy and reproducibility for compliance attestation outweigh the batching savings. A sink that wants its own batching can implement it internally; the gateway's contract with consumers is one event per call.

interface CostEvent {
  readonly type:             "domain.gateway.cost.recorded";
  readonly tenantId:         TenantId;
  readonly family:           ProviderFamily;
  readonly callClass:        CallClass;
  readonly modelInvoked:     string;
  readonly costUsd:          number;
  readonly promptTokens:     number;
  readonly completionTokens: number;
  readonly byokActive:       boolean;
  readonly byokKeyHash?:     string;
  readonly routingDecision:  RoutingDecision;
  readonly timestamp:        string;
}

The full routingDecision shape ships inside the payload so a downstream consumer — typically @pleach/compliance's attestation runtime — has the provenance to attest the call without a round-trip to the audit ledger. raw_provider_cost_usd and markup_pct live inside the decision separately from the marked costUsd, so a consumer that needs the pre-markup figure has it directly.

response.cost.usd is raw_provider_cost × (1 + markupPct). The 20% flat markup is sourced from MODEL_FAMILY_MATRIX; the gateway doesn't compute it independently.

BYOK is header-only

A BYOK key passed to the constructor or to an individual route() call is forwarded to the transport via the request payload and never written anywhere else. The gateway computes a 16-character sha256 fingerprint of the key for three purposes:

• The cache fingerprint key, so the same BYOK gets cache reuse without ever comparing raw keys.
• The byok_key_id field on the routing decision for audit attribution.
• The byokKeyHash field on the emitted cost event.

import { fingerprintByokKey, isSameByokKey } from "@pleach/gateway";

fingerprintByokKey("sk-ant-api03-abc…");
// → "8f3a2b1c4d5e6f78"

isSameByokKey(a, b);
// → true iff fingerprintByokKey(a) === fingerprintByokKey(b)

The 16-character hex slice is opaque — enough entropy to de-dup within a tenant, not enough to reverse. brandedFingerprintByokKey returns the same string under a branded ByokKeyFingerprint type for code that wants to distinguish gateway hashes from arbitrary hex strings at the type layer.

The caller is responsible for never logging the raw key. Once it's fingerprinted, gateway forgets the plain value.

Family-strict cascade on 503

On a transient transport failure (status 502 / 503 / 504, rate-limit 429, timeout, abort), GatewayClient walks the in-family rung ladder via pickNextInFamily(family, currentModel, attempted) from the substrate. The walk follows the same order the runtime uses — synthesize → reasoning → utility → converse — skipping any rung already tried this call. Each attempt produces a CascadeWalk entry on the response.

When every rung fails, the gateway throws GatewayFamilyExhaustedError with the attempted-model list attached. It does not widen across families. That's the gateway-side mirror of the runtime-side Family-Strict Cascade Pivot: cross-family is a consumer decision, made explicitly, not a silent behavior of the routing layer.

try {
  const response = await gateway.route({ family: "anthropic", … });
} catch (err) {
  if (err instanceof GatewayFamilyExhaustedError) {
    // err.family            — "anthropic"
    // err.attemptedModels   — ["claude-sonnet-4-6", "claude-haiku-4-5", …]
    // Consumer chooses: widen to "openai", surface to user, or fail.
  }
}

The graceful familyExhausted: true flag on the response is reserved for a future shape where a transport reports a degraded-but-non-failed rung; Phase A throws on hard exhaustion to keep the error contract honest.

The cascade is bounded at 16 steps so a pathological transport can't infinite-loop the gateway.

allowedFamilies governance

allowedFamilies is the operator-facing governance hook. When set, route() checks the requested family against the allowlist before any transport invocation and throws GatewayFamilyDeniedError on denial. The throw is synchronous-within-the-promise — denied calls never touch the wire, never count against rate budgets, and never emit a cost event.

const gateway = new GatewayClient({
  tenantId:        asTenantId("acme-corp"),
  transports:      transportMap,
  allowedFamilies: new Set(["anthropic", "openai"]),
});

await gateway.route({ family: "deepseek", … });
// → throws GatewayFamilyDeniedError
//   err.family            — "deepseek"
//   err.allowedFamilies   — Set { "anthropic", "openai" }

Use it to enforce per-tenant family policy ("acme-corp is on anthropic-only"), to block a family during a rollback, or to gate a preview family behind an explicit operator opt-in. The mechanism is structural — a tenant on the allowlist cannot reach a family that's not on it, regardless of what the model id string says.

Errors

Three error classes, three failure modes. All thrown via promise rejection from route(); the constructor's TypeError for invalid tenantId is the fourth.

GatewayFamilyDeniedError and GatewayTransportMissingError are configuration failures and never reach the wire. GatewayFamilyExhaustedError is a cascade outcome and carries the attempted-model list so the consumer can decide what (if anything) to widen to.

Tenant scoping

tenantId is required at construction and stamped on every emitted CostEvent. The gateway's tenant-scope contract aligns with the substrate's tenant facet and the multi-tenant deployment pattern — same field, same partition key. The gateway's cost numbers come from the transport's reported usage, not from reading the core audit ledger; the gateway never queries the ledger. A consumer that wants both surfaces partitioned the same way (GROUP BY tenantId) gets that because both stamp the same opaque tenantId, not because the gateway joins through the ledger.

The tenantId field is opaque the same way the audit row's is. A GatewayClient per end customer is the SaaS pattern; a GatewayClient per cost center is the internal-Enterprise pattern (one Anthropic Workspace or OpenAI Project sits behind the gateway, and the cost event partitions spend across teams). The cost-event sink reads the same GROUP BY tenantId either way.

One GatewayClient instance per tenant is the canonical usage pattern. Per-tenant API keys (rotating credentials scoped to the tenant, separate from BYOK) are deferred to a later phase.

Phase A status

What ships in the Phase A cut:

• GatewayClient with route() and close().
• Per-call domain.gateway.cost.recorded event emission with full RoutingDecision payload.
• BYOK fingerprinting (fingerprintByokKey, isSameByokKey, brandedFingerprintByokKey).
• Family-strict cascade-on-503 via pickNextInFamily.
• Operator allowedFamilies allowlist governance.
• Three typed error classes.
• Re-exports of CallClass, ProviderFamily, and RoutingDecision from @pleach/core.
• createGatewayRuntime() contract + routeChatCompletion() body. routeChatCompletion() validates the input shape (tenantId / family / messages) and the allowedProviders governance hook, resolves the family default provider, and returns a structurally-valid RouteChatCompletionOutput. The response is stub-shape — the content field carries a placeholder stub value, usage is zero, and no transport invocation occurs. The shape is lossless against the steady-state output so slice 3+ drops in real transports without consumer churn. routeEmbedding(), getProviderHealth(), and getTrafficStats() continue to throw the slice-1 sentinel.

What's not in Phase A — lands in subsequent passes:

• Concrete HTTP transports. Phase A consumes a caller-supplied GatewayTransport map for GatewayClient, and routeChatCompletion() returns a stub-shape response without invoking any wire transport. Production hosts wire a minimal HTTP wrapper around OpenRouter or their existing native provider client. A bundled OpenRouter + native-Anthropic / native-OpenAI / native-Google adapter set, plus the Bedrock / Azure / Vertex region-pin routing inside routeChatCompletion, arrives in subsequent Phase B slices.
• OTEL llm.invocation span emission. The C7 telemetry rung that records every gateway call as an OTEL span lands once runtime.otel is consumable from @pleach/core's exported surface. Until then, callers thread their own span context if they need it.
• Failover policy primitives. Region pins, native-vs-openrouter transport arbitration, and per-call failover policy objects are v1.x work. Phase A's cascade is family-strict and walk-order-strict; there's no policy knob between them.

Related SKUs

• @pleach/core — the family-locked matrix the gateway routes against. pickNextInFamily, MODEL_FAMILY_MATRIX, and the CallClass / ProviderFamily / RoutingDecision types all live there; gateway re-exports them for convenience.
• @pleach/compliance — downstream consumer of domain.gateway.cost.recorded events. Reads the full RoutingDecision payload for attestation provenance without a round-trip to the audit ledger.
• @pleach/observe — wires the gateway's cost events into OpenTelemetry / Datadog / Honeycomb spans.

For the full SKU map see Which SKU do I need?.

Where to go next