Gateway · OpenTelemetry integration

@pleach/gateway emits two kinds of OpenTelemetry signal:

• Spans. One llm.invocation span per routed call, from GatewayClient.route() and from a transport-served GatewayRuntime.routeChatCompletion().
• Counters. The cost-event adapter records a cost counter and a token counter per CostEvent.

Cost counters

The OTel surface is delivered via the cost-event adapter at @pleach/gateway/cost. It emits two counters per CostEvent:

pleach_gateway_cost_usd        # Counter, double, USD
pleach_gateway_tokens_total    # Counter, long,   {token}

pleach_gateway_tokens_total carries an extra dimension token.kind = "prompt" | "completion" so the prompt-vs-completion breakdown is preserved as a separate dimension rather than collapsed into a single sum.

Wiring

routeChatCompletion's output carries { providerInvoked, modelInvoked, content, usage, costUsd, routingDecisionId } — not tenantId, family, callClass, byokActive, or the full routingDecision. Merge those onto the observation from the request input plus your routing context, then read them in computeCost (see Cost events for the full mapper):

import { metrics } from "@opentelemetry/api"
import { createOtelCostEmitter, createCostMiddleware } from "@pleach/gateway/cost"
import { asTenantId } from "@pleach/gateway"
import type { RouteChatCompletionInput, RouteChatCompletionOutput } from "@pleach/gateway"

const meter = metrics.getMeter("@pleach/gateway")

const emitter = createOtelCostEmitter({ meter })

type CostObservation = RouteChatCompletionOutput & {
  tenantId: string
  family: RouteChatCompletionInput["family"]
  callClass: "synthesize" | "reasoning" | "converse" | "utility"
  byokActive: boolean
}

const cost = createCostMiddleware<CostObservation>({
  emitter,
  computeCost: (route, response) => ({
    type: "domain.gateway.cost.recorded",
    tenantId: asTenantId(response.tenantId),   // non-empty: sourced from input.tenantId
    family: response.family,
    callClass: response.callClass,
    modelInvoked: response.modelInvoked,
    costUsd: response.costUsd,
    promptTokens: response.usage.promptTokens,
    completionTokens: response.usage.completionTokens,
    byokActive: response.byokActive,
    routingDecision: {
      family: response.family,
      call_class: response.callClass,
      selected_model: response.modelInvoked,
      transport: "openrouter",
      byok_active: response.byokActive,
      upgrade_path_invoked: false,
      fallback_chain: [],
      raw_provider_cost_usd: response.costUsd,
      markup_pct: 0,
    },
    timestamp: new Date().toISOString(),
  }),
})

const output = await runtime.routeChatCompletion(input)
void cost({
  route: "chat.completion",
  response: {
    ...output,
    tenantId: input.tenantId,
    family: input.family,
    callClass: "converse",   // host routing context — not on the route output
    byokActive: false,       // host routing context — not on the route output
  },
})

With no transport wired for the resolved provider, routeChatCompletion returns a stub response with costUsd: 0.

Attributes recorded on every measurement

token.kind ("prompt" / "completion") is added to pleach_gateway_tokens_total measurements only.

The OTel SDK is an optional peer

createOtelCostEmitter accepts the meter as a duck-typed argument mirroring @opentelemetry/api's Meter shape. The adapter never imports @opentelemetry/api at module-eval time — the duck-typed shape is lifted to a local OtelMeter interface so callers without an @opentelemetry/api type ambient can still satisfy it (a stub in tests, a custom in-house meter facade).

The factory validates only that the supplied meter exposes createCounter(name, options?):

if (!opts.meter || typeof opts.meter.createCounter !== "function") {
  throw new TypeError(
    "@pleach/gateway/cost/adapters/otel: options.meter must implement createCounter(name, options?)",
  )
}

Override counter names

For deployments that prefix metrics by team or service:

const emitter = createOtelCostEmitter({
  meter,
  costCounterName: "myco.pleach.gateway.cost.usd",
  tokensCounterName: "myco.pleach.gateway.tokens.total",
})

Soft-fail semantics

Per the cost-adapter contract, emit() swallows internal exceptions — a metric backend hiccup MUST NOT propagate up the gateway call:

try {
  costCounter.add(event.costUsd, baseAttrs)
  // ... tokensCounter.add(...) ...
} catch {
  // Soft-fail per cost-adapter contract — never propagate.
}

llm.invocation spans

The gateway emits one span named llm.invocation per routed call, the same span name core's seams emit:

• GatewayClient.route() emits one span per call that reaches a transport.
• GatewayRuntime.routeChatCompletion() emits one span per transport-served call. The stub path, with no transport wired, emits none because no provider was invoked.

Cascade and failover hops are attributes on that one span, not separate spans. The provider call runs with the span as the active OTel context, so spans the transport opens (HTTP client instrumentation, for example) nest under it.

Choosing a tracer

import { trace } from "@opentelemetry/api"
import { GatewayClient } from "@pleach/gateway"

const client = new GatewayClient({
  tenantId,
  transports,
  tracer: trace.getTracer("my-app"), // or null to turn spans off
})

• A tracer you pass wins. It's typed structurally, so any @opentelemetry/api Tracer fits.
• tracer: null disables span emission.
• With no tracer, the gateway lazily imports @opentelemetry/api (an optional peer, ^1.0.0) on the first routed call and uses trace.getTracer("@pleach/gateway"). If the package isn't installed, emission is a silent no-op.

GatewayRuntimeConfig.tracer takes the same values. A throwing tracer never fails a routed call: every tracer and span call is wrapped, and a failure degrades to no span.

The lazy import uses a non-literal specifier so a bundler can't hard-fail on the missing optional peer. Webpack may print a "Critical dependency" warning for it. Pass tracer explicitly, or null, to avoid both the warning and the lookup.

Attributes

The tenant, call-class, family, and token keys come from @pleach/core/otel, so gateway and core traces filter on the same keys. pleach.model_id uses the same spelling core's seam spans use.

GATEWAY_SPAN_ATTRIBUTE_KEYS and LLM_INVOCATION_SPAN_NAME are exported from the package root, along with the GatewayTracerLike and GatewaySpanLike types.

Token, cost, and served-model attributes are set when the call succeeds. A transport result without usage is tolerated: the token attributes are omitted, never set to 0.

Errors

A failed call ends the span with status ERROR. It records only:

• error.type: the error class name, when it's a plain identifier (otherwise "Error"), never the message.
• http.response.status_code: when the error carries a numeric HTTP status.

The status message and the recorded exception both use the fixed text llm invocation failed. Transport error messages can embed the full upstream response body, including prompts or a key echoed back by a proxy, so the raw message is never copied onto the span.

For route(), the span ends after the cost event is emitted, so a cost-sink failure shows as an ERROR span.

callClass on routeChatCompletion()

RouteChatCompletionInput.callClass is optional and defaults to "converse". It must be one of core's ALL_CALL_CLASSES; any other value throws TypeError. It feeds pleach.call_class and the provenance row's call.callClass (and, after a failover, fallbackStep.originalCallClass / attemptCallClass). It's attribution only: GatewayRuntime selects by family and model, not by call class.

Cited source

• packages/gateway/src/cost/adapters/otel.ts — OtelMeter / OtelCounter duck-typed interfaces and createOtelCostEmitter factory.
• packages/gateway/src/cost/CostMiddleware.ts — middleware that composes the OTel adapter with any route response shape.
• packages/gateway/src/otel/llmInvocationSpan.ts — span name, attribute keys, tracer resolution, and error sanitizing.

Where to go next