pleach
Gateway

Gateway · OpenTelemetry integration

OTel surface for `@pleach/gateway` — what's shipped today (cost counters), what's roadmap (tracer + spans).

OpenTelemetry support in @pleach/gateway is partial today. The cost-event substrate ships an OTel adapter that records two counters per call; a dedicated tracer that emits an llm.invocation span per call is roadmap.

This page documents what's actually shipped — be explicit about the gap so consumers wire their own span context until the tracer rung lands.

What ships today: OTel cost counters

The OTel surface is delivered via the cost-event adapter at @pleach/gateway/cost. It emits two counters per CostEvent:

pleach_gateway_cost_usd        # Counter, double, USD
pleach_gateway_tokens_total    # Counter, long,   {token}

pleach_gateway_tokens_total carries an extra dimension token.kind = "prompt" | "completion" so the prompt-vs-completion breakdown is preserved as a separate dimension rather than collapsed into a single sum.

Wiring

routeChatCompletion's output carries { providerInvoked, modelInvoked, content, usage, costUsd, routingDecisionId } — not tenantId, family, callClass, byokActive, or the full routingDecision. Merge those onto the observation from the request input plus your routing context, then read them in computeCost (see Cost events for the full mapper):

import { metrics } from "@opentelemetry/api"
import { createOtelCostEmitter, createCostMiddleware } from "@pleach/gateway/cost"
import { asTenantId } from "@pleach/gateway"
import type { RouteChatCompletionInput, RouteChatCompletionOutput } from "@pleach/gateway"

const meter = metrics.getMeter("@pleach/gateway")

const emitter = createOtelCostEmitter({ meter })

type CostObservation = RouteChatCompletionOutput & {
  tenantId: string
  family: RouteChatCompletionInput["family"]
  callClass: "synthesize" | "reasoning" | "converse" | "utility"
  byokActive: boolean
}

const cost = createCostMiddleware<CostObservation>({
  emitter,
  computeCost: (route, response) => ({
    type: "domain.gateway.cost.recorded",
    tenantId: asTenantId(response.tenantId),   // non-empty: sourced from input.tenantId
    family: response.family,
    callClass: response.callClass,
    modelInvoked: response.modelInvoked,
    costUsd: response.costUsd,
    promptTokens: response.usage.promptTokens,
    completionTokens: response.usage.completionTokens,
    byokActive: response.byokActive,
    routingDecision: {
      family: response.family,
      call_class: response.callClass,
      selected_model: response.modelInvoked,
      transport: "openrouter",
      byok_active: response.byokActive,
      upgrade_path_invoked: false,
      fallback_chain: [],
      raw_provider_cost_usd: response.costUsd,
      markup_pct: 0,
    },
    timestamp: new Date().toISOString(),
  }),
})

const output = await runtime.routeChatCompletion(input)
void cost({
  route: "chat.completion",
  response: {
    ...output,
    tenantId: input.tenantId,
    family: input.family,
    callClass: "converse",   // host routing context — not on the route output
    byokActive: false,       // host routing context — not on the route output
  },
})

routeChatCompletion is the Pack-221 slice-2 stub today: it returns costUsd: 0 and a stub content until the real transport lands.

Attributes recorded on every measurement

AttributeSource
tenant.idevent.tenantId
provider.familyevent.family
call.classevent.callClass
model.invokedevent.modelInvoked
byok.activeevent.byokActive
cost.event.type"domain.gateway.cost.recorded"

token.kind ("prompt" / "completion") is added to pleach_gateway_tokens_total measurements only.

The OTel SDK is an optional peer

createOtelCostEmitter accepts the meter as a duck-typed argument mirroring @opentelemetry/api's Meter shape. The adapter never imports @opentelemetry/api at module-eval time — the duck-typed shape is lifted to a local OtelMeter interface so callers without an @opentelemetry/api type ambient can still satisfy it (a stub in tests, a custom in-house meter facade).

The factory validates only that the supplied meter exposes createCounter(name, options?):

if (!opts.meter || typeof opts.meter.createCounter !== "function") {
  throw new TypeError(
    "@pleach/gateway/cost/adapters/otel: options.meter must implement createCounter(name, options?)",
  )
}

Override counter names

For deployments that prefix metrics by team or service:

const emitter = createOtelCostEmitter({
  meter,
  costCounterName: "myco.pleach.gateway.cost.usd",
  tokensCounterName: "myco.pleach.gateway.tokens.total",
})

Soft-fail semantics

Per the cost-adapter contract, emit() swallows internal exceptions — a metric backend hiccup MUST NOT propagate up the gateway call:

try {
  costCounter.add(event.costUsd, baseAttrs)
  // ... tokensCounter.add(...) ...
} catch {
  // Soft-fail per cost-adapter contract — never propagate.
}

What's roadmap: dedicated tracer + llm.invocation spans

A full OTel tracer integration is v1.x work. The C7 telemetry rung that records every gateway call as an OTel span lands once runtime.otel is consumable from @pleach/core's exported surface.

Until then, callers thread their own span context if they need distributed tracing across the gateway boundary:

import { trace, context, SpanKind } from "@opentelemetry/api"

const tracer = trace.getTracer("my-app")

await tracer.startActiveSpan(
  "llm.invocation",
  { kind: SpanKind.CLIENT, attributes: {
    "tenant.id": tenantId,
    "provider.family": family,
    "call.class": callClass,
    "model.requested": model,
  }},
  async (span) => {
    try {
      const response = await runtime.routeChatCompletion(input)
      span.setAttribute("model.invoked", response.modelInvoked)
      span.setAttribute("cost.usd", response.costUsd)
      span.setAttribute("tokens.prompt", response.usage.promptTokens)
      span.setAttribute("tokens.completion", response.usage.completionTokens)
      // byok.active is not on the route output — set it from your
      // routing context (input / BYOK resolver), not from response.
      return response
    } catch (err) {
      span.recordException(err as Error)
      throw err
    } finally {
      span.end()
    }
  },
)

This shim is not what we want long-term — span attributes are inferred from response shape rather than emitted at the routing decision point, cascade walks are not captured per-rung, and BYOK hash provenance is missing. The dedicated Tracer SKU will own all of this. Track the work under the gateway punch list.

Cited source

  • packages/gateway/src/cost/adapters/otel.ts — OtelMeter / OtelCounter duck-typed interfaces and createOtelCostEmitter factory.
  • packages/gateway/src/cost/CostMiddleware.ts — middleware that composes the OTel adapter with any route response shape.

Where to go next

Cost events

The full CostEmitter contract + the other three adapters (Postgres, Datadog, Honeycomb).

OTel observability

The platform-wide OTel posture — telemetry boundaries, span conventions, and the `runtime.otel` facet roadmap.

`@pleach/gateway` overview

The Phase A GatewayClient — route(), cost events, family-strict cascade, transport seam.

Gateway · Cost events

Per-call cost-event emission — CostEmitter contract, CostMiddleware factory, and four reference adapters (OTel, Postgres, Datadog, Honeycomb).

Gateway · Migration from `@pleach/core`

When to introduce `@pleach/gateway` over direct `@pleach/core` provider use — multi-tenant, BYOK, cost rollup, region failover, identity federation.

On this page

What ships today: OTel cost countersWiringAttributes recorded on every measurementThe OTel SDK is an optional peerOverride counter namesSoft-fail semanticsWhat's roadmap: dedicated tracer + llm.invocation spansCited sourceWhere to go next