# Deployment (/docs/deployment) A production deployment of `@pleach/core` has five concerns: schema migrations against the target database, environment variable wiring, a runtime-construction pattern that fits your hosting model, observability hooks, and a rollback strategy. This page walks each. The goal is to ship without surprises — the substrate has no hidden state, no implicit network calls, and no required external service beyond what you wire. ## Environments [#environments] The runtime supports three named environments distinguished by the `runtimeMode` [fingerprint](/docs/fingerprint) field: | Mode | What changes | | ---------------- | ----------------------------------------------------------------------- | | `production` | Full ledger writes, [cache](/docs/cache) reads enabled, replay disabled | | `replay` | Reads from recorded ledger; provider calls go through fingerprint cache | | `eval-noncached` | Provider calls execute; cache writes disabled (for honest baseline) | Set via runtime config or environment variable (`HARNESS_RUNTIME_MODE`). Production code paths should never default this — pass it explicitly. ## Pre-deployment checklist [#pre-deployment-checklist] * [ ] Schema bundle applied to the production database * [ ] `FEATURE_HARNESS_V2_RUNTIME=true` set (or unset — true is the default) * [ ] `HARNESS_MOCK_MODE` not set (or explicitly `false`) * [ ] Provider credentials in env (`OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, etc.) * [ ] `SUPABASE_URL` and `SUPABASE_SERVICE_ROLE_KEY` set; service key never reaches client bundle * [ ] DevTools hook (`useHarnessDevTools`) gated behind `NODE_ENV !== "production"` * [ ] Plugin set finalized; versions pinned in `package.json` * [ ] `enabledSafetyPolicies` reviewed for each tenant * [ ] Observability sinks wired (next section) * [ ] Rollback plan documented (last section) ## Schema migrations [#schema-migrations] Two paths to apply the schema bundle. ### Supabase CLI [#supabase-cli] ```bash npx pleach init --apply --target ./supabase/migrations --timestamped supabase db push ``` `--timestamped` prefixes each file with `YYYYMMDDHHMMSS_` so the files slot cleanly into Supabase's chronological migration history. ### Manual `psql` [#manual-psql] ```bash npx pleach init --apply --target ./harness-migrations for f in harness-migrations/*.sql; do psql "$DATABASE_URL" -f "$f" done ``` Both paths are idempotent — every file uses `CREATE ... IF NOT EXISTS` and `DROP POLICY IF EXISTS` — but they don't migrate column shapes. Schema evolution lands as new files; running an old apply against a newer database is safe but won't backfill missing fields. See [CLI](/docs/cli) and [Schema](/docs/schema) for the details. ## Runtime construction patterns [#runtime-construction-patterns] The right construction pattern depends on the hosting model. ### Long-lived process (Node, containers) [#long-lived-process-node-containers] Construct one runtime at startup; reuse for every request. ```typescript // server.ts import { SessionRuntime, AiSdkProvider } from "@pleach/core"; import { createOpenRouter } from "@openrouter/ai-sdk-provider"; const openrouter = createOpenRouter({ apiKey: process.env.OPENROUTER_API_KEY! }); export const runtime = new SessionRuntime({ storage: supabaseAdapter, checkpointer: supabaseSaver, provider: new AiSdkProvider({ model: openrouter("anthropic/claude-sonnet-4-5"), maxSteps: 5 }), plugins: productionPlugins, // userId / organizationId set per-request via session scoping below }); // Per-request handler: app.post("/api/chat", async (req, res) => { // Scope the runtime to this request's tenant via the call: const session = await runtime.createSession(/* ... */); // ... }); ``` For multi-tenant deployments where tenant credentials differ, construct per-request — see [Multi-tenant](/docs/multi-tenant). ### Serverless functions (Fluid Compute, Lambda, Cloudflare Workers) [#serverless-functions-fluid-compute-lambda-cloudflare-workers] Two patterns, depending on cold-start sensitivity. **Pattern A — Construct in handler.** Simple; pays runtime construction on every cold start. ```typescript export async function POST(req: Request) { const runtime = new SessionRuntime({/* ... */}); // ... handle request } ``` **Pattern B — Module-scope construct, lazy-init storage.** The runtime object is reused across warm invocations; storage and provider clients lazy-init. ```typescript // app/api/chat/route.ts const runtimePromise = (async () => { const supabase = createClient(/* ... */); return new SessionRuntime({/* ... */}); })(); export async function POST(req: Request) { const runtime = await runtimePromise; // ... } ``` A full route handler that streams the response and forwards the client's abort signal looks like this: ```typescript // app/api/chat/route.ts import { SessionRuntime } from "@pleach/core"; import { SupabaseAdapter } from "@pleach/core/sessions"; import { SupabaseSaver } from "@pleach/core/checkpointing"; export const runtime = "edge"; export async function POST(req: Request) { const { sessionId, message } = (await req.json()) as { sessionId: string; message: string; }; const rt = new SessionRuntime({ storage: new SupabaseAdapter({ client: supabase }), checkpointer: new SupabaseSaver({ client: supabase }), organizationId: "org-acme", userId: "user-7", }); const events = rt.executeMessage(sessionId, message, { abortSignal: req.signal }); const stream = new ReadableStream({ async pull(controller) { const { value, done } = await events.next(); if (done) { controller.close(); return; } controller.enqueue(new TextEncoder().encode(`data: ${JSON.stringify(value)}\n\n`)); }, }); return new Response(stream, { headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache, no-transform" }, }); } ``` For Vercel Fluid Compute specifically: register `waitUntil` with the runtime's durable-flush pipeline so the event log survives function teardown: ```typescript // app/api/chat/route.ts import { setWaitUntilImpl } from "@pleach/core/eventLog"; export async function POST(req: Request, ctx: { waitUntil: (p: Promise) => void }) { setWaitUntilImpl(ctx.waitUntil.bind(ctx)); // ... handle request } ``` See [Event log](/docs/event-log) for the durable-flush contract. ### Edge runtime constraints [#edge-runtime-constraints] Edge runtimes (Cloudflare Workers, Vercel Edge) restrict which Node APIs are available. The substrate avoids Node-only APIs in its hot path — `fingerprint`, [`channels`](/docs/channels), `audit`, and `prompt-builder` are all isomorphic. The Supabase adapter and the Anthropic SDK provider work on edge runtimes. The storage adapter's choice of `crypto.randomUUID()` vs a Node-only UUID generator is the most common edge-incompatibility gotcha; both `MemoryAdapter` and `SupabaseAdapter` use the Web Crypto API explicitly. The other concrete limitation: edge runtimes cap response time per request (Cloudflare Workers' default is \~30s of CPU time per invocation; Vercel Edge gives you \~25s of wall-clock streaming before the connection idle-cuts). A turn that fans out to three subagents each running a 20s tool call exceeds the budget on edge and needs to move to Fluid Compute or a long-lived Node process. Anchor-plan + tool-loop + synthesize turns that complete in under \~20s stay edge-safe; longer-running orchestrations belong on Fluid Compute with `waitUntil` registered via `setWaitUntilImpl`. ## Observability [#observability] The substrate emits two streams ready for observability sinks: ### `runtime.on(event)` [#runtimeonevent] Every `StreamEvent` type also fires as a `SessionRuntime` event. Wire a long-lived subscriber: ```typescript runtime.on("checkpoint.created", (event) => { metrics.increment("checkpoints", { sessionId: event.checkpoint.sessionId }); }); runtime.on("error", (event) => { errors.capture(event.error, { code: event.code }); }); runtime.on("subagent.completed", (event) => { metrics.timing("subagent.duration", event.durationMs); }); ``` ### The audit ledger [#the-audit-ledger] The `ProviderDecisionLedger` write is the per-call telemetry hook that always fires (every LLM call, never dropped). Wire a custom adapter that writes to both your primary store and your observability sink: ```typescript class DualLedger implements ProviderDecisionLedger { constructor( private primary: ProviderDecisionLedger, private telemetry: TelemetryClient, ) {} async recordCall(call: AuditableCall): Promise { this.telemetry.record("llm.call", { model: call.modelId, family: call.family, latency: call.latencyMs, tokens: call.tokenUsage, }); return this.primary.recordCall(call); } } ``` The telemetry write is non-blocking by contract — a failed telemetry call doesn't break the turn. ### OpenTelemetry [#opentelemetry] For OTel-shaped observability, the same pattern wraps the runtime's events and ledger writes into spans. The `@pleach/gateway` SKU ships OTel spans pre-wired; for non-gateway deployments, build the spans in your custom ledger adapter and event subscribers. ## Logging [#logging] The substrate's default loggers write event types and ids — not payloads. For production logging: * Set log level via your standard mechanism. The runtime respects `LOG_LEVEL` if your logger does. * Don't log raw `AuditableCall` rows from custom adapters unless you've wired `PIIRedactor` — raw messages may contain PII. * Stream events that carry user content (`message.delta`, `message.complete`) shouldn't go to long-term logs. Use the [audit ledger](/docs/audit-ledger) as the durable record; treat logs as ephemeral. ## Rollback strategy [#rollback-strategy] Three layers of rollback, in order of granularity. ### 1. Per-session checkpoint rollback [#1-per-session-checkpoint-rollback] The runtime's built-in [time-travel](/docs/time-travel). Use during incident response to revert a single session to a prior point: ```typescript await runtime.checkpoints.rollback(sessionId, "cp_018f..."); ``` Audit-ledger rows for the rolled-back portion stay (append-only contract); the next turn continues from the restored state. The rollback itself writes a new `AuditableCall` row with the rollback target id in the payload, so a regulator's "show every state transition for `session-018f-7a`" query reads both the original calls and the rollback marker — the history is forward-only even when the session state isn't. ### 2. Application-version rollback [#2-application-version-rollback] Standard deploy rollback. The fingerprint includes `pleachVersion`, so rolling back the substrate version invalidates the cache automatically — no stale-cache risk. If you've added prompt contributions or safety policies between versions, the fingerprint changes accordingly; old cache entries are invalidated by construction. ### 3. Schema rollback [#3-schema-rollback] Schema rollback is hard and the substrate doesn't try to make it easy. The append-only contract on the audit ledger means a schema "rollback" is really a forward-fix: ship a new migration that restores the prior shape or adds back a removed column. For schema-shape mistakes caught before production load: drop the table, re-apply the bundle, re-run the migration. For schema-shape mistakes caught after production data has landed: forward-fix only. The audit ledger is the source of truth for what calls were made; the runtime can re-hydrate session state from [`harness_event_log`](/docs/event-log) if `harness_sessions` needs to be rebuilt. ## Feature flag rollouts [#feature-flag-rollouts] The substrate has one master switch: `FEATURE_HARNESS_V2_RUNTIME`. Setting it to `false` disables the runtime — all `/api/harness/*` routes return 503. Useful for emergency disable; not a graceful rollback. For graceful rollouts, wire your own feature flag at runtime construction: ```typescript const runtime = featureFlags.harnessEnabled(req.user) ? new SessionRuntime({/* ... */}) : null; if (!runtime) { return legacyHandler(req); } ``` Roll out by tenant, by user cohort, or by traffic percentage — whatever your flag system supports. ## Health checks [#health-checks] The `/api/harness/health` route returns component-level diagnostics. Cheap; safe to hit from a load balancer: For container workers, point the readiness probe at the same route — the worker is unready until storage and provider checks return `ok`: ```yaml # Dockerfile CMD ["node", "dist/worker.js"] # k8s deployment.yaml (excerpt) readinessProbe: httpGet: { path: /api/harness/health, port: 3000 } initialDelaySeconds: 5 periodSeconds: 10 livenessProbe: httpGet: { path: /api/harness/health, port: 3000 } periodSeconds: 30 ``` ```json { "ok": true, "version": "1.1.0", "components": { "storage": { "ok": true }, "checkpointer": { "ok": true }, "providers": { "ok": true, "configured": ["anthropic"] } } } ``` A non-`ok` response indicates a misconfiguration — typically a missing env var, a database connection failure, or a provider key that doesn't validate. The component-level fields narrow the diagnosis: `components.storage.ok === false` means `SupabaseAdapter` can't reach Postgres (check `SUPABASE_URL`/`SUPABASE_SERVICE_ROLE_KEY` and the project's connection pooler); `components.providers.ok === false` with an empty `configured` array means no provider env var was visible at construction time (the substrate doesn't ship default credentials by design). ## Where to go next [#where-to-go-next]