Audit gates
The CI-time invariant checks that protect the `@pleach/*` substrate — package shape, plugin contracts, tool coverage, event-log integrity, multi-tenant scoping, and graph lattice.
Audit gates are structural checks that protect substrate
invariants — a missing barrel re-export, a tool name no
definition emits, an event type with no scrubber allowlist, a
published dist/ artifact missing a .d.ts. Most gates are
pure source-text scans (.mjs) that run in under a second.
External consumers (plugin authors, sibling SKU authors, hosts
embedding @pleach/core) care about a subset: the ones that
protect the published surface they consume. This page covers
that subset. The full internal gate set — coordinator-process
gates, refactoring scaffolds, soak ledgers gating in-flight
substrate moves — lives in
scripts/audit/
on the source repo. Source there is canonical.
This page is organized by subsystem. To read the same gates by the contract they enforce — which clause of the auditability and replayability promise each one holds up — see CI enforcement of the audit contract.
Run any gate via npm:
npm run audit:<name>Suffix conventions are stable across gates. :strict is the CI
mode (fails on novel drift). :json emits machine-readable
output. :list dumps the inventory. :update-baseline
regenerates the allowlist after intentional drift.
Package shape (gates every consumer should know)
These run before any first-publish and before any 1.x release
tag. They catch silent failure modes invisible to in-monorepo
builds — workspace path aliases that resolve cross-tree locally
but fail in a real npm install, tsup DTS emission that exits 0
but produces no .d.ts, peer-deps not declared in dependencies.
If you're authoring a sibling @pleach/* SKU OR a third-party
plugin that ships its own npm package, these are the gates to
mirror in your own CI.
| Gate | Purpose |
|---|---|
audit:package-export-validation | Every active @pleach/core/* import resolves through package.json:exports. |
audit:package-runtime-deps | Every third-party import in dist/**/*.{js,cjs,mjs} is declared in dependencies, peerDependencies, or optionalDependencies. Catches the silently-undeclared-runtime-peer class. |
audit:package-dist-runtime-aliases | Workspace-alias leak detector. Scans dist/** for @/lib/...-style imports that resolve only inside the monorepo. |
audit:package-dts-emission | Every declared .d.ts path in types, exports[K].types, typesVersions[*][K] exists on disk and is non-empty. Closes the silent-tsup-DTS-failure class. |
audit:package-version-vs-registry | Compares in-tree package.json:version to npm view <pkg> versions --json. Fails on past-publish drift. |
audit:harness-package | @pleach/core substrate is domain-agnostic. Scans for brand, backend-class, scientific-term, product-surface tokens that would couple the substrate to a specific host. |
audit:harness-package:local-clone | Packs @pleach/core via npm pack, installs in an isolated dir, smoke-imports SessionRuntime + createPleachRuntime. Catches resolution failures the in-monorepo scan misses. |
audit:consumer-rehearsal:core | Six-phase end-to-end publish rehearsal. Builds, packs, installs into scratch dir, smoke-tests facets + subpaths + TypeScript consumer surface + examples. |
Plugin contract (gates plugin authors should know)
The plugin substrate is a strict contract between
@pleach/core/plugins and host code. These gates enforce
completeness on both sides — every contributeX hook has a
paired collector accessor on PluginManager, every host
extension stays out of domain-coupling-suspect deep paths.
If you're writing a HarnessPlugin, mirror these in your
plugin's repo.
| Gate | Purpose |
|---|---|
audit:plugin-contract-completeness | For every contributeX hook on HarnessPlugin, verifies a paired collect<X> accessor exists and ≥1 consumer reads from it. Closes the dead-hook drift class. |
audit:harness-plugin-deprecated-usage | No novel usage of @deprecated HarnessPlugin hooks beyond baseline. Hard-deprecated set retires in 1.1.0; soft-deprecated retires in 1.2.0. |
audit:plugin-contract-purity | Host code MUST NOT import from domain-coupling-suspect @pleach/core/<deep-path> surfaces. Inverse of lint:harness-boundary. |
audit:stream-observer-factory-completeness | Every observer factory under observers/*Observer.ts is registered by ≥1 plugin's contributeStreamObservers, and every registration points at an existing factory. |
audit:plugin-hook-category-assigned | Every contributeX hook on HarnessPlugin resolves to exactly one of the nine contribution namespaces; a new hook with no namespace fails the build. |
Tool coverage (gates tool authors should know)
A "reverse-invariant" audit verifies the VALUES (or KEYS) in a runtime lookup table all resolve to canonical tool names. Forward direction (every tool has an intent) is covered by the tool-generation pipeline; these audits close the reverse — every NAMED tool in a map is a REAL tool.
| Gate | Purpose |
|---|---|
audit:intent-tool-map-completeness | Every tool named in INTENT_TOOL_MAP resolves. Drift surfaces as silently-shadowed tools in the LLM-facing condensed catalog. |
audit:tool-alternatives-completeness | Every fallback tool: field in TOOL_ALTERNATIVES resolves. Drift surfaces as silently-broken error-recovery. |
audit:post-tool-phrases-completeness | Every key in POST_TOOL_PHRASES resolves. Drift surfaces as silently-dropped curated conversational nudges. |
audit:decision-hints-completeness | Every tool named in DECISION_HINTS resolves. |
Event log + hash chain
The event log is the canonical record of every session turn. These gates enforce the redaction surface, the hash chain, and the producer/projection contracts.
| Gate | Purpose |
|---|---|
audit:c8-event-type-allowlist-coverage | Every event-type literal in EventLogInput has a matching SCRUBBABLE_FIELDS allowlist entry. Empty array [] is a valid "audit ran; no fields" signal. |
audit:c8-union-member-has-producer | Every EventLogInput union member has at least one producer call site. Reverse-invariant complement to the allowlist audit. |
audit:c9-hash-chain-integrity | Source-text regression on the C9 hash-chain wire-in (verifyChainForChat, generateProof, computeGenesisSeed, etc). |
audit:event-log-manifest-hash-completeness | EventLogWriter.eventToRow stamps manifest_hash on every row. |
Config manifest
The config manifest records which substrate ran each session, content-addressed and referenced by every event-log row. These gates protect its hash determinism and its referential integrity against the event log.
| Gate | Purpose |
|---|---|
audit:plugin-content-hash-stability | Two consecutive manifest builds produce identical per-surface child hashes; a Date.now() leak in any surface builder fails the specific surface, not "something moved." |
audit:config-manifest-fk-constraint-applied | The FK migration carries every canonical clause — REFERENCES, ON DELETE SET NULL, DEFERRABLE INITIALLY DEFERRED. |
audit:config-manifest-referential-integrity | Offline: SessionRuntime wiring reaches the manifest write. Online: zero event rows reference a missing manifest. |
audit:config-manifest-retention-completeness | After a retention GC pass, no surviving event references a dropped manifest — the guard against a GC racing a concurrent event write. |
Multi-tenant
These gates close tenant_id propagation end-to-end — every
tenant-scoped table has RLS, every event-log write stamps the
tenant.
| Gate | Purpose |
|---|---|
audit:tenant-scoping | Every tenant_id-bearing table in supabase/migrations/*.sql has an RLS policy referencing current_tenant(). |
audit:harness-event-log-tenant-id-required | Every harness_event_log write site references tenant_id (or carries a documented opt-out marker). |
Graph + lattice
The compiled agent graph obeys the four-stage lattice
(anchor-plan → tool-loop → synthesize → post-turn, one-way,
with a messageId-guarded synthesize → synthesize retry). These
gates enforce that invariant, the recovery-dispatch surface, and
the AuditableCall row shape.
| Gate | Purpose |
|---|---|
audit:graph-stages | Parses defaultAgentGraph.ts + wiring/ and fails on out-of-lattice edges. Asserts the 44-name NODE_STAGE_MAP and nine-pattern ALLOWED_EDGE_PATTERNS table; node/edge counts are byte-identical PR-to-PR for the same config. |
audit:graph-stages:warn | Warn-only mode. Auto-runs inside prebuild to surface drift without blocking builds. |
audit:edge-inventory-completeness | Every compiled cross-stage edge resolves to a documented entry in the 61-edge inventory; drift fails. |
audit:recovery-dispatch-single-surface | Recovery dispatch lives in exactly one place — the post-turn stream filters. Fails any build that re-introduces a graph-side recovery node or conditional edge. |
audit:graph-stream-filter-substrate-coverage | Every lattice stage has the filter slots the post-stage filters dispatch against. |
audit:graph-barrel-stability | The graph barrel's public-vs-internal export split doesn't drift — public SDK-contract exports stay public, internal-only exports stay internal. |
audit:auditable-call | Composite — runs audit:auditable-call-shape + audit:auditable-call-version to assert the row shape and that AuditRecordVersion hasn't drifted. |
audit:agentic-flag-respected | Every recursive this.streamChat(...) site in turnOrchestrator.ts carries the agentic:false marker + guard, so a new recursive call site can't silently re-enter the agentic loop when single-LLM-call mode (ChatOptions.agentic === false) is requested. A runtime gate that rides the ci:graphnoderef bundle — node stageId is enforced by audit:graph-stages above, not here. |
API protection (host-side)
| Gate | Purpose |
|---|---|
audit:api-protection | Every API route is wrapped with an auth middleware. (Host-side gate; reuse the pattern in your own pages/api/ tree.) |
Domain purity + frontmatter
Substrate hygiene at PR time.
| Gate | Purpose |
|---|---|
audit:domain-string-purity | Complementary to audit:harness-package. Catches host-domain string literals (tool names, brand strings) leaking into the substrate. |
audit:frontmatter | Non-generated .ts/.tsx files have canonical YAML frontmatter (module + role + related_files). :fix auto-normalizes role drift. |
Bundled commands
Composite scripts that chain related gates.
| Command | Runs |
|---|---|
npm run ci:graphnoderef | lint:graphnoderef (boundary + callclass + observer-registration) + audit:graph-stages + audit:agentic-flag-respected + graph tests + audit:auditable-call |
npm run ci:plangen-v2 | Plan-generation v2 ladder tests |
npm run audit:auditable-call | Shape + version checks (see Graph + lattice above) |
Internal coordinator-process gates
The substrate carries additional gates for in-flight refactoring ladders, soak ledgers, slot reservations, and parallel-session coordination. They protect the engineering process, not the shipped surface. External consumers will not run them; mention of them in public docs would be misleading.
If you're operating the substrate (committer access to the
upstream repo, authoring a design pack, running a multi-agent
refactoring fleet), see
scripts/audit/
directly. The script source files carry the canonical purpose +
mode documentation per gate.
Calendar-substitution pattern
Worth knowing as a general technique. When a "let it bake for
N days" soak window has a deterministic failure mode the script
can catch directly, the calendar collapses into a script gate
that runs at PR time. Replace ≥30d of zero fires on probe X
with last 3 canvas batches at probe X count == 0. The script
gate then becomes the permanent regression-detection surface.
The pattern has applied across 8+ internal initiatives. When you see a calendar wall in a design doc, default-search for "what would the script gate look like?" before accepting the wall.
Mirroring gates in your own repo
The upstream gates protect the substrate. Hosts that build on
top of @pleach/* often want the same discipline for their
load-bearing invariants — a per-tenant key never appears in
logs, every domain tool has a matching detector, a feature flag
hasn't been left on past its sunset date.
The pattern is the same:
- Write a script that walks the relevant source and asserts the invariant. Exit non-zero with a structured error message that names the offending file and line.
- Wire it into
package.jsonunder anaudit:<name>script and into the PR-blocking CI job. - Document the gate alongside its sibling gates in your repo's equivalent of this page — the audit only earns its keep when the next contributor knows it exists.
The audit:plugin-contract-completeness script in the upstream
core repo is a small reference shape: it walks the
HarnessPlugin interface, asserts every contribute* hook has
both a typed schema and a wire-check entry, and fails with the
hook name when one is missing. Host-side audits that follow that
shape — one invariant per script, structured failure output,
documented in source-tree-adjacent prose — compose cleanly with
the substrate's existing gate set.
The audit gate is the regression-detection surface; the discipline is that adding a new invariant means adding a new gate, not adding it to a code-review checklist.
Where to go next
CI enforcement of the audit contract
The same gates, indexed by the auditability and replayability clause each one enforces.
Publishing contract
First-publish packaging baseline every @pleach/* SKU shares.
Plugin contract
HarnessPlugin hooks, collectors, and consumer wiring.
Host adapter
Plug a host runtime into @pleach/core via the dep-inversion contract.
Event log
The append-only typed StreamEvent log + its hash chain + projection contract.
Subpath exports
Every named import path under @pleach/core, what ships from it, and whether it's load-bearing.
CI enforcement of the audit contract
The named audit gates that make the auditability and replayability contract load-bearing in CI — what each clause promises, the gate that fails the build when it breaks, and where the gates run.