Swarm agent
An agent that spawns subagents at scale — deep sequential loops, parallel fan-out, supervisor/worker topologies, recursive delegation — with cost ceilings and a session-tree audit walk.
A swarm agent is the shape that pushes the runtime's recursive contract: one user turn fans out into dozens or hundreds of subagents, each of which may spawn its own. Kimi K2-style deep sequential loops, OpenAI Deep Research-style parallel fan-out, AutoGen / LangGraph supervisor topologies, Claude Code-style recursive delegation — different topologies, same buyer cohort, same runtime stress.
The load-bearing pain is the runaway-loop case. A buggy fan-out or a stuck sequential loop can spend a five-figure budget in one user turn. This page walks the cost ceiling at the root-turn boundary, the session-tree audit walk that attributes every descendant call back to the originating turn, and the concurrency primitives a swarm host reaches for.
Related shapes. Research agent for the canonical single-anchor / bounded-subagent pattern — the swarm shape inherits its primitives and pushes them further. Coding agent if the subagents run code in sandboxes. Multi-tenant SaaS agent if one runtime serves many customers, each driving their own swarm.
What you're building
A turn loop whose recursion is bounded by construction, not by hope:
- Every subagent — including subagents-of-subagents — carries
the originating user turn's
turnIdand its ownsubagentDepth. The whole tree is one query against the ledger. - A per-root-turn cost ceiling aborts the turn when cumulative spend crosses the line. The ledger records the cutoff.
- A fan-out cap limits how many subagents one node can spawn at once. A recursion-depth cap limits how deep the tree can go.
- Concurrency primitives —
race,waitAll,waitN,bounded,backpressure— handle the structural cases a swarm host actually hits.
Cost ceiling at the root-turn boundary
The cost ceiling is the load-bearing protection for a swarm.
Today @pleach/core ships the structural caps — maxDepth,
maxConcurrent, maxPerTurn, timeoutMs — as hard ceilings
the substrate enforces independently of consumer config. A
per-root-turn USD ceiling now ships in @pleach/gateway/swarm as
an inline pre-transport guard — see Gateway swarm
governance below. Hosts that don't route
through the gateway enforce the dollar cap out-of-band against the
audit ledger.
import {
SessionRuntime,
SUBAGENT_LIMITS,
AiSdkProvider,
createSupabaseAdapter,
} from "@pleach/core";
const runtime = new SessionRuntime({
storage: createSupabaseAdapter({ client: supabase }),
userId: "user_123",
tenantId: "tenant_abc",
enableSubagentConcurrency: true,
// SUBAGENT_LIMITS.maxConcurrent is the substrate-enforced
// ceiling (3); consumer values above the ceiling clamp down.
maxConcurrentSubagents: SUBAGENT_LIMITS.maxConcurrent,
orchestratorConfig: {
provider: anchorProvider,
},
});
// Out-of-band USD ceiling, polled by the host against the
// audit ledger. `@pleach/compliance/education` ships a
// per-session helper for the single-process case; for swarms
// that span multiple processes, back it with a tenant-scoped
// store and check on each turn boundary.
import { createCostCapPolicy } from "@pleach/compliance/education";
const costCap = createCostCapPolicy({
ceilingUsd: 25,
onBreach: () => runtime.abort(),
});SUBAGENT_LIMITS carries the substrate-side hard caps:
| Cap | Value | Effect when breached |
|---|---|---|
maxDepth | 3 | Spawn rejected — fourth nested runtime never starts |
maxConcurrent | 3 | New spawns queue until a slot frees |
maxPerTurn | 5 | Sixth spawn in a turn is rejected |
timeoutMs | 120_000 | Subagent emits subagent.failed with terminalStatus: "timeout" |
The recursion is bounded structurally, not by per-subagent prompts. A subagent that doesn't know about the cap can't escape it.
Gateway swarm governance
@pleach/gateway/swarm ships the inline cost governance the
out-of-band poll above approximates. The middleware guards each
routing call and throws before transport when the root turn would
cross its ceiling:
createRootTurnCostAggregator(...)— accumulatesCostEvents byrootTurnId. In-memory by default; wire a sharedRootTurnCostStore(Redis / Upstash / DynamoDB) so sub-agents on different processes accumulate to one counter.createCostCeilingMiddleware({ aggregator, maxCostUsdPerRootTurn })— wraps a routing call viaguard({ rootTurnId, invoke }). A call that would push accumulated cost over the ceiling throwsCostCeilingExceededErrorbefore the transport runs — the breaching call never touches the wire. OptionalwarnCostUsdPerRootTurn/warnThresholdfireonWarnat a soft threshold.createCostRateLimitMiddleware({ windowMs?, maxRequests? })— a per-rootTurnIdrequest throttle.process()returns a{ retryAfterMs }deferral once the window cap is met and firesonDeferat most once per window.createFailoverCoordinator(...)— coordinates sibling sub-agents on a failover, withmode: "best-of-N" | "all-or-nothing".classifySwarmTopology(events)— classifies a flat spawn-event list intotree | dag | ring, plusdepth,fanOut, andsubAgentCount.
Two structural events ride the callbacks for operator telemetry:
CostBudgetWarned (soft threshold) and CostRateLimitDeferred
(transient throttle). Both are data-shaped for the event log and OTEL
attributes.
import {
createRootTurnCostAggregator,
createCostCeilingMiddleware,
CostCeilingExceededError,
} from "@pleach/gateway/swarm";
const aggregator = createRootTurnCostAggregator();
const ceiling = createCostCeilingMiddleware({
aggregator,
maxCostUsdPerRootTurn: 25,
warnThreshold: 0.8,
onWarn: (e) => notifyOperator(e),
});
try {
const answer = await ceiling.guard({
rootTurnId,
// invoke returns the call result plus its CostEvent so the
// aggregator can roll spend up by rootTurnId.
invoke: async () => {
const { result, costEvent } = await runRoutedCall(decision);
return { result, costEvent };
},
});
} catch (err) {
if (err instanceof CostCeilingExceededError) {
// root turn halted before transport — err carries accumulated + ceiling
}
}What today ships
Today, @pleach/core ships:
- Recursive
SessionRuntimesessions — every subagent is itself aSessionRuntime, with the same primitives the root has. AuditableCalltyped audit-ledger rows — every spawn writes a row that ties the child session back to the parent turn.parent_turn_idandsubagent_depthcolumns on the audit ledger — the tree walk is one recursive CTE.SUBAGENT_LIMITSfor depth, parallel fanout, and total subagent count.
For the canonical anchor → bounded subagent fan-out pattern, see Research agent.
Roadmap
The deeper swarm surface lands incrementally:
- Per-root-turn cost ceiling. Shipped in
@pleach/gateway/swarmas an inline pre-transport guard — see Gateway swarm governance. A first-classmaxCostUsdPerTurnlimit inside@pleach/core— enforced without routing through the gateway — is still on the roadmap. Until then, gateway-less hosts pollharness_auditable_callsfiltered byparent_turn_idand callruntime.abort()when spend crosses the budget;@pleach/compliance/educationships acreateCostCapPolicy({ ceilingUsd, onBreach })helper for the per-session shape (single-process / in-memory by default; swap in a tenant-DB backing store for multi-process). - Concurrency primitives. A
@pleach/core/spawnsubpath withrace,waitAll,waitN,bounded, andbackpressureprimitives that absorb the topology a swarm host already wrote in user space. Until it lands, hosts compose the same shapes withPromise.all,Promise.race, and a host-side semaphore. - Session-tree audit walker. Shipped as
walkSessionTreein@pleach/compliance/swarm— it walks the tree the CTE above describes and verifies each node's hash chain. See Verifying the tree withwalkSessionTree. - Deterministic spawn-order replay. A
spawnIndexonSpawnTreeStatethat normalizes parallel-spawn ordering for byte-identical replay across the tree. - Per-subagent cost ceilings. A nested cap that lets one branch of the tree have its own budget without blocking the rest. Lands after the root-turn ceiling is in soak.
No dates. Track the upstream package READMEs and CHANGELOG for landing notices.
The session-tree audit walk
Every subagent's audit rows carry parent_turn_id and
subagent_depth. The whole tree — sequential loops, parallel
fan-out, recursive delegation — reads as one recursive CTE.
with recursive tree as (
select turn_id, parent_turn_id, subagent_depth,
tool_name, model_id, input_tokens, output_tokens
from harness_auditable_calls
where turn_id = $1
union all
select c.turn_id, c.parent_turn_id, c.subagent_depth,
c.tool_name, c.model_id, c.input_tokens, c.output_tokens
from harness_auditable_calls c
join tree t on c.parent_turn_id = t.turn_id
)
select
subagent_depth,
count(*) as calls,
sum(input_tokens + output_tokens) as tokens
from tree
group by subagent_depth
order by subagent_depth;The query reads the whole investigation top-down: depth 0 is the anchor turn; depth N is the deepest spawn. A regulator asking "what did the agent do for user turn T" gets the full tree in one read.
Verifying the tree with walkSessionTree
The SQL above rolls the tree up for review. @pleach/compliance/swarm
walks the same tree and verifies each node's hash chain in one pass:
walkSessionTree({ rootSessionId, eventStore, maxDepth? })— BFS over the spawn tree from the root session.maxDepthdefaults to 64. Returns aTreeWalkResult.EventStoreLike— the duck-typed store the walker reads through. One method:listEvents(sessionId, filter?), returning rows insequence_numberascending order.TreeWalkResult—rootSessionId,treeValid(the conjunction of every node'schainValid),nodes,totalRows,totalFailures.TreeWalkNode— per session:sessionId,depth(0 at root),chainValid,rowCount, andfailures(typedChainVerificationErrors).
A per-node chain divergence surfaces via nodes[i].failures and
treeValid: false — the walker only throws on a store-level error. A
malformed tree that links back to an ancestor can't loop; visited
session IDs are tracked in a set.
import { walkSessionTree } from "@pleach/compliance/swarm";
const result = await walkSessionTree({ rootSessionId, eventStore });
if (!result.treeValid) {
// result.nodes[i].failures carries the per-node ChainVerificationErrors
}Topology-neutral primitives
The runtime doesn't have a topology opinion. The same primitives — recursive sessions, the ledger tree, the cost ceiling, the spawn record — absorb four different shapes:
| Topology | What the host writes | What the runtime stamps |
|---|---|---|
| Deep sequential loop | a long tool-loop in one subagent | subagent_depth = 1, many rows |
| Parallel fan-out | N subagents spawned at the anchor | subagent_depth = 1, N siblings |
| Supervisor / worker | supervisor spawns workers per task | subagent_depth = 1 or 2, tree |
| Recursive delegation | subagent spawns its own subagent | subagent_depth = N, deep tree |
A swarm host picks the topology; the runtime makes the audit walk identical across all four.
Where to go next
Subagents
The spawn surface, the SpawnTreeState shape, and SUBAGENT_LIMITS.
Research agent
The canonical anchor → bounded subagent fan-out pattern that the swarm shape extends.
Audit ledger
The harness_auditable_calls schema and the parent_turn_id column the tree walk reads from.
Channels
Per-subagent stream isolation so the UI can render the tree without rebuilding it.
Coding agent
The swarm shape when the subagents run code in sandboxes.
Multi-tenant SaaS agent
Per-tenant runtime construction when each tenant drives its own swarm.
Regulated-domain agent
An agent built for HIPAA, SOC 2, PCI, FedRAMP, or 21 CFR Part 11. Constrained routing, PII gate, append-only ledger keyed for the regulator's query.
License compatibility for procurement
FSL-1.1-Apache-2.0 across all 13 publishable SKUs, with a written future-license clause to Apache-2.0 at the 2-year mark.