# Gateway · Rate limiting (/docs/gateway/rate-limit)



The gateway's rate-limit substrate is the seam between a host's shared
quota store (Redis, Upstash, DynamoDB, Postgres) and the per-tenant /
per-user / per-route throttling decision. Today the package ships a
contract + one in-process reference adapter; production adapters
follow on additive subpaths.

## `RateLimiter` contract [#ratelimiter-contract]

```typescript
interface RateLimitCheckOptions {
  readonly windowMs?: number
  readonly maxRequests?: number
}

interface RateLimitDecision {
  readonly allowed: boolean
  readonly remaining: number
  readonly resetAt: number       // epoch ms; window reset
  readonly retryAfterMs?: number // populated only when !allowed
}

interface RateLimiter {
  check(key: string, opts?: RateLimitCheckOptions): Promise<RateLimitDecision>
}
```

The contract is intentionally minimal: given a key plus optional
per-call overrides, return a decision indicating whether the call is
allowed, the remaining quota in the current window, and the reset
time.

The decision shape mirrors what production adapters (Upstash, AWS
Throttle Tokens) emit. Callers render HTTP `429` responses with
accurate `Retry-After` + `X-RateLimit-*` headers without per-adapter
glue.

### Implementation requirements [#implementation-requirements]

* `check()` MUST be atomic — the count increment + comparison against
  the ceiling MUST happen as a single observable operation. The
  in-process reference adapter achieves this via the single-threaded
  JavaScript event loop; future Redis / Upstash adapters use Lua
  scripts or `INCR`+`EXPIRE` pipelines.
* `check()` MUST NOT throw under normal load — return a conservative
  `allowed: false` if the underlying substrate misbehaves and let the
  caller decide whether to fail open or closed.
* Distinct `key` values MUST be independent — a denied tenant does
  not affect a peer tenant's quota.

## In-process reference adapter [#in-process-reference-adapter]

```typescript
import { createInMemoryRateLimiter } from "@pleach/gateway/rateLimit"

const limiter = createInMemoryRateLimiter({
  defaultWindowMs: 60_000,
  defaultMaxRequests: 100,
})

const decision = await limiter.check(`tenant:${tenantId}`)
if (!decision.allowed) {
  return new Response("Too Many Requests", {
    status: 429,
    headers: {
      "Retry-After": String(Math.ceil((decision.retryAfterMs ?? 0) / 1000)),
      "X-RateLimit-Remaining": "0",
      "X-RateLimit-Reset": String(Math.ceil(decision.resetAt / 1000)),
    },
  })
}
```

### Configuration [#configuration]

```typescript
interface InMemoryRateLimiterConfig {
  readonly defaultWindowMs?: number      // default 60_000
  readonly defaultMaxRequests?: number   // default 60
}
```

Per-call overrides are passed to `check()`:

```typescript
// Tighter limit for an expensive route, same limiter instance.
await limiter.check(`tenant:${tenantId}:expensive-llm`, {
  windowMs: 60_000,
  maxRequests: 5,
})
```

### What it is [#what-it-is]

Fixed-window counter stored in an in-process `Map`. Simpler than a
sliding-window log and sufficient for local dev, single-node
deployments, and unit tests.

### What it is NOT [#what-it-is-not]

**Not distributed-safe.** Counters live in the calling process's
heap. Multiple Node.js instances behind a load balancer will each
track an independent count, so a request might pass the limiter on
instance A while instance B is at the ceiling. For multi-node
production deployments use a shared substrate — see the roadmap below.

**Memory bound is lazy.** Idle keys are reaped only when a `check()`
call observes that the stored window expired. A long tail of one-shot
keys can accumulate; hosts that route via short-lived per-user keys
should prefer a production adapter with explicit eviction policy.

## Roadmap [#roadmap]

Production adapters ship as additive subpaths so the base package
stays dependency-free. None of these are in the current `0.x` cut —
they are planned for `v1.x`:

* `@pleach/gateway/rateLimit/redis` — `ioredis` / `redis` v4+.
  Atomic `INCR`+`EXPIRE` pipeline; sliding-window via Lua script.
* `@pleach/gateway/rateLimit/upstash` — `@upstash/ratelimit`. HTTP
  REST API, fits Workers / Edge runtime.
* `@pleach/gateway/rateLimit/dynamodb` — AWS SDK v3. Conditional
  writes + TTL-based window reset.

Per-tenant-call-class policy (different windows for `utility` vs
`reasoning` vs `synthesize` calls inside one tenant) is also a `v1.x`
item. Today, hosts that need this layer pass distinct keys at the
call site:

```typescript
// Workaround pre-v1.x: encode the call-class in the key.
await limiter.check(`tenant:${tenantId}:callClass:${callClass}`, {
  maxRequests: callClass === "synthesize" ? 10 : 100,
})
```

## Cited source [#cited-source]

* `packages/gateway/src/rateLimit/types.ts` — `RateLimiter` contract.
* `packages/gateway/src/rateLimit/inMemoryLimiter.ts` — fixed-window
  in-process adapter.

## Where to go next [#where-to-go-next]

<Cards>
  <Card title="Cost events" href="/docs/gateway/cost-events" description="Per-call cost emission — pairs with rate limiting on the same per-tenant key dimension." />

  <Card title="BYOK credential routing" href="/docs/gateway/byok" description="Per-tenant key resolution. Rate-limit keys typically share the same tenant scope as BYOK credentials." />

  <Card title="`@pleach/gateway` overview" href="/docs/gateway" description="The Phase A GatewayClient — route(), cost events, family-strict cascade, transport seam." />
</Cards>