# Redis Redis holds every short-lived value: authorization codes, access and refresh tokens, grant chains, sessions, PAR request URIs, device codes, DCR initial-access tokens, replay markers, and rate-limit counters. No raw token is ever a Redis key or value; token keys are keyed hashes and values hold only derived state. The JWT marker and CIMD cache use an unkeyed SHA-256 over a random `jti` or a public document URL respectively, neither of which is secret. ## Purpose This guide covers the key namespaces and their TTLs, the hash-tag scheme that keeps atomic operations on one Cluster slot, the tombstone and durability model, and the three supported topologies. ## Key namespaces The token prefix defaults to `oauth:{state}:` and the rate-limit prefix to `oauth:{rl}:`. Both are configurable but must keep their hash tags and must differ. | Key | Contents | TTL | | --- | --- | --- | | `oauth:{state}:code:` | Single-use authorization code, PKCE-bound. | `ttl.code` (60s), clamped to 60 s (`CODE_TTL_CAP`) | | `oauth:{state}:access:` | Access-token record. | `ttl.access_secs` (default 300s) | | `oauth:{state}:access::act` | Token-exchange `act` claim sidecar. | Lives with the access record | | `oauth:{state}:refresh:` | Refresh-token record or tombstone. | `ttl.refresh` (30d) | | `oauth:{state}:grant:` | Grant-chain record. | Refresh lifetime | | `oauth:{state}:grant::tokens` | Set of member token hashes for cascade revoke. | Lives with the grant | | `oauth:{state}:session:` | Server-side login session. | `ttl.session` (12h) | | `oauth:{state}:session::meta` | Session sidecar: `auth_time`, CSRF secret, bound `request_id`, and the bound post-logout client/URI. | `ttl.session` | | `oauth:{state}:par:` | PAR `request_uri` record. | `ttl.par` (90s) | | `oauth:{state}:pending:` | Validated pending-auth record. | `routes::PENDING_TTL` (300s) | | `oauth:{state}:device:` | Device-code record. | `ttl.device` (600s) | | `oauth:{state}:device::cnf` | Device sender-binding sidecar. | Lives with the device record | | `oauth:{state}:device::poll` | Device poll-throttle marker. | Lives with the device record | | `oauth:{state}:user_code:` | User code to device-hash mapping. | `ttl.device` | | `oauth:{state}:dcr_iat:` | DCR initial-access token budget. | CLI `--ttl` | | `oauth:{state}:dpop_jti:` | DPoP `jti` replay marker. | 300 s (`DPOP_JTI_TTL`; fixed, independent of proof `iat`) | | `oauth:{state}:client_assertion_jti:` | `private_key_jwt` `jti` replay marker. | Remaining assertion validity (`exp + 30s skew - now`), clamped `[1s, 360s]` (300s max lifetime + 2×30s skew) | | `oauth:{state}:rotation:` | Rotation idempotency mapping. | `min(900s, refresh TTL)` | | `oauth:{state}:loginfail:` | Failed-login counter (hashed label; the global per-account cap). | `ui::login::LOCKOUT_WINDOW` (15 min) | | `oauth:{state}:loginfail:ip::` | Composite failed-login counter for one source IP and one account (both halves hashed; locks that IP only). | `ui::login::LOCKOUT_WINDOW` (15 min) | | `oauth:{state}:loginfail:unknown:` | Unknown-label bucket for one source IP (hashed IP only; no per-label keys, so label enumeration cannot spray the keyspace). | `ui::login::LOCKOUT_WINDOW` (15 min) | | `oauth:{state}:totp_replay:` | TOTP used-code replay marker. The fingerprint is an HMAC under the token pepper (never the code itself), so a dump does not reveal the code. | 180 s (`USED_CODE_TTL`; fixed, 6× the 30 s step) | | `oauth:{state}:jwt:` | JWT access-token kill marker (allowlist: a signed token is valid only while this hash is live). Carries `grant_id`, `client_id`, and the raw `subject` (email/user-id; empty for client-credentials), matching the `purge_subject` `subject` decoder. | `ttl.access_secs` (default 300s) | | `oauth:{state}:cimd:` | Cached client-id-metadata document (keyed by a hash of the URL, never the raw URL). | 15 minutes | | `oauth:{state}:cimd:neg:` | Negative cache for an origin-level CIMD fetch failure (network/TLS/non-2xx, timeout, non-JSON). | 30 seconds | | `oauth:{state}:cimd:invalid:` | Short completion marker for a fresh-fetch CIMD validation failure, so waiters do not re-fetch a deterministically invalid document. | 2 seconds | | `oauth:{state}:cimd:lock:` | Single-flight fetcher election for one uncached CIMD URL; a random token is released with a compare-and-delete Lua script. | `cimd_fetch_timeout_secs` + 1 s (2–11 s) | | `oauth:{rl}:ratelimit:` | Fixed-window rate-limit counter. | `rate_limit.window` (60s) | `` is a keyed hash for token lookups: tokens are opaque, so the server can look a record up by re-hashing the presented value but cannot recover the raw value from storage. The `` of a JWT marker and the CIMD `` are unkeyed because their inputs are already a random token id and a public URL. Resource audiences (`aud`, RFC 8707) ride on seven record families as a field named `aud`: `code:`, `access:`, `refresh:`, `grant:`, `par:`, `device:`, and `pending:` (the last is a JSON blob, not a hash). The value is a JSON array of strings, so a URI that itself contains a comma can never collide with a delimiter join. In the hash-backed records it is stored as a JSON string field, so the rotation Lua script's `cjson.decode` sees a string rather than a Lua table. A record written by a replica that predates audiences carries no `aud` field and decodes back to an empty audience, so in-flight records survive a rolling deploy; a present-but-malformed field fails closed as `InvalidData`. The audience is `{state}`-keyspace metadata only. Resource indicators add no key family and no `{rl}` counter; the durable catalog of allowed identifiers lives in Postgres, not Redis. JWT access tokens use an allowlist, not a denylist. Each mint writes a `jwt:` marker (`grant_id`, `client_id`, `subject`, TTL `ttl.access_secs`) in the same atomic script as the grant membership, and a token is live only while that hash exists. Because the marker rides the `{state}` tag, it is single-slot with the access record and the grant set. Three paths delete it: - A grant cascade deletes `jwt:` markers through the third ARGV of the revoke script, together with the member access records and the grant set. - `user disable` sweeps `jwt:` keys by their `subject` field. Grant-bound markers are also covered by the cascade; markers for client-credentials tokens (empty subject) and any marker left outside a live grant rely on this sweep. - `POST /revoke` with a JWT-shaped token deletes the marker, the `access:jwt:` record, and the grant membership after verifying the signature, `iss`, and ownership, then runs a durability barrier. A resource server that verifies locally against `/jwks.json` never reads this keyspace, so it cannot observe a deletion. See [OAUTH.md](OAUTH.md#the-resource-server-trade-off) for that exp-bounded trade-off. ## Hash tags and single-slot atomicity Every token-state key carries the `{state}` hash tag, so one Lua operation that touches several keys lands on a single Cluster slot. Rate-limit counters carry `{rl}`, which puts them in a different slot group and a different shard from token state. This is what keeps two promises: - A multi-key token operation never fails with `CROSSSLOT`. - Counters can never evict tokens, because the two keyspaces are separate and Redis eviction is off by default (`noeviction`, required unless `redis.noeviction_required` is set false). The server asserts both properties at startup and refuses a prefix that lacks its tag. ## Record writes Every hash record is written through one atomic Lua script (`hset_record` → `HSET_TTL`): the fields and the `PEXPIRE` happen in a single script, so no record can ever be observed — or survive a crash — without its TTL. Replay markers are `SET NX PX` plus a durability barrier. ## Rotation, tombstones, and idempotency Refresh rotation is a Lua compare-and-set keyed by a deterministic `rotation_request_id`, derived as an HMAC of the pepper over the old refresh token hash. The script returns one of: - `ROTATED`: the refresh was valid and new tokens were written. - `REPLAY`: the same request id already completed, so the same minted tokens are returned. A lost reply does not cause a double rotation. - `REUSE`: the presented token was already rotated. Its tombstone records the member it rotated to. Reuse also cascades revocation of the whole grant family. - `ABSENT`: unknown token. - `MISMATCH`: issued to a different client. - `NOGRANT`: the grant record is gone (a mint racing a cascade is fenced before any write, so nothing is stored and no cascade runs). The old refresh record is not deleted. It becomes a tombstone with a `used` flag and a `rotated_to` member, kept for the refresh lifetime so reuse stays detectable. The new tokens are minted before the script runs, so a crash during the write cannot mint a token that was never stored. Grant cascade `revoke` drains in a bounded loop: one script pass is atomic but blind to a mint racing the cascade, so the pass repeats until the member set reads empty and a token minted mid-cascade is deleted by a later pass (or refused outright by the rotation `NOGRANT` fence) instead of surviving. ## The durability barrier A rotation, tombstone, or replay-marker write is followed by a durability barrier issued as a separate command after the write returns and before any token is returned to the client. It is never run inside a Lua script or `MULTI`/`EXEC`, where `WAIT` and `WAITAOF` do not block. | Mode | Topology | Barrier | | --- | --- | --- | | `wait` | Sentinel or Cluster | `WAIT 1 `; the write must be acknowledged by at least one replica. | | `waitaof` | Single node | `WAITAOF 1 0 `; the local AOF must be flushed. Falls back to the `appendfsync always` assertion on servers older than Redis 7.2. | | `appendfsync_always` | Single node | Assert `CONFIG GET appendfsync` is `always`. This is instance-wide, not per-write-path. | A barrier below the required level fails closed with `503`, mapped from `StoreError::Durability`. Exception: a refresh rotation whose originating write already applied returns the minted pair as `RotatedUnbarriered` instead of `503` (any other caller, including a `Replay` hit, stays fail-closed). The server-side wait leaves 500 ms of headroom under the client response timeout so an expired barrier surfaces as a clean failure rather than an ambiguous timeout. The start of a claim: with the barrier active, a failover cannot resurrect a rotated token or lose a replay marker. Any residual window only exists where neither barrier nor `appendfsync always` is active, which the configuration refuses for a single node. ## Topologies Exactly one topology is selected at load time. ### Single One node from `redis.url`, using the driver's `ConnectionManager`. Durability must be `waitaof` or `appendfsync_always`; `wait` is rejected because there are no replicas. ### Sentinel A set of `host:port` nodes plus `master_name`. The driver's built-in Sentinel support does not reliably follow a promoted master after a failover, so the server hand-builds master re-resolution: when the current master fails, it asks the Sentinels for the new master and reconnects. Durability is `wait`. ### Cluster A set of seed nodes. The server uses native slot refresh, so `MOVED` and `ASK` redirects recover without a restart. Multi-key token operations stay single-slot through the `{state}` tag. Durability is `wait`. ## Error classification and retries Errors are classified as `Transient` or `Permanent`. - Transient: I/O errors, timeouts, dropped connections, and `LOADING`, `TRYAGAIN`, `MASTERDOWN`, `CLUSTERDOWN`, `MOVED`, `ASK`, `NOSCRIPT`. - Permanent: `OOM` under noeviction, `READONLY`, auth failures, `CROSSSLOT`, type errors, and anything unrecognized. Only idempotent reads and `SET NX` or idempotent Lua compare-and-set are retried, and only on transient errors, under a bounded budget with jittered backoff. Non-idempotent operations such as single-use consumes and `INCR` are never retried. On an indeterminate post-`EXEC` result the caller re-reads the rotation mapping or the record instead of replaying the write. ## Security properties - Token keys are keyed hashes; the JWT marker and CIMD cache use an unkeyed SHA-256 over a random `jti` or a public URL. A Redis dump does not reveal a usable token. - `noeviction` and AOF are required by default and asserted at startup unless the matching opt-out flag (`redis.noeviction_required` or `redis.aof_required`) is set false. - Token state and rate-limit counters are separated, so a counter cannot evict a token. - Replay markers are written atomically with `SET NX`, so a replayed `jti` is rejected. - A failed durability barrier fails the request closed rather than returning a weakly durable token. ## Edge cases - A Redis flush loses every ephemeral value but not clients, users, scopes, or redirect URIs. Clients registered through DCR can still be managed, because the registration access token hash is in Postgres. - A rotation after a failover does not double-rotate: the mapping key resolves the retry to the original minted tokens. - A `CROSSSLOT` error fails closed. It is never worked around by splitting an atomic operation across slots. ## Subject purge (`user disable`) `d-oauth-server user disable --email ` revokes the user's ephemeral state before the Postgres `disabled` flag flips (fail closed: a Redis outage leaves the user enabled). The purge deletes every record whose `subject` field is the canonical email or the user-id string: - `session:` records plus their `:meta` sidecars. - The `loginfail:` account counter plus every `loginfail:ip:*:` composite bucket for the normalized email. The `loginfail:unknown:` buckets carry no account label and are never touched. - `code:` authorization-code records. - `device:` records plus their `:cnf`/`:poll` sidecars and the `user_code:` mappings that point at them. - `grant:` families via the grant cascade (member token keys, the `:tokens` set, the record, and member `:act` sidecars), plus standalone `access:` (with `:act`) and `refresh:` records. - `jwt:` allowlist markers whose `subject` field matches. The grant cascade already removes markers for grant-bound tokens; this sweep covers a marker whose grant set is already gone. Client-credentials markers carry an empty subject, so the subject decoder never matches them; they expire with `ttl.access_secs`. Records without a `subject` field are never touched: `par:`, `pending:`, `dpop_jti:`, `client_assertion_jti:`, `rotation:`, `totp_replay:`, and the `{rl}` rate-limit counters. Like the DCR `DELETE` revocation, the purge scans with `SCAN MATCH * COUNT 100` and filters on the raw hash field. On Cluster topologies a scan covers the contacted node only (Single and Sentinel scan fully). `user unlock --email` sweeps the same `loginfail` keyspace the same way (direct `DEL` of the account counter plus `SCAN MATCH` over the per-IP composite buckets), so the same Cluster caveat applies: buckets living on other nodes survive the sweep. A password change or TOTP re-enrollment reuses the same session sweep scoped to the subject minus the session in hand (`purge_sibling_sessions` in the same file): sibling `session:` records plus their `:meta` sidecars are deleted, everything else is untouched, and no new key layout is introduced. ## Operational runbook Configure Redis with persistence and eviction off. A minimal single-node configuration looks like: ```conf appendonly yes appendfsync everysec maxmemory-policy noeviction min-replicas-to-write 1 min-replicas-max-lag 10 ``` For a single node without replicas, use `appendfsync always` or Redis 7.2+ so `WAITAOF` is available, and set `redis.durability = waitaof`. On startup the server runs `PING`, checks `maxmemory-policy`, and checks `appendonly`. A violation is a startup failure, not a warning, unless the matching opt-out flag (`redis.noeviction_required` or `redis.aof_required`) is set false. To inspect a key family, use `SCAN` with the prefix; never use `KEYS` in a request path. ## File and directory references - `src/store/mod.rs` for the key layout, Lua scripts, and the barrier. - `src/store/tokens.rs`, `src/store/grants.rs`, `src/store/codes.rs`, `src/store/sessions.rs`, `src/store/par.rs`, `src/store/device.rs`, `src/store/dcr.rs`, `src/store/ratelimit.rs` for the per-family records. - `src/protocol/purge.rs` for the `user disable` subject purge. - `src/config/redis.rs` for topology, durability, and prefix validation. - `src/state/redis.rs`, `src/state/sentinel.rs` for connection management.