Ver código fonte

Hardened container configuration

Efren Yevale Varela 1 mês atrás
pai
commit
3f4acb4e8b

+ 1 - 1
deploy/k8s/kustomization.yaml

@@ -36,4 +36,4 @@ images:
 # Ingress while `D_OAUTH_TLS__CLIENT_CA_PATH` is set fails
 # `scripts/check_manifests.py`. See docs/DEPLOYMENT.md. Pod egress and ingress
 # are constrained by `networkpolicy.yaml` (default-deny egress except DNS +
-# Postgres + Redis).
+# Postgres/Redis scoped to their namespace).

+ 18 - 9
deploy/k8s/networkpolicy.yaml

@@ -9,10 +9,18 @@
 # - DNS only, to the `kube-system` namespace on port 53 (UDP + TCP). Adjust if
 #   CoreDNS runs elsewhere; NodeLocal DNSCache also lives in `kube-system`, so
 #   it is covered.
-# - Postgres (5432) and Redis (6379) anywhere else, with link-local
-#   `169.254.0.0/16` excluded so the cloud metadata endpoint
-#   (`169.254.169.254`) can never be reached. If Postgres/Redis run in-cluster,
-#   tighten the port-scoped `ipBlock` to their namespace/pod selector.
+# - Postgres (5432) and Redis (6379) scoped to the namespace that hosts them.
+#   The reference ConfigMap DSNs point at `*.d-oauth.svc.cluster.local`, so the
+#   default selects the `d-oauth` namespace. There is deliberately NO
+#   `ipBlock: 0.0.0.0/0` peer: that would leave both ports as an exfiltration
+#   channel to any internet host. Point the selector at the data namespace if
+#   it differs, and AND in a `podSelector` for their labels to admit only the
+#   database and cache pods rather than the whole namespace.
+# - A managed database outside the cluster cannot be named by a selector. Give
+#   its private endpoint a specific `ipBlock` CIDR (never `0.0.0.0/0`) or, on a
+#   CNI with FQDN policy (Cilium `toFQDNs`), the DNS name. Every `ipBlock` peer
+#   must exclude `169.254.0.0/16` so the cloud metadata endpoint
+#   (`169.254.169.254`) can never be reached.
 # - Everything else, including the Kubernetes API and arbitrary HTTPS
 #   exfiltration, is denied. The app never calls the API and runs with
 #   `automountServiceAccountToken: false`.
@@ -54,12 +62,13 @@ spec:
           protocol: UDP
         - port: 53
           protocol: TCP
-    # Postgres + Redis ports only; link-local (metadata) excluded.
+    # Postgres + Redis, scoped to their namespace (the reference deployment
+    # runs the data services alongside the app in `d-oauth`). No internet-wide
+    # `ipBlock`: these ports are not an exfiltration channel.
     - to:
-        - ipBlock:
-            cidr: 0.0.0.0/0
-            except:
-              - 169.254.0.0/16
+        - namespaceSelector:
+            matchLabels:
+              kubernetes.io/metadata.name: d-oauth
       ports:
         - port: 5432
           protocol: TCP

+ 5 - 2
docs/CHANGELOG.md

@@ -40,8 +40,11 @@ While the version stays below `1.0.0`, a minor bump may carry a breaking change.
   `0400` tmpfs files under `/run/secrets` (read through
   `D_OAUTH_TOKEN_PEPPER_FILE` / `D_OAUTH_TOTP_ENCRYPTION_KEY_FILE`, mirrored
   in `compose.yaml`), the image reference is digest-pinned, a NetworkPolicy
-  default-denies pod egress except DNS plus Postgres/Redis, and
-  `automountServiceAccountToken` is `false`.
+  default-denies pod egress except DNS plus Postgres/Redis scoped to their
+  namespace (no internet-wide `0.0.0.0/0` peer; `scripts/check_manifests.py`
+  rejects one), and `automountServiceAccountToken` is `false`. Remote JAR
+  `request_uri` is documented as unsupported under the default-deny, with the
+  Cilium FQDN opt-in path in `docs/DEPLOYMENT.md`.
 
 ### Changed
 

+ 53 - 21
docs/DEPLOYMENT.md

@@ -323,27 +323,57 @@ default-deny for those pods on any CNI that enforces NetworkPolicy (Calico,
 Cilium, and similar). A CNI that ignores NetworkPolicy gets no benefit from the
 manifest; verify the cluster CNI applies it.
 
-- Egress admits DNS to the `kube-system` namespace on port 53 (UDP and TCP)
-  and the Postgres (5432) and Redis (6379) ports to any address except
-  `169.254.0.0/16`. The link-local exclusion means a cloud metadata endpoint
-  such as `169.254.169.254` can never be reached, closing the SSRF-to-metadata
-  and arbitrary-exfiltration paths.
+- Egress admits DNS to the `kube-system` namespace on port 53 (UDP and TCP) and
+  Postgres (5432) and Redis (6379) scoped to the namespace that hosts them —
+  `d-oauth` in the reference deployment, matching the ConfigMap DSNs
+  (`*.d-oauth.svc.cluster.local`). It carries no `ipBlock: 0.0.0.0/0` peer, so
+  neither port is an exfiltration channel to an arbitrary internet host;
+  `scripts/check_manifests.py` rejects one if it is reintroduced.
+- Tightening further: AND a `podSelector` for the database/cache pod labels into
+  the same `to` entry to admit only those pods rather than their whole namespace
+  (a whole-namespace selector is the portable choice across CNIs; a
+  `podSelector` is tighter but depends on the CNI resolving a Service backend
+  for egress policy).
+- A managed database outside the cluster cannot be named by a selector. Give its
+  private endpoint a specific `ipBlock` CIDR (never `0.0.0.0/0`) or, on a CNI
+  with FQDN policy (Cilium `toFQDNs`), the DNS name. Every `ipBlock` peer must
+  exclude `169.254.0.0/16` so the cloud metadata endpoint (`169.254.169.254`)
+  can never be reached.
 - Ingress admits only the public TLS port (8080) and the plaintext health port
   (8081). The Service still publishes only 8080.
-- If Postgres and Redis run in-cluster behind known labels, tighten the
-  port-scoped `ipBlock` to their `namespaceSelector`/`podSelector`. If CoreDNS
-  runs outside `kube-system`, adjust the DNS rule. A gateway in another
-  namespace is still admitted because the ingress rule constrains ports, not
-  sources.
+- If the data services run in another namespace, point the egress selector at
+  it. If CoreDNS runs outside `kube-system`, adjust the DNS rule. A gateway in
+  another namespace is still admitted because the ingress rule constrains
+  ports, not sources.
 
 The policy deliberately denies outbound HTTPS (443). The JAR `request_uri`
-fetch (`src/jose/jar/fetch.rs`) retrieves a pre-registered request object from
-a publicly routable HTTPS host, so it cannot reach its target under the shipped
-default-deny. A deployment that relies on remote request objects must add a
-scoped egress rule to that specific host on 443 — never a blanket 443
-allowance, which would restore arbitrary exfiltration. A deployment that does
-not can leave the default-deny in place and treat remote `request_uri` as
-unavailable. The tension is recorded in [TODO.md](TODO.md).
+fetch (`src/jose/jar/fetch.rs`) retrieves a pre-registered request object from a
+publicly routable HTTPS host, so remote request objects are unavailable under
+the shipped default-deny and the fetch fails closed with
+`invalid_request_object`. A deployment that relies on them must allow the
+specific request-object host on 443 — but base Kubernetes `NetworkPolicy`
+cannot name a hostname, so this needs a CNI with FQDN policy (Cilium) or an
+egress proxy/gateway; a blanket 443 allowance is never correct, because it
+restores arbitrary exfiltration. On Cilium, an additive policy such as:
+
+```yaml
+apiVersion: cilium.io/v2
+kind: CiliumNetworkPolicy
+metadata:
+  name: d-oauth-request-uri
+  namespace: d-oauth
+spec:
+  endpointSelector:
+    matchLabels:
+      app.kubernetes.io/name: d-oauth-server
+  egress:
+    - toFQDNs:
+        - matchName: request-objects.example.com
+      toPorts:
+        - ports:
+            - port: "443"
+              protocol: TCP
+```
 
 ### Database access and backups
 
@@ -466,8 +496,9 @@ The image pipeline is gated on:
   rootfs, no capabilities, and no shell, but still bypass the application's own
   auth; restrict `pods/exec` with RBAC and audit it (see
   [OPERATIONS.md](OPERATIONS.md)).
-- A NetworkPolicy default-denies pod egress except DNS, Postgres, and Redis,
-  and admits ingress only on the two pod ports.
+- A NetworkPolicy default-denies pod egress except DNS and the namespace
+  hosting Postgres/Redis scoped to ports 5432/6379, forbids an internet-wide
+  `0.0.0.0/0` peer, and admits ingress only on the two pod ports.
 - Health routes are unreachable from the public listener, so the probe port is
   not a second API surface.
 - Liveness never touches a dependency, so a dependency outage does not trigger
@@ -527,8 +558,9 @@ d-oauth-server healthcheck --url http://127.0.0.1:8081/health/live
   probes on the plaintext health listener, PDB + topology spread,
   `automountServiceAccountToken: false`, every secret file-mounted),
   `migrate-job.yaml` (exactly-once schema migration, never per replica),
-  `networkpolicy.yaml` (default-deny egress except DNS/Postgres/Redis, ingress
-  to the two pod ports), `pdb.yaml` (`minAvailable: 2`), `kustomization.yaml`
+  `networkpolicy.yaml` (default-deny egress except DNS + Postgres/Redis scoped
+  to their namespace, no `0.0.0.0/0` peer; ingress to the two pod ports),
+  `pdb.yaml` (`minAvailable: 2`), `kustomization.yaml`
   (committed placeholder digest; the release pipeline pins the real image
   digest here), and `tls-secret.example.yaml` (how to provision `/certs`,
   never applied). No Ingress is shipped: mTLS requires L4/TCP passthrough.

+ 4 - 0
docs/OAUTH.md

@@ -313,6 +313,10 @@ Reference: RFC 9101 §4.1 and §5.2, RFC 9126 §2 and §3.
 - A `request_uri` is a one-time, client-bound PAR handle or an exactly
   pre-registered URI. It is fetched only with the SSRF guard: https only, no
   redirects, global IPs only, size and time capped.
+- A remote (pre-registered, non-PAR) `request_uri` is fetched over outbound
+  HTTPS, which the shipped cluster network policy default-denies. It is
+  available only when the deployment adds a scoped egress allowance for the
+  specific host; see [DEPLOYMENT.md](DEPLOYMENT.md#network-policy).
 
 ## Security properties
 

+ 2 - 2
docs/SECURITY.md

@@ -59,12 +59,12 @@ class, or runtime setting).
 | Secret exfiltration from the process environment | Secrets are injected at runtime, not baked into image layers; the pepper/TOTP key live outside the database; the Deployment mounts every secret — pepper, TOTP key, and the database/Redis passwords — as `0400` tmpfs files under `/run/secrets` (read through their `*_FILE` variables), and the passwordless DSNs come from the ConfigMap, so no secret enters the environment | An attacker with code execution can still read the mounted secret files under `/run/secrets`; that is the same capability the server has. |
 | Exposed management surface | The plaintext health listener (8081) is unauthenticated but not published in the Service (only 8080 is) | A misconfigured Service/Ingress could expose it; it must never be internet-facing. |
 | Host access via mounted volumes | `/certs` is mounted read-only at mode `0400`; no Docker socket is mounted | Operator misconfiguration. |
-| Unrestricted pod egress (SSRF to metadata, exfiltration) | `networkpolicy.yaml` default-denies egress except DNS (53 to `kube-system`) and Postgres/Redis (5432/6379), with link-local `169.254.0.0/16` excluded; ingress is limited to pod ports 8080/8081 | Enforcement depends on a NetworkPolicy-capable CNI; a cluster that ignores NetworkPolicy gets no benefit. The default-deny also blocks the JAR `request_uri` HTTPS fetch — a deployment that needs it must allow the specific host on 443 (see `docs/DEPLOYMENT.md`). |
+| Unrestricted pod egress (SSRF to metadata, exfiltration) | `networkpolicy.yaml` default-denies egress except DNS (53 to `kube-system`) and Postgres/Redis (5432/6379) scoped to the namespace hosting them — no internet-wide `0.0.0.0/0` peer; any `ipBlock` peer excludes `169.254.0.0/16`; ingress is limited to pod ports 8080/8081 | Enforcement depends on a NetworkPolicy-capable CNI; a cluster that ignores NetworkPolicy gets no benefit. A managed database outside the cluster needs a specific CIDR or a CNI FQDN policy. The default-deny also blocks the JAR `request_uri` HTTPS fetch — remote request objects are unsupported unless the deployment adds a scoped allowance (see `docs/DEPLOYMENT.md`). |
 | Operator `exec` into a running pod | No shell and a non-root, read-only-rootfs container limit what runs; every secret is a file the container user already reads, and no secret is in the environment | `pods/exec` (and `docker exec`) bypass the application's own authorization; restrict it with RBAC and audit it (`docs/OPERATIONS.md`). |
 | Kubernetes API access from the pod | `automountServiceAccountToken: false`; the binary never calls the API | A future feature that needs the API would have to re-enable it deliberately. |
 | Image/registry compromise | Digest pinning, cosign verification, and a committed digest reference with no shippable `:latest`; CI writes the signed build digest into `kustomization.yaml` | A registry compromise before the digest is recorded still ships; the pin step must run. |
 
-Last reviewed: 2026-10-04. File-mounted DB/Redis passwords, digest pinning,
+Last reviewed: 2026-10-05. File-mounted DB/Redis passwords, digest pinning,
 NetworkPolicy, `automountServiceAccountToken: false`, and scheduled fuzzing are
 implemented in the shipped manifests and workflows; the remaining mitigations
 are tracked in `docs/TODO.md`.

+ 4 - 41
docs/TODO.md

@@ -2,10 +2,10 @@
 
 This is a backlog of deferred hardening work. Each item is a candidate to be
 planned and implemented later; none of them is a current defect that makes the
-server unusable. The items came out of a threat-model review that asked two
-questions: what an attacker can still read or crack from a full dump of the
-PostgreSQL database, and what an attacker already inside the image's trust
-boundary can reach.
+server unusable. The items came out of a threat-model review of what an attacker
+can still read or crack from a full dump of the PostgreSQL database; the
+image-trust-boundary items that review surfaced have since been closed (see
+[SECURITY.md](SECURITY.md) and [DEPLOYMENT.md](DEPLOYMENT.md)).
 
 This document only records the gaps. It does not describe anything that is
 already implemented, and it does not propose code changes here.
@@ -35,45 +35,12 @@ list is deliberately short and ordered by value.
   key, which makes rotation incremental and limits a single leak's blast radius.
 - **Files.** `src/crypto/totp.rs`, `src/keys.rs`.
 
-## Container and host hardening
-
-The attack surface and the controls already in place are documented in
-[SECURITY.md](SECURITY.md); this section lists only the mitigations that remain.
-It answers a second threat-model question: if an attacker is already inside the
-image's trust boundary, how could they reach the container or the host, and how
-should we reduce that risk? The remaining vectors are narrow.
-
-### 3. NetworkPolicy versus remote `request_uri`
-
-- **Risk.** The default-deny egress blocks outbound HTTPS, so the JAR
-  `request_uri` fetch (`src/jose/jar/fetch.rs`) cannot reach a publicly routable
-  request-object host. A deployment either relies on a feature that silently
-  fails, or loosens egress further than intended.
-- **Proposed change.** Decide per deployment: allow the specific request-object
-  host on 443 with a scoped rule, or document that remote `request_uri` is
-  unavailable under the shipped policy. A blanket 443 allowance is not an
-  option because it restores arbitrary exfiltration.
-- **Files.** `deploy/k8s/networkpolicy.yaml`, `docs/DEPLOYMENT.md`.
-
-### 4. Narrow Postgres/Redis egress to their destination
-
-- **Risk.** `networkpolicy.yaml` allows egress to *any* address on ports 5432 and
-  6379 (link-local excluded), so those ports are an exfiltration channel to an
-  arbitrary internet host, not only the database and cache.
-- **Proposed change.** Where Postgres and Redis run in-cluster, replace the
-  `ipBlock: 0.0.0.0/0` peer with their `namespaceSelector`/`podSelector` so only
-  the real endpoints are reachable; keep a port-scoped rule only for an external
-  managed database.
-- **Files.** `deploy/k8s/networkpolicy.yaml`, `docs/DEPLOYMENT.md`.
-
 ## Priority
 
 Ranked by value against effort:
 
 - **Medium:** items 1 and 2. Item 1 is medium/high effort because it needs a
   migration plus a blind index with uniqueness and rotation handling.
-- **Container/host hardening:** item 3 is the concrete correctness risk; item 4
-  is the egress-narrowing follow-up.
 
 Recommendation: implement item 1 next. It closes the largest plaintext
 disclosure; item 2 then reduces the cost of a future key compromise.
@@ -84,7 +51,3 @@ disclosure; item 2 then reduces the cost of a future key compromise.
 - Is the token pepper and TOTP key held in mounted files or a secrets manager
   in the target deployment? This decides whether envelope encryption (item 2)
   is worth building at all.
-- Which target environments rely on remote JAR `request_uri` objects? This
-  decides whether item 3 needs a scoped egress rule or only documentation.
-- Do Postgres and Redis run in-cluster? This decides whether item 4 can narrow
-  egress to their namespace/selector.

+ 34 - 10
scripts/check_manifests.py

@@ -25,9 +25,10 @@ contract WITHOUT a live cluster:
   may embed a password (it belongs in the Secret, read through the
   `*_PASSWORD_FILE` form).
 - NetworkPolicy: must select `app.kubernetes.io/name: d-oauth-server`, enable
-  `Ingress` + `Egress`, admit ingress only on 8080/8081, and allow egress only
-  on 53 (DNS), 5432 (Postgres), and 6379 (Redis) with `169.254.0.0/16`
-  excluded (cloud metadata).
+  `Ingress` + `Egress`, admit ingress only on 8080/8081, allow egress only on
+  53 (DNS), 5432 (Postgres), and 6379 (Redis), forbid an internet-wide
+  `0.0.0.0/0` egress peer, and require any `ipBlock` peer to exclude
+  `169.254.0.0/16` (cloud metadata).
 - No baked secrets: no PEM blocks in manifests; `REPLACE_ME` placeholders
   only inside `secret.yaml`; no secret-shaped keys in the ConfigMap.
 - L7-vs-L4 mTLS constraint: an L7 Ingress that terminates TLS while
@@ -48,6 +49,7 @@ requires every workload image to be digest-pinned (`@sha256:<64 hex>`).
 from __future__ import annotations
 
 import argparse
+import ipaddress
 import re
 import sys
 from pathlib import Path
@@ -284,6 +286,20 @@ def policy_ports(rule: dict) -> set[int]:
   return ports
 
 
+def ip_network_contains(cidr: str, inner: str) -> bool:
+  try:
+    outer = ipaddress.ip_network(cidr)
+    sub = ipaddress.ip_network(inner)
+  except ValueError:
+    return False
+  if outer.version != sub.version:
+    return False
+  return (
+    int(sub.network_address) >= int(outer.network_address)
+    and int(sub.broadcast_address) <= int(outer.broadcast_address)
+  )
+
+
 def check_networkpolicy(doc: dict, fails: Failures) -> None:
   """Item 9: default-deny ingress/egress for the app pods."""
   name = doc.get("metadata", {}).get("name", "?")
@@ -306,22 +322,30 @@ def check_networkpolicy(doc: dict, fails: Failures) -> None:
       f"{PUBLIC_PORT} and {HEALTH_PORT}"
     )
   egress_ports: set[int] = set()
-  metadata_excluded = False
   for rule in spec.get("egress") or []:
     if not isinstance(rule, dict):
       continue
     egress_ports |= policy_ports(rule)
     for peer in rule.get("to") or []:
       block = (peer or {}).get("ipBlock") or {}
-      if METADATA_CIDR in (block.get("except") or []):
-        metadata_excluded = True
+      cidr = block.get("cidr")
+      if not cidr:
+        continue
+      if cidr in ("0.0.0.0/0", "::/0"):
+        fails.add(
+          f"networkpolicy/{name}: egress peer {cidr} is internet-wide; scope "
+          "Postgres/Redis to a namespace/pod selector or a specific CIDR"
+        )
+      elif ip_network_contains(cidr, METADATA_CIDR) and METADATA_CIDR not in (
+        block.get("except") or []
+      ):
+        fails.add(
+          f"networkpolicy/{name}: egress ipBlock {cidr} must exclude "
+          f"{METADATA_CIDR} (cloud metadata)"
+        )
   for port in REQUIRED_EGRESS_PORTS:
     if port not in egress_ports:
       fails.add(f"networkpolicy/{name}: egress must allow port {port}")
-  if not metadata_excluded:
-    fails.add(
-      f"networkpolicy/{name}: egress must exclude {METADATA_CIDR} (cloud metadata)"
-    )
 
 
 def check_deployment(doc: dict, fails: Failures, strict_digest: bool) -> None: