Bläddra i källkod

Added debug log

Efren Yevale Varela 1 månad sedan
förälder
incheckning
f420c8ccf6

+ 1 - 0
.gitignore

@@ -7,6 +7,7 @@ audio/
 audit/
 triage/
 brief/
+debug/
 models/
 speakers/
 *.onnx

+ 1 - 1
AGENTS.md

@@ -109,7 +109,7 @@ publish-only.
 ## Conventions & gotchas
 
 - Never commit gitignored runtime output: `config.toml`, `transcripts/`, `audio/`, `audit/`,
-  `triage/`, `brief/`, `speakers/` (biometric), `models/`, `.omo/`.
+  `triage/`, `brief/`, `debug/`, `speakers/` (biometric), `models/`, `.omo/`.
 - New assistant tools: register in `assistant/tools.py`. Model-supplied arguments are JSON-parsed and
   schema-validated before running; set `side_effect` and `requires_confirmation` for anything
   non-read-only.

+ 6 - 0
config.example.toml

@@ -136,3 +136,9 @@ join_timeout_s    = 2.0
 write    = false
 dir      = "brief"
 filename = "brief.md"
+
+[debug]
+enabled           = false
+dir               = "debug"
+heartbeat         = true
+max_bytes_per_day = 50000000

+ 21 - 0
docs/CONFIGURATION.md

@@ -160,6 +160,12 @@ join_timeout_s    = 2.0                      # Embedding-lane join timeout befor
 write    = false                            # false = brief shown in the TUI only; true = also persist to disk
 dir      = "brief"                          # Destination directory (cwd-relative); gitignored
 filename = "brief.md"                       # Snapshot filename inside dir
+
+[debug]                                     # Structured health-signal log (no transcript text; off by default)
+enabled          = false                    # false = no debug lines, zero overhead (DebugSink is None)
+dir              = "debug"                  # Destination directory (cwd-relative); gitignored, one JSONL per day
+heartbeat        = true                     # true = ~1 line/s heartbeat signals; false = anomalies only
+max_bytes_per_day = 50000000                # Best-effort per-day byte cap; writes past the cap are skipped
 ```
 
 `[speaker]` adds opt-in speaker identification: a dedicated in-process sherpa-onnx embedding lane
@@ -243,6 +249,21 @@ display-only: the TUI renders it and nothing is written to disk. With `write = t
 assistant snapshots the current brief as plain text to `dir/filename` (default
 `brief/brief.md`, cwd-relative, gitignored); clearing the brief truncates the file.
 
+`[debug]` persists per-component health signals to `debug/YYYY-MM-DD.jsonl` (one JSON
+object per line, `ensure_ascii=False`) so the next silent freeze is diagnosable from disk.
+Signals only — no transcript text, no audio, no prompt content: RMS/queue/timing fields such
+as `device_ok`, `worker_stall_s`, `queue_depth`, `inference_depth`, plus anomaly lines for
+capture reopens, inference-queue `queue-full` / `eviction-drop`, `llm-timeout`, worker
+`worker-stall`, triage `triage-brief-failure`, and supervisor/child exit lines. With
+`enabled = false` (the default) no sink is built and the suite stays disk-clean; with
+`heartbeat = false` only anomalies are written. `max_bytes_per_day` is a best-effort guard:
+writes past the cap are skipped with one warning (never crash, never rotate). `debug/` lives
+on local storage (a slow/NFS/FUSE mount can block the only capturer) and is gitignored like
+`transcripts/`, `audit/`, and `triage/`; clean it manually with `rm debug/*`. The assistant
+constructs its sink via `debug_sink_factory` threaded through `runner.main` (`None` when
+disabled), and every assistant emit runs after its `with self._lock` block — never under the
+RLock — so a slow write cannot stall the ticker + event reader.
+
 Upgrading from a v5-draft config: `socket`, `model`, `predict_timeout_s`, `checkpoint`,
 `subfolder`, and `max_len` are **not** `[triage]` keys and are silently ignored by `load_config`
 (per-field `.get()`). `checkpoint` is superseded by `default_checkpoint`; the others have no

+ 2 - 0
docs/DEVELOPMENT.md

@@ -22,6 +22,7 @@ Tests cover:
 - **Assistant**: mode machine, command readiness/timeouts, tool schema validation, LLM client/history trimming, audit log
 - **Triage & brief**: Laya lane, questions-file validation, signal normalization, the fail-open `substantive_update` gate, single-outstanding `brief-update` coalescing, and the durable brief file
 - **Speakers**: prototype matching, mint/confirm/adapt guards, daemon wiring, assistant roster and naming, offline pre-enrollment (`tests/test_speaker_enroll.py`)
+- **Debug**: `DebugSink` JSONL lines, disabled-means-None, emit-throw safety, per-day byte cap, concurrent-emit integrity (`tests/test_debug.py`); assistant lock-free emits via `debug_sink_factory` (`tests/test_assistant_service.py`)
 - **TUI**: pure view-state reducer and control-socket client
 
 ## Code structure
@@ -49,6 +50,7 @@ src/stt_daemon/
 ├── daemon.py            - Orchestrator (threads, queues, event emission)
 ├── events.py            - NDJSON event envelope + codec
 ├── bus.py               - Publish-only event bus (socket, ring replay)
+├── debug.py             - Structured health-signal log (`DebugSink`, `debug/YYYY-MM-DD.jsonl`)
 ├── assistant/
 │   ├── modes.py         - Mode state machine (idle/command_armed/executing/chat)
 │   ├── service.py       - Modes, command buffer, readiness, timeouts

+ 30 - 0
docs/TROUBLESHOOTING.md

@@ -124,6 +124,36 @@ export HF_HOME=/path/to/cache
 uv run stt-daemon --model small
 ```
 
+## Frozen with no transcripts (silent freeze)
+
+With `[debug] enabled = true`, the debug log is the first place to look — it is the only
+structured record (child stdout/stderr is discarded, so there is no raw stderr complement).
+Each line is one JSON object in `debug/YYYY-MM-DD.jsonl`; no transcript text is stored.
+
+```bash
+tail -n 100 debug/$(date +%F).jsonl
+```
+
+Read the tail in this order:
+
+1. **`device_ok`**: `false` means the capture device went away — check `pactl list short
+   sources`, restart PulseAudio, and re-run (see "Audio device not responding" above).
+2. **`worker_stall_s`**: compare against `[assistant] worker_stall_ms` (default 15000).
+   A rising stall with a growing `queue_depth` means transcription is falling behind —
+   use a smaller model or GPU (see "Transcription backlog" above). A `worker-stall`
+   signal line marks the threshold crossing.
+3. **Capture `reopen` lines**: repeated reopen attempt/fail lines mean the source keeps
+   dropping; fix the device rather than raising queue sizes.
+4. **Supervisor exit lines**: a child early-exit / control-socket timeout / first-child-exit
+   code line means the supervisor already reaped a child — restart via
+   `uv run stt-daemon up --with-tui` and check which child exited first.
+5. **Assistant `queue-full` / `eviction-drop` / `llm-timeout` / `triage-brief-failure`**:
+   backpressure or LLM-side stalls on the assistant lane, not a capture problem.
+
+When the system is healthy again, clear the log manually (`rm debug/*`); there is no
+automatic retention job. Keep `debug/` on local storage — a slow/NFS/FUSE mount can block
+the only capturer.
+
 ## Speaker identification
 
 See [SPEAKERS.md](SPEAKERS.md) for the feature reference.

+ 16 - 0
src/stt_daemon/assistant/runner.py

@@ -96,6 +96,21 @@ def main() -> int:
         on_result=lambda gen, payload: holder["svc"]._on_triage_result(gen, payload),
     )
     brief_file = BriefFile(cfg.brief.write, cfg.brief.dir, cfg.brief.filename)
+    _debug_factory = None
+    debug_cfg = getattr(cfg, "debug", None)
+    if debug_cfg is not None and bool(getattr(debug_cfg, "enabled", False)):
+        try:
+            from ..debug import DebugSink
+        except Exception:  # noqa: BLE001 - debug stays disabled without the module
+            DebugSink = None  # type: ignore[assignment]
+        if DebugSink is not None:
+            debug_dir = str(getattr(debug_cfg, "dir", "debug"))
+
+            def _make_debug_sink():
+                return DebugSink(debug_dir, "assistant")
+
+            _debug_factory = _make_debug_sink
+
     service = AssistantService(
         cfg,
         subscriber,
@@ -108,6 +123,7 @@ def main() -> int:
         brief_fn=brief_fn,
         brief_reset=brief_reset,
         brief_file=brief_file,
+        debug_sink_factory=_debug_factory,
     )
     holder["svc"] = service
     if session is not None:

+ 79 - 0
src/stt_daemon/assistant/service.py

@@ -67,6 +67,7 @@ class AssistantService:
         brief_fn: Callable[..., str] | None = None,
         brief_reset: Callable[[], None] | None = None,
         brief_file: Any = None,
+        debug_sink_factory: Callable[[], Any] | None = None,
     ):
         """Create the service.
 
@@ -85,6 +86,10 @@ class AssistantService:
             brief_fn: Callable turning a brief prompt into plain-text brief.
             brief_reset: Callable clearing the brief engine session on clear.
             brief_file: Durable brief writer (duck-typed `write`/`clear`).
+            debug_sink_factory: Zero-arg factory returning a DebugSink-like
+                object with `emit(type, **fields)`; None disables debug lines.
+                The sink is total-swallow, and every emit call is placed
+                AFTER any `with self._lock` block (never under RLock).
         """
         self.cfg = cfg
         self._subscriber = subscriber
@@ -162,6 +167,44 @@ class AssistantService:
         self._reader: threading.Thread | None = None
         self._ticker: threading.Thread | None = None
         self._inference = InferenceQueue(size=cfg.llm.inference_queue_size)
+        # Debug signal sink (None when `[debug] enabled = false`). Built once
+        # here via the factory; every `_debug_emit` below is best-effort and
+        # runs AFTER any `with self._lock` block — never under the RLock, so
+        # a slow write cannot stall the ticker + event reader.
+        if debug_sink_factory is not None:
+            try:
+                self._debug = debug_sink_factory()
+            except Exception:  # noqa: BLE001 - debug must never break startup
+                logger.warning("debug sink factory failed; continuing disabled")
+                self._debug = None
+        else:
+            self._debug = None
+
+    def _debug_emit(self, etype: str, **fields: Any) -> None:
+        """Best-effort debug signal (never raises; never holds `self._lock`).
+
+        Callers must invoke this AFTER leaving any `with self._lock` block.
+        No chat text, tool args, or prompt content may be passed in `fields`.
+        """
+        sink = self._debug
+        if sink is None:
+            return
+        try:
+            sink.emit(etype, **fields)
+        except Exception:  # noqa: BLE001 - sink is total-swallow by contract
+            pass
+
+    def _inference_full_before(self) -> bool:
+        """True when the inference lane is at capacity (an admit evicts)."""
+        try:
+            size = int(self._inference.size)
+        except Exception:  # noqa: BLE001 - defensive; a bad size means unknown
+            return False
+        try:
+            used = self._inference.depth() + (1 if self._inference.in_flight() else 0)
+        except Exception:  # noqa: BLE001 - depth probe must not break dispatch
+            return False
+        return used >= size
 
     # ------------------------------------------------------------------ lifecycle
 
@@ -342,9 +385,13 @@ class AssistantService:
                 run=lambda: self._run_execution(text, edited),
                 priority=-1,
             )
+            full_before = self._inference_full_before()
             if not self._inference.submit(job):
+                self._debug_emit("queue-full", job="command")
                 self.finish()
                 return EXECUTE_BUSY
+            if full_before:
+                self._debug_emit("eviction-drop", job="command")
         return text
 
     def _run_execution(self, text: str, edited: bool) -> None:
@@ -357,6 +404,8 @@ class AssistantService:
             except Exception as e:  # noqa: BLE001 - an executor crash must not kill the service
                 logger.error("Executor error: %s", e)
                 result = ExecutionResult(ok=False, result=f"executor error: {e}")
+                if isinstance(e, TimeoutError):
+                    self._debug_emit("llm-timeout", job="command")
 
         latency_ms = int((self._now() - start) * 1000)
         self._emit(
@@ -600,6 +649,17 @@ class AssistantService:
             inference_depth=self._inference.depth(),
             inference_in_flight=self._inference.in_flight(),
         )
+        try:
+            stall_limit = float(self.cfg.assistant.worker_stall_ms) / 1000.0
+        except (TypeError, ValueError):
+            stall_limit = 15.0
+        if self._worker_stall_s > stall_limit:
+            self._debug_emit(
+                "worker-stall",
+                worker_stall_s=round(self._worker_stall_s, 3),
+                queue_depth=self._queue_depth,
+                device_ok=self._device_ok,
+            )
 
     def _on_status(self, now: float) -> None:
         # Reconnect rule: the replayed ring is informational. Never rebuild speech
@@ -711,9 +771,12 @@ class AssistantService:
         if not text:
             return
         job = InferenceJob(name="chat", run=lambda: self._run_chat(text))
+        full_before = self._inference_full_before()
         if self._inference.submit(job):
             with self._lock:
                 self._chat_queue_full = False
+            if full_before:
+                self._debug_emit("eviction-drop", job="chat")
             return
         with self._lock:
             was_full = self._chat_queue_full
@@ -721,6 +784,7 @@ class AssistantService:
             self._pending_chat.insert(0, text)
         if not was_full:
             self._emit(ERROR_EVENT, code="queue_full", message="chat queue full")
+        self._debug_emit("queue-full", job="chat")
 
     def _run_chat(self, text: str) -> None:
         self._emit(CHAT_MESSAGE_EVENT, role="user", text=text)
@@ -732,6 +796,8 @@ class AssistantService:
         except Exception as e:  # noqa: BLE001 - an LLM failure must not kill the service
             logger.error("Chat error: %s", e)
             self._emit(ERROR_EVENT, code="llm_error", message=str(e))
+            if isinstance(e, TimeoutError):
+                self._debug_emit("llm-timeout", job="chat")
             return
         if reply:
             self._emit(CHAT_MESSAGE_EVENT, role="assistant", text=reply)
@@ -845,6 +911,7 @@ class AssistantService:
                 if self._brief_token == token:
                     self._brief_outstanding = False
             self._triage_dump.finalize(ts)
+            self._debug_emit("queue-full", job="brief-update")
             return False
         return True
 
@@ -913,6 +980,18 @@ class AssistantService:
         except Exception as e:  # noqa: BLE001 - a brief failure must not kill the worker
             logger.error("Triage brief error: %s", e)
             requeued = True
+            try:
+                err_kind = type(e).__name__
+            except Exception:  # noqa: BLE001 - debug fields must never raise
+                err_kind = "error"
+            self._debug_emit(
+                "triage-brief-failure",
+                error=err_kind,
+                generation=generation,
+                deltas=len(deltas),
+            )
+            if isinstance(e, TimeoutError):
+                self._debug_emit("llm-timeout", job="brief-update")
         finally:
             with self._lock:
                 stale = generation != self._triage_generation

+ 23 - 0
src/stt_daemon/config.py

@@ -22,6 +22,7 @@ KNOWN_SECTIONS = {
     "triage",
     "speaker",
     "brief",
+    "debug",
 }
 
 
@@ -221,6 +222,15 @@ class BriefConfig:
     filename: str = "brief.md"
 
 
+@dataclass
+class DebugConfig:
+    """Debug signal-log configuration (JSONL diagnostics, gitignored)."""
+    enabled: bool = False
+    dir: str = "debug"
+    heartbeat: bool = True
+    max_bytes_per_day: int = 50000000
+
+
 @dataclass
 class SinkConfig:
     """Virtual audio sink configuration."""
@@ -246,6 +256,7 @@ class Config:
     triage: TriageConfig = field(default_factory=TriageConfig)
     speaker: SpeakerConfig = field(default_factory=SpeakerConfig)
     brief: BriefConfig = field(default_factory=BriefConfig)
+    debug: DebugConfig = field(default_factory=DebugConfig)
 
 
 def _optional_int(value: Any) -> int | None:
@@ -603,4 +614,16 @@ def load_config(path: str | None = None) -> Config:
                 filename=brief_data.get("filename", cfg.brief.filename),
             )
 
+        # Debug section
+        if "debug" in data:
+            debug_data = data["debug"]
+            cfg.debug = DebugConfig(
+                enabled=debug_data.get("enabled", cfg.debug.enabled),
+                dir=debug_data.get("dir", cfg.debug.dir),
+                heartbeat=debug_data.get("heartbeat", cfg.debug.heartbeat),
+                max_bytes_per_day=debug_data.get(
+                    "max_bytes_per_day", cfg.debug.max_bytes_per_day
+                ),
+            )
+
     return cfg

+ 87 - 14
src/stt_daemon/daemon.py

@@ -237,6 +237,7 @@ class Daemon:
         transcriber_factory: Callable[[], Any] | None = None,
         sink_factory: Callable[[], Any] | None = None,
         embedder_factory: Callable[[], Any] | None = None,
+        debug_sink_factory: Callable[[], Any] | None = None,
     ):
         """Initialize the daemon.
 
@@ -248,6 +249,8 @@ class Daemon:
             transcriber_factory: Override transcriber creation (used by tests).
             sink_factory: Override transcript sink creation (used by tests).
             embedder_factory: Override speaker embedder creation (used by tests).
+            debug_sink_factory: Override debug signal sink creation (used by
+                tests). Distinct from sink_factory, which builds TranscriptSink.
         """
         self.cfg = cfg
         self._capture_factory = capture_factory or (lambda: open_capture(self.cfg.audio))
@@ -255,6 +258,8 @@ class Daemon:
         self._transcriber_factory = transcriber_factory or self._build_transcriber
         self._sink_factory = sink_factory or self._build_sink
         self._embedder_factory = embedder_factory or self._build_embedder
+        self._debug_sink_factory = debug_sink_factory
+        self.debug_sink: Any = None
 
         # Set up logging
         self.logger = logging.getLogger("stt_daemon")
@@ -324,6 +329,26 @@ class Daemon:
 
         return build_embedder(self.cfg)
 
+    def _debug_config(self) -> Any:
+        """Return the [debug] section, or None when the config predates it."""
+        return getattr(self.cfg, "debug", None)
+
+    def _build_debug_sink(self) -> Any:
+        """Default debug sink builder (lazy import; duck-typed by callers)."""
+        from .debug import DebugSink
+
+        return DebugSink(getattr(self._debug_config(), "dir", "debug"), "daemon")
+
+    def _debug_emit(self, type_: str, **fields: Any) -> None:
+        """Best-effort debug signal line; never raises, never blocks shutdown."""
+        sink = self.debug_sink
+        if sink is None:
+            return
+        try:
+            sink.emit(type_, **fields)
+        except Exception as e:  # noqa: BLE001 -- debug must never break the daemon
+            self.logger.warning("Debug sink emit failed (%s)", e)
+
     def _initialize(self) -> None:
         """Initialize all components and start publishing events."""
         self.logger.info("Initializing audio capture...")
@@ -357,9 +382,23 @@ class Daemon:
         self._started_at = time.time()
         self._start_publisher()
         self._start_speaker_lane()
+        self._start_debug_sink()
 
         self.logger.info("Daemon ready. Listening...")
 
+    def _start_debug_sink(self) -> None:
+        """Construct the debug signal sink when [debug] is enabled (None otherwise)."""
+        self.debug_sink = None
+        debug_cfg = self._debug_config()
+        if debug_cfg is None or not getattr(debug_cfg, "enabled", False):
+            return
+        try:
+            factory = self._debug_sink_factory or self._build_debug_sink
+            self.debug_sink = factory()
+        except Exception as e:  # noqa: BLE001 -- boundary: debug must not crash init
+            self.logger.warning("Debug sink disabled (%s)", e)
+            self.debug_sink = None
+
     def _start_speaker_lane(self) -> None:
         """Build the speaker store + embedding lane (no-op when disabled).
 
@@ -445,6 +484,9 @@ class Daemon:
         for attempt in range(1, attempts + 1):
             if self._stop_event.is_set():
                 return False
+            self._debug_emit(
+                "capture_reopen_attempt", attempt=attempt, attempts=attempts
+            )
             try:
                 if self.capture is not None:
                     try:
@@ -455,11 +497,20 @@ class Daemon:
                 self.logger.warning(
                     "Capture reopened on attempt %d/%d", attempt, attempts
                 )
+                self._debug_emit(
+                    "capture_reopen_ok", attempt=attempt, attempts=attempts
+                )
                 return True
             except Exception as e:  # noqa: BLE001 -- boundary: retry, then give up
                 self.logger.warning(
                     "Capture reopen attempt %d/%d failed (%s)", attempt, attempts, e
                 )
+                self._debug_emit(
+                    "capture_reopen_fail",
+                    attempt=attempt,
+                    attempts=attempts,
+                    error=str(e),
+                )
                 if attempt < attempts:
                     self._stop_event.wait(delay_s)
         return False
@@ -551,6 +602,7 @@ class Daemon:
                     self.logger.debug("Capture close on reader exit failed (%s)", e)
             if failed and not self._stop_event.is_set():
                 self.logger.error("Reader thread exited unexpectedly; requesting shutdown")
+                self._debug_emit("reader_fatal")
                 self.request_stop()
 
     def _on_segment_finalized(self, utt: Utterance) -> None:
@@ -582,6 +634,12 @@ class Daemon:
                     emitted=False,
                     end_reason=dropped.reason,
                 )
+                self._debug_emit(
+                    "queue_drop",
+                    dropped_seg_id=dropped.seg_id,
+                    end_reason=dropped.reason,
+                    queue_size=self.segment_queue.maxsize,
+                )
             try:
                 self.segment_queue.put(utt, block=False)
                 return True
@@ -597,28 +655,31 @@ class Daemon:
         now = time.time()
         rms = math.sqrt(sum(self._rms_window) / len(self._rms_window)) if self._rms_window else 0.0
         busy = self._in_flight or self.segment_queue.qsize() > 0
-        self._publish(
-            events.HEARTBEAT,
-            in_flight=self._in_flight,
-            segments_done=self._segments_done,
-            worker_stall_s=(
+        payload = {
+            "in_flight": self._in_flight,
+            "segments_done": self._segments_done,
+            "worker_stall_s": (
                 round(now - self._last_worker_activity_ts, 3)
                 if busy and self._last_worker_activity_ts
                 else 0.0
             ),
-            queue_depth=self.segment_queue.qsize(),
-            frames_since_last=self._frames_since_heartbeat,
-            rms=round(rms, 5),
-            capture=self.cfg.audio.device,
-            device_ok=self.capture.healthy() if self.capture else False,
-            model_loaded=self.transcriber is not None,
-            uptime_s=round(now - self._started_at, 3),
-            last_transcript_age_s=(
+            "queue_depth": self.segment_queue.qsize(),
+            "frames_since_last": self._frames_since_heartbeat,
+            "rms": round(rms, 5),
+            "capture": self.cfg.audio.device,
+            "device_ok": self.capture.healthy() if self.capture else False,
+            "model_loaded": self.transcriber is not None,
+            "uptime_s": round(now - self._started_at, 3),
+            "last_transcript_age_s": (
                 round(now - self._last_transcript_ts, 3)
                 if self._last_transcript_ts
                 else 0.0
             ),
-        )
+        }
+        self._publish(events.HEARTBEAT, **payload)
+        debug_cfg = self._debug_config()
+        if debug_cfg is not None and getattr(debug_cfg, "heartbeat", True):
+            self._debug_emit("heartbeat", **payload)
         self._frames_since_heartbeat = 0
 
     def _check_backlog(self) -> None:
@@ -630,6 +691,9 @@ class Daemon:
                 size,
                 self.segment_queue.maxsize,
             )
+            self._debug_emit(
+                "backlog_warn", size=size, maxsize=self.segment_queue.maxsize
+            )
             self._backlog_warned = True
         elif size == 0 and self._backlog_warned:
             self._backlog_warned = False
@@ -657,6 +721,7 @@ class Daemon:
                 self.logger.debug("Drain on worker exit failed (%s)", e)
             if failed and not self._stop_event.is_set():
                 self.logger.error("Worker thread exited unexpectedly; requesting shutdown")
+                self._debug_emit("worker_fatal")
                 self.request_stop()
 
     def _process(self, utt: Utterance) -> None:
@@ -740,6 +805,7 @@ class Daemon:
             outcome = future.result(timeout=self.cfg.speaker.join_timeout_s)
         except Exception as e:  # noqa: BLE001 -- TimeoutError or lane failure -> "none"
             self.logger.debug("Speaker lookup failed (%s)", e)
+            self._debug_emit("speaker_fallback", error=str(e))
             return None, "none", "?", None
         try:
             return self._speaker_fields(outcome)
@@ -806,5 +872,12 @@ class Daemon:
             self.sink.close()
         if self.publisher:
             self.publisher.close()
+        if self.debug_sink is not None:
+            try:
+                self.debug_sink.close()
+            except Exception as e:  # noqa: BLE001 -- shutdown path must not raise
+                self.logger.debug("Debug sink close failed (%s)", e)
+            finally:
+                self.debug_sink = None
 
         self.logger.info("Daemon stopped.")

+ 128 - 0
src/stt_daemon/debug.py

@@ -0,0 +1,128 @@
+"""Per-component health-signal log (signals only, never transcripts/audio).
+
+Persists structured health signals to ``dir/YYYY-MM-DD.jsonl`` so the next
+silent freeze is diagnosable from disk. No transcript text, no audio paths,
+no prompt content -- numeric/boolean signal fields only.
+
+Freeze-safety invariant: :class:`DebugSink` is a leaf lock. It never calls
+back into the daemon, the assistant service, or any other component, so the
+internal :class:`threading.Lock` cannot participate in an ABBA deadlock.
+``emit()`` total-swallows every exception and latches a disabled circuit
+breaker, because a raise in the daemon reader is misclassified as capture
+failure (reopen churn/shutdown) and a raise in the worker is fatal.
+"""
+
+import json
+import logging
+import os
+import threading
+from datetime import datetime
+from pathlib import Path
+
+logger = logging.getLogger(__name__)
+
+_VALID_PROCS = ("daemon", "assistant", "supervisor")
+_FORBIDDEN_FIELDS = ("text", "audio", "prompt")
+
+
+class DebugSink:
+    """Append-only per-day JSONL signal log, safe to share across threads.
+
+    State invariant: this sink owns no reference to any daemon/service
+    object and performs no callbacks -- it is a leaf lock. The lock is held
+    only for a day-rollover ``os.open`` (when needed), a single ``os.write``
+    of the fully serialized line, and the byte-counter increment.
+    """
+
+    def __init__(self, dir, proc, max_bytes_per_day=50_000_000, now_fn=None):
+        """Create the sink, making ``dir`` once.
+
+        Args:
+            dir: Destination directory for daily JSONL files.
+            proc: One of ``daemon`` | ``assistant`` | ``supervisor``.
+            max_bytes_per_day: Best-effort per-date cap; writes past the cap
+                are skipped with a single warning (never crash, never rotate).
+            now_fn: Injectable clock returning a ``datetime`` (tests only).
+
+        Raises:
+            ValueError: If ``proc`` is not a known component name.
+        """
+        if proc not in _VALID_PROCS:
+            raise ValueError(f"unknown proc: {proc!r} (expected one of {_VALID_PROCS})")
+        self._proc = proc
+        self._dir = Path(dir)
+        self._max = max_bytes_per_day
+        self._now = now_fn or datetime.now
+        self._lock = threading.Lock()
+        self._fd = None
+        self._date = None
+        self._bytes = 0
+        self._disabled = False
+        self._cap_warned = False
+        try:
+            self._dir.mkdir(parents=True, exist_ok=True)
+        except Exception as e:  # noqa: BLE001 - debug logging is best effort
+            self._disabled = True
+            logger.warning("debug log disabled (mkdir failed): %s", e)
+
+    def _open_day(self, today):
+        """Open today's file, closing the previous day's fd. Caller holds the lock."""
+        if self._fd is not None:
+            try:
+                os.close(self._fd)
+            except OSError:
+                pass
+            self._fd = None
+        path = str(self._dir / f"{today}.jsonl")
+        self._fd = os.open(path, os.O_WRONLY | os.O_CREAT | os.O_APPEND, 0o644)
+        self._date = today
+        self._bytes = 0
+        self._cap_warned = False
+
+    def emit(self, type, **fields):
+        """Append one signal record. Never raises.
+
+        The line is serialized BEFORE acquiring the lock; the lock covers
+        only the day-rollover open (when needed), one ``os.write`` of the
+        full line, and the byte-counter increment. The first I/O failure
+        latches a disabled circuit breaker: it warns once and every
+        subsequent emit is a no-op. ``text``/``audio``/``prompt`` fields are
+        silently dropped (signals only).
+        """
+        try:
+            if self._disabled:
+                return
+            for key in _FORBIDDEN_FIELDS:
+                fields.pop(key, None)
+            now = self._now()
+            # Serialize before opening/locking: a failure here must not leave
+            # a partial line in the file for the next record to append onto.
+            record = {"ts": now.isoformat(), "proc": self._proc, "type": type, **fields}
+            data = (json.dumps(record, ensure_ascii=False) + "\n").encode("utf-8")
+            with self._lock:
+                if self._disabled:
+                    return
+                today = now.strftime("%Y-%m-%d")
+                if today != self._date:
+                    self._open_day(today)
+                if self._bytes + len(data) > self._max:
+                    if not self._cap_warned:
+                        self._cap_warned = True
+                        logger.warning("debug log cap reached for %s; skipping", today)
+                    return
+                assert self._fd is not None
+                os.write(self._fd, data)
+                self._bytes += len(data)
+        except Exception as e:  # noqa: BLE001 - debug logging must never break callers
+            with self._lock:
+                self._disabled = True
+                if self._fd is not None:
+                    try:
+                        os.close(self._fd)
+                    except OSError:
+                        pass
+                    self._fd = None
+            logger.warning("debug log disabled after write failure: %s", e)
+
+    def close(self):
+        """Clean up (no-op; per-day fds are reclaimed on rollover/exit)."""

+ 112 - 10
src/stt_daemon/supervisor.py

@@ -54,6 +54,7 @@ class Supervisor:
         wait_timeout_s: float = 15.0,
         grace_s: float = 5.0,
         poll_s: float = 0.1,
+        debug_sink_factory=None,
     ):
         """Initialize the supervisor.
 
@@ -70,6 +71,9 @@ class Supervisor:
             wait_timeout_s: Timeout for waiting on assistant control socket.
             grace_s: Time to wait for children to exit after terminate().
             poll_s: Time between child poll() checks.
+            debug_sink_factory: Override debug signal sink creation (used by
+                tests). Buffering is in-memory; real I/O only happens in
+                _flush_debug, after _terminate_all has reaped children.
         """
         self.cfg = cfg
         self.with_tui = with_tui
@@ -80,6 +84,9 @@ class Supervisor:
         self.poll_s = poll_s
         self.sleep = sleep
         self.clock = clock
+        self._debug_sink_factory = debug_sink_factory
+        self._debug_sink = None
+        self._debug_buffer: list = []
 
         # Default spawn wraps subprocess.Popen
         if spawn is None:
@@ -93,6 +100,65 @@ class Supervisor:
         else:
             self.spawn = spawn
 
+    def _record_debug(self, type_: str, **fields) -> None:
+        """Buffer a debug signal (pure append; safe anywhere, performs no I/O)."""
+        self._debug_buffer.append((type_, fields))
+
+    def _build_debug_sink(self):
+        """Default debug sink builder (lazy import; duck-typed by callers)."""
+        from .debug import DebugSink
+
+        debug_cfg = getattr(self.cfg, "debug", None)
+        sink_dir = getattr(debug_cfg, "dir", "debug")
+        return DebugSink(sink_dir, "supervisor")
+
+    def _flush_debug(self) -> None:
+        """Emit buffered signals, each individually wrapped; never raises.
+
+        Only call after _terminate_all has returned, so a slow or throwing
+        sink can neither orphan children nor delay terminate-to-kill.
+        """
+        if not self._debug_buffer:
+            return
+        pending, self._debug_buffer = self._debug_buffer, []
+        debug_cfg = getattr(self.cfg, "debug", None)
+        if debug_cfg is None or not getattr(debug_cfg, "enabled", False):
+            return
+        try:
+            if self._debug_sink is None:
+                factory = self._debug_sink_factory or self._build_debug_sink
+                self._debug_sink = factory()
+            sink = self._debug_sink
+        except Exception as e:  # noqa: BLE001 -- debug must not break shutdown
+            logger.warning("Debug sink disabled (%s)", e)
+            self._debug_sink = None
+            return
+        if sink is None:
+            return
+        for type_, fields in pending:
+            try:
+                sink.emit(type_, **fields)
+            except Exception as e:  # noqa: BLE001 -- one bad line drops, rest flow
+                logger.warning("Debug sink emit failed (%s)", e)
+
+    def _terminate_and_flush(self, procs: list) -> None:
+        """Reap children, then record the shutdown summary and flush debug I/O."""
+        alive_before = [proc for proc in procs if proc.poll() is None]
+        start = self.clock()
+        self._terminate_all(procs)
+        elapsed = self.clock() - start
+        if alive_before and elapsed >= self.grace_s:
+            self._record_debug(
+                "kill_escalation",
+                pids=[getattr(proc, "pid", None) for proc in alive_before],
+                grace_s=self.grace_s,
+            )
+        self._record_debug(
+            "shutdown",
+            codes=[proc.poll() for proc in procs],
+        )
+        self._flush_debug()
+
     def run(self) -> int:
         """Run the supervisor until a child exits or a signal is received.
 
@@ -112,11 +178,12 @@ class Supervisor:
         try:
             daemon_proc = self._spawn_daemon()
             if daemon_proc is None:
+                self._flush_debug()
                 return 1
 
             assistant_proc = self._spawn_assistant()
             if assistant_proc is None:
-                self._terminate_all([daemon_proc])
+                self._terminate_and_flush([daemon_proc])
                 return 1
 
             tui_proc = None
@@ -131,12 +198,12 @@ class Supervisor:
                             child_exit_code = code
                             break
                     logger.error("Failed to bring up TUI; terminating all children")
-                    self._terminate_all([daemon_proc, assistant_proc])
+                    self._terminate_and_flush([daemon_proc, assistant_proc])
                     return child_exit_code if child_exit_code is not None else 1
                 tui_proc = self._spawn_tui()
                 if tui_proc is None:
                     logger.error("Failed to spawn TUI; terminating all children")
-                    self._terminate_all([daemon_proc, assistant_proc])
+                    self._terminate_and_flush([daemon_proc, assistant_proc])
                     return 1
 
             procs = [daemon_proc, assistant_proc]
@@ -153,14 +220,19 @@ class Supervisor:
                             "Child process exited with code %d; shutting down", code
                         )
                         first_exit_code = code
+                        self._record_debug(
+                            "child_exit",
+                            pid=getattr(proc, "pid", None),
+                            code=code,
+                        )
                         stop.set()
                         break
 
                 if not stop.is_set():
                     self.sleep(self.poll_s)
 
-            # Shutdown all remaining children
-            self._terminate_all(procs)
+            # Shutdown all remaining children (debug I/O only after reaping)
+            self._terminate_and_flush(procs)
 
             # Return first child's exit code or 0 for signal-driven shutdown
             return first_exit_code if first_exit_code is not None else 0
@@ -201,9 +273,14 @@ class Supervisor:
             argv = [sys.executable, "-m", "stt_daemon", "run"]
             self._add_common_flags(argv)
             logger.info("Spawning daemon: %s", " ".join(argv))
-            return self._spawn_with_output(argv)
+            proc = self._spawn_with_output(argv)
+            self._record_debug(
+                "spawn", child="daemon", argv=list(argv), pid=getattr(proc, "pid", None)
+            )
+            return proc
         except Exception as e:
             logger.error("Failed to spawn daemon: %s", e)
+            self._record_debug("spawn_fail", child="daemon", error=str(e))
             return None
 
     def _spawn_assistant(self) -> subprocess.Popen | None:
@@ -215,9 +292,17 @@ class Supervisor:
             if self.verbose:
                 argv.append("-v")
             logger.info("Spawning assistant: %s", " ".join(argv))
-            return self._spawn_with_output(argv)
+            proc = self._spawn_with_output(argv)
+            self._record_debug(
+                "spawn",
+                child="assistant",
+                argv=list(argv),
+                pid=getattr(proc, "pid", None),
+            )
+            return proc
         except Exception as e:
             logger.error("Failed to spawn assistant: %s", e)
+            self._record_debug("spawn_fail", child="assistant", error=str(e))
             return None
 
     def _spawn_tui(self) -> subprocess.Popen | None:
@@ -227,9 +312,14 @@ class Supervisor:
             if "config" in self.overrides:
                 argv.extend(["--config", self.overrides["config"]])
             logger.info("Spawning TUI: %s", " ".join(argv))
-            return self._spawn_with_output(argv, redirect_output=False)
+            proc = self._spawn_with_output(argv, redirect_output=False)
+            self._record_debug(
+                "spawn", child="tui", argv=list(argv), pid=getattr(proc, "pid", None)
+            )
+            return proc
         except Exception as e:
             logger.error("Failed to spawn TUI: %s", e)
+            self._record_debug("spawn_fail", child="tui", error=str(e))
             return None
 
     def _add_common_flags(self, argv: list[str]) -> None:
@@ -276,8 +366,14 @@ class Supervisor:
 
             # Check if any child exited early
             for proc in procs:
-                if proc.poll() is not None:
+                code = proc.poll()
+                if code is not None:
                     logger.warning("Child exited during control socket wait")
+                    self._record_debug(
+                        "child_early_exit",
+                        pid=getattr(proc, "pid", None),
+                        code=code,
+                    )
                     return False
 
             if stop.is_set():
@@ -287,6 +383,7 @@ class Supervisor:
             self.sleep(self.poll_s)
 
         logger.warning("Timeout waiting for control socket: %s", assistant_socket)
+        self._record_debug("socket_timeout", socket=str(assistant_socket))
         return False
 
     def _terminate_all(self, procs: list) -> None:
@@ -324,9 +421,14 @@ def run_up(
     with_tui: bool = False,
     overrides: dict[str, str] | None = None,
     verbose: bool = False,
+    debug_sink_factory=None,
 ) -> int:
     """Convenience function to run the supervisor."""
     supervisor = Supervisor(
-        cfg, with_tui=with_tui, overrides=overrides, verbose=verbose
+        cfg,
+        with_tui=with_tui,
+        overrides=overrides,
+        verbose=verbose,
+        debug_sink_factory=debug_sink_factory,
     )
     return supervisor.run()

+ 90 - 0
tests/test_config.py

@@ -956,3 +956,93 @@ provider = "bogus"
     assert cfg.speaker.provider == "cpu"
     assert "Unknown speaker provider" in caplog.text
     assert "bogus" in caplog.text
+
+
+def test_debug_defaults():
+    """The [debug] section defaults to disabled with signal-log defaults."""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        import os
+        old_cwd = os.getcwd()
+        try:
+            os.chdir(tmpdir)
+            cfg = load_config()
+        finally:
+            os.chdir(old_cwd)
+
+    assert cfg.debug.enabled is False
+    assert cfg.debug.dir == "debug"
+    assert cfg.debug.heartbeat is True
+    assert cfg.debug.max_bytes_per_day == 50000000
+
+
+def test_debug_config_fields_are_parsed():
+    """Every [debug] key is applied from TOML."""
+    config_toml = """
+[debug]
+enabled = true
+dir = "dbg"
+heartbeat = false
+max_bytes_per_day = 1000
+"""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        config_path = Path(tmpdir) / "config.toml"
+        config_path.write_text(config_toml)
+
+        cfg = load_config(str(config_path))
+
+    assert cfg.debug.enabled is True
+    assert cfg.debug.dir == "dbg"
+    assert cfg.debug.heartbeat is False
+    assert cfg.debug.max_bytes_per_day == 1000
+
+
+def test_debug_section_not_warned(caplog):
+    """An explicit [debug] section loads without an unknown-section warning."""
+    config_toml = """
+[debug]
+enabled = false
+"""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        config_path = Path(tmpdir) / "config.toml"
+        config_path.write_text(config_toml)
+
+        with caplog.at_level("WARNING", logger="stt_daemon.config"):
+            cfg = load_config(str(config_path))
+
+    assert cfg.debug.enabled is False
+    assert "debug" not in caplog.text
+
+
+def test_unknown_debug_key_is_ignored():
+    """An unknown key inside [debug] is silently ignored (per-field .get)."""
+    config_toml = """
+[debug]
+enabled = true
+bogus_key = 42
+"""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        config_path = Path(tmpdir) / "config.toml"
+        config_path.write_text(config_toml)
+
+        cfg = load_config(str(config_path))
+
+    assert cfg.debug.enabled is True
+    assert not hasattr(cfg.debug, "bogus_key")
+    assert cfg.debug.dir == "debug"
+
+
+def test_unknown_top_level_still_warns(caplog):
+    """An unknown top-level section still warns after [debug] exists."""
+    config_toml = """
+[debugg]
+enabled = true
+"""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        config_path = Path(tmpdir) / "config.toml"
+        config_path.write_text(config_toml)
+
+        with caplog.at_level("WARNING", logger="stt_daemon.config"):
+            cfg = load_config(str(config_path))
+
+    assert "debugg" in caplog.text
+    assert cfg.debug.enabled is False

+ 488 - 1
tests/test_daemon_events.py

@@ -1,5 +1,6 @@
 """Tests for the daemon's publish-only event stream (Phase 1)."""
 
+import concurrent.futures
 import logging
 import os
 import shutil
@@ -7,6 +8,7 @@ import socket
 import tempfile
 import threading
 import time
+from types import SimpleNamespace
 
 import numpy as np
 import pytest
@@ -15,6 +17,8 @@ from stt_daemon import events
 from stt_daemon.bus import EventSubscriber
 from stt_daemon.config import Config
 from stt_daemon.daemon import Daemon
+from stt_daemon.segmenter import Utterance
+from stt_daemon.supervisor import Supervisor
 from stt_daemon.transcriber import TranscriptResult
 
 FRAME_SAMPLES = 480
@@ -165,7 +169,7 @@ def _base_config(sock_path):
     return cfg
 
 
-def _start_daemon(cfg, capture, vad, transcriber):
+def _start_daemon(cfg, capture, vad, transcriber, debug_sink_factory=None):
     sink = FakeSink()
     daemon = Daemon(
         cfg,
@@ -174,6 +178,7 @@ def _start_daemon(cfg, capture, vad, transcriber):
         vad_factory=lambda _aggr: vad,
         transcriber_factory=lambda: transcriber,
         sink_factory=lambda: sink,
+        debug_sink_factory=debug_sink_factory,
     )
     daemon._initialize()
     subscriber = EventSubscriber(cfg.events.socket, reconnect_delay=0.05)
@@ -387,6 +392,488 @@ def test_inbound_data_on_subscriber_socket_is_ignored(sock_path):
     daemon.stop()
 
 
+class FakeDebugSink:
+    """In-memory DebugSink stand-in (duck-typed emit/close, thread-safe)."""
+
+    def __init__(self, dir="debug", proc="daemon"):
+        self.dir = dir
+        self.proc = proc
+        self.emitted = []
+        self.closed = False
+        self._lock = threading.Lock()
+
+    def emit(self, type_, **fields):
+        with self._lock:
+            self.emitted.append({"type": type_, **fields})
+
+    def types(self):
+        with self._lock:
+            return [e["type"] for e in self.emitted]
+
+    def of(self, type_):
+        with self._lock:
+            return [e for e in self.emitted if e["type"] == type_]
+
+    def close(self):
+        self.closed = True
+
+
+class BoomDebugSink(FakeDebugSink):
+    """Debug sink whose emit always throws (total-swallow check)."""
+
+    def emit(self, type_, **fields):
+        raise RuntimeError("disk gone")
+
+
+def _enable_debug(cfg, enabled=True, heartbeat=True):
+    cfg.debug = SimpleNamespace(enabled=enabled, heartbeat=heartbeat, dir="debug")
+
+
+def _wait_until(predicate, timeout=5.0):
+    deadline = time.time() + timeout
+    while time.time() < deadline:
+        if predicate():
+            return True
+        time.sleep(0.01)
+    return predicate()
+
+
+def test_debug_sink_none_when_disabled(sock_path):
+    cfg = _base_config(sock_path)
+    daemon = Daemon(
+        cfg,
+        log_level=logging.WARNING,
+        capture_factory=lambda: FakeCapture([_frame()]),
+        vad_factory=lambda _aggr: FakeVad([]),
+        transcriber_factory=lambda: FakeTranscriber(),
+        sink_factory=lambda: FakeSink(),
+    )
+    daemon._initialize()
+    assert daemon.debug_sink is None
+    daemon.stop()
+
+
+def test_debug_heartbeat_lines_carry_health_payload(sock_path):
+    cfg = _base_config(sock_path)
+    cfg.events.heartbeat_ms = 40
+    _enable_debug(cfg)
+    debug = FakeDebugSink()
+    frames = [_frame() for _ in range(40)]
+    daemon, collector, subscriber, _ = _start_daemon(
+        cfg,
+        FakeCapture(frames, delay=0.02),
+        FakeVad([]),
+        FakeTranscriber(),
+        debug_sink_factory=lambda: debug,
+    )
+
+    assert daemon.debug_sink is debug
+    assert _wait_until(lambda: bool(debug.of("heartbeat")), timeout=5.0)
+
+    beat = debug.of("heartbeat")[0]
+    assert beat["model_loaded"] is True
+    for field in ("queue_depth", "device_ok", "worker_stall_s", "rms", "uptime_s"):
+        assert field in beat
+
+    subscriber.close()
+    daemon.stop()
+    assert debug.closed is True
+
+
+def test_debug_heartbeat_gate_off_leaves_only_anomalies(sock_path):
+    cfg = _base_config(sock_path)
+    cfg.events.heartbeat_ms = 40
+    _enable_debug(cfg, heartbeat=False)
+    debug = FakeDebugSink()
+    daemon, collector, subscriber, _ = _start_daemon(
+        cfg,
+        FakeCapture([_frame() for _ in range(20)], delay=0.02),
+        FakeVad([]),
+        FakeTranscriber(),
+        debug_sink_factory=lambda: debug,
+    )
+
+    assert _wait_until(
+        lambda: any(e["type"] == events.HEARTBEAT for e in collector.events),
+        timeout=5.0,
+    )
+    assert daemon.debug_sink is debug
+    assert debug.of("heartbeat") == []
+
+    subscriber.close()
+    daemon.stop()
+
+
+def test_debug_reopen_attempt_success_and_fail_lines(sock_path):
+    cfg = _base_config(sock_path)
+    _enable_debug(cfg)
+    debug = FakeDebugSink()
+    daemon = Daemon(
+        cfg,
+        log_level=logging.WARNING,
+        capture_factory=lambda: FakeCapture([_frame()]),
+        vad_factory=lambda _aggr: FakeVad([]),
+        transcriber_factory=lambda: FakeTranscriber(),
+        sink_factory=lambda: FakeSink(),
+        debug_sink_factory=lambda: debug,
+    )
+    daemon._initialize()
+
+    calls = []
+
+    def flaky_factory():
+        calls.append(1)
+        if len(calls) == 1:
+            raise RuntimeError("no device")
+        return FakeCapture([_frame()])
+
+    daemon._capture_factory = flaky_factory
+    assert daemon._reopen_capture(attempts=2, delay_s=0.0) is True
+    assert debug.types().count("capture_reopen_attempt") == 2
+    fails = debug.of("capture_reopen_fail")
+    assert len(fails) == 1
+    assert "no device" in fails[0]["error"]
+    assert len(debug.of("capture_reopen_ok")) == 1
+
+    def dead_factory():
+        raise RuntimeError("still gone")
+
+    daemon._capture_factory = dead_factory
+    assert daemon._reopen_capture(attempts=2, delay_s=0.0) is False
+    assert debug.types().count("capture_reopen_attempt") == 4
+    assert len(debug.of("capture_reopen_fail")) == 3
+
+    daemon.stop()
+
+
+def test_debug_capture_failure_emits_fatal_and_stops(sock_path):
+    cfg = _base_config(sock_path)
+    _enable_debug(cfg)
+    debug = FakeDebugSink()
+
+    class BoomCapture(FakeCapture):
+        def frames(self):
+            raise RuntimeError("device lost")
+            yield
+
+    daemon = Daemon(
+        cfg,
+        log_level=logging.WARNING,
+        capture_factory=lambda: BoomCapture([]),
+        vad_factory=lambda _aggr: FakeVad([]),
+        transcriber_factory=lambda: FakeTranscriber(),
+        sink_factory=lambda: FakeSink(),
+        debug_sink_factory=lambda: debug,
+    )
+    daemon._initialize()
+
+    def dead_factory():
+        raise RuntimeError("device lost")
+
+    daemon._capture_factory = dead_factory
+    daemon._start_workers()
+
+    assert daemon._stop_event.wait(timeout=10.0)
+    daemon.stop()
+    assert "capture_reopen_attempt" in debug.types()
+    assert "reader_fatal" in debug.types()
+
+
+def test_debug_clean_exhaustion_emits_no_fatal(sock_path):
+    cfg = _base_config(sock_path)
+    _enable_debug(cfg)
+    debug = FakeDebugSink()
+    daemon, collector, subscriber, _ = _start_daemon(
+        cfg,
+        FakeCapture([_frame()]),
+        FakeVad([False]),
+        FakeTranscriber(),
+        debug_sink_factory=lambda: debug,
+    )
+
+    reader = daemon._reader_thread
+    assert reader is not None
+    assert _wait_until(lambda: not reader.is_alive(), timeout=5.0)
+    assert not daemon._stop_event.is_set()
+    assert "reader_fatal" not in debug.types()
+    assert "worker_fatal" not in debug.types()
+
+    subscriber.close()
+    daemon.stop()
+
+
+def test_debug_queue_drop_and_backlog_warn(sock_path):
+    cfg = _base_config(sock_path)
+    cfg.daemon.queue_size = 2
+    cfg.daemon.backlog_warn_at = 1
+    _enable_debug(cfg)
+    debug = FakeDebugSink()
+    daemon = Daemon(
+        cfg,
+        log_level=logging.WARNING,
+        capture_factory=lambda: FakeCapture([_frame()]),
+        vad_factory=lambda _aggr: FakeVad([]),
+        transcriber_factory=lambda: FakeTranscriber(),
+        sink_factory=lambda: FakeSink(),
+        debug_sink_factory=lambda: debug,
+    )
+    daemon._initialize()
+
+    def _utt(i):
+        return Utterance(
+            audio=np.zeros(160, dtype=np.float32),
+            duration_s=0.01,
+            started_at=time.time(),
+            rms=0.0,
+            reason="end_of_turn",
+            seg_id=i,
+        )
+
+    assert daemon._enqueue(_utt(1)) is True
+    assert daemon._enqueue(_utt(2)) is True
+    assert daemon._enqueue(_utt(3)) is True  # drops seg 1
+    daemon._check_backlog()
+
+    drops = debug.of("queue_drop")
+    assert len(drops) == 1
+    assert drops[0]["dropped_seg_id"] == 1
+    warns = debug.of("backlog_warn")
+    assert len(warns) == 1
+    assert warns[0]["size"] == 2
+
+    daemon.stop()
+
+
+def test_debug_speaker_fallback_on_lane_failure(sock_path):
+    cfg = _base_config(sock_path)
+    cfg.speaker.join_timeout_s = 0.05
+    _enable_debug(cfg)
+    debug = FakeDebugSink()
+    daemon = Daemon(
+        cfg,
+        log_level=logging.WARNING,
+        capture_factory=lambda: FakeCapture([_frame()]),
+        vad_factory=lambda _aggr: FakeVad([]),
+        transcriber_factory=lambda: FakeTranscriber(),
+        sink_factory=lambda: FakeSink(),
+        debug_sink_factory=lambda: debug,
+    )
+    daemon._initialize()
+
+    failed: concurrent.futures.Future = concurrent.futures.Future()
+    failed.set_exception(RuntimeError("lane died"))
+    assert daemon._resolve_speaker(failed) == (None, "none", "?", None)
+
+    pending: concurrent.futures.Future = concurrent.futures.Future()
+    assert daemon._resolve_speaker(pending) == (None, "none", "?", None)
+
+    fallbacks = debug.of("speaker_fallback")
+    assert len(fallbacks) == 2
+    assert "lane died" in fallbacks[0]["error"]
+
+    daemon.stop()
+
+
+def test_debug_sink_closed_last_in_stop(sock_path):
+    cfg = _base_config(sock_path)
+    _enable_debug(cfg)
+    order = []
+    transcriber = FakeTranscriber()
+    transcriber.close = lambda: order.append("transcriber")
+    sink = FakeSink()
+    sink.close = lambda: order.append("sink")
+    debug = FakeDebugSink()
+    orig_debug_close = debug.close
+
+    def tracked_debug_close() -> None:
+        order.append("debug")
+        orig_debug_close()
+
+    debug.close = tracked_debug_close
+    daemon = Daemon(
+        cfg,
+        log_level=logging.WARNING,
+        capture_factory=lambda: FakeCapture([_frame()]),
+        vad_factory=lambda _aggr: FakeVad([]),
+        transcriber_factory=lambda: transcriber,
+        sink_factory=lambda: sink,
+        debug_sink_factory=lambda: debug,
+    )
+    daemon._initialize()
+    assert daemon.publisher is not None
+    orig_pub_close = daemon.publisher.close
+
+    def tracked_pub_close() -> None:
+        order.append("publisher")
+        orig_pub_close()
+
+    daemon.publisher.close = tracked_pub_close
+
+    daemon.stop()
+
+    assert order == ["transcriber", "sink", "publisher", "debug"]
+    assert debug.closed is True
+    assert daemon.debug_sink is None
+
+
+def test_debug_factory_throw_disables_sink(sock_path):
+    cfg = _base_config(sock_path)
+    _enable_debug(cfg)
+
+    def boom_factory():
+        raise RuntimeError("no dir")
+
+    daemon = Daemon(
+        cfg,
+        log_level=logging.WARNING,
+        capture_factory=lambda: FakeCapture([_frame()]),
+        vad_factory=lambda _aggr: FakeVad([]),
+        transcriber_factory=lambda: FakeTranscriber(),
+        sink_factory=lambda: FakeSink(),
+        debug_sink_factory=boom_factory,
+    )
+    daemon._initialize()
+    assert daemon.debug_sink is None
+    daemon.stop()
+
+
+class FakeProc:
+    """Minimal Popen stand-in for supervisor debug tests."""
+
+    _pid_counter = 20000
+
+    def __init__(self, exit_on_poll=None, stubborn=False):
+        self.pid = FakeProc._pid_counter
+        FakeProc._pid_counter += 1
+        self._exit_on_poll = exit_on_poll
+        self._stubborn = stubborn
+        self.returncode = None
+        self.terminated = False
+        self.killed = False
+
+    def poll(self):
+        if self.returncode is None and self._exit_on_poll is not None:
+            self.returncode = self._exit_on_poll
+        return self.returncode
+
+    def terminate(self):
+        self.terminated = True
+        if not self._stubborn and self.returncode is None:
+            self.returncode = -15
+
+    def kill(self):
+        self.killed = True
+        if self.returncode is None:
+            self.returncode = -9
+
+    def wait(self, timeout=None):
+        if self.returncode is None:
+            self.returncode = 0
+        return self.returncode
+
+
+def _supervisor_spawn(procs):
+    def fake_spawn(argv, *, redirect_output=False):
+        joined = " ".join(argv)
+        if "stt_daemon.assistant" in joined:
+            return procs["assistant"]
+        return procs["daemon"]
+
+    return fake_spawn
+
+
+def test_supervisor_debug_spawn_exit_shutdown_lines():
+    cfg = Config()
+    _enable_debug(cfg)
+    debug = FakeDebugSink(proc="supervisor")
+    procs = {"daemon": FakeProc(exit_on_poll=3), "assistant": FakeProc()}
+    supervisor = Supervisor(
+        cfg,
+        with_tui=False,
+        spawn=_supervisor_spawn(procs),
+        sleep=lambda s: None,
+        debug_sink_factory=lambda: debug,
+    )
+
+    assert supervisor.run() == 3
+    assert debug.types() == ["spawn", "spawn", "child_exit", "shutdown"]
+    assert debug.emitted[2]["code"] == 3
+    assert debug.emitted[3]["codes"] == [3, -15]
+
+
+def test_supervisor_debug_kill_escalation_line():
+    cfg = Config()
+    _enable_debug(cfg)
+    debug = FakeDebugSink(proc="supervisor")
+    daemon_proc = FakeProc(stubborn=True)
+    procs = {"daemon": daemon_proc, "assistant": FakeProc(exit_on_poll=1)}
+    supervisor = Supervisor(
+        cfg,
+        with_tui=False,
+        spawn=_supervisor_spawn(procs),
+        sleep=lambda s: None,
+        grace_s=0,
+        debug_sink_factory=lambda: debug,
+    )
+
+    assert supervisor.run() == 1
+    assert daemon_proc.killed is True
+    kills = debug.of("kill_escalation")
+    assert len(kills) == 1
+    assert kills[0]["pids"] == [daemon_proc.pid]
+
+
+def test_supervisor_debug_socket_timeout_line(tmp_path):
+    cfg = Config()
+    cfg.assistant.control_socket = str(tmp_path / "missing.sock")
+    _enable_debug(cfg)
+    debug = FakeDebugSink(proc="supervisor")
+    procs = {"daemon": FakeProc(), "assistant": FakeProc()}
+    supervisor = Supervisor(
+        cfg,
+        with_tui=True,
+        spawn=_supervisor_spawn(procs),
+        sleep=lambda s: None,
+        wait_timeout_s=0,
+        debug_sink_factory=lambda: debug,
+    )
+
+    assert supervisor.run() == 1
+    assert len(debug.of("socket_timeout")) == 1
+    assert debug.of("shutdown")
+
+
+def test_supervisor_debug_disabled_never_builds_sink():
+    cfg = Config()
+    calls = []
+    procs = {"daemon": FakeProc(exit_on_poll=2), "assistant": FakeProc()}
+    supervisor = Supervisor(
+        cfg,
+        with_tui=False,
+        spawn=_supervisor_spawn(procs),
+        sleep=lambda s: None,
+        debug_sink_factory=lambda: calls.append(1),
+    )
+
+    assert supervisor.run() == 2
+    assert calls == []
+
+
+def test_supervisor_debug_emit_throw_never_breaks_shutdown():
+    cfg = Config()
+    _enable_debug(cfg)
+    procs = {"daemon": FakeProc(exit_on_poll=4), "assistant": FakeProc()}
+    supervisor = Supervisor(
+        cfg,
+        with_tui=False,
+        spawn=_supervisor_spawn(procs),
+        sleep=lambda s: None,
+        debug_sink_factory=lambda: BoomDebugSink(proc="supervisor"),
+    )
+
+    assert supervisor.run() == 4
+
+
 def test_request_stop_shuts_down_run_loop(tmp_path):
     """request_stop() (the signal-handler path) must unwind run() cleanly."""
     sock_path = str(tmp_path / "events.sock")

+ 220 - 0
tests/test_debug.py

@@ -0,0 +1,220 @@
+"""Tests for stt_daemon.debug.DebugSink (per-file fakes, no conftest.py)."""
+
+import json
+import logging
+import os
+import threading
+from datetime import datetime
+from pathlib import Path
+
+import pytest
+
+from stt_daemon.debug import DebugSink
+
+
+def _read_lines(path):
+    with open(path, encoding="utf-8") as f:
+        return [json.loads(line) for line in f if line.strip()]
+
+
+def test_emit_writes_valid_jsonl_per_day(tmp_path):
+    sink = DebugSink(str(tmp_path / "debug"), "daemon")
+    sink.emit("heartbeat", queue_depth=0, device_ok=True)
+    sink.emit("capture_reopen", attempt=1)
+
+    day = datetime.now().strftime("%Y-%m-%d")
+    records = _read_lines(tmp_path / "debug" / f"{day}.jsonl")
+    assert len(records) == 2
+    assert records[0]["proc"] == "daemon"
+    assert records[0]["type"] == "heartbeat"
+    assert records[0]["queue_depth"] == 0
+    assert records[0]["device_ok"] is True
+    assert "ts" in records[0]
+    assert records[1]["type"] == "capture_reopen"
+
+
+def test_forbidden_fields_never_persisted(tmp_path):
+    sink = DebugSink(str(tmp_path / "debug"), "assistant")
+    sink.emit("heartbeat", text="secret transcript", audio="clip.wav", prompt="sys", rms=0.1)
+
+    day = datetime.now().strftime("%Y-%m-%d")
+    records = _read_lines(tmp_path / "debug" / f"{day}.jsonl")
+    assert len(records) == 1
+    assert "text" not in records[0]
+    assert "audio" not in records[0]
+    assert "prompt" not in records[0]
+    assert records[0]["rms"] == 0.1
+
+
+def test_invalid_proc_rejected(tmp_path):
+    with pytest.raises(ValueError):
+        DebugSink(str(tmp_path), "worker")
+
+
+@pytest.mark.parametrize("proc", ["daemon", "assistant", "supervisor"])
+def test_all_procs_accepted(tmp_path, proc):
+    sink = DebugSink(str(tmp_path / proc), proc)
+    sink.emit("heartbeat")
+    day = datetime.now().strftime("%Y-%m-%d")
+    records = _read_lines(tmp_path / proc / f"{day}.jsonl")
+    assert records[0]["proc"] == proc
+
+
+def test_disabled_path_creates_no_files(tmp_path):
+    # Disabled path = caller holds None instead of a sink (zero overhead).
+    debug = None
+    target = tmp_path / "debug"
+    if debug is not None:
+        debug.emit("heartbeat")
+    assert not target.exists()
+
+
+def test_heartbeat_gating_respected_by_caller_contract(tmp_path):
+    # Gating lives with the caller: heartbeat lines are skipped unless enabled,
+    # anomaly lines always flow.
+    class FakeCaller:
+        def __init__(self, sink, heartbeat):
+            self.sink = sink
+            self.heartbeat = heartbeat
+
+        def tick(self, is_heartbeat, type, **fields):
+            if is_heartbeat and not self.heartbeat:
+                return
+            self.sink.emit(type, **fields)
+
+    sink = DebugSink(str(tmp_path / "debug"), "daemon")
+    caller = FakeCaller(sink, heartbeat=False)
+    caller.tick(True, "heartbeat", queue_depth=0)
+    caller.tick(False, "capture_reopen", attempt=1)
+
+    day = datetime.now().strftime("%Y-%m-%d")
+    records = _read_lines(tmp_path / "debug" / f"{day}.jsonl")
+    assert [r["type"] for r in records] == ["capture_reopen"]
+
+    caller.heartbeat = True
+    caller.tick(True, "heartbeat", queue_depth=0)
+    records = _read_lines(tmp_path / "debug" / f"{day}.jsonl")
+    assert [r["type"] for r in records] == ["capture_reopen", "heartbeat"]
+
+
+def test_mkdir_failure_disables_without_raising(tmp_path, caplog):
+    blocker = tmp_path / "blocker"
+    blocker.write_text("not a dir")
+    with caplog.at_level(logging.WARNING):
+        sink = DebugSink(str(blocker / "debug"), "daemon")
+    assert sink._disabled
+    assert "mkdir failed" in caplog.text
+    sink.emit("heartbeat")  # no-op, must not raise
+
+
+def test_open_failure_warns_not_raises_and_breaker_latches(tmp_path, caplog, monkeypatch):
+    sink = DebugSink(str(tmp_path / "debug"), "daemon")
+
+    def _boom(path, flags, mode=0o644):
+        raise OSError("disk gone")
+
+    monkeypatch.setattr(os, "open", _boom)
+    with caplog.at_level(logging.WARNING):
+        sink.emit("heartbeat", queue_depth=1)
+    assert "write failure" in caplog.text
+
+    # Breaker latched: even with os.open restored, emits are silent no-ops.
+    monkeypatch.undo()
+    caplog.clear()
+    with caplog.at_level(logging.WARNING):
+        sink.emit("heartbeat", queue_depth=2)
+    assert caplog.text == ""
+    day = datetime.now().strftime("%Y-%m-%d")
+    assert not (tmp_path / "debug" / f"{day}.jsonl").exists()
+
+
+def test_write_failure_warns_once_then_latches(tmp_path, caplog, monkeypatch):
+    sink = DebugSink(str(tmp_path / "debug"), "daemon")
+    sink.emit("heartbeat", seq=0)
+
+    calls = {"n": 0}
+    real_write = os.write
+
+    def _flaky(fd, data):
+        calls["n"] += 1
+        if calls["n"] == 1:
+            raise OSError("transient")
+        return real_write(fd, data)
+
+    monkeypatch.setattr(os, "write", _flaky)
+    with caplog.at_level(logging.WARNING):
+        sink.emit("heartbeat", seq=1)
+    assert "write failure" in caplog.text
+
+    monkeypatch.undo()
+    caplog.clear()
+    with caplog.at_level(logging.WARNING):
+        sink.emit("heartbeat", seq=2)
+    assert caplog.text == ""
+
+    day = datetime.now().strftime("%Y-%m-%d")
+    records = _read_lines(tmp_path / "debug" / f"{day}.jsonl")
+    assert [r.get("seq") for r in records] == [0]
+
+
+def test_concurrent_emits_produce_no_partial_lines(tmp_path):
+    sink = DebugSink(str(tmp_path / "debug"), "daemon")
+    threads = 8
+    per_thread = 100
+
+    def _work(tid):
+        for i in range(per_thread):
+            sink.emit("heartbeat", tid=tid, seq=i, pad="x" * 200)
+
+    workers = [threading.Thread(target=_work, args=(t,)) for t in range(threads)]
+    for w in workers:
+        w.start()
+    for w in workers:
+        w.join()
+
+    day = datetime.now().strftime("%Y-%m-%d")
+    path = tmp_path / "debug" / f"{day}.jsonl"
+    with open(path, encoding="utf-8") as f:
+        raw = f.readlines()
+    assert len(raw) == threads * per_thread
+    seen = set()
+    for line in raw:
+        record = json.loads(line)  # raises on any partial/interleaved line
+        seen.add((record["tid"], record["seq"]))
+    assert len(seen) == threads * per_thread
+
+
+def test_cap_skips_past_limit_with_single_warning(tmp_path, caplog):
+    sink = DebugSink(str(tmp_path / "debug"), "daemon", max_bytes_per_day=200)
+    with caplog.at_level(logging.WARNING):
+        for i in range(20):
+            sink.emit("heartbeat", seq=i, pad="y" * 50)
+    warnings = [r for r in caplog.records if "cap reached" in r.message]
+    assert len(warnings) == 1
+
+    day = datetime.now().strftime("%Y-%m-%d")
+    size = Path(tmp_path / "debug" / f"{day}.jsonl").stat().st_size
+    assert size <= 200
+
+
+def test_cap_counter_resets_on_rollover(tmp_path):
+    clock = {"now": datetime(2026, 9, 29, 23, 59, 59)}
+    sink = DebugSink(str(tmp_path / "debug"), "daemon", max_bytes_per_day=200,
+                     now_fn=lambda: clock["now"])
+    for i in range(20):  # exhaust day one's cap
+        sink.emit("heartbeat", seq=i, pad="z" * 50)
+
+    clock["now"] = datetime(2026, 9, 30, 0, 0, 1)
+    sink.emit("heartbeat", seq="next-day")
+    records = _read_lines(tmp_path / "debug" / "2026-09-30.jsonl")
+    assert [r.get("seq") for r in records] == ["next-day"]
+
+
+def test_close_is_noop(tmp_path):
+    sink = DebugSink(str(tmp_path / "debug"), "supervisor")
+    sink.emit("spawn", pid=123)
+    sink.close()
+    sink.emit("exit", code=0)
+    day = datetime.now().strftime("%Y-%m-%d")
+    records = _read_lines(tmp_path / "debug" / f"{day}.jsonl")
+    assert [r["type"] for r in records] == ["spawn", "exit"]