STT-Assistant/docs/DEVELOPMENT.md

Development

Contributor reference: how to run the tests, where the code lives, how to lint and build, and the limits the design accepts on purpose.

Testing

Run the test suite (no audio hardware required):

uv run pytest -v

Tests cover:

• Pre-roll buffer preservation: confirm the first frames are not cut
• Silence finalization: utterances end after N silence frames
• Min/max utterance duration: blips are discarded, long utterances are finalized
• Pause-aligned chunking: long speech splits at pauses (end_reason = "pause")
• Config loading and language normalization
• Provisioning: setup/ensure/teardown, orphan-name cleanup, and clear idempotency
• Event bus: envelope codec, ring replay, subscriber fan-out
• Assistant: mode machine, command readiness/timeouts, tool schema validation, LLM client/history trimming, audit log, and the chat_capture/triage_chat capture gating
• Triage & brief: Laya lane, questions-file validation, signal normalization, the fail-open substantive_update gate, the periodic periodic_s accumulating-speech trigger, single-outstanding brief-update coalescing, the code-owned append-only FactLedger (dedupe/eviction/render), the stateless brief engine, and the durable brief file
• Speakers: prototype matching, mint/confirm/adapt guards, daemon wiring, assistant roster and naming, offline pre-enrollment (tests/test_speaker_enroll.py)
• Debug: DebugSink JSONL lines, disabled-means-None, emit-throw safety, per-day byte cap, concurrent-emit integrity (tests/test_debug.py); assistant lock-free emits via debug_sink_factory (tests/test_assistant_service.py)
• TUI: pure view-state reducer and control-socket client

Code structure

src/stt_daemon/
├── __init__.py          - Package metadata
├── __main__.py          - Entry point for `python -m stt_daemon`
├── cli.py               - Argument parsing and entry point (`stt-daemon`)
├── ctl.py               - One-shot control client (`stt-ctl`, hotkeys)
├── config.py            - Config loading (TOML)
├── audio.py             - Audio capture (parec, sounddevice)
├── pulse.py             - PulseAudio/pactl wrapper (devices, modules, sink checks)
├── provisioner.py       - Virtual sink + loopback lifecycle (setup/ensure/teardown)
├── state.py             - Provisioning state file (audio.json)
├── picker.py            - Interactive device picker (questionary / numbered menu)
├── segmenter.py         - VAD + pre-roll buffer + utterance segmentation (core)
├── transcriber.py       - Faster-Whisper wrapper
├── speakers.py          - Speaker prototype store, matcher, mint/adapt guards
├── embedder.py          - sherpa-onnx embedding extractor (lazy import, build_embedder)
├── enroll.py            - Offline pre-enrollment from WAV files (`speaker-enroll`)
├── enroll_assist.py     - Assisted seed-based enrollment review (`speaker-enroll-seed`)
├── evaluate.py          - Labeled-clip accuracy scorer (`speaker-eval`)
├── output.py            - Sinks (stdout, JSONL, optional WAV)
├── daemon.py            - Orchestrator (threads, queues, event emission)
├── events.py            - NDJSON event envelope + codec
├── bus.py               - Publish-only event bus (socket, ring replay)
├── debug.py             - Structured health-signal log (`DebugSink`, `troubleshoot/debug/YYYY-MM-DD.jsonl`)
├── assistant/
│   ├── modes.py         - Mode state machine (idle/command_armed/executing/chat)
│   ├── service.py       - Modes, command buffer, readiness, timeouts
│   ├── control.py       - Control socket (viewers, commands, broadcast)
│   ├── executor.py      - NullExecutor / LlmExecutor (tool loop)
│   ├── tools.py         - Tool registry + schema validation + get_datetime
│   ├── llm.py           - llama.cpp client, chat session (block trimming), stateless brief engine
│   ├── inference.py     - Single-lane LLM queue (chat/command/brief-update)
│   ├── triage.py        - Triage analyzer (normalize, flag, ledger prompt/parse) + code-owned FactLedger
│   ├── laya_lane.py     - In-process Laya lane (loader + single-flight worker)
│   ├── triage_dump.py   - Opt-in per-triage I/O dump (`dump_io`)
│   ├── brief_file.py    - Opt-in durable brief snapshot writer (`[brief]`)
│   ├── audit.py         - Daily action audit log
│   ├── runner.py        - `stt-assistant` process
│   └── __main__.py      - Entry point for `python -m stt_daemon.assistant`
└── tui/
    ├── state.py         - Pure view-state reducer
    ├── client.py        - Control-socket client
    └── app.py           - Textual app + `stt-tui` entry point

tests/                    - 1028 tests, no audio hardware or live LLM required

scripts/
├── audio-sink.sh        - Thin wrapper over `stt-daemon setup|clear|list`
└── hotkeys/             - sxhkd / xbindkeys examples

Running linters

uv run ruff check .

Building/packaging

# Build sdist + wheel (hatchling backend, installed in an isolated env by uv)
uv build

Limitations & Design Decisions

1. Mono only: Stereo support can be added, but speech is typically mono in transcription pipelines
2. Single language per transcription: Whisper detects one language per utterance; mixed-language speech is attributed to the dominant language
3. LLM is optional and separate: the daemon itself is transcription-only; the stt-assistant process adds chat/commands/tools on top, and can be disabled with [llm].enabled = false
4. PulseAudio/PipeWire-Pulse: The default parec backend assumes UNIX-like systems with PulseAudio. macOS/Windows should use sounddevice
5. Real-time: Latency depends on frame duration (30ms), silence threshold (default 700ms), and model size. Expect ~1–3 second latency with small model on CPU
6. Thread-safe queues: Reader thread (audio) is separate from worker thread (transcription) so audio never blocks waiting for transcription