STT-Assistant/docs/DEVELOPMENT.md

Development

Contributor reference: how to run the tests, where the code lives, how to lint and build, and the limits the design accepts on purpose.

Testing

Run the test suite (no audio hardware required):

uv run pytest -v

Tests cover:

• Pre-roll buffer preservation: confirm the first frames are not cut
• Silence finalization: utterances end after N silence frames
• Min/max utterance duration: blips are discarded, long utterances are finalized
• Pause-aligned chunking: long speech splits at pauses (end_reason = "pause")
• Config loading and language normalization
• Provisioning: setup/ensure/teardown, orphan-name cleanup, and clear idempotency
• Event bus: envelope codec, ring replay, subscriber fan-out
• Assistant: mode machine, command readiness/timeouts, tool schema validation, LLM client/history trimming, audit log
• TUI: pure view-state reducer and control-socket client

Code structure

src/stt_daemon/
├── __init__.py          - Package metadata
├── __main__.py          - Entry point for `python -m stt_daemon`
├── cli.py               - Argument parsing and entry point (`stt-daemon`)
├── ctl.py               - One-shot control client (`stt-ctl`, hotkeys)
├── config.py            - Config loading (TOML)
├── audio.py             - Audio capture (parec, sounddevice)
├── pulse.py             - PulseAudio/pactl wrapper (devices, modules, sink checks)
├── provisioner.py       - Virtual sink + loopback lifecycle (setup/ensure/teardown)
├── state.py             - Provisioning state file (audio/audio.json)
├── picker.py            - Interactive device picker (questionary / numbered menu)
├── segmenter.py         - VAD + pre-roll buffer + utterance segmentation (core)
├── transcriber.py       - Faster-Whisper wrapper
├── output.py            - Sinks (stdout, JSONL, optional WAV)
├── daemon.py            - Orchestrator (threads, queues, event emission)
├── events.py            - NDJSON event envelope + codec
├── bus.py               - Publish-only event bus (socket, ring replay)
├── assistant/
│   ├── modes.py         - Mode state machine (idle/command_armed/executing/chat)
│   ├── service.py       - Modes, command buffer, readiness, timeouts
│   ├── control.py       - Control socket (viewers, commands, broadcast)
│   ├── executor.py      - NullExecutor / LlmExecutor (tool loop)
│   ├── tools.py         - Tool registry + schema validation + get_datetime
│   ├── llm.py           - llama.cpp client + chat session (block trimming)
│   ├── audit.py         - Daily action audit log
│   ├── runner.py        - `stt-assistant` process
│   └── __main__.py      - Entry point for `python -m stt_daemon.assistant`
└── tui/
    ├── state.py         - Pure view-state reducer
    ├── client.py        - Control-socket client
    └── app.py           - Textual app + `stt-tui` entry point

tests/                    - 359 tests, no audio hardware or live LLM required

scripts/
├── audio-sink.sh        - Thin wrapper over `stt-daemon setup|clear|list`
└── hotkeys/             - sxhkd / xbindkeys examples

Running linters

uv run ruff check .

Building/packaging

# Build sdist + wheel (hatchling backend, installed in an isolated env by uv)
uv build

Limitations & Design Decisions

1. Mono only: Stereo support can be added, but speech is typically mono in transcription pipelines
2. Single language per transcription: Whisper detects one language per utterance; mixed-language speech is attributed to the dominant language
3. LLM is optional and separate: the daemon itself is transcription-only; the stt-assistant process adds chat/commands/tools on top, and can be disabled with [llm].enabled = false
4. PulseAudio/PipeWire-Pulse: The default parec backend assumes UNIX-like systems with PulseAudio. macOS/Windows should use sounddevice
5. Real-time: Latency depends on frame duration (30ms), silence threshold (default 700ms), and model size. Expect ~1–3 second latency with small model on CPU
6. Thread-safe queues: Reader thread (audio) is separate from worker thread (transcription) so audio never blocks waiting for transcription