STT-Assistant/AGENTS.md

AGENTS.md

Always-on speech-to-text daemon plus a separate voice-assistant layer. Python 3.11+ (dev env is 3.13, but requires-python = ">=3.11" and ruff targets py311 — don't use 3.12+ syntax), uv, src/ layout. stt_daemon/ is the only package; console scripts live in pyproject.toml (stt-daemon, stt-assistant, stt-ctl, stt-tui).

Commands

uv sync                                                  # install into .venv (see pywhispercpp below)
uv run pytest -v                                         # full suite (hardware- and network-free)
uv run pytest tests/test_segmenter.py::test_name -v      # single test
uv run ruff check .                                      # only static check (no mypy/pyright)
uv build                                                 # sdist + wheel (hatchling)

uv run stt-daemon up --with-tui                          # supervise all three processes (main launcher)

There is no CI, pre-commit, Makefile, or task runner. Run ruff + pytest manually before finishing.

Process architecture (don't break this)

Three processes wired by UNIX sockets under $XDG_RUNTIME_DIR (fallback /tmp):

• stt-daemon (cli.py) — capture → pre-roll/VAD → segment → transcribe → stdout + JSONL. Publish-only on stt-daemon.sock: it never accepts inbound messages; lifecycle is signals only.
• stt-assistant (assistant/runner.py) — subscribes to that bus, owns all state (modes, command buffer, chat history, LLM, tools, audit), serves stt-assistant.sock.
• stt-tui / stt-ctl — clients of the assistant control socket only.

stt-daemon up [--with-tui] (supervisor.py) launches these as child processes. With --with-tui, TUI startup is mandatory (no silent headless fallback: socket timeout or early child exit kills all), and the supervisor restores the tty's ISIG on exit so a killed TUI doesn't leave Ctrl+C dead.

Stateful logic belongs in the assistant. Two single-worker-thread lanes exist by design — do not parallelize either: transcription (faster-whisper isn't thread-safe) and all LLM calls (assistant/inference.py InferenceQueue; commands submit with priority -1 to preempt chat).

stt-daemon with no subcommand runs the daemon (legacy --check / --list-sources still work).

Config

• Loaded from config.toml in the current working directory only (no env var, no XDG lookup); --config overrides. A missing file means dataclass defaults.
• config.example.toml is the shipped template (values only); docs/CONFIGURATION.md is the annotated reference. Keep the dataclass, example, and docs in sync when adding a key.
• Mapping is manual — no pydantic/dynaconf. Adding a key means editing the dataclass and load_config() in config.py. Unknown top-level sections warn and are ignored; unknown keys inside a known section are silently ignored (dict.get per field).
• Env vars: STT_DAEMON_STATE (provisioning state; default audio/audio.json in the cwd), XDG_RUNTIME_DIR (sockets), HF_HOME (Whisper cache, honored by faster-whisper/huggingface).
• Never treat the live config.toml as a source of defaults; it is gitignored user state.

Transcription backends

• [whisper].backend picks faster-whisper (default) or pywhispercpp through build_transcriber() in transcriber.py. device/compute_type apply to faster-whisper only; pywhispercpp ignores them (GPU comes from the built wheel, quantization from the ggml model).
• pywhispercpp is a local, uncommitted CUDA build: vendor/.gitignore is *, the cp313-only wheel is not tracked, and [tool.uv.sources] points at that path. A fresh clone on Python 3.13 fails uv sync until the wheel is rebuilt — see docs/PYWHISPERCPP.md. Never delete vendor/.

Testing

• tests/ has 359 tests, all hardware- and network-free. Preserve that: inject fakes instead of touching real pactl, audio devices, or an LLM.
• Key seams: Daemon(...) accepts capture_factory, vad_factory, transcriber_factory, sink_factory; PulseAudio takes an injectable runner; AssistantService takes time_fn, executor, audit.
• No conftest.py — per-file fakes are the convention. Tests are the behavioral spec; read them before changing behavior.

Conventions & gotchas

• Never commit gitignored runtime output: config.toml, transcripts/, audio/, audit/, .omo/.
• New assistant tools: register in assistant/tools.py. Model-supplied arguments are JSON-parsed and schema-validated before running; set side_effect and requires_confirmation for anything non-read-only.
• Whisper models must be multilingual — do not use .en variants (defaults auto-detect es/en).
• docs/ holds the full references (config, CLI, audio setup, assistant, output, troubleshooting, development); .omo/plans/ holds local design plans (gitignored).