Home Esplora Aiuto
Accedi
efren
/
STT-Assistant
1
0
Forka 0
File Problemi 0 Pull Requests 0 Wiki
Albero (Tree): 4dd96d5181
Rami (Branch) Tag
STT-Assistant
/
AGENTS.md

AGENTS.md 7.2 KB
Cronologia Originale

AGENTS.md

Always-on speech-to-text daemon plus a separate voice-assistant layer. Python 3.11+ (dev env is 3.13, but requires-python = ">=3.11" and ruff targets py311 — don't use 3.12+ syntax), uv, src/ layout. stt_daemon/ is the only package; console scripts live in pyproject.toml (stt-daemon, stt-assistant, stt-ctl, stt-tui).

Commands

uv sync                                                  # install into .venv (see pywhispercpp below)
uv sync --extra portaudio                                # sounddevice capture backend only
uv sync --extra triage                                   # optional: in-process Laya triage lane
uv sync --extra speaker                                  # optional: in-process speaker embedding lane
# extras are exclusive under uv sync: pass both flags (--extra speaker --extra triage) to keep both
uv run pytest -v                                         # full suite (hardware- and network-free)
uv run pytest tests/test_segmenter.py::test_name -v      # single test
uv run ruff check .                                      # only static check: E,F,W, line-length 100, no typechecker
uv build                                                 # sdist + wheel (hatchling)

uv run stt-daemon up --with-tui                          # supervise all three processes (main launcher)
uv run stt-daemon setup && uv run stt-daemon run         # manual flow without supervisor

There is no CI, pre-commit, Makefile, or task runner. Run ruff + pytest manually before finishing.

Process architecture (don't break this)

Three processes wired by UNIX sockets under $XDG_RUNTIME_DIR (fallback /tmp):

  • stt-daemon (cli.py) — capture → pre-roll/VAD → segment → transcribe → stdout + JSONL. Publish-only on stt-daemon.sock: it never accepts inbound messages; lifecycle is signals only.
  • stt-assistant (assistant/runner.py) — subscribes to that bus, owns all state (modes, command buffer, chat history, LLM, tools, audit), serves stt-assistant.sock.
  • stt-tui / stt-ctl — clients of the assistant control socket only.

stt-daemon up [--with-tui] (supervisor.py) launches these as child processes. With --with-tui, TUI startup is mandatory (no silent headless fallback: socket timeout or early child exit kills all), and the supervisor restores the tty's ISIG on exit so a killed TUI doesn't leave Ctrl+C dead.

Stateful logic belongs in the assistant (the speaker embedding lane is the daemon-side exception: it owns the prototype store). Four single-worker-thread lanes exist by design, and none may be parallelized: transcription (faster-whisper isn't thread-safe), the speaker embedding lane (in-process in stt-daemon: src/stt_daemon/speakers.py + src/stt_daemon/embedder.py, owns the SpeakerStore, drains the rename inbox), all LLM calls (assistant/inference.py InferenceQueue; commands submit with priority -1 to preempt chat), and the Laya triage lane (assistant/laya_lane.py: one background thread plus one loader, running in-process in the assistant). Neither the speaker embedding lane nor the Laya lane is a process or a socket, and neither changes supervisor spawn counts (2 without the TUI, 3 with it); the daemon socket stays publish-only.

stt-daemon with no subcommand runs the daemon (legacy --check / --list-sources still work).

Config

  • Loaded from config.toml in the current working directory only (no env var, no XDG lookup); --config overrides. A missing file means dataclass defaults.
  • config.example.toml is the shipped template (values only); docs/CONFIGURATION.md is the annotated reference. Keep the dataclass, example, and docs in sync when adding a key.
  • Mapping is manual — no pydantic/dynaconf. Adding a key means editing the dataclass and load_config() in config.py. Unknown top-level sections warn and are ignored; unknown keys inside a known section are silently ignored (dict.get per field).
  • Env vars: STT_DAEMON_STATE (provisioning state; default audio/audio.json in the cwd), XDG_RUNTIME_DIR (sockets), HF_HOME (Whisper cache, honored by faster-whisper/huggingface).
  • [triage] is the display-only signal panel, off by default (enabled = false). Its questions_file (questions.json in the cwd) is private user state and is gitignored, like config.toml.
  • [speaker] is opt-in per-utterance speaker identification, off by default, backed by the SpeakerConfig dataclass (12th known section; 14 keys: enabled, model_path, provider, threshold_id, threshold_new, min_enroll_s, rms_floor, margin, mint_cooldown_s, confirm_s, max_prototypes, provisional_ttl_h, store_dir, join_timeout_s). Needs uv sync --extra speaker plus a model under models/. Full reference: docs/SPEAKERS.md.
  • Never treat the live config.toml as a source of defaults; it is gitignored user state.

Transcription backends

  • [whisper].backend picks faster-whisper (default) or pywhispercpp through build_transcriber() in transcriber.py. device/compute_type apply to faster-whisper only; pywhispercpp ignores them (GPU comes from the built wheel, quantization from the ggml model).
  • pywhispercpp is a local, uncommitted CUDA build: vendor/.gitignore is *, the cp313-only wheel is not tracked, and [tool.uv.sources] points at that path. A fresh clone on Python 3.13 fails uv sync until the wheel is rebuilt — see docs/PYWHISPERCPP.md. Never delete vendor/.

Testing

  • tests/ has 701 tests, all hardware- and network-free. Preserve that: inject fakes instead of touching real pactl, audio devices, an LLM, or sherpa_onnx (the speaker lane uses a fake embedder; no ONNX in tests).
  • Key seams: Daemon(...) accepts capture_factory, vad_factory, transcriber_factory, embedder_factory, sink_factory; PulseAudio takes an injectable runner; AssistantService takes time_fn, executor, audit.
  • Speaker coverage lives in tests/test_speakers.py, tests/test_daemon_speaker.py, tests/test_assistant_speaker.py, tests/test_enroll_assist.py (the offline seed helper src/stt_daemon/enroll_assist.py), and tests/test_evaluate.py (the speaker-eval scorer src/stt_daemon/evaluate.py).
  • No conftest.py — per-file fakes are the convention. Tests are the behavioral spec; read them before changing behavior.
  • The default suite is green without the optional triage and speaker extras. Install triage (uv sync --extra triage) only to exercise the Laya measurement test, which otherwise skips; speaker (uv sync --extra speaker) is needed only for real ONNX embeddings, never the suite.

Conventions & gotchas

  • Never commit gitignored runtime output: config.toml, transcripts/, audio/, audit/, speakers/ (biometric), models/, .omo/.
  • New assistant tools: register in assistant/tools.py. Model-supplied arguments are JSON-parsed and schema-validated before running; set side_effect and requires_confirmation for anything non-read-only.
  • Whisper models must be multilingual — do not use .en variants (defaults auto-detect es/en).
  • docs/ holds the full references (config, CLI, audio setup, assistant, speaker identification, output, troubleshooting, development); .omo/plans/ holds local design plans (gitignored).