|
|
@@ -1,17 +1,16 @@
|
|
|
# STT Daemon
|
|
|
|
|
|
-An always-on speech-to-text daemon that continuously listens to a PulseAudio/PipeWire audio source, detects voice activity with webrtcvad, and transcribes utterances to Spanish/English text using Faster-Whisper.
|
|
|
+An always-on speech-to-text daemon that listens to a PulseAudio/PipeWire source, detects voice activity, and transcribes utterances to Spanish/English text, with a separate voice-assistant layer for commands and chat on top.
|
|
|
|
|
|
## Overview
|
|
|
|
|
|
-This daemon is designed to:
|
|
|
-
|
|
|
- **Capture audio continuously** from PulseAudio/PipeWire sources
|
|
|
-- **Preserve the start of speech** using a pre-roll (ring) buffer, so the first phoneme is never cut when voice detection fires
|
|
|
+- **Preserve the start of speech** with a pre-roll (ring) buffer, so the first phoneme is never cut when voice detection fires
|
|
|
- **Segment utterances** with configurable silence thresholds
|
|
|
-- **Transcribe** each utterance with Faster-Whisper (supports multilingual, Spanish, English, or auto-detect)
|
|
|
-- **Output to stdout** (human-readable) and/or JSONL files (for logging/downstream processing)
|
|
|
+- **Transcribe** each utterance with Faster-Whisper (multilingual: Spanish, English, or auto-detect)
|
|
|
+- **Output** to stdout (human-readable) and/or JSONL files
|
|
|
- **Run non-blocking** with worker threads so audio keeps flowing during transcription
|
|
|
+- **Act on speech** through the separate `stt-assistant` process (modes, commands, chat, tools, audit)
|
|
|
|
|
|
### Architecture
|
|
|
|
|
|
@@ -37,9 +36,7 @@ stt-assistant (subscribes; owns modes, command buffer, chat, LLM, tools)
|
|
|
stt-tui (viewer) / stt-ctl (hotkeys) audit/YYYY-MM-DD.jsonl ←┘
|
|
|
```
|
|
|
|
|
|
-The daemon never accepts input: it only transcribes and publishes. Everything
|
|
|
-stateful lives in the assistant, which is a separate process (see
|
|
|
-[Assistant Layer](#assistant-layer)).
|
|
|
+The daemon never accepts input: it only transcribes and publishes. Everything stateful lives in the assistant, a separate process (see [docs/ASSISTANT.md](docs/ASSISTANT.md)).
|
|
|
|
|
|
## Prerequisites
|
|
|
|
|
|
@@ -48,7 +45,7 @@ stateful lives in the assistant, which is a separate process (see
|
|
|
- **PulseAudio or PipeWire-Pulse**: `pactl`, `parec` command-line tools
|
|
|
- On Ubuntu/Debian: `apt-get install pulseaudio-utils` or `apt-get install pipewire pipewire-pulse`
|
|
|
- On Fedora: `dnf install pulseaudio-utils`
|
|
|
- - On macOS: Install via Homebrew (though PulseAudio is Linux-first; `sounddevice` backend recommended on macOS)
|
|
|
+ - On macOS: Install via Homebrew (though PulseAudio is Linux-first; the `sounddevice` backend is recommended on macOS)
|
|
|
|
|
|
- **FFmpeg** (optional, used by Faster-Whisper for format conversion)
|
|
|
- On Ubuntu/Debian: `apt-get install ffmpeg`
|
|
|
@@ -73,828 +70,50 @@ uv sync
|
|
|
uv sync --extra portaudio
|
|
|
```
|
|
|
|
|
|
-## Configuration
|
|
|
-
|
|
|
-### Quick Start
|
|
|
+## Quick Start
|
|
|
|
|
|
Copy the example config and edit as needed:
|
|
|
|
|
|
```bash
|
|
|
cp config.example.toml config.toml
|
|
|
-# Edit config.toml to adjust settings (see section below)
|
|
|
-```
|
|
|
-
|
|
|
-### Configuration Reference
|
|
|
-
|
|
|
-The config file is TOML. Here are all available options with defaults.
|
|
|
-`config.example.toml` ships values only; this section is the annotated reference.
|
|
|
-
|
|
|
-```toml
|
|
|
-[audio]
|
|
|
-backend = "parec" # "parec" (PulseAudio by name) | "sounddevice"
|
|
|
-device = "VirtualMicSink.monitor" # PulseAudio source name (parec) or device index/name (sounddevice)
|
|
|
-sample_rate = 16000 # Hz
|
|
|
-channels = 1 # Mono only
|
|
|
-frame_ms = 30 # webrtcvad requires 10, 20, or 30
|
|
|
-gain = 1.0 # Amplitude scaling (1.0 = no change, >1 = louder)
|
|
|
-
|
|
|
-[vad]
|
|
|
-aggressiveness = 2 # 0–3: higher = fewer false positives (speech misses more)
|
|
|
-pre_roll_ms = 400 # Ring buffer duration: keep this many ms of pre-speech
|
|
|
-speech_start_ms = 90 # Consecutive speech frames to confirm speech start
|
|
|
-silence_ms = 700 # Consecutive silence to end a turn
|
|
|
-min_utterance_ms = 300 # Discard utterances with less actual speech than this
|
|
|
-max_utterance_s = 30.0 # Hard cap: force-finalize a very long utterance
|
|
|
-chunk_pause_ms = 350 # Once soft_max_s is reached, end a chunk at a pause this long
|
|
|
-soft_max_s = 10.0 # After this much speech, chunk-ends become eligible at pauses
|
|
|
-rms_floor = 0.008 # Drop segments quieter than this RMS (noise-triggered blips)
|
|
|
-trailing_trim_ms = 200 # Keep only this much trailing silence (less hallucination-prone padding)
|
|
|
-
|
|
|
-[whisper]
|
|
|
-backend = "faster-whisper" # "faster-whisper" (CTranslate2) | "pywhispercpp" (whisper.cpp; see docs/PYWHISPERCPP.md)
|
|
|
-model = "small" # "tiny"/"base"/"small"/"medium"/"large" (multilingual; do NOT use `.en` variants)
|
|
|
-device = "cpu" # "cpu" or "cuda"
|
|
|
-compute_type = "int8" # "int8" (CPU) or "float16" (CUDA, slower but more accurate)
|
|
|
-language = "auto" # "auto" (detect), "en", "es", or None/""/"none"
|
|
|
-beam_size = 1 # 1 = fast greedy, >1 = slower beam search
|
|
|
-vad_filter = false # Use Whisper's built-in VAD (we already have webrtcvad)
|
|
|
-cpu_threads = 0 # CPU inference threads (0 = CTranslate2's automatic choice).
|
|
|
- # On many-core CPUs, set this to the physical core count
|
|
|
- # (e.g. 16 on a Ryzen 9 8945HX) - measurably faster than auto.
|
|
|
-no_speech_prob_max = 0.6 # Drop segments Whisper scores as non-speech above this
|
|
|
-avg_logprob_min = -1.0 # Drop segments with confidence below this
|
|
|
-language_strategy = "static" # "static" (detect per utterance) | "sticky" (detect once, pin)
|
|
|
-sticky_redetect_logprob = -0.8 # Unpin and re-detect if segment avg_logprob falls below this.
|
|
|
- # Segments below avg_logprob_min (-1.0) are dropped; between
|
|
|
- # -1.0 and -0.8 they are accepted but trigger re-detection.
|
|
|
- # No accepted segments also clears the pin for re-detection.
|
|
|
-
|
|
|
-[sink] # Virtual audio sink configuration
|
|
|
-sink_name = "VirtualMicSink" # null-sink name
|
|
|
-remap_source_name = "VirtualMic" # Microphone source name
|
|
|
-loopback_latency_msec = 1 # Module latency (ms)
|
|
|
-monitor_output = false # Loop sink.monitor → default sink (OFF = no feedback risk)
|
|
|
-
|
|
|
-[output]
|
|
|
-stdout = true # Print transcripts to stdout
|
|
|
-jsonl = false # Persist transcripts to JSONL files (off by default for agent-only use)
|
|
|
-jsonl_dir = "transcripts" # JSONL destination (one file per day); only used if jsonl = true
|
|
|
-save_audio = false # Also write each transcribed utterance as a 16 kHz mono WAV
|
|
|
-audio_dir = "audio" # WAV destination (YYYY-MM-DD subfolders); only used if save_audio
|
|
|
-
|
|
|
-[daemon]
|
|
|
-queue_size = 50 # Max utterances queued for transcription (oldest dropped when full)
|
|
|
-backlog_warn_at = 5 # Warn when the queue backs up this far (worker falling behind)
|
|
|
-
|
|
|
-[events] # Publish-only event bus
|
|
|
-socket = "" # Default: $XDG_RUNTIME_DIR/stt-daemon.sock
|
|
|
-heartbeat_ms = 1000 # Heartbeat interval (progress-driven, from the capture loop)
|
|
|
-ring_size = 200 # Recent events replayed to a newly attached subscriber
|
|
|
-
|
|
|
-[assistant] # Controller process
|
|
|
-control_socket = "" # Default: $XDG_RUNTIME_DIR/stt-assistant.sock
|
|
|
-stall_timeout_ms = 3000 # No heartbeat for this long -> STT considered offline
|
|
|
-worker_stall_ms = 15000 # Heartbeats arriving but a transcription runs this long -> degraded
|
|
|
-command_arm_timeout_s = 10.0 # Disarm a command if no turn is captured in this time
|
|
|
-chat_idle_timeout_s = 300.0 # Leave chat mode after this much silence
|
|
|
-turn_gap_s = 1.5 # Quiet gap that ends a turn
|
|
|
-require_viewer = true # Refuse to execute commands with no TUI attached
|
|
|
-chat_exit_phrases = ["end conversation", "that's all", "stop listening"]
|
|
|
-
|
|
|
-[audit] # Executed actions only (commands/chats are volatile)
|
|
|
-enabled = true # audit/YYYY-MM-DD.jsonl
|
|
|
-dir = "audit"
|
|
|
-
|
|
|
-[llm] # local llama.cpp (OpenAI-compatible)
|
|
|
-enabled = true
|
|
|
-endpoint = "http://localhost:9931/v1/chat/completions"
|
|
|
-temperature = 0.7
|
|
|
-max_tokens = 1024 # max output per request; reasoning models spend tokens before the answer, so keep generous
|
|
|
-timeout_s = 120.0
|
|
|
-cache_prompt = true # reuse the KV prefix (append-only history keeps it warm)
|
|
|
-max_context_tokens = 131072 # the server's --ctx-size (llama.cpp `GET /props` -> n_ctx); the prompt budget derives from it
|
|
|
-compaction_safety_margin = 256 # tokens reserved for the token estimate error + template/tool overhead
|
|
|
-compaction_enabled = true # summarize old turns instead of deleting them
|
|
|
-compaction_trigger_ratio = 0.8 # compact when the prompt reaches this fraction of the budget
|
|
|
-compaction_keep_turns = 6 # recent turns kept verbatim
|
|
|
-compaction_max_tokens = 512 # summary length cap
|
|
|
-compaction_timeout_s = 30.0 # summarizer timeout, so a compaction cannot hold the inference lane
|
|
|
-compaction_input_tokens = 8192 # summarizer input cap before chunk-and-fold
|
|
|
-compaction_system_prompt = "You compress a conversation. Preserve names, numbers, decisions, user preferences, and unresolved questions. Omit pleasantries. Output only the summary."
|
|
|
-inference_queue_size = 8 # bounded single-lane inference queue
|
|
|
-trim_block_turns = 4 # fallback block trimming, not per-turn, to limit re-prefill
|
|
|
-max_tool_iterations = 3
|
|
|
-system_prompt_chat = "You are a concise voice assistant. Reply in 1-3 sentences."
|
|
|
-system_prompt_command = "You translate a spoken request into a single tool call when one applies."
|
|
|
-```
|
|
|
-
|
|
|
-`backend` picks the transcription engine. `faster-whisper` is the default, so nothing changes
|
|
|
-unless you opt in. `pywhispercpp` selects the whisper.cpp backend (GPU via the vendored CUDA
|
|
|
-wheel; see [docs/PYWHISPERCPP.md](docs/PYWHISPERCPP.md)). `model`, `language`, `beam_size`,
|
|
|
-`cpu_threads`, `no_speech_prob_max`, `avg_logprob_min`, `language_strategy`, and
|
|
|
-`sticky_redetect_logprob` apply to both; `device` and `compute_type` apply only to
|
|
|
-`faster-whisper`; `vad_filter` is ignored (with a warning) on `pywhispercpp`. For `pywhispercpp`
|
|
|
-the ggml quantization is chosen by the model name (e.g. `small`, `small-q5_1`).
|
|
|
-
|
|
|
-`max_prompt_tokens` was removed: the prompt budget is derived as
|
|
|
-`max_context_tokens - max_tokens - compaction_safety_margin`, so the window and the trim budget
|
|
|
-can never disagree. A legacy `max_prompt_tokens` key is ignored with a warning.
|
|
|
-
|
|
|
-### Audio Source Setup
|
|
|
-
|
|
|
-By default, the daemon listens to `VirtualMicSink.monitor` (the monitor of the virtual sink). Apps route audio into `VirtualMicSink`, and the daemon captures the mix.
|
|
|
-
|
|
|
-#### Option 1: Use the CLI with interactive setup (recommended)
|
|
|
-
|
|
|
-```bash
|
|
|
-# Show devices and create a virtual sink (interactive picker)
|
|
|
-uv run stt-daemon setup
|
|
|
-
|
|
|
-# Check status
|
|
|
-uv run stt-daemon list
|
|
|
-
|
|
|
-# In another terminal, start the daemon
|
|
|
-uv run stt-daemon run
|
|
|
-
|
|
|
-# The daemon auto-uses the provisioned sink's monitor as the capture device
|
|
|
-# (no need to manually specify --device)
|
|
|
-
|
|
|
-# Route audio:
|
|
|
-# - In pavucontrol (GUI), drag apps' "Recording" side to "VirtualMicSink"
|
|
|
-# - Or use pactl: pactl move-sink-input <app-index> VirtualMicSink
|
|
|
-
|
|
|
-# Stop when done
|
|
|
-uv run stt-daemon clear --yes
|
|
|
-```
|
|
|
-
|
|
|
-#### Option 2: Use the provided shell wrapper
|
|
|
-
|
|
|
-For convenience, `scripts/audio-sink.sh` is a thin wrapper around the Python CLI:
|
|
|
-
|
|
|
-```bash
|
|
|
-scripts/audio-sink.sh start # → uv run stt-daemon setup
|
|
|
-scripts/audio-sink.sh status # → uv run stt-daemon list
|
|
|
-scripts/audio-sink.sh stop # → uv run stt-daemon clear --yes
|
|
|
-```
|
|
|
-
|
|
|
-#### Option 3: Manual PulseAudio setup
|
|
|
-
|
|
|
-If you prefer manual control or want to understand what's happening:
|
|
|
-
|
|
|
-```bash
|
|
|
-# Get your default microphone source
|
|
|
-default_source=$(pactl get-default-source)
|
|
|
-echo $default_source # e.g., alsa_input.usb-...
|
|
|
-
|
|
|
-# Create null sink
|
|
|
-pactl load-module module-null-sink sink_name=VirtualMicSink \
|
|
|
- sink_properties=device.description=Virtual-Mic-Sink
|
|
|
-
|
|
|
-# Expose the sink's monitor as a microphone
|
|
|
-pactl load-module module-remap-source \
|
|
|
- master=VirtualMicSink.monitor \
|
|
|
- source_name=VirtualMic \
|
|
|
- source_properties=device.description=Virtual-Microphone
|
|
|
-
|
|
|
-# Route your microphone into the sink
|
|
|
-pactl load-module module-loopback source="$default_source" sink=VirtualMicSink latency_msec=1
|
|
|
-
|
|
|
-# (Optional) Monitor the sink output on your speakers
|
|
|
-pactl load-module module-loopback source=VirtualMicSink.monitor latency_msec=1
|
|
|
-
|
|
|
-# Now tell the daemon to use it
|
|
|
-uv run stt-daemon --device VirtualMicSink.monitor
|
|
|
-
|
|
|
-# Clean up (unload modules and remove the file):
|
|
|
-pactl unload-module <sink_id>
|
|
|
-pactl unload-module <source_id>
|
|
|
-pactl unload-module <loopback_id_1>
|
|
|
-pactl unload-module <loopback_id_2>
|
|
|
-```
|
|
|
-
|
|
|
-#### Option 3: Use a physical microphone directly
|
|
|
-
|
|
|
-If you have a single physical mic and don't need the virtual routing:
|
|
|
-
|
|
|
-```bash
|
|
|
-# List available sources
|
|
|
-pactl list short sources
|
|
|
-
|
|
|
-# Start daemon with your source
|
|
|
-uv run stt-daemon --device alsa_input.usb-C-Media_Electronics_Inc._USB_Audio_Device-00.mono-fallback
|
|
|
-```
|
|
|
-
|
|
|
-#### Option 4: Use sounddevice backend
|
|
|
-
|
|
|
-If you prefer not to use PulseAudio (e.g., on macOS or Windows):
|
|
|
-
|
|
|
-```bash
|
|
|
-uv sync --extra portaudio
|
|
|
-uv run stt-daemon --backend sounddevice --device 0 # device 0 = default input
|
|
|
-```
|
|
|
-
|
|
|
-## Running the Daemon
|
|
|
-
|
|
|
-### Subcommands
|
|
|
-
|
|
|
-The `stt-daemon` CLI supports these subcommands. Every subcommand also accepts the
|
|
|
-common overrides `--config`, `--device`, `--backend`, `--model`, `--language`, and
|
|
|
-`-v/--verbose` (for example `stt-daemon setup --device ...`).
|
|
|
-
|
|
|
-#### `setup` — Create virtual sink and loopbacks
|
|
|
-
|
|
|
-Interactive setup (shows device picker):
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon setup
|
|
|
-```
|
|
|
-
|
|
|
-Non-interactive setup (specify sources directly):
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon setup --sources "alsa_input.usb-Foo,alsa_output.pci-0000.monitor"
|
|
|
-```
|
|
|
-
|
|
|
-This creates:
|
|
|
-- A null sink (by default `VirtualMicSink`)
|
|
|
-- A remap-source to expose the sink's monitor as a microphone (by default `VirtualMic`)
|
|
|
-- Loopback modules routing each selected source into the sink
|
|
|
-- A state file tracking all module IDs for later cleanup
|
|
|
-
|
|
|
-#### `run` — Start the daemon (auto-provisions if needed)
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon run
|
|
|
-```
|
|
|
-
|
|
|
-If no saved provisioning exists and the sink is missing, the daemon will:
|
|
|
-1. Show the interactive device picker
|
|
|
-2. Create the sink and loopbacks (if sources are selected)
|
|
|
-3. Start listening and transcribing
|
|
|
-
|
|
|
-You can also override settings:
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon run --device "alsa_input.usb-..." --language es --model base
|
|
|
-uv run stt-daemon run -v # Verbose logging
|
|
|
-uv run stt-daemon run --check # Test audio capture + model, then exit
|
|
|
```
|
|
|
|
|
|
-#### `list` — Show devices and provisioning status
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon list
|
|
|
-```
|
|
|
-
|
|
|
-Output shows:
|
|
|
-- Input devices (real microphones)
|
|
|
-- Output devices (speakers/headphones; captured via their `.monitor` source)
|
|
|
-- Current provisioning state (if a sink has been created)
|
|
|
-
|
|
|
-#### `clear` — Remove virtual sink and loopbacks
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon clear
|
|
|
-# or without confirmation prompt:
|
|
|
-uv run stt-daemon clear --yes
|
|
|
-```
|
|
|
-
|
|
|
-This unloads all tracked modules and removes the state file. If
|
|
|
-`audio/audio.json` is already gone (reboot, manual
|
|
|
-delete, crash), `clear --yes` still finds orphaned modules by name
|
|
|
-(`VirtualMicSink`/`VirtualMic` via `pactl list short modules`) and unloads
|
|
|
-them in reverse order (loopbacks → remap-source → null-sink), so the
|
|
|
-command is idempotent — safe to re-run and to recover from
|
|
|
-`Sink 'VirtualMicSink' already exists but is not tracked`.
|
|
|
-
|
|
|
-#### `up` — Launch daemon, assistant, and optionally TUI
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon up # Start daemon + assistant (separate processes)
|
|
|
-uv run stt-daemon up --with-tui # Also launch the interactive TUI viewer
|
|
|
-```
|
|
|
-
|
|
|
-This subcommand runs the entire system as a supervised set of child processes:
|
|
|
-
|
|
|
-- **daemon** — transcription engine (listens to audio, publishes transcriptions)
|
|
|
-- **assistant** — controller (subscribes to transcriptions, serves the control socket)
|
|
|
-- **TUI** (optional) — viewer/controller UI (connects to the assistant via control socket)
|
|
|
-
|
|
|
-The supervisor manages their lifetimes: if any child exits or Ctrl+C is pressed, all
|
|
|
-remaining children are cleanly shut down. Without `--with-tui`, you can still run
|
|
|
-a separate `uv run stt-tui` in another terminal to view and control the assistant.
|
|
|
-
|
|
|
-With `--with-tui`, the daemon and assistant output is discarded so it cannot corrupt
|
|
|
-the screen, while the TUI keeps the terminal. Without `--with-tui`, all output goes to
|
|
|
-the terminal. The supervisor writes nothing to disk.
|
|
|
-
|
|
|
-Like all subcommands, `up` also accepts `--config`, `--device`, `--backend`,
|
|
|
-`--model`, `--language`, and `-v` (verbose) flags.
|
|
|
-
|
|
|
-Example workflow:
|
|
|
-
|
|
|
-```bash
|
|
|
-# One-command startup with TUI (everything stops when you quit the TUI)
|
|
|
-uv run stt-daemon up --with-tui
|
|
|
-
|
|
|
-# Or, start daemon + assistant only
|
|
|
-uv run stt-daemon up
|
|
|
-
|
|
|
-# Then in another terminal, launch the TUI whenever you want
|
|
|
-uv run stt-tui
|
|
|
-```
|
|
|
-
|
|
|
-#### (Legacy) `--list-sources` flag
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon --list-sources
|
|
|
-# Equivalent to: uv run stt-daemon list
|
|
|
-```
|
|
|
-
|
|
|
-#### (Legacy) `--check` flag
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon --check
|
|
|
-# Equivalent to: uv run stt-daemon run --check
|
|
|
-```
|
|
|
-
|
|
|
-### Interactive Device Picker
|
|
|
-
|
|
|
-When you run `setup` or `run` without `--sources`, the daemon shows an interactive picker:
|
|
|
-
|
|
|
-- **TTY mode** (terminal): Uses `questionary` to show a checkbox UI. Pre-checked items are from the previously saved selection.
|
|
|
-- **Non-TTY mode** (piped input): Shows a numbered menu; enter space/comma-separated indices, `all`, `none`, or blank (to keep previous selection).
|
|
|
-
|
|
|
-Device labels:
|
|
|
-- `[IN] <description> (<name>)` — Input device (real microphone)
|
|
|
-- `[OUT] <description> (<name>)` — Output device (monitor of a sink)
|
|
|
-
|
|
|
-### Running the daemon
|
|
|
-
|
|
|
-The simplest workflow:
|
|
|
-
|
|
|
-```bash
|
|
|
-# Create/setup the sink (interactive)
|
|
|
-uv run stt-daemon setup
|
|
|
-
|
|
|
-# Start the daemon in another terminal
|
|
|
-# (auto-uses the provisioned sink)
|
|
|
-uv run stt-daemon run
|
|
|
-
|
|
|
-# Stop and remove the sink when done
|
|
|
-uv run stt-daemon clear --yes
|
|
|
-```
|
|
|
-
|
|
|
-### Output examples
|
|
|
-
|
|
|
-**Stdout** (human-readable):
|
|
|
-```
|
|
|
-[14:23:45] (es) Hola, cómo estás? [rms=0.041 end_of_turn]
|
|
|
-[14:23:52] (en) What time is it? [rms=0.038 end_of_turn]
|
|
|
-[14:24:10] (es) Gracias por tu ayuda. [rms=0.052 pause]
|
|
|
-```
|
|
|
-
|
|
|
-**JSONL** (one line per utterance; only written when `jsonl = true`):
|
|
|
-```json
|
|
|
-{"ts": "2025-09-20T14:23:45.123456", "lang": "es", "lang_prob": 0.95, "duration_s": 2.5, "rms": 0.04123, "end_reason": "end_of_turn", "text": "Hola, cómo estás?", "transcribe_s": 0.42}
|
|
|
-{"ts": "2025-09-20T14:23:52.654321", "lang": "en", "lang_prob": 0.98, "duration_s": 1.8, "rms": 0.03811, "end_reason": "pause", "text": "What time is it?", "transcribe_s": 0.31}
|
|
|
-```
|
|
|
-
|
|
|
-`duration_s` is the length of the utterance audio; `transcribe_s` is the wall-clock
|
|
|
-time the worker spent transcribing it (inference latency), rounded to milliseconds.
|
|
|
-Their ratio is the real-time factor. The field is omitted if the timing was not
|
|
|
-measured (e.g. `emit()` called directly without `transcribe_s`).
|
|
|
-
|
|
|
-`end_reason` is one of `end_of_turn` (full `silence_ms` elapsed), `pause` (chunked at
|
|
|
-`chunk_pause_ms` because it passed `soft_max_s`), `hard_cap` (`max_utterance_s`
|
|
|
-reached), or `flush` (shutdown mid-utterance).
|
|
|
-
|
|
|
-**Audio files** — when `save_audio = true`, each transcribed utterance is also
|
|
|
-written as a 16 kHz mono WAV under `audio_dir/YYYY-MM-DD/`, and the record gains
|
|
|
-an `"audio"` field with its path:
|
|
|
-```json
|
|
|
-{..., "end_reason": "pause", "text": "Hola", "audio": "audio/2025-09-20/142345-123-pause.wav"}
|
|
|
-```
|
|
|
-Saving is best-effort: if a write fails (e.g. disk full), the transcript is still
|
|
|
-emitted and a warning is logged.
|
|
|
-
|
|
|
-> ⚠️ **Disk space** — saved audio is **never rotated or cleaned up** automatically.
|
|
|
-> At 16 kHz mono 16-bit it costs 32 KB/s, i.e. **~1.9 MB/min, ~115 MB/hour,
|
|
|
-> ~2.8 GB/day** of *continuous speech*. Silence is not written, so real usage is
|
|
|
-> lower — but an always-on daemon in a talkative room still accumulates quickly.
|
|
|
-> Point `audio_dir` at a volume with headroom and prune it yourself (e.g. a cron
|
|
|
-> job removing files older than N days), and leave `save_audio = false` unless you
|
|
|
-> actually need the clips for auditing.
|
|
|
-
|
|
|
-## Assistant Layer
|
|
|
-
|
|
|
-The assistant is a **separate process** that turns transcriptions into actions and
|
|
|
-conversation. The daemon stays a dumb, reliable publisher; the assistant owns all state.
|
|
|
-
|
|
|
-### Topology
|
|
|
-
|
|
|
-- `stt-daemon` → publish-only event socket (`$XDG_RUNTIME_DIR/stt-daemon.sock`, mode `0600`). It never accepts input.
|
|
|
-- `stt-assistant` → subscribes to that stream and serves a control socket (`$XDG_RUNTIME_DIR/stt-assistant.sock`, mode `0600`).
|
|
|
-- `stt-tui` / `stt-ctl` → clients of the assistant.
|
|
|
-
|
|
|
-### Running
|
|
|
-
|
|
|
-#### Simplest: single command with supervisor
|
|
|
+Then launch the whole system with one command:
|
|
|
|
|
|
```bash
|
|
|
uv run stt-daemon up --with-tui
|
|
|
```
|
|
|
|
|
|
-This starts daemon, assistant, and TUI as child processes under a supervisor.
|
|
|
-When you quit the TUI, everything stops cleanly. Daemon and assistant output is
|
|
|
-discarded while the TUI owns the screen; nothing is written to disk.
|
|
|
+That supervises the daemon, the assistant, and the TUI together; quitting the TUI stops everything cleanly. The manual flow is the same three steps without the supervisor:
|
|
|
|
|
|
-#### Traditional: three terminals
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon # transcription + event bus
|
|
|
-uv run stt-assistant # controller: modes, LLM, tools, audit, control socket
|
|
|
-uv run stt-tui # viewer/controller TUI (or `stt-tui --dump 10` headless)
|
|
|
-```
|
|
|
-
|
|
|
-Or start daemon + assistant once, then connect viewers as needed:
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon up # daemon + assistant
|
|
|
-uv run stt-tui # optional: viewer in another terminal
|
|
|
-uv run stt-tui --dump 5 # optional: headless event dump (5 seconds)
|
|
|
-```
|
|
|
-
|
|
|
-### Requirements
|
|
|
-
|
|
|
-A local llama.cpp server exposing an OpenAI-compatible `/v1/chat/completions` with a
|
|
|
-tool-calling model (default endpoint `http://localhost:9931/v1/chat/completions`).
|
|
|
-Set `[llm].enabled = false` to run without an LLM (commands then report
|
|
|
-"no executor configured").
|
|
|
-
|
|
|
-### Modes
|
|
|
-
|
|
|
-| mode | what it does |
|
|
|
-|---|---|
|
|
|
-| `idle` | listening; nothing is sent to the model |
|
|
|
-| `command_armed` | the next complete turn becomes a command; the text accumulates live |
|
|
|
-| `executing` | a command is running through the tool loop |
|
|
|
-| `chat` | every completed turn is sent to the model; no tools |
|
|
|
-
|
|
|
-Command and chat modes are mutually exclusive.
|
|
|
-
|
|
|
-### The two-press command flow
|
|
|
-
|
|
|
-1. Arm (`stt-ctl command-start`, or `a` in the TUI).
|
|
|
-2. Speak; the transcript fills the buffer.
|
|
|
-3. Optionally edit with `e` (freezes the buffer, so ASR mistakes can be corrected before the model sees them).
|
|
|
-4. Execute (`stt-ctl command-execute` or `Enter`), or cancel (`Esc`).
|
|
|
-
|
|
|
-Execution is gated on **readiness**: no speech open, the last finalized segment's
|
|
|
-`segment_done` has arrived, and STT is healthy. With `require_viewer = true`, execute is
|
|
|
-refused unless a TUI is attached.
|
|
|
-
|
|
|
-### Control protocol
|
|
|
-
|
|
|
-`stt-ctl` verbs (one message each): `command-start`, `command-execute`, `command-cancel`,
|
|
|
-`command-edit --text "..."`, `chat-toggle`, `clear`, `status`.
|
|
|
-
|
|
|
-Clients attach by sending `subscribe` first (`viewer: true` for the TUI, which makes the
|
|
|
-assistant treat it as a watcher for the `require_viewer` gate).
|
|
|
-
|
|
|
-Assistant → clients: `mode`, `command_text`, `transcript`, `speech_start`/`speech_stop`,
|
|
|
-`tool_call`, `tool_result`, `chat_message`, `cleared`, `heartbeat`, `error`, plus the direct
|
|
|
-replies `ack` (command accepted) and `status`.
|
|
|
-
|
|
|
-### Audit
|
|
|
-
|
|
|
-Only **actions** are persisted, to `audit/YYYY-MM-DD.jsonl`:
|
|
|
-
|
|
|
-```json
|
|
|
-{"ts":"...","mode":"command","text":"what time is it","edited":false,
|
|
|
- "tool":"get_datetime","args":{},"side_effect":"read_only","requires_confirmation":false,
|
|
|
- "confirmed":null,"ok":true,"result":"...","latency_ms":412}
|
|
|
-```
|
|
|
-
|
|
|
-Commands and chat history are volatile — held in memory and discarded.
|
|
|
-
|
|
|
-### Conversation memory (compaction)
|
|
|
-
|
|
|
-Chat history lives in memory only. To avoid overflowing the model's context, the assistant
|
|
|
-**compacts** rather than deletes: when the prompt reaches `compaction_trigger_ratio` of the
|
|
|
-derived budget (`max_context_tokens - max_tokens - compaction_safety_margin`), the oldest turns
|
|
|
-are summarized by the same endpoint (using `compaction_system_prompt`, capped at
|
|
|
-`compaction_max_tokens`) and replaced by one rolling summary, while the most recent
|
|
|
-`compaction_keep_turns` stay verbatim. Large evictions are summarized in chunks and folded.
|
|
|
-
|
|
|
-The summary is injected as a second `system` message, so the prefix (`system` + summary) is
|
|
|
-**stable between compactions** — a compaction invalidates the server's prefix cache *once*, and
|
|
|
-subsequent turns are append-only and cache-friendly. If summarization fails (error, empty reply,
|
|
|
-or a truncated summary) the assistant falls back to deleting the oldest turns in blocks
|
|
|
-(`trim_block_turns`); compaction is never the reason a request exceeds the window. Each
|
|
|
-compaction emits a `chat_message` with `role: "note"` so the pane shows why the context changed —
|
|
|
-the note is UI-only and never sent to the model.
|
|
|
-
|
|
|
-A reply cut off by the output limit (`finish_reason: "length"`) surfaces as
|
|
|
-`error{code:"truncated"}` instead of disappearing silently; an empty truncated turn is not stored.
|
|
|
-
|
|
|
-### Inference queue
|
|
|
-
|
|
|
-Every LLM call — chat turns, command executions, and the compaction summarizer — runs through a
|
|
|
-single worker thread, bounded by `inference_queue_size`. This serializes requests to the one
|
|
|
-llama.cpp server. Commands jump ahead of chat turns; when the queue is full an execute is refused
|
|
|
-with `error{code:"busy"}` and the oldest queued chat turn is dropped. Leaving chat mode discards
|
|
|
-queued chat turns.
|
|
|
-
|
|
|
-### TUI keys
|
|
|
-
|
|
|
-`a` arm · `Enter` execute · `Esc` cancel · `e` edit · `t` chat toggle · `c` clear · `q` quit
|
|
|
-
|
|
|
-`c` asks for confirmation, then wipes the conversation / command-output pane and resets the assistant's in-memory chat history (the LLM session), while staying in the current mode. Transcripts are not affected.
|
|
|
-
|
|
|
-**Live queue counters:** The pane borders show real-time queue depths. `transcripts [seg N]` displays the number of audio segments queued for transcription (the daemon's backlog). `conversation / command output [inf N]` shows the number of LLM inference jobs queued, with `· busy` appearing while one is running. These counters refresh about once per second, driven by the heartbeat cadence.
|
|
|
-
|
|
|
-**Selecting / copying text:** the TUI captures mouse events (Textual enables mouse
|
|
|
-reporting by default), so dragging does **not** select text the way it normally does in
|
|
|
-your terminal. Hold `Shift` while dragging to bypass the TUI and use your terminal's
|
|
|
-native selection and copy.
|
|
|
-
|
|
|
-### Hotkeys
|
|
|
-
|
|
|
-See [`scripts/hotkeys/`](scripts/hotkeys/README.md) for `sxhkd` and `xbindkeys` examples
|
|
|
-that bind those commands to global keys.
|
|
|
-
|
|
|
-## Virtual Sink Provisioning: How It Works
|
|
|
-
|
|
|
-The daemon uses PulseAudio's module system to create a virtual microphone from multiple sources:
|
|
|
-
|
|
|
-### What Gets Created
|
|
|
-
|
|
|
-1. **Null Sink** (`module-null-sink`, e.g., `VirtualMicSink`)
|
|
|
- - A sink that doesn't play anywhere; apps route audio into it instead.
|
|
|
-
|
|
|
-2. **Remap Source** (`module-remap-source`, e.g., `VirtualMic`)
|
|
|
- - Exposes the null-sink's monitor as a microphone so the daemon can capture from it.
|
|
|
-
|
|
|
-3. **Loopback Modules** (`module-loopback`, one per selected source)
|
|
|
- - Routes each selected input (microphone, speaker monitor, etc.) into the null-sink.
|
|
|
-
|
|
|
-### State Persistence
|
|
|
-
|
|
|
-When you run `uv run stt-daemon setup`, the daemon:
|
|
|
-- Creates all modules and records their numeric IDs
|
|
|
-- Saves the state to `audio/audio.json` (or `$STT_DAEMON_STATE`)
|
|
|
-- Allows `run` to re-provision if the sink is missing (e.g., after reboot)
|
|
|
-- Allows `clear` to unload the modules it created — and falls back to
|
|
|
- name-based discovery (`sink_name`/`remap_source_name` in
|
|
|
- `pactl list short modules`) when `audio.json` is missing, so orphaned
|
|
|
- `VirtualMicSink`/`VirtualMic` sinks are still cleaned up (e.g., after a
|
|
|
- reboot or manual state-file delete)
|
|
|
-
|
|
|
-The state file is JSON and can be edited manually if needed, but the daemon handles it automatically.
|
|
|
-
|
|
|
-### Feedback Risk
|
|
|
-
|
|
|
-By default, `monitor_output = false`, so the sink's output is **not** looped back to the default speaker. If you enable it, speakers will play back everything captured in the sink—including your own voice—causing audio feedback. Keep it OFF unless you specifically want monitoring.
|
|
|
-
|
|
|
-## Pre-roll Buffer: Why It Matters
|
|
|
-
|
|
|
-The **pre-roll buffer** is a ring buffer that keeps the last N milliseconds (default 400ms) of audio BEFORE speech is detected. When the VAD confirms speech has started, those pre-roll frames are prepended to the utterance.
|
|
|
-
|
|
|
-**Without pre-roll:** The first phoneme of "hello" might be cut off—you'd get "llo" or "elo" because the VAD fires AFTER the first sound is already gone.
|
|
|
-
|
|
|
-**With pre-roll:** The entire "hello" is captured, including the initial "h" and "e", because the ring buffer preserved them.
|
|
|
-
|
|
|
-### How it works (under the hood)
|
|
|
-
|
|
|
-1. **Silence phase**: Frames are pushed into a ring buffer (maxlen = pre_roll_ms / frame_ms). Old frames fall off.
|
|
|
-2. **Speech detected**: VAD returns True for consecutive `speech_start_ms` worth of frames.
|
|
|
-3. **Speech confirmed**: `_begin()` is called. The entire ring buffer contents are copied into the segment, then regular frames append to it.
|
|
|
-4. **End of turn**: after `silence_ms` of silence the segment is finalized and transcribed.
|
|
|
-
|
|
|
-### Long speech and silence
|
|
|
-
|
|
|
-Two extra rules keep long monologues from becoming one huge blob, and keep
|
|
|
-noise from being transcribed:
|
|
|
-
|
|
|
-- **Pause-aligned chunking**: once a segment reaches `soft_max_s` (10s) of speech,
|
|
|
- a shorter `chunk_pause_ms` (350ms) pause ends a *chunk*. The cut lands on a
|
|
|
- natural phrase boundary, so no word is split and no overlap/duplication occurs.
|
|
|
- `max_utterance_s` (30s) remains the last-resort cap for someone who never pauses.
|
|
|
-- **Noise suppression**: `min_utterance_ms` drops blips, `rms_floor` drops
|
|
|
- segments too quiet to be speech, `trailing_trim_ms` strips hallucination-prone
|
|
|
- trailing silence before transcription, and Whisper segments below
|
|
|
- `no_speech_prob_max` / `avg_logprob_min` are discarded.
|
|
|
-
|
|
|
-## Troubleshooting
|
|
|
-
|
|
|
-### "Device not found"
|
|
|
-
|
|
|
-```
|
|
|
-Error: Device 'VirtualMicSink.monitor' not found. Available sources: [...]
|
|
|
-To create VirtualMicSink, run: scripts/audio-sink.sh start
|
|
|
-```
|
|
|
-
|
|
|
-**Fix:**
|
|
|
```bash
|
|
|
-scripts/audio-sink.sh start
|
|
|
+uv run stt-daemon setup # create the virtual sink (interactive picker)
|
|
|
+uv run stt-daemon run # start listening and transcribing
|
|
|
+uv run stt-daemon clear --yes # tear down the sink when done
|
|
|
```
|
|
|
|
|
|
-Then route audio into VirtualMicSink and try again.
|
|
|
+See [docs/CLI.md](docs/CLI.md) for every subcommand and [docs/CONFIGURATION.md](docs/CONFIGURATION.md) for the config reference.
|
|
|
|
|
|
-### "Can't select or copy text in the TUI"
|
|
|
+## Documentation
|
|
|
|
|
|
-**Cause:** Textual enables terminal mouse reporting, so mouse events are consumed by the
|
|
|
-app instead of the terminal, and native drag-selection never starts.
|
|
|
-
|
|
|
-**Fix:** hold `Shift` while dragging to force your terminal's native selection, then copy
|
|
|
-as usual. (Alternatively, launching the app with mouse support disabled —
|
|
|
-`app.run(mouse=False)` — restores plain selection but loses in-app scrolling/clicks.)
|
|
|
-
|
|
|
-### High latency / slow transcription
|
|
|
-
|
|
|
-By default, the daemon uses `compute_type = "int8"` and `device = "cpu"`, which trades speed for accuracy. To speed up:
|
|
|
-
|
|
|
-1. **Pin `cpu_threads` to the physical core count**: `cpu_threads = 16` in `[whisper]`
|
|
|
- (CTranslate2's automatic choice is often slower than an explicit count on many-core CPUs)
|
|
|
-2. **Reduce model size**: `--model base` or `--model tiny`
|
|
|
-3. **Enable GPU** (if you have CUDA): `--device cuda` with `compute_type float16`
|
|
|
-4. **Reduce beam_size**: Edit config to `beam_size = 1` (already default, very fast)
|
|
|
-
|
|
|
-Example:
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon --model base --device cuda
|
|
|
-```
|
|
|
-
|
|
|
-### "Transcription backlog reached N of M"
|
|
|
-
|
|
|
-This warning means the single transcription worker is falling behind: utterances
|
|
|
-are arriving faster than they are being transcribed, and once the queue is full
|
|
|
-the oldest ones get dropped. The worker is serial by design (faster-whisper's
|
|
|
-model isn't thread-safe), so fix it by making transcription faster rather than by
|
|
|
-queueing more:
|
|
|
-
|
|
|
-1. **Smaller/faster model**: `--model base` or `tiny`
|
|
|
-2. **GPU**: `--device cuda` with `compute_type float16`
|
|
|
-3. **Raise `backlog_warn_at`** or `queue_size` in `[daemon]` only to make the
|
|
|
- warning/dropping less likely — it hides the symptom, not the cause.
|
|
|
-
|
|
|
-### Spanish / English / Other languages
|
|
|
-
|
|
|
-The Faster-Whisper model is **multilingual by default**. It auto-detects language or can be forced:
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run stt-daemon --language es # Force Spanish
|
|
|
-uv run stt-daemon --language en # Force English
|
|
|
-uv run stt-daemon --language auto # Auto-detect (default)
|
|
|
-```
|
|
|
-
|
|
|
-The output includes the detected language tag: `[14:23:45] (es) ...`.
|
|
|
-
|
|
|
-### Recognizing Spanish vs. English better
|
|
|
-
|
|
|
-- **Force the language**: Use `--language es` or `--language en`
|
|
|
-- **Use a larger model**: `--model medium` or `--model large` (slower, more accurate)
|
|
|
-- **Tune VAD aggressiveness**: Lower values (0–1) capture more, higher values (2–3) are stricter
|
|
|
-
|
|
|
-### Audio device not responding / parec hangs
|
|
|
-
|
|
|
-If `parec` seems stuck:
|
|
|
-
|
|
|
-1. Check if the source exists: `pactl list short sources`
|
|
|
-2. Kill the daemon (Ctrl+C)
|
|
|
-3. Verify PulseAudio is running: `systemctl status pulseaudio` or check journalctl
|
|
|
-4. Restart PulseAudio if needed: `systemctl restart pulseaudio`
|
|
|
-5. Re-run the daemon
|
|
|
-
|
|
|
-### Sounddevice backend issues
|
|
|
-
|
|
|
-If using `--backend sounddevice` and getting import errors:
|
|
|
-
|
|
|
-```bash
|
|
|
-uv sync --extra portaudio
|
|
|
-```
|
|
|
-
|
|
|
-Then verify available devices:
|
|
|
-
|
|
|
-```bash
|
|
|
-python -c "import sounddevice as sd; print(sd.query_devices())"
|
|
|
-```
|
|
|
-
|
|
|
-Use the device index or name as `--device 0` or `--device "Microphone (...)`.
|
|
|
-
|
|
|
-### No transcriptions appearing
|
|
|
-
|
|
|
-1. Check VAD is firing: `uv run stt-daemon -v` (verbose logging)
|
|
|
-2. Verify audio is loud enough: `uv run stt-daemon --check` (check RMS)
|
|
|
-3. Reduce `aggressiveness` (0–1 instead of 2–3) to detect quieter speech
|
|
|
-4. Check `min_utterance_ms` isn't too high (default 300ms should be fine)
|
|
|
-
|
|
|
-### Whisper model not loading
|
|
|
-
|
|
|
-```
|
|
|
-Error loading model: ...
|
|
|
-```
|
|
|
-
|
|
|
-The model is downloaded on first run (~1.5 GB for `small`). Check:
|
|
|
-
|
|
|
-1. Disk space: `df -h` (needs ~2 GB free)
|
|
|
-2. Internet: Model is downloaded from Hugging Face
|
|
|
-3. HF cache location: Models go to `~/.cache/huggingface/hub/` by default
|
|
|
-
|
|
|
-You can override the cache:
|
|
|
-
|
|
|
-```bash
|
|
|
-export HF_HOME=/path/to/cache
|
|
|
-uv run stt-daemon --model small
|
|
|
-```
|
|
|
+- [docs/CONFIGURATION.md](docs/CONFIGURATION.md) — annotated TOML reference for every config key.
|
|
|
+- [docs/AUDIO-SETUP.md](docs/AUDIO-SETUP.md) — virtual sink creation and audio routing options.
|
|
|
+- [docs/CLI.md](docs/CLI.md) — `stt-daemon` subcommands and common overrides.
|
|
|
+- [docs/ASSISTANT.md](docs/ASSISTANT.md) — the assistant layer: modes, command flow, LLM, tools, audit.
|
|
|
+- [docs/OUTPUT.md](docs/OUTPUT.md) — stdout, JSONL, and WAV output formats and fields.
|
|
|
+- [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) — common errors and how to fix them.
|
|
|
+- [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) — code structure, testing, linting, and packaging.
|
|
|
+- [docs/PYWHISPERCPP.md](docs/PYWHISPERCPP.md) — building the `pywhispercpp` GPU backend from source.
|
|
|
|
|
|
## Testing
|
|
|
|
|
|
-Run the test suite (no audio hardware required):
|
|
|
+Run the full suite with no audio hardware or live LLM required:
|
|
|
|
|
|
```bash
|
|
|
uv run pytest -v
|
|
|
```
|
|
|
|
|
|
-Tests cover:
|
|
|
-
|
|
|
-- **Pre-roll buffer** preservation: confirm the first frames are not cut
|
|
|
-- **Silence finalization**: utterances end after N silence frames
|
|
|
-- **Min/max utterance duration**: blips are discarded, long utterances are finalized
|
|
|
-- **Pause-aligned chunking**: long speech splits at pauses (`end_reason = "pause"`)
|
|
|
-- **Config loading and language normalization**
|
|
|
-- **Provisioning**: `setup`/`ensure`/`teardown`, orphan-name cleanup, and `clear` idempotency
|
|
|
-- **Event bus**: envelope codec, ring replay, subscriber fan-out
|
|
|
-- **Assistant**: mode machine, command readiness/timeouts, tool schema validation, LLM client/history trimming, audit log
|
|
|
-- **TUI**: pure view-state reducer and control-socket client
|
|
|
-
|
|
|
-## Development
|
|
|
-
|
|
|
-### Code structure
|
|
|
-
|
|
|
-```
|
|
|
-src/stt_daemon/
|
|
|
-├── __init__.py - Package metadata
|
|
|
-├── __main__.py - Entry point for `python -m stt_daemon`
|
|
|
-├── cli.py - Argument parsing and entry point (`stt-daemon`)
|
|
|
-├── ctl.py - One-shot control client (`stt-ctl`, hotkeys)
|
|
|
-├── config.py - Config loading (TOML)
|
|
|
-├── audio.py - Audio capture (parec, sounddevice)
|
|
|
-├── pulse.py - PulseAudio/pactl wrapper (devices, modules, sink checks)
|
|
|
-├── provisioner.py - Virtual sink + loopback lifecycle (setup/ensure/teardown)
|
|
|
-├── state.py - Provisioning state file (audio/audio.json)
|
|
|
-├── picker.py - Interactive device picker (questionary / numbered menu)
|
|
|
-├── segmenter.py - VAD + pre-roll buffer + utterance segmentation (core)
|
|
|
-├── transcriber.py - Faster-Whisper wrapper
|
|
|
-├── output.py - Sinks (stdout, JSONL, optional WAV)
|
|
|
-├── daemon.py - Orchestrator (threads, queues, event emission)
|
|
|
-├── events.py - NDJSON event envelope + codec
|
|
|
-├── bus.py - Publish-only event bus (socket, ring replay)
|
|
|
-├── assistant/
|
|
|
-│ ├── modes.py - Mode state machine (idle/command_armed/executing/chat)
|
|
|
-│ ├── service.py - Modes, command buffer, readiness, timeouts
|
|
|
-│ ├── control.py - Control socket (viewers, commands, broadcast)
|
|
|
-│ ├── executor.py - NullExecutor / LlmExecutor (tool loop)
|
|
|
-│ ├── tools.py - Tool registry + schema validation + get_datetime
|
|
|
-│ ├── llm.py - llama.cpp client + chat session (block trimming)
|
|
|
-│ ├── audit.py - Daily action audit log
|
|
|
-│ ├── runner.py - `stt-assistant` process
|
|
|
-│ └── __main__.py - Entry point for `python -m stt_daemon.assistant`
|
|
|
-└── tui/
|
|
|
- ├── state.py - Pure view-state reducer
|
|
|
- ├── client.py - Control-socket client
|
|
|
- └── app.py - Textual app + `stt-tui` entry point
|
|
|
-
|
|
|
-tests/ - 206 tests, no audio hardware or live LLM required
|
|
|
-
|
|
|
-scripts/
|
|
|
-├── audio-sink.sh - Thin wrapper over `stt-daemon setup|clear|list`
|
|
|
-└── hotkeys/ - sxhkd / xbindkeys examples
|
|
|
-```
|
|
|
-
|
|
|
-### Running linters
|
|
|
-
|
|
|
-```bash
|
|
|
-uv run ruff check .
|
|
|
-```
|
|
|
-
|
|
|
-### Building/packaging
|
|
|
-
|
|
|
-```bash
|
|
|
-# Build sdist + wheel (hatchling backend, installed in an isolated env by uv)
|
|
|
-uv build
|
|
|
-```
|
|
|
-
|
|
|
-## Limitations & Design Decisions
|
|
|
-
|
|
|
-1. **Mono only**: Stereo support can be added, but speech is typically mono in transcription pipelines
|
|
|
-2. **Single language per transcription**: Whisper detects one language per utterance; mixed-language speech is attributed to the dominant language
|
|
|
-3. **LLM is optional and separate**: the daemon itself is transcription-only; the `stt-assistant` process adds chat/commands/tools on top, and can be disabled with `[llm].enabled = false`
|
|
|
-4. **PulseAudio/PipeWire-Pulse**: The default `parec` backend assumes UNIX-like systems with PulseAudio. macOS/Windows should use `sounddevice`
|
|
|
-5. **Real-time**: Latency depends on frame duration (30ms), silence threshold (default 700ms), and model size. Expect ~1–3 second latency with `small` model on CPU
|
|
|
-6. **Thread-safe queues**: Reader thread (audio) is separate from worker thread (transcription) so audio never blocks waiting for transcription
|
|
|
+See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for what the tests cover.
|
|
|
|
|
|
## References
|
|
|
|
