# STT Daemon

An always-on speech-to-text daemon that continuously listens to a PulseAudio/PipeWire audio source, detects voice activity with webrtcvad, and transcribes utterances to Spanish/English text using Faster-Whisper.

## Overview

This daemon is designed to:

- **Capture audio continuously** from PulseAudio/PipeWire sources
- **Preserve the start of speech** using a pre-roll (ring) buffer, so the first phoneme is never cut when voice detection fires
- **Segment utterances** with configurable silence thresholds
- **Transcribe** each utterance with Faster-Whisper (supports multilingual, Spanish, English, or auto-detect)
- **Output to stdout** (human-readable) and/or JSONL files (for logging/downstream processing)
- **Run non-blocking** with worker threads so audio keeps flowing during transcription

### Architecture

```
Audio Source (PulseAudio)
    ↓
ParecCapture / SoundDeviceCapture
    ↓
Pre-roll Ring Buffer (VAD lookback)
    ↓
WebRTC-VAD (voice activity detection)
    ↓
FrameSegmenter (utterance boundaries)
    ↓
Queue (segment_queue)
    ↓
Faster-Whisper (transcription)
    ↓
TranscriptSink (stdout + transcripts/YYYY-MM-DD.jsonl) + EventPublisher
    ↓  publish-only UNIX socket ($XDG_RUNTIME_DIR/stt-daemon.sock)
stt-assistant  (subscribes; owns modes, command buffer, chat, LLM, tools)
    ↓  control socket ($XDG_RUNTIME_DIR/stt-assistant.sock)
stt-tui (viewer) / stt-ctl (hotkeys)              audit/YYYY-MM-DD.jsonl ←┘
```

The daemon never accepts input: it only transcribes and publishes. Everything
stateful lives in the assistant, which is a separate process (see
[Assistant Layer](#assistant-layer)).

## Prerequisites

### System packages

- **PulseAudio or PipeWire-Pulse**: `pactl`, `parec` command-line tools
  - On Ubuntu/Debian: `apt-get install pulseaudio-utils` or `apt-get install pipewire pipewire-pulse`
  - On Fedora: `dnf install pulseaudio-utils`
  - On macOS: Install via Homebrew (though PulseAudio is Linux-first; `sounddevice` backend recommended on macOS)

- **FFmpeg** (optional, used by Faster-Whisper for format conversion)
  - On Ubuntu/Debian: `apt-get install ffmpeg`

### Python

- **Python 3.11+** (tested on 3.13.5)
- **uv** package manager (https://docs.astral.sh/uv/)

## Installation

Clone the repo and install with `uv`:

```bash
# From the repo root (wherever you cloned it)
cd /path/to/STT-TTS

# Sync dependencies (installs into .venv)
uv sync

# (Optional) If you want to use the sounddevice backend (requires portaudio):
uv sync --extra portaudio
```

## Configuration

### Quick Start

Copy the example config and edit as needed:

```bash
cp config.example.toml config.toml
# Edit config.toml to adjust settings (see section below)
```

### Configuration Reference

The config file is TOML. Here are all available options with defaults.
`config.example.toml` ships values only; this section is the annotated reference.

```toml
[audio]
backend = "parec"                     # "parec" (PulseAudio by name) | "sounddevice"
device  = "VirtualMicSink.monitor"    # PulseAudio source name (parec) or device index/name (sounddevice)
sample_rate = 16000                   # Hz
channels    = 1                       # Mono only
frame_ms    = 30                      # webrtcvad requires 10, 20, or 30
gain        = 1.0                     # Amplitude scaling (1.0 = no change, >1 = louder)

[vad]
aggressiveness   = 2                  # 0–3: higher = fewer false positives (speech misses more)
pre_roll_ms      = 400                # Ring buffer duration: keep this many ms of pre-speech
speech_start_ms  = 90                 # Consecutive speech frames to confirm speech start
silence_ms       = 700                # Consecutive silence to end a turn
min_utterance_ms = 300                # Discard utterances with less actual speech than this
max_utterance_s  = 30.0               # Hard cap: force-finalize a very long utterance
chunk_pause_ms   = 350                # Once soft_max_s is reached, end a chunk at a pause this long
soft_max_s       = 10.0               # After this much speech, chunk-ends become eligible at pauses
rms_floor        = 0.008              # Drop segments quieter than this RMS (noise-triggered blips)
trailing_trim_ms = 200                # Keep only this much trailing silence (less hallucination-prone padding)

[whisper]
model                   = "small"               # "tiny"/"base"/"small"/"medium"/"large" (multilingual; do NOT use `.en` variants)
device                  = "cpu"                 # "cpu" or "cuda"
compute_type            = "int8"                # "int8" (CPU) or "float16" (CUDA, slower but more accurate)
language                = "auto"                # "auto" (detect), "en", "es", or None/""/"none"
beam_size               = 1                     # 1 = fast greedy, >1 = slower beam search
vad_filter              = false                 # Use Whisper's built-in VAD (we already have webrtcvad)
cpu_threads             = 0                     # CPU inference threads (0 = CTranslate2's automatic choice).
                                                # On many-core CPUs, set this to the physical core count
                                                # (e.g. 16 on a Ryzen 9 8945HX) - measurably faster than auto.
no_speech_prob_max      = 0.6                   # Drop segments Whisper scores as non-speech above this
avg_logprob_min         = -1.0                  # Drop segments with confidence below this
language_strategy       = "static"              # "static" (detect per utterance) | "sticky" (detect once, pin)
sticky_redetect_logprob = -0.8                  # Unpin and re-detect if segment avg_logprob falls below this.
                                                # Segments below avg_logprob_min (-1.0) are dropped; between
                                                # -1.0 and -0.8 they are accepted but trigger re-detection.
                                                # No accepted segments also clears the pin for re-detection.

[sink]                                # Virtual audio sink configuration
sink_name             = "VirtualMicSink"       # null-sink name
remap_source_name     = "VirtualMic"           # Microphone source name
loopback_latency_msec = 1                      # Module latency (ms)
monitor_output        = false                  # Loop sink.monitor → default sink (OFF = no feedback risk)

[output]
stdout     = true                     # Print transcripts to stdout
jsonl      = false                    # Persist transcripts to JSONL files (off by default for agent-only use)
jsonl_dir  = "transcripts"            # JSONL destination (one file per day); only used if jsonl = true
save_audio = false                    # Also write each transcribed utterance as a 16 kHz mono WAV
audio_dir  = "audio"                  # WAV destination (YYYY-MM-DD subfolders); only used if save_audio

[daemon]
queue_size      = 50                        # Max utterances queued for transcription (oldest dropped when full)
backlog_warn_at = 5                         # Warn when the queue backs up this far (worker falling behind)

[events]                                    # Publish-only event bus
socket       = ""                           # Default: $XDG_RUNTIME_DIR/stt-daemon.sock
heartbeat_ms = 1000                         # Heartbeat interval (progress-driven, from the capture loop)
ring_size    = 200                          # Recent events replayed to a newly attached subscriber

[assistant]                                 # Controller process
control_socket        = ""                  # Default: $XDG_RUNTIME_DIR/stt-assistant.sock
stall_timeout_ms      = 3000                # No heartbeat for this long -> STT considered offline
worker_stall_ms       = 15000               # Heartbeats arriving but a transcription runs this long -> degraded
command_arm_timeout_s = 10.0                # Disarm a command if no turn is captured in this time
chat_idle_timeout_s   = 300.0               # Leave chat mode after this much silence
turn_gap_s            = 1.5                 # Quiet gap that ends a turn
require_viewer        = true                # Refuse to execute commands with no TUI attached
chat_exit_phrases     = ["end conversation", "that's all", "stop listening"]

[audit]                                     # Executed actions only (commands/chats are volatile)
enabled = true                              # audit/YYYY-MM-DD.jsonl
dir     = "audit"

[llm]                                       # local llama.cpp (OpenAI-compatible)
enabled              = true
endpoint             = "http://localhost:9931/v1/chat/completions"
temperature          = 0.7
max_tokens           = 1024                 # reasoning models spend tokens before the answer; keep generous
timeout_s            = 120.0
cache_prompt         = true                 # reuse the KV prefix (append-only history keeps it warm)
max_prompt_tokens    = 64000                # ~half of the 131072 context
trim_block_turns     = 4                    # trim history in blocks, not per-turn, to limit re-prefill
max_tool_iterations  = 3
system_prompt_chat   = "You are a concise voice assistant. Reply in 1-3 sentences."
system_prompt_command = "You translate a spoken request into a single tool call when one applies."
```

### Audio Source Setup

By default, the daemon listens to `VirtualMicSink.monitor` (the monitor of the virtual sink). Apps route audio into `VirtualMicSink`, and the daemon captures the mix.

#### Option 1: Use the CLI with interactive setup (recommended)

```bash
# Show devices and create a virtual sink (interactive picker)
uv run stt-daemon setup

# Check status
uv run stt-daemon list

# In another terminal, start the daemon
uv run stt-daemon run

# The daemon auto-uses the provisioned sink's monitor as the capture device
# (no need to manually specify --device)

# Route audio:
#   - In pavucontrol (GUI), drag apps' "Recording" side to "VirtualMicSink"
#   - Or use pactl: pactl move-sink-input <app-index> VirtualMicSink

# Stop when done
uv run stt-daemon clear --yes
```

#### Option 2: Use the provided shell wrapper

For convenience, `scripts/audio-sink.sh` is a thin wrapper around the Python CLI:

```bash
scripts/audio-sink.sh start   # → uv run stt-daemon setup
scripts/audio-sink.sh status  # → uv run stt-daemon list
scripts/audio-sink.sh stop    # → uv run stt-daemon clear --yes
```

#### Option 3: Manual PulseAudio setup

If you prefer manual control or want to understand what's happening:

```bash
# Get your default microphone source
default_source=$(pactl get-default-source)
echo $default_source  # e.g., alsa_input.usb-...

# Create null sink
pactl load-module module-null-sink sink_name=VirtualMicSink \
    sink_properties=device.description=Virtual-Mic-Sink

# Expose the sink's monitor as a microphone
pactl load-module module-remap-source \
    master=VirtualMicSink.monitor \
    source_name=VirtualMic \
    source_properties=device.description=Virtual-Microphone

# Route your microphone into the sink
pactl load-module module-loopback source="$default_source" sink=VirtualMicSink latency_msec=1

# (Optional) Monitor the sink output on your speakers
pactl load-module module-loopback source=VirtualMicSink.monitor latency_msec=1

# Now tell the daemon to use it
uv run stt-daemon --device VirtualMicSink.monitor

# Clean up (unload modules and remove the file):
pactl unload-module <sink_id>
pactl unload-module <source_id>
pactl unload-module <loopback_id_1>
pactl unload-module <loopback_id_2>
```

#### Option 3: Use a physical microphone directly

If you have a single physical mic and don't need the virtual routing:

```bash
# List available sources
pactl list short sources

# Start daemon with your source
uv run stt-daemon --device alsa_input.usb-C-Media_Electronics_Inc._USB_Audio_Device-00.mono-fallback
```

#### Option 4: Use sounddevice backend

If you prefer not to use PulseAudio (e.g., on macOS or Windows):

```bash
uv sync --extra portaudio
uv run stt-daemon --backend sounddevice --device 0  # device 0 = default input
```

## Running the Daemon

### Subcommands

The `stt-daemon` CLI supports these subcommands. Every subcommand also accepts the
common overrides `--config`, `--device`, `--backend`, `--model`, `--language`, and
`-v/--verbose` (for example `stt-daemon setup --device ...`).

#### `setup` — Create virtual sink and loopbacks

Interactive setup (shows device picker):

```bash
uv run stt-daemon setup
```

Non-interactive setup (specify sources directly):

```bash
uv run stt-daemon setup --sources "alsa_input.usb-Foo,alsa_output.pci-0000.monitor"
```

This creates:
- A null sink (by default `VirtualMicSink`)
- A remap-source to expose the sink's monitor as a microphone (by default `VirtualMic`)
- Loopback modules routing each selected source into the sink
- A state file tracking all module IDs for later cleanup

#### `run` — Start the daemon (auto-provisions if needed)

```bash
uv run stt-daemon run
```

If no saved provisioning exists and the sink is missing, the daemon will:
1. Show the interactive device picker
2. Create the sink and loopbacks (if sources are selected)
3. Start listening and transcribing

You can also override settings:

```bash
uv run stt-daemon run --device "alsa_input.usb-..." --language es --model base
uv run stt-daemon run -v  # Verbose logging
uv run stt-daemon run --check  # Test audio capture + model, then exit
```

#### `list` — Show devices and provisioning status

```bash
uv run stt-daemon list
```

Output shows:
- Input devices (real microphones)
- Output devices (speakers/headphones; captured via their `.monitor` source)
- Current provisioning state (if a sink has been created)

#### `clear` — Remove virtual sink and loopbacks

```bash
uv run stt-daemon clear
# or without confirmation prompt:
uv run stt-daemon clear --yes
```

This unloads all tracked modules and removes the state file. If
`audio/audio.json` is already gone (reboot, manual
delete, crash), `clear --yes` still finds orphaned modules by name
(`VirtualMicSink`/`VirtualMic` via `pactl list short modules`) and unloads
them in reverse order (loopbacks → remap-source → null-sink), so the
command is idempotent — safe to re-run and to recover from
`Sink 'VirtualMicSink' already exists but is not tracked`.

#### `up` — Launch daemon, assistant, and optionally TUI

```bash
uv run stt-daemon up              # Start daemon + assistant (separate processes)
uv run stt-daemon up --with-tui   # Also launch the interactive TUI viewer
```

This subcommand runs the entire system as a supervised set of child processes:

- **daemon** — transcription engine (listens to audio, publishes transcriptions)
- **assistant** — controller (subscribes to transcriptions, serves the control socket)
- **TUI** (optional) — viewer/controller UI (connects to the assistant via control socket)

The supervisor manages their lifetimes: if any child exits or Ctrl+C is pressed, all
remaining children are cleanly shut down. Without `--with-tui`, you can still run
a separate `uv run stt-tui` in another terminal to view and control the assistant.

With `--with-tui`, the daemon and assistant output is discarded so it cannot corrupt
the screen, while the TUI keeps the terminal. Without `--with-tui`, all output goes to
the terminal. The supervisor writes nothing to disk.

Like all subcommands, `up` also accepts `--config`, `--device`, `--backend`,
`--model`, `--language`, and `-v` (verbose) flags.

Example workflow:

```bash
# One-command startup with TUI (everything stops when you quit the TUI)
uv run stt-daemon up --with-tui

# Or, start daemon + assistant only
uv run stt-daemon up

# Then in another terminal, launch the TUI whenever you want
uv run stt-tui
```

#### (Legacy) `--list-sources` flag

```bash
uv run stt-daemon --list-sources
# Equivalent to: uv run stt-daemon list
```

#### (Legacy) `--check` flag

```bash
uv run stt-daemon --check
# Equivalent to: uv run stt-daemon run --check
```

### Interactive Device Picker

When you run `setup` or `run` without `--sources`, the daemon shows an interactive picker:

- **TTY mode** (terminal): Uses `questionary` to show a checkbox UI. Pre-checked items are from the previously saved selection.
- **Non-TTY mode** (piped input): Shows a numbered menu; enter space/comma-separated indices, `all`, `none`, or blank (to keep previous selection).

Device labels:
- `[IN] <description> (<name>)` — Input device (real microphone)
- `[OUT] <description> (<name>)` — Output device (monitor of a sink)

### Running the daemon

The simplest workflow:

```bash
# Create/setup the sink (interactive)
uv run stt-daemon setup

# Start the daemon in another terminal
# (auto-uses the provisioned sink)
uv run stt-daemon run

# Stop and remove the sink when done
uv run stt-daemon clear --yes
```

### Output examples

**Stdout** (human-readable):
```
[14:23:45] (es) Hola, cómo estás?  [rms=0.041 end_of_turn]
[14:23:52] (en) What time is it?  [rms=0.038 end_of_turn]
[14:24:10] (es) Gracias por tu ayuda.  [rms=0.052 pause]
```

**JSONL** (one line per utterance; only written when `jsonl = true`):
```json
{"ts": "2025-09-20T14:23:45.123456", "lang": "es", "lang_prob": 0.95, "duration_s": 2.5, "rms": 0.04123, "end_reason": "end_of_turn", "text": "Hola, cómo estás?", "transcribe_s": 0.42}
{"ts": "2025-09-20T14:23:52.654321", "lang": "en", "lang_prob": 0.98, "duration_s": 1.8, "rms": 0.03811, "end_reason": "pause", "text": "What time is it?", "transcribe_s": 0.31}
```

`duration_s` is the length of the utterance audio; `transcribe_s` is the wall-clock
time the worker spent transcribing it (inference latency), rounded to milliseconds.
Their ratio is the real-time factor. The field is omitted if the timing was not
measured (e.g. `emit()` called directly without `transcribe_s`).

`end_reason` is one of `end_of_turn` (full `silence_ms` elapsed), `pause` (chunked at
`chunk_pause_ms` because it passed `soft_max_s`), `hard_cap` (`max_utterance_s`
reached), or `flush` (shutdown mid-utterance).

**Audio files** — when `save_audio = true`, each transcribed utterance is also
written as a 16 kHz mono WAV under `audio_dir/YYYY-MM-DD/`, and the record gains
an `"audio"` field with its path:
```json
{..., "end_reason": "pause", "text": "Hola", "audio": "audio/2025-09-20/142345-123-pause.wav"}
```
Saving is best-effort: if a write fails (e.g. disk full), the transcript is still
emitted and a warning is logged.

> ⚠️ **Disk space** — saved audio is **never rotated or cleaned up** automatically.
> At 16 kHz mono 16-bit it costs 32 KB/s, i.e. **~1.9 MB/min, ~115 MB/hour,
> ~2.8 GB/day** of *continuous speech*. Silence is not written, so real usage is
> lower — but an always-on daemon in a talkative room still accumulates quickly.
> Point `audio_dir` at a volume with headroom and prune it yourself (e.g. a cron
> job removing files older than N days), and leave `save_audio = false` unless you
> actually need the clips for auditing.

## Assistant Layer

The assistant is a **separate process** that turns transcriptions into actions and
conversation. The daemon stays a dumb, reliable publisher; the assistant owns all state.

### Topology

- `stt-daemon` → publish-only event socket (`$XDG_RUNTIME_DIR/stt-daemon.sock`, mode `0600`). It never accepts input.
- `stt-assistant` → subscribes to that stream and serves a control socket (`$XDG_RUNTIME_DIR/stt-assistant.sock`, mode `0600`).
- `stt-tui` / `stt-ctl` → clients of the assistant.

### Running

#### Simplest: single command with supervisor

```bash
uv run stt-daemon up --with-tui
```

This starts daemon, assistant, and TUI as child processes under a supervisor.
When you quit the TUI, everything stops cleanly. Daemon and assistant output is
discarded while the TUI owns the screen; nothing is written to disk.

#### Traditional: three terminals

```bash
uv run stt-daemon        # transcription + event bus
uv run stt-assistant     # controller: modes, LLM, tools, audit, control socket
uv run stt-tui           # viewer/controller TUI  (or `stt-tui --dump 10` headless)
```

Or start daemon + assistant once, then connect viewers as needed:

```bash
uv run stt-daemon up             # daemon + assistant
uv run stt-tui                   # optional: viewer in another terminal
uv run stt-tui --dump 5          # optional: headless event dump (5 seconds)
```

### Requirements

A local llama.cpp server exposing an OpenAI-compatible `/v1/chat/completions` with a
tool-calling model (default endpoint `http://localhost:9931/v1/chat/completions`).
Set `[llm].enabled = false` to run without an LLM (commands then report
"no executor configured").

### Modes

| mode | what it does |
|---|---|
| `idle` | listening; nothing is sent to the model |
| `command_armed` | the next complete turn becomes a command; the text accumulates live |
| `executing` | a command is running through the tool loop |
| `chat` | every completed turn is sent to the model; no tools |

Command and chat modes are mutually exclusive.

### The two-press command flow

1. Arm (`stt-ctl command-start`, or `a` in the TUI).
2. Speak; the transcript fills the buffer.
3. Optionally edit with `e` (freezes the buffer, so ASR mistakes can be corrected before the model sees them).
4. Execute (`stt-ctl command-execute` or `Enter`), or cancel (`Esc`).

Execution is gated on **readiness**: no speech open, the last finalized segment's
`segment_done` has arrived, and STT is healthy. With `require_viewer = true`, execute is
refused unless a TUI is attached.

### Control protocol

`stt-ctl` verbs (one message each): `command-start`, `command-execute`, `command-cancel`,
`command-edit --text "..."`, `chat-toggle`, `status`.

Clients attach by sending `subscribe` first (`viewer: true` for the TUI, which makes the
assistant treat it as a watcher for the `require_viewer` gate).

Assistant → clients: `mode`, `command_text`, `transcript`, `speech_start`/`speech_stop`,
`tool_call`, `tool_result`, `chat_message`, `heartbeat`, `error`, plus the direct
replies `ack` (command accepted) and `status`.

### Audit

Only **actions** are persisted, to `audit/YYYY-MM-DD.jsonl`:

```json
{"ts":"...","mode":"command","text":"what time is it","edited":false,
 "tool":"get_datetime","args":{},"side_effect":"read_only","requires_confirmation":false,
 "confirmed":null,"ok":true,"result":"...","latency_ms":412}
```

Commands and chat history are volatile — held in memory and discarded.

### TUI keys

`a` arm · `Enter` execute · `Esc` cancel · `e` edit · `t` chat toggle · `q` quit

**Selecting / copying text:** the TUI captures mouse events (Textual enables mouse
reporting by default), so dragging does **not** select text the way it normally does in
your terminal. Hold `Shift` while dragging to bypass the TUI and use your terminal's
native selection and copy.

### Hotkeys

See [`scripts/hotkeys/`](scripts/hotkeys/README.md) for `sxhkd` and `xbindkeys` examples
that bind those commands to global keys.

## Virtual Sink Provisioning: How It Works

The daemon uses PulseAudio's module system to create a virtual microphone from multiple sources:

### What Gets Created

1. **Null Sink** (`module-null-sink`, e.g., `VirtualMicSink`)
   - A sink that doesn't play anywhere; apps route audio into it instead.
   
2. **Remap Source** (`module-remap-source`, e.g., `VirtualMic`)
   - Exposes the null-sink's monitor as a microphone so the daemon can capture from it.
   
3. **Loopback Modules** (`module-loopback`, one per selected source)
   - Routes each selected input (microphone, speaker monitor, etc.) into the null-sink.

### State Persistence

When you run `uv run stt-daemon setup`, the daemon:
- Creates all modules and records their numeric IDs
- Saves the state to `audio/audio.json` (or `$STT_DAEMON_STATE`)
- Allows `run` to re-provision if the sink is missing (e.g., after reboot)
- Allows `clear` to unload the modules it created — and falls back to
  name-based discovery (`sink_name`/`remap_source_name` in
  `pactl list short modules`) when `audio.json` is missing, so orphaned
  `VirtualMicSink`/`VirtualMic` sinks are still cleaned up (e.g., after a
  reboot or manual state-file delete)

The state file is JSON and can be edited manually if needed, but the daemon handles it automatically.

### Feedback Risk

By default, `monitor_output = false`, so the sink's output is **not** looped back to the default speaker. If you enable it, speakers will play back everything captured in the sink—including your own voice—causing audio feedback. Keep it OFF unless you specifically want monitoring.

## Pre-roll Buffer: Why It Matters

The **pre-roll buffer** is a ring buffer that keeps the last N milliseconds (default 400ms) of audio BEFORE speech is detected. When the VAD confirms speech has started, those pre-roll frames are prepended to the utterance.

**Without pre-roll:** The first phoneme of "hello" might be cut off—you'd get "llo" or "elo" because the VAD fires AFTER the first sound is already gone.

**With pre-roll:** The entire "hello" is captured, including the initial "h" and "e", because the ring buffer preserved them.

### How it works (under the hood)

1. **Silence phase**: Frames are pushed into a ring buffer (maxlen = pre_roll_ms / frame_ms). Old frames fall off.
2. **Speech detected**: VAD returns True for consecutive `speech_start_ms` worth of frames.
3. **Speech confirmed**: `_begin()` is called. The entire ring buffer contents are copied into the segment, then regular frames append to it.
4. **End of turn**: after `silence_ms` of silence the segment is finalized and transcribed.

### Long speech and silence

Two extra rules keep long monologues from becoming one huge blob, and keep
noise from being transcribed:

- **Pause-aligned chunking**: once a segment reaches `soft_max_s` (10s) of speech,
  a shorter `chunk_pause_ms` (350ms) pause ends a *chunk*. The cut lands on a
  natural phrase boundary, so no word is split and no overlap/duplication occurs.
  `max_utterance_s` (30s) remains the last-resort cap for someone who never pauses.
- **Noise suppression**: `min_utterance_ms` drops blips, `rms_floor` drops
  segments too quiet to be speech, `trailing_trim_ms` strips hallucination-prone
  trailing silence before transcription, and Whisper segments below
  `no_speech_prob_max` / `avg_logprob_min` are discarded.

## Troubleshooting

### "Device not found"

```
Error: Device 'VirtualMicSink.monitor' not found. Available sources: [...]
To create VirtualMicSink, run: scripts/audio-sink.sh start
```

**Fix:**
```bash
scripts/audio-sink.sh start
```

Then route audio into VirtualMicSink and try again.

### "Can't select or copy text in the TUI"

**Cause:** Textual enables terminal mouse reporting, so mouse events are consumed by the
app instead of the terminal, and native drag-selection never starts.

**Fix:** hold `Shift` while dragging to force your terminal's native selection, then copy
as usual. (Alternatively, launching the app with mouse support disabled —
`app.run(mouse=False)` — restores plain selection but loses in-app scrolling/clicks.)

### High latency / slow transcription

By default, the daemon uses `compute_type = "int8"` and `device = "cpu"`, which trades speed for accuracy. To speed up:

1. **Pin `cpu_threads` to the physical core count**: `cpu_threads = 16` in `[whisper]`
   (CTranslate2's automatic choice is often slower than an explicit count on many-core CPUs)
2. **Reduce model size**: `--model base` or `--model tiny`
3. **Enable GPU** (if you have CUDA): `--device cuda` with `compute_type float16`
4. **Reduce beam_size**: Edit config to `beam_size = 1` (already default, very fast)

Example:

```bash
uv run stt-daemon --model base --device cuda
```

### "Transcription backlog reached N of M"

This warning means the single transcription worker is falling behind: utterances
are arriving faster than they are being transcribed, and once the queue is full
the oldest ones get dropped. The worker is serial by design (faster-whisper's
model isn't thread-safe), so fix it by making transcription faster rather than by
queueing more:

1. **Smaller/faster model**: `--model base` or `tiny`
2. **GPU**: `--device cuda` with `compute_type float16`
3. **Raise `backlog_warn_at`** or `queue_size` in `[daemon]` only to make the
   warning/dropping less likely — it hides the symptom, not the cause.

### Spanish / English / Other languages

The Faster-Whisper model is **multilingual by default**. It auto-detects language or can be forced:

```bash
uv run stt-daemon --language es   # Force Spanish
uv run stt-daemon --language en   # Force English
uv run stt-daemon --language auto # Auto-detect (default)
```

The output includes the detected language tag: `[14:23:45] (es) ...`.

### Recognizing Spanish vs. English better

- **Force the language**: Use `--language es` or `--language en`
- **Use a larger model**: `--model medium` or `--model large` (slower, more accurate)
- **Tune VAD aggressiveness**: Lower values (0–1) capture more, higher values (2–3) are stricter

### Audio device not responding / parec hangs

If `parec` seems stuck:

1. Check if the source exists: `pactl list short sources`
2. Kill the daemon (Ctrl+C)
3. Verify PulseAudio is running: `systemctl status pulseaudio` or check journalctl
4. Restart PulseAudio if needed: `systemctl restart pulseaudio`
5. Re-run the daemon

### Sounddevice backend issues

If using `--backend sounddevice` and getting import errors:

```bash
uv sync --extra portaudio
```

Then verify available devices:

```bash
python -c "import sounddevice as sd; print(sd.query_devices())"
```

Use the device index or name as `--device 0` or `--device "Microphone (...)`.

### No transcriptions appearing

1. Check VAD is firing: `uv run stt-daemon -v` (verbose logging)
2. Verify audio is loud enough: `uv run stt-daemon --check` (check RMS)
3. Reduce `aggressiveness` (0–1 instead of 2–3) to detect quieter speech
4. Check `min_utterance_ms` isn't too high (default 300ms should be fine)

### Whisper model not loading

```
Error loading model: ...
```

The model is downloaded on first run (~1.5 GB for `small`). Check:

1. Disk space: `df -h` (needs ~2 GB free)
2. Internet: Model is downloaded from Hugging Face
3. HF cache location: Models go to `~/.cache/huggingface/hub/` by default

You can override the cache:

```bash
export HF_HOME=/path/to/cache
uv run stt-daemon --model small
```

## Testing

Run the test suite (no audio hardware required):

```bash
uv run pytest -v
```

Tests cover:

- **Pre-roll buffer** preservation: confirm the first frames are not cut
- **Silence finalization**: utterances end after N silence frames
- **Min/max utterance duration**: blips are discarded, long utterances are finalized
- **Pause-aligned chunking**: long speech splits at pauses (`end_reason = "pause"`)
- **Config loading and language normalization**
- **Provisioning**: `setup`/`ensure`/`teardown`, orphan-name cleanup, and `clear` idempotency
- **Event bus**: envelope codec, ring replay, subscriber fan-out
- **Assistant**: mode machine, command readiness/timeouts, tool schema validation, LLM client/history trimming, audit log
- **TUI**: pure view-state reducer and control-socket client

## Development

### Code structure

```
src/stt_daemon/
├── __init__.py          - Package metadata
├── __main__.py          - Entry point for `python -m stt_daemon`
├── cli.py               - Argument parsing and entry point (`stt-daemon`)
├── ctl.py               - One-shot control client (`stt-ctl`, hotkeys)
├── config.py            - Config loading (TOML)
├── audio.py             - Audio capture (parec, sounddevice)
├── pulse.py             - PulseAudio/pactl wrapper (devices, modules, sink checks)
├── provisioner.py       - Virtual sink + loopback lifecycle (setup/ensure/teardown)
├── state.py             - Provisioning state file (audio/audio.json)
├── picker.py            - Interactive device picker (questionary / numbered menu)
├── segmenter.py         - VAD + pre-roll buffer + utterance segmentation (core)
├── transcriber.py       - Faster-Whisper wrapper
├── output.py            - Sinks (stdout, JSONL, optional WAV)
├── daemon.py            - Orchestrator (threads, queues, event emission)
├── events.py            - NDJSON event envelope + codec
├── bus.py               - Publish-only event bus (socket, ring replay)
├── assistant/
│   ├── modes.py         - Mode state machine (idle/command_armed/executing/chat)
│   ├── service.py       - Modes, command buffer, readiness, timeouts
│   ├── control.py       - Control socket (viewers, commands, broadcast)
│   ├── executor.py      - NullExecutor / LlmExecutor (tool loop)
│   ├── tools.py         - Tool registry + schema validation + get_datetime
│   ├── llm.py           - llama.cpp client + chat session (block trimming)
│   ├── audit.py         - Daily action audit log
│   ├── runner.py        - `stt-assistant` process
│   └── __main__.py      - Entry point for `python -m stt_daemon.assistant`
└── tui/
    ├── state.py         - Pure view-state reducer
    ├── client.py        - Control-socket client
    └── app.py           - Textual app + `stt-tui` entry point

tests/                    - 206 tests, no audio hardware or live LLM required

scripts/
├── audio-sink.sh        - Thin wrapper over `stt-daemon setup|clear|list`
└── hotkeys/             - sxhkd / xbindkeys examples
```

### Running linters

```bash
uv run ruff check .
```

### Building/packaging

```bash
# Build sdist + wheel (hatchling backend, installed in an isolated env by uv)
uv build
```

## Limitations & Design Decisions

1. **Mono only**: Stereo support can be added, but speech is typically mono in transcription pipelines
2. **Single language per transcription**: Whisper detects one language per utterance; mixed-language speech is attributed to the dominant language
3. **LLM is optional and separate**: the daemon itself is transcription-only; the `stt-assistant` process adds chat/commands/tools on top, and can be disabled with `[llm].enabled = false`
4. **PulseAudio/PipeWire-Pulse**: The default `parec` backend assumes UNIX-like systems with PulseAudio. macOS/Windows should use `sounddevice`
5. **Real-time**: Latency depends on frame duration (30ms), silence threshold (default 700ms), and model size. Expect ~1–3 second latency with `small` model on CPU
6. **Thread-safe queues**: Reader thread (audio) is separate from worker thread (transcription) so audio never blocks waiting for transcription

## References

- **Faster-Whisper**: https://github.com/SYSTRAN/faster-whisper
- **WebRTC VAD**: https://github.com/wiseman/py-webrtcvad
- **PulseAudio**: https://www.freedesktop.org/wiki/Software/PulseAudio/

## License

This project is ISC-licensed. See [`LICENSE.md`](LICENSE.md).

---

Made with ❤️ for Efren. Questions? Open an issue or reach out.