STT-Assistant

STT Daemon

An always-on speech-to-text daemon that continuously listens to a PulseAudio/PipeWire audio source, detects voice activity with webrtcvad, and transcribes utterances to Spanish/English text using Faster-Whisper.

Overview

This daemon is designed to:

• Capture audio continuously from PulseAudio/PipeWire sources
• Preserve the start of speech using a pre-roll (ring) buffer, so the first phoneme is never cut when voice detection fires
• Segment utterances with configurable silence thresholds
• Transcribe each utterance with Faster-Whisper (supports multilingual, Spanish, English, or auto-detect)
• Output to stdout (human-readable) and/or JSONL files (for logging/downstream processing)
• Run non-blocking with worker threads so audio keeps flowing during transcription

Architecture

Audio Source (PulseAudio)
    ↓
ParecCapture / SoundDeviceCapture
    ↓
Pre-roll Ring Buffer (VAD lookback)
    ↓
WebRTC-VAD (voice activity detection)
    ↓
FrameSegmenter (utterance boundaries)
    ↓
Queue (segment_queue)
    ↓
Faster-Whisper (transcription)
    ↓
TranscriptSink (stdout + transcripts/YYYY-MM-DD.jsonl) + EventPublisher
    ↓  publish-only UNIX socket ($XDG_RUNTIME_DIR/stt-daemon.sock)
stt-assistant  (subscribes; owns modes, command buffer, chat, LLM, tools)
    ↓  control socket ($XDG_RUNTIME_DIR/stt-assistant.sock)
stt-tui (viewer) / stt-ctl (hotkeys)              audit/YYYY-MM-DD.jsonl ←┘

The daemon never accepts input: it only transcribes and publishes. Everything stateful lives in the assistant, which is a separate process (see Assistant Layer).

Prerequisites

System packages

• PulseAudio or PipeWire-Pulse: pactl, parec command-line tools

  • On Ubuntu/Debian: apt-get install pulseaudio-utils or apt-get install pipewire pipewire-pulse
  • On Fedora: dnf install pulseaudio-utils
  • On macOS: Install via Homebrew (though PulseAudio is Linux-first; sounddevice backend recommended on macOS)

• FFmpeg (optional, used by Faster-Whisper for format conversion)

  • On Ubuntu/Debian: apt-get install ffmpeg

Python

• Python 3.11+ (tested on 3.13.5)
• uv package manager (https://docs.astral.sh/uv/)

Installation

Clone the repo and install with uv:

# From the repo root (wherever you cloned it)
cd /path/to/STT-TTS

# Sync dependencies (installs into .venv)
uv sync

# (Optional) If you want to use the sounddevice backend (requires portaudio):
uv sync --extra portaudio

Configuration

Quick Start

Copy the example config and edit as needed:

cp config.example.toml config.toml
# Edit config.toml to adjust settings (see section below)

Configuration Reference

The config file is TOML. Here are all available options with defaults. config.example.toml ships values only; this section is the annotated reference.

[audio]
backend = "parec"                     # "parec" (PulseAudio by name) | "sounddevice"
device  = "VirtualMicSink.monitor"    # PulseAudio source name (parec) or device index/name (sounddevice)
sample_rate = 16000                   # Hz
channels    = 1                       # Mono only
frame_ms    = 30                      # webrtcvad requires 10, 20, or 30
gain        = 1.0                     # Amplitude scaling (1.0 = no change, >1 = louder)

[vad]
aggressiveness   = 2                  # 0–3: higher = fewer false positives (speech misses more)
pre_roll_ms      = 400                # Ring buffer duration: keep this many ms of pre-speech
speech_start_ms  = 90                 # Consecutive speech frames to confirm speech start
silence_ms       = 700                # Consecutive silence to end a turn
min_utterance_ms = 300                # Discard utterances with less actual speech than this
max_utterance_s  = 30.0               # Hard cap: force-finalize a very long utterance
chunk_pause_ms   = 350                # Once soft_max_s is reached, end a chunk at a pause this long
soft_max_s       = 10.0               # After this much speech, chunk-ends become eligible at pauses
rms_floor        = 0.008              # Drop segments quieter than this RMS (noise-triggered blips)
trailing_trim_ms = 200                # Keep only this much trailing silence (less hallucination-prone padding)

[whisper]
model                   = "small"               # "tiny"/"base"/"small"/"medium"/"large" (multilingual; do NOT use `.en` variants)
device                  = "cpu"                 # "cpu" or "cuda"
compute_type            = "int8"                # "int8" (CPU) or "float16" (CUDA, slower but more accurate)
language                = "auto"                # "auto" (detect), "en", "es", or None/""/"none"
beam_size               = 1                     # 1 = fast greedy, >1 = slower beam search
vad_filter              = false                 # Use Whisper's built-in VAD (we already have webrtcvad)
cpu_threads             = 0                     # CPU inference threads (0 = CTranslate2's automatic choice).
                                                # On many-core CPUs, set this to the physical core count
                                                # (e.g. 16 on a Ryzen 9 8945HX) - measurably faster than auto.
no_speech_prob_max      = 0.6                   # Drop segments Whisper scores as non-speech above this
avg_logprob_min         = -1.0                  # Drop segments with confidence below this
language_strategy       = "static"              # "static" (detect per utterance) | "sticky" (detect once, pin)
sticky_redetect_logprob = -0.8                  # Unpin and re-detect if segment avg_logprob falls below this.
                                                # Segments below avg_logprob_min (-1.0) are dropped; between
                                                # -1.0 and -0.8 they are accepted but trigger re-detection.
                                                # No accepted segments also clears the pin for re-detection.

[sink]                                # Virtual audio sink configuration
sink_name             = "VirtualMicSink"       # null-sink name
remap_source_name     = "VirtualMic"           # Microphone source name
loopback_latency_msec = 1                      # Module latency (ms)
monitor_output        = false                  # Loop sink.monitor → default sink (OFF = no feedback risk)

[output]
stdout     = true                     # Print transcripts to stdout
jsonl_dir  = "transcripts"            # Write JSONL files (one per day) to this dir
save_audio = false                    # Also write each transcribed utterance as a 16 kHz mono WAV
audio_dir  = "audio"                  # WAV destination (YYYY-MM-DD subfolders); only used if save_audio

[daemon]
queue_size      = 50                        # Max utterances queued for transcription (oldest dropped when full)
backlog_warn_at = 5                         # Warn when the queue backs up this far (worker falling behind)

[events]                                    # Publish-only event bus
socket       = ""                           # Default: $XDG_RUNTIME_DIR/stt-daemon.sock
heartbeat_ms = 1000                         # Heartbeat interval (progress-driven, from the capture loop)
ring_size    = 200                          # Recent events replayed to a newly attached subscriber

[assistant]                                 # Controller process
control_socket        = ""                  # Default: $XDG_RUNTIME_DIR/stt-assistant.sock
stall_timeout_ms      = 3000                # No heartbeat for this long -> STT considered offline
worker_stall_ms       = 15000               # Heartbeats arriving but a transcription runs this long -> degraded
command_arm_timeout_s = 10.0                # Disarm a command if no turn is captured in this time
chat_idle_timeout_s   = 300.0               # Leave chat mode after this much silence
turn_gap_s            = 1.5                 # Quiet gap that ends a turn
require_viewer        = true                # Refuse to execute commands with no TUI attached
chat_exit_phrases     = ["end conversation", "that's all", "stop listening"]

[audit]                                     # Executed actions only (commands/chats are volatile)
enabled = true                              # audit/YYYY-MM-DD.jsonl
dir     = "audit"

[llm]                                       # local llama.cpp (OpenAI-compatible)
enabled              = true
endpoint             = "http://localhost:9931/v1/chat/completions"
temperature          = 0.7
max_tokens           = 1024                 # reasoning models spend tokens before the answer; keep generous
timeout_s            = 120.0
cache_prompt         = true                 # reuse the KV prefix (append-only history keeps it warm)
max_prompt_tokens    = 64000                # ~half of the 131072 context
trim_block_turns     = 4                    # trim history in blocks, not per-turn, to limit re-prefill
max_tool_iterations  = 3
system_prompt_chat   = "You are a concise voice assistant. Reply in 1-3 sentences."
system_prompt_command = "You translate a spoken request into a single tool call when one applies."

Audio Source Setup

By default, the daemon listens to VirtualMicSink.monitor (the monitor of the virtual sink). Apps route audio into VirtualMicSink, and the daemon captures the mix.

Option 1: Use the CLI with interactive setup (recommended)

# Show devices and create a virtual sink (interactive picker)
uv run stt-daemon setup

# Check status
uv run stt-daemon list

# In another terminal, start the daemon
uv run stt-daemon run

# The daemon auto-uses the provisioned sink's monitor as the capture device
# (no need to manually specify --device)

# Route audio:
#   - In pavucontrol (GUI), drag apps' "Recording" side to "VirtualMicSink"
#   - Or use pactl: pactl move-sink-input <app-index> VirtualMicSink

# Stop when done
uv run stt-daemon clear --yes

Option 2: Use the provided shell wrapper

For convenience, scripts/audio-sink.sh is a thin wrapper around the Python CLI:

scripts/audio-sink.sh start   # → uv run stt-daemon setup
scripts/audio-sink.sh status  # → uv run stt-daemon list
scripts/audio-sink.sh stop    # → uv run stt-daemon clear --yes

Option 3: Manual PulseAudio setup

If you prefer manual control or want to understand what's happening:

# Get your default microphone source
default_source=$(pactl get-default-source)
echo $default_source  # e.g., alsa_input.usb-...

# Create null sink
pactl load-module module-null-sink sink_name=VirtualMicSink \
    sink_properties=device.description=Virtual-Mic-Sink

# Expose the sink's monitor as a microphone
pactl load-module module-remap-source \
    master=VirtualMicSink.monitor \
    source_name=VirtualMic \
    source_properties=device.description=Virtual-Microphone

# Route your microphone into the sink
pactl load-module module-loopback source="$default_source" sink=VirtualMicSink latency_msec=1

# (Optional) Monitor the sink output on your speakers
pactl load-module module-loopback source=VirtualMicSink.monitor latency_msec=1

# Now tell the daemon to use it
uv run stt-daemon --device VirtualMicSink.monitor

# Clean up (unload modules and remove the file):
pactl unload-module <sink_id>
pactl unload-module <source_id>
pactl unload-module <loopback_id_1>
pactl unload-module <loopback_id_2>

Option 3: Use a physical microphone directly

If you have a single physical mic and don't need the virtual routing:

# List available sources
pactl list short sources

# Start daemon with your source
uv run stt-daemon --device alsa_input.usb-C-Media_Electronics_Inc._USB_Audio_Device-00.mono-fallback

Option 4: Use sounddevice backend

If you prefer not to use PulseAudio (e.g., on macOS or Windows):

uv sync --extra portaudio
uv run stt-daemon --backend sounddevice --device 0  # device 0 = default input

Running the Daemon

Subcommands

The stt-daemon CLI supports these subcommands. Every subcommand also accepts the common overrides --config, --device, --backend, --model, --language, and -v/--verbose (for example stt-daemon setup --device ...).

setup — Create virtual sink and loopbacks

Interactive setup (shows device picker):

uv run stt-daemon setup

Non-interactive setup (specify sources directly):

uv run stt-daemon setup --sources "alsa_input.usb-Foo,alsa_output.pci-0000.monitor"

This creates:

• A null sink (by default VirtualMicSink)
• A remap-source to expose the sink's monitor as a microphone (by default VirtualMic)
• Loopback modules routing each selected source into the sink
• A state file tracking all module IDs for later cleanup

run — Start the daemon (auto-provisions if needed)

uv run stt-daemon run

If no saved provisioning exists and the sink is missing, the daemon will:

1. Show the interactive device picker
2. Create the sink and loopbacks (if sources are selected)
3. Start listening and transcribing

You can also override settings:

uv run stt-daemon run --device "alsa_input.usb-..." --language es --model base
uv run stt-daemon run -v  # Verbose logging
uv run stt-daemon run --check  # Test audio capture + model, then exit

list — Show devices and provisioning status

uv run stt-daemon list

Output shows:

• Input devices (real microphones)
• Output devices (speakers/headphones; captured via their .monitor source)
• Current provisioning state (if a sink has been created)

clear — Remove virtual sink and loopbacks

uv run stt-daemon clear
# or without confirmation prompt:
uv run stt-daemon clear --yes

This unloads all tracked modules and removes the state file. If ~/.local/state/stt-daemon/audio.json is already gone (reboot, manual delete, crash), clear --yes still finds orphaned modules by name (VirtualMicSink/VirtualMic via pactl list short modules) and unloads them in reverse order (loopbacks → remap-source → null-sink), so the command is idempotent — safe to re-run and to recover from Sink 'VirtualMicSink' already exists but is not tracked.

(Legacy) --list-sources flag

uv run stt-daemon --list-sources
# Equivalent to: uv run stt-daemon list

(Legacy) --check flag

uv run stt-daemon --check
# Equivalent to: uv run stt-daemon run --check

Interactive Device Picker

When you run setup or run without --sources, the daemon shows an interactive picker:

• TTY mode (terminal): Uses questionary to show a checkbox UI. Pre-checked items are from the previously saved selection.
• Non-TTY mode (piped input): Shows a numbered menu; enter space/comma-separated indices, all, none, or blank (to keep previous selection).

Device labels:

• [IN] <description> (<name>) — Input device (real microphone)
• [OUT] <description> (<name>) — Output device (monitor of a sink)

Running the daemon

The simplest workflow:

# Create/setup the sink (interactive)
uv run stt-daemon setup

# Start the daemon in another terminal
# (auto-uses the provisioned sink)
uv run stt-daemon run

# Stop and remove the sink when done
uv run stt-daemon clear --yes

Output examples

Stdout (human-readable):

[14:23:45] (es) Hola, cómo estás?  [rms=0.041 end_of_turn]
[14:23:52] (en) What time is it?  [rms=0.038 end_of_turn]
[14:24:10] (es) Gracias por tu ayuda.  [rms=0.052 pause]

JSONL (one line per utterance):

{"ts": "2025-09-20T14:23:45.123456", "lang": "es", "lang_prob": 0.95, "duration_s": 2.5, "rms": 0.04123, "end_reason": "end_of_turn", "text": "Hola, cómo estás?"}
{"ts": "2025-09-20T14:23:52.654321", "lang": "en", "lang_prob": 0.98, "duration_s": 1.8, "rms": 0.03811, "end_reason": "pause", "text": "What time is it?"}

end_reason is one of end_of_turn (full silence_ms elapsed), pause (chunked at chunk_pause_ms because it passed soft_max_s), hard_cap (max_utterance_s reached), or flush (shutdown mid-utterance).

Audio files — when save_audio = true, each transcribed utterance is also written as a 16 kHz mono WAV under audio_dir/YYYY-MM-DD/, and the record gains an "audio" field with its path:

{..., "end_reason": "pause", "text": "Hola", "audio": "audio/2025-09-20/142345-123-pause.wav"}

Saving is best-effort: if a write fails (e.g. disk full), the transcript is still emitted and a warning is logged.

⚠️ Disk space — saved audio is never rotated or cleaned up automatically. At 16 kHz mono 16-bit it costs 32 KB/s, i.e. ~1.9 MB/min, ~115 MB/hour, ~2.8 GB/day of continuous speech. Silence is not written, so real usage is lower — but an always-on daemon in a talkative room still accumulates quickly. Point audio_dir at a volume with headroom and prune it yourself (e.g. a cron job removing files older than N days), and leave save_audio = false unless you actually need the clips for auditing.

Assistant Layer

The assistant is a separate process that turns transcriptions into actions and conversation. The daemon stays a dumb, reliable publisher; the assistant owns all state.

Topology

• stt-daemon → publish-only event socket ($XDG_RUNTIME_DIR/stt-daemon.sock, mode 0600). It never accepts input.
• stt-assistant → subscribes to that stream and serves a control socket ($XDG_RUNTIME_DIR/stt-assistant.sock, mode 0600).
• stt-tui / stt-ctl → clients of the assistant.

Running (three terminals)

uv run stt-daemon        # transcription + event bus
uv run stt-assistant     # controller: modes, LLM, tools, audit, control socket
uv run stt-tui           # viewer/controller TUI  (or `stt-tui --dump 10` headless)

Requirements

A local llama.cpp server exposing an OpenAI-compatible /v1/chat/completions with a tool-calling model (default endpoint http://localhost:9931/v1/chat/completions). Set [llm].enabled = false to run without an LLM (commands then report "no executor configured").

Modes

mode	what it does
idle	listening; nothing is sent to the model
command_armed	the next complete turn becomes a command; the text accumulates live
executing	a command is running through the tool loop
chat	every completed turn is sent to the model; no tools

Command and chat modes are mutually exclusive.

The two-press command flow

1. Arm (stt-ctl command-start, or a in the TUI).
2. Speak; the transcript fills the buffer.
3. Optionally edit with e (freezes the buffer, so ASR mistakes can be corrected before the model sees them).
4. Execute (stt-ctl command-execute or Enter), or cancel (Esc).

Execution is gated on readiness: no speech open, the last finalized segment's segment_done has arrived, and STT is healthy. With require_viewer = true, execute is refused unless a TUI is attached.

Control protocol

stt-ctl verbs (one message each): command-start, command-execute, command-cancel, command-edit --text "...", chat-toggle, status.

Clients attach by sending subscribe first (viewer: true for the TUI, which makes the assistant treat it as a watcher for the require_viewer gate).

Assistant → clients: mode, command_text, transcript, speech_start/speech_stop, tool_call, tool_result, chat_message, heartbeat, error, plus the direct replies ack (command accepted) and status.

Audit

Only actions are persisted, to audit/YYYY-MM-DD.jsonl:

{"ts":"...","mode":"command","text":"what time is it","edited":false,
 "tool":"get_datetime","args":{},"side_effect":"read_only","requires_confirmation":false,
 "confirmed":null,"ok":true,"result":"...","latency_ms":412}

Commands and chat history are volatile — held in memory and discarded.

TUI keys

a arm · Enter execute · Esc cancel · e edit · t chat toggle · q quit

Hotkeys

See scripts/hotkeys/ for sxhkd and xbindkeys examples that bind those commands to global keys.

Virtual Sink Provisioning: How It Works

The daemon uses PulseAudio's module system to create a virtual microphone from multiple sources:

What Gets Created

1. Null Sink (module-null-sink, e.g., VirtualMicSink)

   • A sink that doesn't play anywhere; apps route audio into it instead.
     

2. Remap Source (module-remap-source, e.g., VirtualMic)

   • Exposes the null-sink's monitor as a microphone so the daemon can capture from it.
     

3. Loopback Modules (module-loopback, one per selected source)

   • Routes each selected input (microphone, speaker monitor, etc.) into the null-sink.

State Persistence

When you run uv run stt-daemon setup, the daemon:

• Creates all modules and records their numeric IDs
• Saves the state to ~/.local/state/stt-daemon/audio.json (or $STT_DAEMON_STATE)
• Allows run to re-provision if the sink is missing (e.g., after reboot)
• Allows clear to unload the modules it created — and falls back to name-based discovery (sink_name/remap_source_name in pactl list short modules) when audio.json is missing, so orphaned VirtualMicSink/VirtualMic sinks are still cleaned up (e.g., after a reboot or manual state-file delete)

The state file is JSON and can be edited manually if needed, but the daemon handles it automatically.

Feedback Risk

By default, monitor_output = false, so the sink's output is not looped back to the default speaker. If you enable it, speakers will play back everything captured in the sink—including your own voice—causing audio feedback. Keep it OFF unless you specifically want monitoring.

Pre-roll Buffer: Why It Matters

The pre-roll buffer is a ring buffer that keeps the last N milliseconds (default 400ms) of audio BEFORE speech is detected. When the VAD confirms speech has started, those pre-roll frames are prepended to the utterance.

Without pre-roll: The first phoneme of "hello" might be cut off—you'd get "llo" or "elo" because the VAD fires AFTER the first sound is already gone.

With pre-roll: The entire "hello" is captured, including the initial "h" and "e", because the ring buffer preserved them.

How it works (under the hood)

1. Silence phase: Frames are pushed into a ring buffer (maxlen = pre_roll_ms / frame_ms). Old frames fall off.
2. Speech detected: VAD returns True for consecutive speech_start_ms worth of frames.
3. Speech confirmed: _begin() is called. The entire ring buffer contents are copied into the segment, then regular frames append to it.
4. End of turn: after silence_ms of silence the segment is finalized and transcribed.

Long speech and silence

Two extra rules keep long monologues from becoming one huge blob, and keep noise from being transcribed:

• Pause-aligned chunking: once a segment reaches soft_max_s (10s) of speech, a shorter chunk_pause_ms (350ms) pause ends a chunk. The cut lands on a natural phrase boundary, so no word is split and no overlap/duplication occurs. max_utterance_s (30s) remains the last-resort cap for someone who never pauses.
• Noise suppression: min_utterance_ms drops blips, rms_floor drops segments too quiet to be speech, trailing_trim_ms strips hallucination-prone trailing silence before transcription, and Whisper segments below no_speech_prob_max / avg_logprob_min are discarded.

Troubleshooting

"Device not found"

Error: Device 'VirtualMicSink.monitor' not found. Available sources: [...]
To create VirtualMicSink, run: scripts/audio-sink.sh start

Fix:

scripts/audio-sink.sh start

Then route audio into VirtualMicSink and try again.

High latency / slow transcription

By default, the daemon uses compute_type = "int8" and device = "cpu", which trades speed for accuracy. To speed up:

1. Pin cpu_threads to the physical core count: cpu_threads = 16 in [whisper] (CTranslate2's automatic choice is often slower than an explicit count on many-core CPUs)
2. Reduce model size: --model base or --model tiny
3. Enable GPU (if you have CUDA): --device cuda with compute_type float16
4. Reduce beam_size: Edit config to beam_size = 1 (already default, very fast)

Example:

uv run stt-daemon --model base --device cuda

"Transcription backlog reached N of M"

This warning means the single transcription worker is falling behind: utterances are arriving faster than they are being transcribed, and once the queue is full the oldest ones get dropped. The worker is serial by design (faster-whisper's model isn't thread-safe), so fix it by making transcription faster rather than by queueing more:

1. Smaller/faster model: --model base or tiny
2. GPU: --device cuda with compute_type float16
3. Raise backlog_warn_at or queue_size in [daemon] only to make the warning/dropping less likely — it hides the symptom, not the cause.

Spanish / English / Other languages

The Faster-Whisper model is multilingual by default. It auto-detects language or can be forced:

uv run stt-daemon --language es   # Force Spanish
uv run stt-daemon --language en   # Force English
uv run stt-daemon --language auto # Auto-detect (default)

The output includes the detected language tag: [14:23:45] (es) ....

Recognizing Spanish vs. English better

• Force the language: Use --language es or --language en
• Use a larger model: --model medium or --model large (slower, more accurate)
• Tune VAD aggressiveness: Lower values (0–1) capture more, higher values (2–3) are stricter

Audio device not responding / parec hangs

If parec seems stuck:

1. Check if the source exists: pactl list short sources
2. Kill the daemon (Ctrl+C)
3. Verify PulseAudio is running: systemctl status pulseaudio or check journalctl
4. Restart PulseAudio if needed: systemctl restart pulseaudio
5. Re-run the daemon

Sounddevice backend issues

If using --backend sounddevice and getting import errors:

uv sync --extra portaudio

Then verify available devices:

python -c "import sounddevice as sd; print(sd.query_devices())"

Use the device index or name as --device 0 or --device "Microphone (...).

No transcriptions appearing

1. Check VAD is firing: uv run stt-daemon -v (verbose logging)
2. Verify audio is loud enough: uv run stt-daemon --check (check RMS)
3. Reduce aggressiveness (0–1 instead of 2–3) to detect quieter speech
4. Check min_utterance_ms isn't too high (default 300ms should be fine)

Whisper model not loading

Error loading model: ...

The model is downloaded on first run (~1.5 GB for small). Check:

1. Disk space: df -h (needs ~2 GB free)
2. Internet: Model is downloaded from Hugging Face
3. HF cache location: Models go to ~/.cache/huggingface/hub/ by default

You can override the cache:

export HF_HOME=/path/to/cache
uv run stt-daemon --model small

Testing

Run the test suite (no audio hardware required):

uv run pytest -v

Tests cover:

• Pre-roll buffer preservation: confirm the first frames are not cut
• Silence finalization: utterances end after N silence frames
• Min/max utterance duration: blips are discarded, long utterances are finalized
• Pause-aligned chunking: long speech splits at pauses (end_reason = "pause")
• Config loading and language normalization
• Provisioning: setup/ensure/teardown, orphan-name cleanup, and clear idempotency
• Event bus: envelope codec, ring replay, subscriber fan-out
• Assistant: mode machine, command readiness/timeouts, tool schema validation, LLM client/history trimming, audit log
• TUI: pure view-state reducer and control-socket client

Development

Code structure

src/stt_daemon/
├── __init__.py          - Package metadata
├── __main__.py          - Entry point for `python -m stt_daemon`
├── cli.py               - Argument parsing and entry point (`stt-daemon`)
├── ctl.py               - One-shot control client (`stt-ctl`, hotkeys)
├── config.py            - Config loading (TOML)
├── audio.py             - Audio capture (parec, sounddevice)
├── pulse.py             - PulseAudio/pactl wrapper (devices, modules, sink checks)
├── provisioner.py       - Virtual sink + loopback lifecycle (setup/ensure/teardown)
├── state.py             - Provisioning state file (~/.local/state/stt-daemon/audio.json)
├── picker.py            - Interactive device picker (questionary / numbered menu)
├── segmenter.py         - VAD + pre-roll buffer + utterance segmentation (core)
├── transcriber.py       - Faster-Whisper wrapper
├── output.py            - Sinks (stdout, JSONL, optional WAV)
├── daemon.py            - Orchestrator (threads, queues, event emission)
├── events.py            - NDJSON event envelope + codec
├── bus.py               - Publish-only event bus (socket, ring replay)
├── assistant/
│   ├── modes.py         - Mode state machine (idle/command_armed/executing/chat)
│   ├── service.py       - Modes, command buffer, readiness, timeouts
│   ├── control.py       - Control socket (viewers, commands, broadcast)
│   ├── executor.py      - NullExecutor / LlmExecutor (tool loop)
│   ├── tools.py         - Tool registry + schema validation + get_datetime
│   ├── llm.py           - llama.cpp client + chat session (block trimming)
│   ├── audit.py         - Daily action audit log
│   ├── runner.py        - `stt-assistant` process
│   └── __main__.py      - Entry point for `python -m stt_daemon.assistant`
└── tui/
    ├── state.py         - Pure view-state reducer
    ├── client.py        - Control-socket client
    └── app.py           - Textual app + `stt-tui` entry point

tests/                    - 206 tests, no audio hardware or live LLM required

scripts/
├── audio-sink.sh        - Thin wrapper over `stt-daemon setup|clear|list`
└── hotkeys/             - sxhkd / xbindkeys examples

Running linters

uv run ruff check .

Building/packaging

# Build sdist + wheel (hatchling backend, installed in an isolated env by uv)
uv build

Limitations & Design Decisions

1. Mono only: Stereo support can be added, but speech is typically mono in transcription pipelines
2. Single language per transcription: Whisper detects one language per utterance; mixed-language speech is attributed to the dominant language
3. LLM is optional and separate: the daemon itself is transcription-only; the stt-assistant process adds chat/commands/tools on top, and can be disabled with [llm].enabled = false
4. PulseAudio/PipeWire-Pulse: The default parec backend assumes UNIX-like systems with PulseAudio. macOS/Windows should use sounddevice
5. Real-time: Latency depends on frame duration (30ms), silence threshold (default 700ms), and model size. Expect ~1–3 second latency with small model on CPU
6. Thread-safe queues: Reader thread (audio) is separate from worker thread (transcription) so audio never blocks waiting for transcription

References

• Faster-Whisper: https://github.com/SYSTRAN/faster-whisper
• WebRTC VAD: https://github.com/wiseman/py-webrtcvad
• PulseAudio: https://www.freedesktop.org/wiki/Software/PulseAudio/

License

This project is ISC-licensed. See LICENSE.md.

---

Made with ❤️ for Efren. Questions? Open an issue or reach out.