# STT Daemon

An always-on speech-to-text daemon that listens to a PulseAudio/PipeWire source, detects voice activity, and transcribes utterances to Spanish/English text, with a separate voice-assistant layer for commands and chat on top.

## Overview

- **Capture audio continuously** from PulseAudio/PipeWire sources
- **Preserve the start of speech** with a pre-roll (ring) buffer, so the first phoneme is never cut when voice detection fires
- **Segment utterances** with configurable silence thresholds
- **Transcribe** each utterance with Faster-Whisper (multilingual: Spanish, English, or auto-detect)
- **Output** to stdout (human-readable) and/or JSONL files
- **Run non-blocking** with worker threads so audio keeps flowing during transcription
- **Act on speech** through the separate `stt-assistant` process (modes, commands, chat, tools, audit)

### Architecture

```
Audio Source (PulseAudio)
    ↓
ParecCapture / SoundDeviceCapture
    ↓
Pre-roll Ring Buffer (VAD lookback)
    ↓
WebRTC-VAD (voice activity detection)
    ↓
FrameSegmenter (utterance boundaries)
    ↓
Queue (segment_queue)
    ↓
Faster-Whisper (transcription)
    ↓
TranscriptSink (stdout + <session_dir>/<id>/transcripts.jsonl) + EventPublisher
    ↓  publish-only UNIX socket ($XDG_RUNTIME_DIR/stt-daemon.sock)
stt-assistant  (subscribes; owns modes, command buffer, chat, LLM, tools)
    ↓  control socket ($XDG_RUNTIME_DIR/stt-assistant.sock)
stt-tui (viewer) / stt-ctl (hotkeys)     troubleshoot/audit/YYYY-MM-DD.jsonl ←┘
```

The daemon never accepts input: it only transcribes and publishes. Everything stateful lives in the assistant, a separate process (see [docs/ASSISTANT.md](docs/ASSISTANT.md)).

## Prerequisites

### System packages

- **PulseAudio or PipeWire-Pulse**: `pactl`, `parec` command-line tools
  - On Ubuntu/Debian: `apt-get install pulseaudio-utils` or `apt-get install pipewire pipewire-pulse`
  - On Fedora: `dnf install pulseaudio-utils`
  - On macOS: Install via Homebrew (though PulseAudio is Linux-first; the `sounddevice` backend is recommended on macOS)

- **FFmpeg** (optional, used by Faster-Whisper for format conversion)
  - On Ubuntu/Debian: `apt-get install ffmpeg`

### Python

- **Python 3.11+** (tested on 3.13.5)
- **uv** package manager (https://docs.astral.sh/uv/)

## Installation

Clone the repo and install with `uv`:

```bash
# From the repo root (wherever you cloned it)
cd /path/to/STT-TTS

# Sync dependencies (installs into .venv)
uv sync

# (Optional) If you want to use the sounddevice backend (requires portaudio):
uv sync --extra portaudio

# (Optional) Enable the triage panel (installs the pinned in-process Laya lane):
uv sync --extra triage

# (Optional) Enable speaker identification (installs sherpa-onnx):
uv sync --extra speaker
```

The `triage` extra is opt-in: the default sync does not install Laya, and the test suite stays green without it.

The `speaker` extra is likewise opt-in and **off by default**. To enable speaker identification,
run `uv sync --extra speaker` and place `nemo_en_titanet_small.onnx` under `models/` (gitignored):

```bash
curl -L -o models/nemo_en_titanet_small.onnx \
  "https://github.com/k2-fsa/sherpa-onnx/releases/download/speaker-recongition-models/nemo_en_titanet_small.onnx"
```

Then set `model_path` and `enabled = true` under `[speaker]` in `config.toml`. Without the extra
and model, the default sync is ONNX-free and speaker status stays `"none"`. Note that
`uv sync --extra speaker` is exclusive: it pins the environment to the default deps plus
`speaker`, which drops the `triage` extra and its stack. To keep both, pass both flags:
`uv sync --extra speaker --extra triage`. See
[docs/SPEAKERS.md](docs/SPEAKERS.md) for the full reference. You can also pre-enroll a known
speaker from existing WAV files with `stt-daemon speaker-enroll`; see the pre-enrollment
section of [docs/SPEAKERS.md](docs/SPEAKERS.md#pre-enroll-from-a-wav-file). For a guided flow,
`stt-daemon speaker-enroll-seed` finds the clips closest to a seed you trust and lets you review
each one before enrolling; see
[docs/SPEAKERS.md](docs/SPEAKERS.md#assisted-enrollment-from-a-seed-clip). Profiles can be tuned by
re-running assisted enrollment (prototypes append, and the oldest is evicted at the cap), and the
store can be wiped and rebuilt; see [docs/SPEAKERS.md](docs/SPEAKERS.md) for both. Assisted review
now flags near-duplicate and likely-other-speaker candidates and both enrollment commands strip
near-duplicates from the profile automatically, while `stt-daemon speaker-eval --labeled DIR`
scores the store against labeled clips; see
[docs/SPEAKERS.md](docs/SPEAKERS.md#measuring-accuracy-speaker-eval).

## Quick Start

Copy the example config and edit as needed:

```bash
cp config.example.toml config.toml
```

To turn on the triage panel, also copy the questions file and enable the section:

```bash
cp questions.example.json questions.json
# then set `enabled = true` under `[triage]` in config.toml
```

`questions.json` is private user state and is gitignored, like `config.toml`. See [docs/TRIAGE.md](docs/TRIAGE.md) for the panel and the questions format.

Then launch the whole system with one command:

```bash
uv run stt-daemon up --with-tui
```

That supervises the daemon, the assistant, and the TUI together; quitting the TUI stops everything cleanly. The manual flow is the same three steps without the supervisor:

```bash
uv run stt-daemon setup       # create the virtual sink (interactive picker)
uv run stt-daemon run         # start listening and transcribing
uv run stt-daemon clear --yes # tear down the sink when done
```

See [docs/CLI.md](docs/CLI.md) for every subcommand and [docs/CONFIGURATION.md](docs/CONFIGURATION.md) for the config reference.

## Documentation

- [docs/CONFIGURATION.md](docs/CONFIGURATION.md) — annotated TOML reference for every config key.
- [docs/AUDIO-SETUP.md](docs/AUDIO-SETUP.md) — virtual sink creation and audio routing options.
- [docs/CLI.md](docs/CLI.md) — `stt-daemon` subcommands and common overrides.
- [docs/ASSISTANT.md](docs/ASSISTANT.md) — the assistant layer: modes, command flow, LLM, tools, audit.
- [docs/TRIAGE.md](docs/TRIAGE.md) — the triage panel: Laya lane, the living brief, and questions file.
- [docs/SPEAKERS.md](docs/SPEAKERS.md) — speaker identification: enrollment, thresholds, store, and commands.
- [docs/OUTPUT.md](docs/OUTPUT.md) — stdout, JSONL, and WAV output formats and fields.
- [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) — common errors and how to fix them.
- [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) — code structure, testing, linting, and packaging.
- [docs/PYWHISPERCPP.md](docs/PYWHISPERCPP.md) — building the `pywhispercpp` GPU backend from source.

## Testing

Run the full suite with no audio hardware or live LLM required:

```bash
uv run pytest -v
```

See [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md) for what the tests cover.

## References

- **Faster-Whisper**: https://github.com/SYSTRAN/faster-whisper
- **WebRTC VAD**: https://github.com/wiseman/py-webrtcvad
- **PulseAudio**: https://www.freedesktop.org/wiki/Software/PulseAudio/

## License

This project is ISC-licensed. See [`LICENSE.md`](LICENSE.md).