STT-Assistant/docs/ASSISTANT.md

Assistant Layer

The assistant is a separate process that turns transcriptions into actions and conversation. The daemon stays a dumb, reliable publisher; the assistant owns all state.

Topology

• stt-daemon → publish-only event socket ($XDG_RUNTIME_DIR/stt-daemon.sock, mode 0600). It never accepts input.
• stt-assistant → subscribes to that stream and serves a control socket ($XDG_RUNTIME_DIR/stt-assistant.sock, mode 0600).
• stt-tui / stt-ctl → clients of the assistant.

Running

Simplest: single command with supervisor

uv run stt-daemon up --with-tui

This starts daemon, assistant, and TUI as child processes under a supervisor. When you quit the TUI, everything stops cleanly. Daemon and assistant output is discarded while the TUI owns the screen; nothing is written to disk.

Traditional: three terminals

uv run stt-daemon        # transcription + event bus
uv run stt-assistant     # controller: modes, LLM, tools, audit, control socket
uv run stt-tui           # viewer/controller TUI  (or `stt-tui --dump 10` headless)

Or start daemon + assistant once, then connect viewers as needed:

uv run stt-daemon up             # daemon + assistant
uv run stt-tui                   # optional: viewer in another terminal
uv run stt-tui --dump 5          # optional: headless event dump (5 seconds)

Requirements

A local llama.cpp server exposing an OpenAI-compatible /v1/chat/completions with a tool-calling model (default endpoint http://localhost:9931/v1/chat/completions). Set [llm].enabled = false to run without an LLM (commands then report "no executor configured").

Modes

mode	what it does
idle	listening; nothing is sent to the model
command_armed	the next complete turn becomes a command; the text accumulates live
executing	a command is running through the tool loop
chat	every completed turn is sent to the model; no tools

Command and chat modes are mutually exclusive.

The two-press command flow

1. Arm (stt-ctl command-start, or a in the TUI).
2. Speak; the transcript fills the buffer.
3. Optionally edit with e (freezes the buffer, so ASR mistakes can be corrected before the model sees them).
4. Execute (stt-ctl command-execute or Enter), or cancel (Esc).

Execution is gated on readiness: no speech open, the last finalized segment's segment_done has arrived, and STT is healthy. With require_viewer = true, execute is refused unless a TUI is attached.

Control protocol

stt-ctl verbs (one message each): command-start, command-execute, command-cancel, command-edit --text "...", chat-toggle, clear, status.

Clients attach by sending subscribe first (viewer: true for the TUI, which makes the assistant treat it as a watcher for the require_viewer gate).

Assistant → clients: mode, command_text, transcript, speech_start/speech_stop, tool_call, tool_result, chat_message, cleared, heartbeat, error, plus the direct replies ack (command accepted) and status.

Audit

Only actions are persisted, to audit/YYYY-MM-DD.jsonl:

{"ts":"...","mode":"command","text":"what time is it","edited":false,
 "tool":"get_datetime","args":{},"side_effect":"read_only","requires_confirmation":false,
 "confirmed":null,"ok":true,"result":"...","latency_ms":412}

Commands and chat history are volatile — held in memory and discarded.

Conversation memory (compaction)

Chat history lives in memory only. To avoid overflowing the model's context, the assistant compacts rather than deletes: when the prompt reaches compaction_trigger_ratio of the derived budget (max_context_tokens - max_tokens - compaction_safety_margin), the oldest turns are summarized by the same endpoint (using compaction_system_prompt, capped at compaction_max_tokens) and replaced by one rolling summary, while the most recent compaction_keep_turns stay verbatim. Large evictions are summarized in chunks and folded.

The summary is injected as a second system message, so the prefix (system + summary) is stable between compactions — a compaction invalidates the server's prefix cache once, and subsequent turns are append-only and cache-friendly. If summarization fails (error, empty reply, or a truncated summary) the assistant falls back to deleting the oldest turns in blocks (trim_block_turns); compaction is never the reason a request exceeds the window. Each compaction emits a chat_message with role: "note" so the pane shows why the context changed — the note is UI-only and never sent to the model.

A reply cut off by the output limit (finish_reason: "length") surfaces as error{code:"truncated"} instead of disappearing silently; an empty truncated turn is not stored.

Inference queue

Every LLM call — chat turns, command executions, and the compaction summarizer — runs through a single worker thread, bounded by inference_queue_size. This serializes requests to the one llama.cpp server. Commands jump ahead of chat turns; when the queue is full an execute is refused with error{code:"busy"} and the oldest queued chat turn is dropped. Leaving chat mode discards queued chat turns.

TUI keys

a arm · Enter execute · Esc cancel · e edit · t chat toggle · c clear · q quit

c asks for confirmation, then wipes the conversation / command-output pane and resets the assistant's in-memory chat history (the LLM session), while staying in the current mode. Transcripts are not affected.

Live queue counters: The pane borders show real-time queue depths. transcripts [seg N] displays the number of audio segments queued for transcription (the daemon's backlog). conversation / command output [inf N] shows the number of LLM inference jobs queued, with · busy appearing while one is running. These counters refresh about once per second, driven by the heartbeat cadence.

Selecting / copying text: the TUI captures mouse events (Textual enables mouse reporting by default), so dragging does not select text the way it normally does in your terminal. Hold Shift while dragging to bypass the TUI and use your terminal's native selection and copy.

Hotkeys

See scripts/hotkeys/ for sxhkd and xbindkeys examples that bind those commands to global keys.