STT-Assistant/docs/CONFIGURATION.md

Configuration

Annotated reference for every stt-daemon configuration key; config.example.toml ships the values only.

Quick Start

Copy the example config and edit as needed:

cp config.example.toml config.toml
# Edit config.toml to adjust settings (see section below)

Configuration Reference

The config file is TOML. Here are all available options with defaults. config.example.toml ships values only; this section is the annotated reference.

[audio]
backend = "parec"                     # "parec" (PulseAudio by name) | "sounddevice"
device  = "VirtualMicSink.monitor"    # PulseAudio source name (parec) or device index/name (sounddevice)
sample_rate = 16000                   # Hz
channels    = 1                       # Mono only
frame_ms    = 30                      # webrtcvad requires 10, 20, or 30
gain        = 1.0                     # Amplitude scaling (1.0 = no change, >1 = louder)

[vad]
aggressiveness   = 2                  # 0–3: higher = fewer false positives (speech misses more)
pre_roll_ms      = 400                # Ring buffer duration: keep this many ms of pre-speech
speech_start_ms  = 90                 # Consecutive speech frames to confirm speech start
silence_ms       = 700                # Consecutive silence to end a turn
min_utterance_ms = 300                # Discard utterances with less actual speech than this
max_utterance_s  = 30.0               # Hard cap: force-finalize a very long utterance
chunk_pause_ms   = 350                # Once soft_max_s is reached, end a chunk at a pause this long
soft_max_s       = 10.0               # After this much speech, chunk-ends become eligible at pauses
rms_floor        = 0.008              # Drop segments quieter than this RMS (noise-triggered blips)
trailing_trim_ms = 200                # Keep only this much trailing silence (less hallucination-prone padding)

[whisper]
backend                 = "faster-whisper"     # "faster-whisper" (CTranslate2) | "pywhispercpp" (whisper.cpp; see docs/PYWHISPERCPP.md)
model                   = "small"               # "tiny"/"base"/"small"/"medium"/"large" (multilingual; do NOT use `.en` variants)
device                  = "cpu"                 # "cpu" or "cuda"
compute_type            = "int8"                # "int8" (CPU) or "float16" (CUDA, slower but more accurate)
language                = "auto"                # "auto" (detect), "en", "es", or None/""/"none"
beam_size               = 1                     # 1 = fast greedy, >1 = slower beam search
vad_filter              = false                 # Use Whisper's built-in VAD (we already have webrtcvad)
cpu_threads             = 0                     # CPU inference threads (0 = CTranslate2's automatic choice).
                                                # On many-core CPUs, set this to the physical core count
                                                # (e.g. 16 on a Ryzen 9 8945HX) - measurably faster than auto.
no_speech_prob_max      = 0.6                   # Drop segments Whisper scores as non-speech above this
avg_logprob_min         = -1.0                  # Drop segments with confidence below this
language_strategy       = "static"              # "static" (detect per utterance) | "sticky" (detect once, pin)
sticky_redetect_logprob = -0.8                  # Unpin and re-detect if segment avg_logprob falls below this.
                                                # Segments below avg_logprob_min (-1.0) are dropped; between
                                                # -1.0 and -0.8 they are accepted but trigger re-detection.
                                                # No accepted segments also clears the pin for re-detection.

[sink]                                # Virtual audio sink configuration
sink_name             = "VirtualMicSink"       # null-sink name
remap_source_name     = "VirtualMic"           # Microphone source name
loopback_latency_msec = 1                      # Module latency (ms)
monitor_output        = false                  # Loop sink.monitor → default sink (OFF = no feedback risk)

[output]
stdout          = true               # Print transcripts to stdout
jsonl           = false              # Persist transcripts to JSONL files (off by default for agent-only use)
session_dir     = "transcripts"      # Conversation root: one folder per conversation, named by the first transcription's YYYY-MM-DD-HH-MM-SS
save_audio      = false              # Also write each transcribed utterance as a 16 kHz mono WAV under <session_dir>/<id>/audio/
session_gap_min = 5.0                # Quiet gap in minutes before a new conversation starts; <= 0 disables rollover

[daemon]
queue_size      = 50                        # Max utterances queued for transcription (oldest dropped when full)
backlog_warn_at = 5                         # Warn when the queue backs up this far (worker falling behind)

[events]                                    # Publish-only event bus
socket       = ""                           # Default: $XDG_RUNTIME_DIR/stt-daemon.sock
heartbeat_ms = 1000                         # Heartbeat interval (progress-driven, from the capture loop)
ring_size    = 200                          # Recent events replayed to a newly attached subscriber

[assistant]                                 # Controller process
control_socket        = ""                  # Default: $XDG_RUNTIME_DIR/stt-assistant.sock
stall_timeout_ms      = 3000                # No heartbeat for this long -> STT considered offline
worker_stall_ms       = 15000               # Heartbeats arriving but a transcription runs this long -> degraded
command_arm_timeout_s = 10.0                # Disarm a command if no turn is captured in this time
chat_idle_timeout_s   = 300.0               # Leave chat mode after this much silence
turn_gap_s            = 1.5                 # Quiet gap that ends a turn
require_viewer        = true                # Refuse to execute commands with no TUI attached
chat_exit_phrases     = ["end conversation", "that's all", "stop listening"]

[audit]                                     # Executed actions only (commands/chats are volatile)
enabled = true                              # troubleshoot/audit/YYYY-MM-DD.jsonl
dir     = "troubleshoot/audit"

[llm]                                       # local llama.cpp (OpenAI-compatible)
enabled                  = true
endpoint                 = "http://localhost:9931/v1/chat/completions"
model                    = ""               # value sent as the request `model`; empty = omit (llama.cpp ignores it, hosted OpenAI-compatible APIs require it)
api_key_env              = ""               # name of the env var holding the bearer token; empty = no Authorization header (never put the secret here)
session_id               = ""               # value sent in `session_header`; stable per lane, e.g. "stt-chat"
session_header           = ""               # header name carrying the session id; OpenCode Go requires "x-opencode-session" (400 MissingSessionID without it)
user_agent               = ""               # User-Agent header; OpenCode Go asks clients to self-identify (generic SDK names discouraged)
reasoning_effort         = ""               # OpenAI-style thinking level sent as `reasoning_effort` (e.g. "low"/"medium"/"high"); empty = omit
extra_body               = {}               # provider-specific request-body passthrough, merged last (e.g. { enable_thinking = true, chat_template_kwargs = { enable_thinking = true } })
temperature              = 0.7
max_tokens               = 1024             # max output per request; reasoning models spend tokens before the answer, so keep generous
timeout_s                = 120.0
cache_prompt             = true             # reuse the KV prefix (append-only history keeps it warm)
chat_slot_id             = ""               # optional llama.cpp id_slot for chat; empty/unset omits it (server needs -np 2)
command_slot_id          = ""               # optional llama.cpp id_slot for command tool-calls; empty/unset omits it (server needs -np 2)
max_context_tokens       = 131072           # the server's --ctx-size (llama.cpp `GET /props` -> n_ctx); the prompt budget derives from it
compaction_safety_margin = 256              # tokens reserved for the token estimate error + template/tool overhead
compaction_enabled       = true             # summarize old turns instead of deleting them
compaction_trigger_ratio = 0.8              # compact when the prompt reaches this fraction of the budget
compaction_keep_turns    = 6                # recent turns kept verbatim
compaction_max_tokens    = 512              # summary length cap
compaction_timeout_s     = 30.0             # summarizer timeout, so a compaction cannot hold the inference lane
compaction_input_tokens  = 8192             # summarizer input cap before chunk-and-fold
compaction_system_prompt = "You compress a conversation. Preserve names, numbers, decisions, user preferences, and unresolved questions. Omit pleasantries. Output only the summary."
brief_compaction_system_prompt = "You compress a triage briefing conversation. Preserve every opportunity, blocker, owner, responsibility, decision, deadline, and open question; drop pleasantries. Output only the summary."
inference_queue_size     = 8                # bounded single-lane inference queue
trim_block_turns         = 4                # fallback block trimming, not per-turn, to limit re-prefill
max_tool_iterations      = 3
system_prompt_chat       = "You are a concise voice assistant. Reply in 1-3 sentences."
system_prompt_command    = "You translate a spoken request into a single tool call when one applies."
triage_system_prompt     = "You maintain a living brief of a business conversation. The caller sends CURRENT LEDGER plus CURRENT BRIEF plus NEW UTTERANCES. Reply with exactly two sections in this order: NEW FACTS: first, then BRIEF:. Facts go first so a truncated reply still preserves extraction (truncation safety). In NEW FACTS:, emit one fact per line as '- <fact>', or the single line 'none' when the new utterances add no new facts. Never repeat facts already listed in CURRENT LEDGER. In BRIEF:, return the UPDATED brief as plain text only, with no markdown fences and no JSON. Open with a Summary section: a rolling overview of the conversation so far, one to a few sentences for a short session and growing to a short paragraph or two as the session extends. Preserve prior content unless the new utterances contradict it; merge duplicates; prune stale items; keep owners, responsibilities, and deadlines explicit. Use short sections: Summary, Opportunities, Blockers/Risks, Responsibilities, Decisions, Action Items, Deadlines, Open Questions. Never emit a 'Prior Details:' section. Output the two sections directly; never emit tool calls, function-call syntax, or special wrapper tokens."

[triage]                                    # Display-only signal panel (Laya lane; no socket, no extra process)
enabled              = false                # Master gate: false = no lane, no panel, behavior identical to today
questions_file       = "questions.json"     # Laya questions JSON in the cwd; missing/invalid -> no_questions
window               = 500                    # Utterances available per classification; the char budget is the real limiter
state_char_budget    = 3000                 # Char cap (~750 tokens) for the condensed state sent to Laya
periodic_s           = 30.0                 # Accumulating-speech cadence (s): while utterances keep arriving without a settled turn, re-run triage every N seconds; <= 0 disables
predict_max_tokens   = 1024                 # Max tokens sent to Laya; the multilingual checkpoint supports 1024; state shares this budget with the ~192-token question head (~830-token state room)
confidence_field     = "answer_confidence"  # Gating confidence: "answer_confidence" (calibrated max(p)) | "confidence"
escalate_threshold   = 0.6                  # noul answers at or above this p are flagged (⚠ marker only)
choice_min_confidence = 0.85                # choice answers below this confidence are flagged (⚠ marker only)
extract_enabled      = true                 # Master switch for the brief engine (plain-text brief update after each Laya pass)
extract_max_tokens   = 3072                 # Brief reply token cap; the brief is then capped to brief_max_chars characters
extract_timeout_s    = 30.0                 # Brief request timeout
extract_temperature  = 0.2                  # Brief request temperature (deterministic extraction; chat keeps [llm].temperature)
brief_slot_id        = ""                   # optional llama.cpp id_slot for the brief engine; empty/unset omits it (server needs -np 2)
brief_gate_enabled   = true                 # fail-open gate: update the brief only when the top signal's p >= brief_gate_threshold
brief_gate_threshold = 0.5                  # gate threshold; the top signal at/above this p triggers a brief update
brief_max_chars      = 4000                 # display cap for the brief (truncated, never dropped) when over this
brief_pending_char_budget = 3000            # pending transcript chars that force an immediate flush of a coalesced update
ledger_max_facts     = 200                  # cap on code-owned durable facts; oldest are evicted (logged) past this
fact_max_chars       = 240                  # per-fact character cap; over-long facts are truncated
device               = ""                   # "" = auto -> None (Laya picks the multilingual checkpoint's default device)
preload              = ["multilingual"]     # Fixed default: only the multilingual checkpoint is preloaded
default_checkpoint   = "multilingual"       # Router default; multilingual is pinned for every call
dump_io              = false                # Opt-in triage I/O dump: one JSON per triage under troubleshoot/triage/YYYY-MM-DD/HHMMSS-mmm.json (Laya request/response + LLM request/response; best-effort, gitignored)

[speaker]                                   # Speaker identification (sherpa-onnx embedding lane; off by default)
enabled           = false                    # Master switch: false = no embedding lane, every record status "none"
model_path        = ""                       # Path to the sherpa-onnx embedding .onnx (e.g. "models/nemo_en_titanet_small.onnx"); empty/unset disables
provider          = "cpu"                    # "cpu" | "cuda"; "cuda" needs a CUDA-enabled sherpa wheel, else warns and falls back to "cpu"
threshold_id      = 0.60                     # T_id: cosine score at/above which a turn matches an existing prototype
threshold_new     = 0.45                     # T_new: cosine score below which a turn may mint a new provisional
min_enroll_s      = 2.0                      # Mint/adapt duration gate: utterances shorter than this never create/adapt
rms_floor         = 0.0                      # Mint/adapt level gate: utterances quieter than this RMS never create/adapt
margin            = 0.04                     # Best-2nd margin: require top1 - top2 >= this before adapt/mint decisions
mint_cooldown_s   = 60.0                     # Cooldown between mints (no second mint within this many seconds)
confirm_s         = 12.0                     # Cumulative speech seconds a provisional needs to become confirmed
max_prototypes    = 8                        # Cap on stored prototypes per speaker
provisional_ttl_h = 24.0                     # Prune provisionals that never reach confirm_s within this many hours (startup)
store_dir         = "speakers"               # Store root (cwd-relative); gitignored (biometric artifacts)
join_timeout_s    = 2.0                      # Embedding-lane join timeout before emitting the transcript

[brief]                                      # Durable brief artifact (plain-text markdown snapshot; off by default)
write = false                                # false = brief shown in the TUI only; true = also persist under the conversation folder

[debug]                                     # Structured health-signal log (no transcript text; off by default)
enabled          = false                    # false = no debug lines, zero overhead (DebugSink is None)
dir              = "troubleshoot/debug"    # Destination directory (cwd-relative); gitignored, one JSONL per day
heartbeat        = true                     # true = ~1 line/s heartbeat signals; false = anomalies only
max_bytes_per_day = 50000000                # Best-effort per-day byte cap; writes past the cap are skipped

[speaker] adds opt-in speaker identification: a dedicated in-process sherpa-onnx embedding lane maps each utterance to a vector, which is matched (max over stored prototypes) against the local store_dir store to label the turn. With enabled = false (the default) the lane is never built and every record keeps speaker_status = "none". With enabled = true but no model_path (or a model that fails to load) the feature disables gracefully: it warns once and never crashes the daemon.

Place the embedding model, nemo_en_titanet_small.onnx (CC-BY-4.0, 192-dim), under models/, which is gitignored; set model_path = "models/nemo_en_titanet_small.onnx". Download it with (note the upstream release tag's spelling, speaker-recongition-models):

curl -L -o models/nemo_en_titanet_small.onnx \
  "https://github.com/k2-fsa/sherpa-onnx/releases/download/speaker-recongition-models/nemo_en_titanet_small.onnx"

The installed onnxruntime exposes only CPUExecutionProvider, so provider = "cpu" is the default and the recommended value; provider = "cuda" requires a CUDA-enabled sherpa-onnx wheel (not onnxruntime-gpu beside the CPU wheel) and otherwise warns and falls back to "cpu". See SPEAKERS.md for enrollment, thresholds, the store layout, and the stt-ctl commands.

threshold_id, threshold_new, margin, and the mint gates are model- and channel-dependent. The defaults are a starting point only — calibrate them on real far-field audio from the actual microphone/room before trusting the labels, and use the per-record scores to tune.

backend picks the transcription engine. faster-whisper is the default, so nothing changes unless you opt in. pywhispercpp selects the whisper.cpp backend (GPU via the vendored CUDA wheel; see PYWHISPERCPP.md). model, language, beam_size, cpu_threads, no_speech_prob_max, avg_logprob_min, language_strategy, and sticky_redetect_logprob apply to both; device and compute_type apply only to faster-whisper; vad_filter is ignored (with a warning) on pywhispercpp. For pywhispercpp the ggml quantization is chosen by the model name (e.g. small, small-q5_1).

max_prompt_tokens was removed: the prompt budget is derived as max_context_tokens - max_tokens - compaction_safety_margin, so the window and the trim budget can never disagree. A legacy max_prompt_tokens key is ignored with a warning.

[triage] adds a display-only signal panel. On each settled turn in any mode, independent of chat, the assistant classifies the last window transcribed utterances (from the STT stream, not the chat history) with a local Laya multilingual checkpoint on a dedicated in-process lane, then, when extract_enabled is true, asks the [llm] lane on a dedicated brief conversation to fold the turn into a living plain-text brief (on a Laya error the prompt carries transcript state only). A settled turn means speech stopped plus quiet for [assistant].turn_gap_s; because a long unbroken monologue never settles, periodic_s (default 30 seconds) additionally re-runs classification every N seconds while new utterances keep arriving (a repeated window is still skipped), so the panel stays current mid-stream. Set periodic_s = 0 to require a settled turn again. extract_enabled stays the brief engine's master switch: with it false the panel still classifies but no brief update is requested. device = "" means auto: it maps to None, so Laya picks the default device for the multilingual checkpoint. preload = ["multilingual"] is the fixed default and default_checkpoint = "multilingual"; the multilingual checkpoint is pinned for every call (there is no .en English-only variant). predict_max_tokens is the Laya token budget (the multilingual checkpoint supports 1024); the condensed state shares it with the ~192-token question head, leaving ~830 tokens of room for state at the default. confidence_field selects which of Laya's two confidence values drives the per-item flagged/⚠ marker, with answer_confidence (the calibrated max(p)) as the default. With enabled = false (the default) the lane is never built and the system behaves exactly as before. dump_io = true additionally writes one JSON file per triage under troubleshoot/triage/YYYY-MM-DD/HHMMSS-mmm.json (local time, millisecond precision, same naming scheme as the audio files) containing the Laya request/response and the LLM request/response; it is off by default, best-effort (a dump failure never disturbs triage), and gitignored.

[llm].triage_system_prompt drives the brief engine: it is the static system message, and the user turn carries the code-owned fact ledger (CURRENT LEDGER), the volatile narrative (CURRENT BRIEF), and the new transcript segment (NEW UTTERANCES). The engine keeps no conversation history, so the prompt stays flat instead of growing. The model must return exactly two sections: NEW FACTS: (only genuinely-new durable facts, one per - line, or none) and BRIEF: (the volatile narrative sections). It never rewrites the ledger. Code owns the ledger: it parses the reply, appends new facts with conservative dedupe, evicts oldest past ledger_max_facts (logged), truncates facts past fact_max_chars, and renders the final brief as narrative + a Prior Details: section. A recorded fact can therefore only disappear via a logged cap eviction, never because the model dropped it. The model reply is bounded by [triage].extract_max_tokens (default 3072), extract_timeout_s (default 30.0), and extract_temperature (default 0.2, deterministic extraction; [llm].temperature still governs chat/command), and the rendered brief is capped to brief_max_chars characters. NEW FACTS: is emitted first so a length cutoff loses narrative (regenerated each pass) rather than a durable fact; the final fact line is discarded when the reply is truncated. Because the brief engine keeps no history, [llm].brief_compaction_system_prompt and the [llm] compaction settings do not apply to it (they still govern chat/command).

The gate brief_gate_enabled (default true) is fail-open: a turn updates the brief only when the top signal's calibrated p is at least brief_gate_threshold (default 0.5), and any missing or non-numeric signal passes. brief_pending_char_budget bounds how many pending transcript characters accumulate before a coalesced update is flushed, so a burst of turns does not stack one in-flight brief job after another.

chat_slot_id, command_slot_id, and brief_slot_id pin each engine's request to a specific llama.cpp id_slot. When unset (the default) the id_slot field is omitted from the request body; setting any of them requires the server to expose slot support (llama-server -np 2 or more), and the server rejects an out-of-range slot.

[brief] controls persistence of that living brief. With write = false (the default) it stays display-only: the TUI renders it and nothing is written to disk. With write = true the assistant snapshots the current brief as plain text to <session_dir>/<YYYY-MM-DD-HH-MM-SS>/brief.md (cwd-relative, gitignored): one folder per conversation, named by the first transcription's YYYY-MM-DD-HH-MM-SS, holding that conversation's transcript JSONL, audio, brief, and screenshots, while the troubleshoot/ diagnostics remain global. Starting a new conversation resets the brief; clearing the brief truncates the file.

[debug] persists per-component health signals to troubleshoot/debug/YYYY-MM-DD.jsonl (one JSON object per line, ensure_ascii=False) so the next silent freeze is diagnosable from disk. Signals only — no transcript text, no audio, no prompt content: RMS/queue/timing fields such as device_ok, worker_stall_s, queue_depth, inference_depth, plus anomaly lines for capture reopens, inference-queue queue-full / eviction-drop, llm-timeout, worker worker-stall, triage lifecycle (triage-submit / triage-skip / triage-result / triage-brief with a trigger/reason, plus triage-brief-failure), and supervisor/child exit lines. With enabled = false (the default) no sink is built and the suite stays disk-clean; with heartbeat = false only anomalies are written. max_bytes_per_day is a best-effort guard: writes past the cap are skipped with one warning (never crash, never rotate). troubleshoot/debug/ lives on local storage (a slow/NFS/FUSE mount can block the only capturer); the whole troubleshoot/ tree is gitignored like transcripts/; clean it manually with rm -r troubleshoot/. The assistant constructs its sink via debug_sink_factory threaded through runner.main (None when disabled), and every assistant emit runs after its with self._lock block — never under the RLock — so a slow write cannot stall the ticker + event reader.

Upgrading from a v5-draft config: socket, model, predict_timeout_s, checkpoint, subfolder, and max_len are not [triage] keys and are silently ignored by load_config (per-field .get()). checkpoint is superseded by default_checkpoint; the others have no successor. The 8192-token long-document max_len capability is out of scope for this feature.

For audio source selection, virtual sink creation, and audio routing, see AUDIO-SETUP.md.