STT-Assistant/docs/TRIAGE.md

Triage Panel

The triage panel is a display-only signal readout in the assistant TUI. On each settled turn (no chat mode required) the assistant classifies the most recent transcribed utterances with a local Laya multilingual model running in-process, and then always asks the existing [llm] lane for one structured JSON extraction. Both results land in a right-hand pane. Nothing here triggers a mode, a command, or a tool: the panel only shows what the model inferred.

The feature is off by default ([triage].enabled = false), and the extra it needs is opt-in, so a plain uv sync and the existing test suite behave exactly as before.

What the panel shows

• A right-hand TUI pane, composed only when [triage].enabled = true. With the default false there is no widget at all.
• signals after a settled turn: one line per question, rendered from the normalized answer.
  • choice: the selected label plus the gating confidence, for example category: sales (0.91).
  • noul: a ten-cell bar filled from p, the probability, for example ███████ churn_risk: 0.72.
  • score: a ten-cell bar filled from the clamped integer level plus the level label.
  • A ⚠ mark follows any item the analyzer flagged.
• content after the LLM extraction: the parsed JSON is merged over the signals as a content key, and the panel appends the summary plus up to three entries each from opportunities, risks, and action_items.
• A reason line while the lane is not ok, for example when the model is still loading or the questions.json file is missing. The panel never renders blank.
• Cleared on reset: clear empties the pane. Leaving chat mode does not.

The panel is a view over events (stage of signals, content, or cleared) plus the cached triage_state / triage_reason in status(). It never feeds anything back into the assistant.

How the cascade works

settled turn (any mode)
    -> newest transcribed-utterance window (window entries, condensed under state_char_budget)
    -> Laya multilingual classification (in-process lane)
    -> signals (emitted; flagged per should_escalate) ----------+
    -> extract_enabled and an extract function present?         |
         -> one triage-extract job on the LLM InferenceQueue    |
              (all normalized signals in the prompt; on a Laya  |
               error the prompt carries transcript state only)  |
              -> parsed JSON merged as content ------------------+

1. Trigger. After the chat dispatch block in the assistant tick, triage runs on any settled turn in any mode (idle, chat, or command), independent of chat. It runs when the triage components are wired and the turn has ended (turn_ended(): speech stopped plus quiet for turn_gap_s). A repeated window is skipped, so an identical set of utterances never re-gates.
2. Window. The window is the newest window transcribed utterances from the STT stream, not the chat conversation history. build_state condenses it newest-first under state_char_budget and restores chronological order, with - prefixed lines.
3. Classification. The condensed state, the loaded questions, the last transcript language, and the current triage generation go to the Laya lane. The lane runs one multilingual predict at a time in the background. The raw answers come back on the lane thread, and the analyzer normalizes them into signals ({order, items, escalated, input_tokens}). Stale generations are dropped before the service stores or emits anything.
4. Escalation. should_escalate is true when any one of three arms fires:
   • noul: p >= escalate_threshold
   • choice: the gating confidence is < choice_min_confidence
   • score: expected >= (levels - 1) * 0.5, where levels is the number of score criteria Escalation only drives the per-item flagged flag and the ⚠ marker in the panel; it no longer gates extraction.
5. Extraction. After every Laya pass, when extraction is enabled and an extract function is configured, the service submits one triage-extract job to the existing LLM InferenceQueue. The extraction prompt carries the condensed transcript state plus all normalized signals in order (rendered as words under a Signals: header); when there are no signal lines the signals block is omitted and the prompt is state plus the JSON-only instruction. On a Laya error (no payload) the LLM still runs, with transcript state only and no signals event emitted. If the queue is full the job is refused with submit_if_room: no queued chat job is ever evicted, and the signals simply stand. The worker re-checks the generation before and after the blocking LLM call, parses the reply (strict JSON, fences stripped, lists capped), and merges it as content over the signals. A parse failure keeps the signals and emits no content.
6. Reset. clear bumps the triage generation, clears the stored signals and the ambient buffer, drops any pending lane request and any queued triage-extract job, and emits the cleared stage. An in-flight result that lands after the bump is suppressed. Leaving chat mode does not reset the panel or its buffer.

Lanes: nothing is parallelized

The assistant has exactly three single-worker lanes, and you should not parallelize any of them:

lane	what runs on it
transcription	one worker; faster-whisper is not thread-safe
LLM InferenceQueue	one worker; all chat, command, and triage-extract calls
Laya	one worker plus one loader thread; one predict at a time

The triage feature adds no process, no UNIX socket, and no supervisor child. The Laya part is a dedicated in-process lane with a background loader thread (lazy import, preload, one throwaway warmup predict) and a single-flight worker. That keeps the 0.1 s tick and the LLM lane unblocked, which multiplexing Laya onto the InferenceQueue would not. The supervisor still spawns 2 children headless and 3 with the TUI.

Questions file

Triage reads a bare Laya questions dict: a JSON object whose keys are question ids (qid) and whose values are question definitions.

Schema

{
  "category": {
    "type": "choice",
    "instructions": "What is the user mainly talking about?",
    "criteria": {
      "support": "the user needs help",
      "sales": "the user asks about pricing",
      "other": "anything else"
    }
  },
  "urgency": {
    "type": "choice",
    "instructions": "How urgent does the user sound?",
    "criteria": ["low", "medium", "high"]
  },
  "churn_risk": {
    "type": "noul",
    "instructions": "The user is thinking about leaving or cancelling."
  }
}

Each definition needs type and instructions. criteria depends on the type:

type	criteria	rules
choice	dict or list, required	non-empty; a dict maps each label to a description
score	list, required	non-empty; must not contain null; entries are the level labels in order
noul	dict, optional	if present, keys must be only "true" and/or "false" (case-insensitive); omit it for a plain yes/no question

Anything else fails validation. The whole file is rejected (and triage reports no_questions) when the top level is not a non-empty object, when any definition is not an object, when type is not one of choice / score / noul, when instructions is missing, or when the criteria above are malformed. load_questions returns None instead of raising, so a bad file never takes the assistant down.

Getting a valid file

A ready-made file ships in the repo:

cp questions.example.json questions.json

The example mixes choice and noul questions (category, urgency, frustration, churn_risk, needs_action, needs_clarification, contains_idea, business_opportunity, positive_sentiment). questions.json is gitignored user state; edit it in place. The file path is relative to the assistant's working directory (questions_file, default questions.json).

Install and enable

The Laya dependency lives behind an optional extra, pinned to one exact revision:

# 1. Install the triage extra (adds laya)
uv sync --extra triage

# 2. Provide the questions file
cp questions.example.json questions.json

# 3. Turn it on
#    [triage]
#    enabled = true

The extra (triage = ["laya"]) resolves through [tool.uv.sources], which pins Laya to revision 970dc8c5f63d7b886a68409493f37d569424f933 (version 0.3.20). This revision is recorded in the plan and asserted by the measurement test below. The pin is exact on purpose: Laya is an early project, and API drift between revisions is real.

The multilingual checkpoint is pinned for every call (default_checkpoint = "multilingual", preload = ["multilingual"], and a per-request model="multilingual"). There is no English-only .en variant and none is selected. Run triage-enabled from the normal supervised flow, uv run stt-daemon up --with-tui; no extra command is needed.

Multilingual pin and the calibration caveat

Because the multilingual checkpoint is pinned, every classification uses it. Multilingual runs at temperature 1.0 upstream, so its confidence scores are uncalibrated out of the box. The default thresholds (escalate_threshold = 0.6, choice_min_confidence = 0.85) are starting points, not measured values. Measure on your own transcripts before trusting the escalation behavior, and lower or raise the thresholds to match what you observe.

The measurement lives in tests/test_laya_lane.py::test_multilingual_pin_lang_and_resources, gated by pytest.importorskip("laya"), so the default suite skips it green and the extra enables it. With the extra installed:

uv run pytest tests/test_laya_lane.py::test_multilingual_pin_lang_and_resources -v -s

That test builds a multilingual Router, runs one lang="es" predict, and records the wall latency, router.loaded, and the process RSS delta, and it checks the installed Laya version and pin. The numbers it prints are environment-specific (CPU, GPU, drivers, disk), so they are not repeated here; read them from your own run. The test asserts only sanity and the pin, never a benchmark target.

Configuration reference

All keys live under [triage] unless noted. The defaults are what you get with no config file.

key	default	meaning
enabled	false	Master gate. false means no lane, no panel, and behavior identical to before.
questions_file	"questions.json"	Laya questions JSON in the working directory. Missing or invalid means no_questions.
window	500	Recent transcribed utterances available per classification; the char budget is the real limiter.
state_char_budget	3000	Character cap (~750 tokens) for the condensed state sent to Laya.
predict_max_tokens	1024	Max tokens sent to Laya; the multilingual checkpoint supports 1024; state shares this budget with the ~192-token question head (~830-token state room).
confidence_field	"answer_confidence"	Which Laya confidence feeds the per-item flagged/⚠ marker: answer_confidence (calibrated max(p)) or confidence (normalized entropy).
escalate_threshold	0.6	A noul answer at or above this p is flagged (⚠ marker only).
choice_min_confidence	0.85	A choice answer below this confidence is flagged (⚠ marker only).
extract_enabled	true	Master switch for the LLM JSON extraction, which runs after every Laya pass.
extract_max_tokens	512	Token cap for the extraction reply.
extract_timeout_s	30.0	Timeout for the extraction request.
device	""	Empty string means auto, which maps to None so Laya picks the checkpoint's default device. Any other value is passed through.
preload	["multilingual"]	Checkpoints preloaded at lane startup. The fixed default preloads the multilingual checkpoint only.
default_checkpoint	"multilingual"	Router default and the per-request model for every call. Multilingual is pinned.

[llm].triage_system_prompt is the strict JSON-only instruction used by the extraction cascade. It must return {summary, opportunities[], doubts[], action_items[{what, why}], risks[]} and nothing else.

Upgrading from a v5-draft config

An earlier draft of this feature used an external service with its own transaction keys. Those keys are not [triage] keys and are silently ignored by load_config because it reads each field with a per-field .get():

stale key	status
socket	no successor
model	no successor; the checkpoint is pinned to default_checkpoint
predict_timeout_s	no successor
checkpoint	superseded by default_checkpoint
subfolder	no successor
max_len	no successor; the 8192-token long-document capability is out of scope

Remove them from config.toml to avoid confusion, but they will not change behavior if left in place. There is also no act_probability anywhere; escalation uses the gating confidence selected by confidence_field.