Triage Panel

The triage panel is a display-only signal readout in the assistant TUI. On each settled turn (no chat mode required) the assistant classifies the most recent transcribed utterances with a local Laya multilingual model running in-process, then folds the turn into a living plain-text brief on the existing [llm] lane through a dedicated brief conversation. Signals and brief land in a right-hand pane. Nothing here triggers a mode, a command, or a tool: the panel only shows what the model inferred.

The feature is off by default ([triage].enabled = false), and the extra it needs is opt-in, so a plain uv sync and the existing test suite behave exactly as before.

What the panel shows

A right-hand TUI pane, composed only when [triage].enabled = true. With the default false there is no widget at all.
signals after a settled turn: one line per question, rendered from the normalized answer.
- choice: the selected label plus the gating confidence, for example decision_status: made (0.91).
- noul: a ten-cell bar filled from p, the probability, for example ███████ blocker: 0.72.
- score: a ten-cell bar filled from the clamped integer level plus the level label.
- A ⚠ mark follows any item the analyzer flagged.
content after a brief update: the plain-text brief arrives as a content stage and the panel appends it below the signals. The brief is written by the model, so the panel shows it as-is (truncated only when it exceeds brief_max_chars).
A reason line while the lane is not ok, for example when the model is still loading or the questions.json file is missing. The panel never renders blank.
A last-triage clock as the first line whenever a real triage is shown: the server-side time of the last triage in the transcript pane's HH:MM:SS format and bold cyan style.
Cleared on reset: clear empties the pane. Leaving chat mode does not.

The panel is a view over events (stage of signals, content, or cleared) plus the cached triage_state / triage_reason / questions in status(). It never feeds anything back into the assistant.

Pressing s in the TUI swaps the pane to a structured view of the loaded questions.json and pressing it again swaps back. The view renders one block per qid in file order — the qid, its type, the instructions, and the criteria (choice labels, a label-to-description map, ordered score levels, or the true/false descriptions of a noul question). The questions come from the assistant's validated payload (the same dict Laya uses), not from disk, and it is a no-op when [triage].enabled = false.

How the brief works

settled turn (any mode)
    -> newest transcribed-utterance window (window entries, condensed under state_char_budget)
    -> Laya multilingual classification (in-process lane)
    -> signals (emitted; flagged per should_escalate)
    -> extract_enabled and the fail-open gate pass?
         -> one brief-update job on the LLM InferenceQueue
              (pending speaker-labeled deltas + Focus signal words;
               a Laya error still runs with the deltas only)
              -> plain-text brief merged as content

Trigger. After the chat dispatch block in the assistant tick, triage runs on any settled turn in any mode (idle, chat, or command), independent of chat. It runs when the triage components are wired and the turn has ended (turn_ended(): speech stopped plus quiet for turn_gap_s). A repeated window is skipped, so an identical set of utterances never re-gates.
Window. The window is the newest window transcribed utterances from the STT stream, not the chat conversation history. build_state condenses it newest-first under state_char_budget and restores chronological order, with - prefixed lines.
Classification. The condensed state, the loaded questions, the last transcript language, and the current triage generation go to the Laya lane. The lane runs one multilingual predict at a time in the background. The raw answers come back on the lane thread, and the analyzer normalizes them into signals ({order, items, escalated, input_tokens}). Stale generations are dropped before the service stores or emits anything.
Escalation. should_escalate is true when any one of three arms fires:
- noul: p >= escalate_threshold
- choice: the gating confidence is < choice_min_confidence
- score: expected >= (levels - 1) * 0.5, where levels is the number of score criteria Escalation only drives the per-item flagged flag and the ⚠ marker in the panel; it drives neither the gate nor the brief.
Brief update. extract_enabled is the brief engine's master switch: with it false the panel still classifies and shows signals, but no brief is requested. When it is true and a brief function is wired, the fail-open gate decides whether the turn updates the brief. The gate reads the substantive_update noul arm of the signals: the update runs when p >= brief_gate_threshold (default 0.5), and any missing or non-numeric signal passes. The service queues one brief-update job on the existing LLM lane. The job carries the speaker-labeled pending deltas (the transcripts that arrived since the last accepted update) plus the current signal words under a Focus: header; a Laya error still runs with the deltas only. [llm].triage_system_prompt drives a dedicated brief conversation that keeps its own history; the reply is a plain-text brief, not JSON (fences stripped, then truncated to brief_max_chars, never dropped). The accepted brief is emitted as the content stage, whose payload key is brief (the event carries the merged signals plus brief), and, when [brief].write = true, snapshotted to disk.
Compaction. The brief conversation is its own ConversationEngine, so it compacts its history independently, using [llm].brief_compaction_system_prompt and the [llm] compaction settings. When it fires, the summary is injected as a second system message, which resets that engine's prompt prefix: the server's prefix cache is invalidated once for it, and the summarizer call can hold the shared LLM lane for up to compaction_timeout_s.
Reset. clear bumps the triage generation, clears the stored signals and the ambient buffer, drops any pending lane request and any queued brief-update job, truncates the optional durable brief file, and emits the cleared stage. An in-flight result that lands after the bump is suppressed, so a stale brief writes neither the pane nor the file. Leaving chat mode does not reset the panel or its buffer. A new conversation also resets the brief, which starts fresh in the new session folder, without clearing chat or command state.

Single-outstanding job and coalescing

Only one brief-update job may exist at a time. The service tracks a token and an outstanding flag under its lock:

If a job is already outstanding, the new turn is refused: its signals still show in the panel, but no second job is stacked and the turn's deltas stay in the pending buffer.
The job is submitted with submit_if_room, so a full queue never evicts a queued chat turn just to run a brief.
On refusal, an empty "" reply, or an exception, the deltas are re-queued at the front of the buffer and the outstanding flag is cleared, so the next settled turn can retry.
On success, if deltas accumulated while the job was in flight, a coalesced flush runs immediately and dispatches the next job with the latest signals.

The buffer is bounded by brief_pending_char_budget (default 3000 characters): once full, the oldest deltas are dropped, so a long quiet stretch can't grow it without limit. A clear mid-flight makes the in-flight result stale; it is discarded and writes nothing.

Lanes: nothing is parallelized

The assistant has exactly three single-worker lanes, and you should not parallelize any of them:

lane	what runs on it
transcription	one worker; faster-whisper is not thread-safe
LLM `InferenceQueue`	one worker; all chat, command, and `brief-update` calls
Laya	one worker plus one loader thread; one `predict` at a time

The triage feature adds no process, no UNIX socket, and no supervisor child. The Laya part is a dedicated in-process lane with a background loader thread (lazy import, preload, one throwaway warmup predict) and a single-flight worker. That keeps the 0.1 s tick and the LLM lane unblocked, which multiplexing Laya onto the InferenceQueue would not. The supervisor still spawns 2 children headless and 3 with the TUI.

The brief engine runs on the shared LLM lane, so its work is visible in the TUI: the triage pane border shows [inf N] with the queue depth and · busy while a job is running, the same counter the chat/command pane shows. Brief jobs are non-evictable, so a full queue refuses the brief instead of dropping a queued chat turn.

Questions file

Triage reads a bare Laya questions dict: a JSON object whose keys are question ids (qid) and whose values are question definitions.

Schema

{
  "category": {
    "type": "choice",
    "instructions": "What is the user mainly talking about?",
    "criteria": {
      "support": "the user needs help",
      "sales": "the user asks about pricing",
      "other": "anything else"
    }
  },
  "urgency": {
    "type": "choice",
    "instructions": "How urgent does the user sound?",
    "criteria": ["low", "medium", "high"]
  },
  "churn_risk": {
    "type": "noul",
    "instructions": "The user is thinking about leaving or cancelling."
  }
}

Each definition needs type and instructions. criteria depends on the type:

type	`criteria`	rules
`choice`	dict or list, required	non-empty; a dict maps each label to a description
`score`	list, required	non-empty; must not contain `null`; entries are the level labels in order
`noul`	dict, optional	if present, keys must be only `"true"` and/or `"false"` (case-insensitive); omit it for a plain yes/no question

Anything else fails validation. The whole file is rejected (and triage reports no_questions) when the top level is not a non-empty object, when any definition is not an object, when type is not one of choice / score / noul, when instructions is missing, or when the criteria above are malformed. load_questions returns None instead of raising, so a bad file never takes the assistant down.

Getting a valid file

A ready-made file ships in the repo:

cp questions.example.json questions.json

The example is the eight-question brief set, in file order:

qid	type	brief mapping
`substantive_update`	`noul`	the fail-open gate; not a brief section itself
`opportunity`	`noul`	Opportunities
`blocker`	`noul`	Blockers/Risks
`decision_status`	`choice`	Decisions
`action_item`	`noul`	Action Items
`responsibility_clarity`	`choice`	Responsibilities
`deadline`	`noul`	Deadlines
`open_question`	`noul`	Open Questions

substantive_update is the gate: when the model returns a p below brief_gate_threshold, the turn is still classified and shown, but the brief isn't asked to update. The gate is fail-open, so a missing or non-numeric answer passes. questions.json is gitignored user state; edit it in place. The file path is relative to the assistant's working directory (questions_file, default questions.json).

Install and enable

The Laya dependency lives behind an optional extra, pinned to one exact revision:

# 1. Install the triage extra (adds laya)
uv sync --extra triage

# 2. Provide the questions file
cp questions.example.json questions.json

# 3. Turn it on
#    [triage]
#    enabled = true

The extra (triage = ["laya"]) resolves through [tool.uv.sources], which pins Laya to revision 970dc8c5f63d7b886a68409493f37d569424f933 (version 0.3.20). This revision is recorded in the plan and asserted by the measurement test below. The pin is exact on purpose: Laya is an early project, and API drift between revisions is real.

The multilingual checkpoint is pinned for every call (default_checkpoint = "multilingual", preload = ["multilingual"], and a per-request model="multilingual"). There is no English-only .en variant and none is selected. Run triage-enabled from the normal supervised flow, uv run stt-daemon up --with-tui; no extra command is needed.

Multilingual pin and the calibration caveat

Because the multilingual checkpoint is pinned, every classification uses it. Multilingual runs at temperature 1.0 upstream, so its confidence scores are uncalibrated out of the box. The default thresholds (escalate_threshold = 0.6, choice_min_confidence = 0.85) are starting points, not measured values. Measure on your own transcripts before trusting the ⚠ flagging (the thresholds drive neither the gate nor the brief), and lower or raise the thresholds to match what you observe.

The measurement lives in tests/test_laya_lane.py::test_multilingual_pin_lang_and_resources, gated by pytest.importorskip("laya"), so the default suite skips it green and the extra enables it. With the extra installed:

uv run pytest tests/test_laya_lane.py::test_multilingual_pin_lang_and_resources -v -s

That test builds a multilingual Router, runs one lang="es" predict, and records the wall latency, router.loaded, and the process RSS delta, and it checks the installed Laya version and pin. The numbers it prints are environment-specific (CPU, GPU, drivers, disk), so they are not repeated here; read them from your own run. The test asserts only sanity and the pin, never a benchmark target.

Configuration reference

All keys live under [triage] unless noted. The defaults are what you get with no config file.

key	default	meaning
`enabled`	`false`	Master gate. `false` means no lane, no panel, and behavior identical to before.
`questions_file`	`"questions.json"`	Laya questions JSON in the working directory. Missing or invalid means `no_questions`.
`window`	`500`	Recent transcribed utterances available per classification; the char budget is the real limiter.
`state_char_budget`	`3000`	Character cap (~750 tokens) for the condensed state sent to Laya.
`predict_max_tokens`	`1024`	Max tokens sent to Laya; the multilingual checkpoint supports 1024; state shares this budget with the ~192-token question head (~830-token state room).
`confidence_field`	`"answer_confidence"`	Which Laya confidence feeds the per-item `flagged`/`⚠` marker: `answer_confidence` (calibrated `max(p)`) or `confidence` (normalized entropy).
`escalate_threshold`	`0.6`	A `noul` answer at or above this `p` is flagged (`⚠` marker only).
`choice_min_confidence`	`0.85`	A `choice` answer below this confidence is flagged (`⚠` marker only).
`extract_enabled`	`true`	Master switch for the brief engine: with it false the panel classifies but no brief is requested.
`extract_max_tokens`	`1536`	Token cap for the brief reply; the brief is then truncated to `brief_max_chars`.
`extract_timeout_s`	`30.0`	Timeout for the brief request.
`brief_slot_id`	unset	Optional llama.cpp `id_slot` for the brief engine; when unset the `id_slot` field is omitted (needs a server started with `-np 2`).
`brief_gate_enabled`	`true`	Fail-open gate: update the brief only when the `substantive_update` `p` is at least `brief_gate_threshold`.
`brief_gate_threshold`	`0.5`	Gate threshold for `substantive_update`; the update runs at or above this `p`.
`brief_max_chars`	`4000`	Display cap for the brief; over this it is truncated, never dropped.
`brief_pending_char_budget`	`3000`	Pending transcript-character budget before a coalesced update is flushed; the oldest deltas are dropped past it.
`device`	`""`	Empty string means auto, which maps to `None` so Laya picks the checkpoint's default device. Any other value is passed through.
`preload`	`["multilingual"]`	Checkpoints preloaded at lane startup. The fixed default preloads the multilingual checkpoint only.
`default_checkpoint`	`"multilingual"`	Router default and the per-request `model` for every call. Multilingual is pinned.
`dump_io`	`false`	Opt-in I/O dump under `triage/YYYY-MM-DD/HHMMSS-mmm.json`; a coalesced update writes two files (see below).

[llm].triage_system_prompt drives the brief conversation: given the current brief (its previous reply, if any) and the new segment, it must return the updated brief as plain text only, with no markdown fences and no JSON. The reply is bounded by extract_max_tokens and extract_timeout_s, then truncated to brief_max_chars. [llm].brief_compaction_system_prompt is the summarizer used when the brief conversation's own history is compacted.

[brief] is a separate section that controls durability of the living brief. write (default false) gates disk persistence only: with it false the brief stays in the TUI, and with write = true each accepted brief is snapshotted to <session_dir>/<id>/brief.md inside the active conversation folder (falling back to brief/brief.md, cwd-relative and gitignored, when no session is active) and truncated on clear. A new conversation resets the brief, so each session folder gets a fresh one.

Opt-in I/O dump (`dump_io`)

With [triage].dump_io = true (default false) each triage writes a self-contained JSON file under triage/YYYY-MM-DD/HHMMSS-mmm.json — local time with millisecond precision, the same naming scheme as the audio files (<session_dir>/<id>/audio/HHMMSS-mmm-<reason>.wav). A file holds the Laya request, the Laya response, the LLM request (endpoint URL plus the full chat-completion body), and the LLM response (raw envelope), keyed by the ts carried on the triage events:

{
  "ts": 1759073916.19,
  "iso": "2026-09-28T09:38:36.412000-06:00",
  "laya": {"request": {...}, "response": {...}, "error": null},
  "llm": {"request": {"url": "...", "body": {...}}, "response": {...}, "error": null}
}

A turn whose brief is coalesced (the previous brief-update was still in flight, so the turn was refused and its deltas were flushed later) produces two files, not one. The turn's own record carries only the laya half (llm is null); the later flush dispatch writes a separate record carrying only the llm half (laya is null), keyed by its own ts. Same-millisecond records get a -N suffix on the second and later files (HHMMSS-mmm-1.json), so the first record keeps the exact HHMMSS-mmm.json shape and no record overwrites another.

When the brief does not run (the master switch is off, the gate fails, or no brief function is wired) the file still gets written with only the laya half (llm is null); on a Laya error the file records laya.response = null with laya.error = "predict failed". The directory is fixed as triage/ (cwd-relative, like the session folders and audit/) — there is no second config key for it. Files contain conversation text, so they are gitignored private debug output. Only the triage Laya predict and the triage brief call are dumped (never the loader warmup predict or non-triage LLM calls), and dumping is best-effort: a dump failure is logged and swallowed, never disturbing triage behavior or events.

Upgrading from the JSON extraction

Earlier versions of this panel asked the LLM for a strict JSON extraction ({summary, opportunities[], doubts[], action_items[{what, why}], risks[]}) and rendered the parsed fields in the panel. That is replaced by the plain-text brief described above.

[triage].extract_enabled still exists and is still the master switch, but it now gates the brief update, not a JSON extraction.
[triage].extract_max_tokens (default raised to 1536) is the brief reply cap, and extract_timeout_s is the brief request timeout.
[llm].triage_system_prompt is now a plain-text brief instruction; the shipped default no longer asks for JSON.
The panel's content stage now carries a brief string instead of merged JSON.
[brief] (new section) optionally persists the brief to disk; write is false by default, so nothing changes on disk unless you opt in.

Upgrading from a v5-draft config

An earlier draft of this feature used an external service with its own transaction keys. Those keys are not [triage] keys and are silently ignored by load_config because it reads each field with a per-field .get():

stale key	status
`socket`	no successor
`model`	no successor; the checkpoint is pinned to `default_checkpoint`
`predict_timeout_s`	no successor
`checkpoint`	superseded by `default_checkpoint`
`subfolder`	no successor
`max_len`	no successor; the 8192-token long-document capability is out of scope

Remove them from config.toml to avoid confusion, but they will not change behavior if left in place. There is also no act_probability anywhere; the per-item flagged/⚠ marker uses the gating confidence selected by confidence_field.

TRIAGE.md 21 KB Historique Raw