Mohamed Boudra
135a3b4c9e
* fix(app): find text in assistant messages that streamed in the open chat Chat find failed with the generic "check the host connection" text whenever the oldest hit sat in a response that streamed while the chat was open and had more than one Markdown block. Presentation rendered such a message as one row per block with ids `<messageId>:block:n`, while hydrated history stayed one row with the raw message id, and every message-addressed DOM lookup (find reveal, scroll-to-message, history reveal, the render-cap expansion) only knew the raw id. The reveal step polled for a row that never existed and gave up after five seconds. The defect was the dual rendering, so this removes it: every assistant message is a block group however it arrived, single-paragraph messages included, so a row id is never a message id. Rows carry `data-message-id`, `getStreamItemMessageId` is the one way from a row to its message, and find, scroll-to-message, history reveal and selection copy address messages through it. Chat find force-mounts every row of the selected message under virtualization so counts span the whole message, and pins the chosen occurrence to its row while the scroll settles. Link reference definitions fold into their neighbouring block so history never paints an empty row. Find failures now say what went wrong (connection, history changed, could not show the match) instead of blaming the connection for everything. * fix(app): let the Markdown parser decide which blocks are link definitions A hand-rolled CommonMark regex rejected destinations markdown-it accepts, such as escaped spaces, so those definitions still became their own empty row and the neighbouring paragraph lost its link. The block splitter now parses the block with the same parser: no tokens plus registered references means definitions only. The two new chat find journeys move their mechanics into named helpers so the test bodies read as intent.
7.6 KiB
Agent stream performance
How assistant text gets from a provider to the screen, and why it is paced on the way. Read this before changing packages/server/src/server/agent/agent-stream-coalescer.ts, the reducer queue in packages/app/src/timeline/session-stream-reducers.ts, or the reveal in packages/app/src/hooks/use-revealed-text.ts.
For terminal output, which is a separate pipeline with separate budgets, see terminal-performance.md.
The pipeline
provider deltas (every provider streams incrementally)
→ AgentStreamCoalescer (daemon, leading + trailing, ≤1 message per 60ms per agent)
→ recordTimeline: one canonical row per flushed item
→ agent_stream ws message
→ reducer queue (app, one commit per frame) → session store
→ source-item plugin transforms → native Markdown blocks / tool grouping
→ paced reveal (app, per displayed item) → paint
Every provider delivers incremental text, so there is no provider that needs special handling: Claude via includePartialMessages, Codex via agent_message_delta, ACP agents via agent_message_chunk, Pi and OMP via text_delta.
Why the reveal is paced
Arrival is lumpy and there is no fixing that at the source. A 60ms coalescing window carries however many characters the model produced in those 60ms, which swings by an order of magnitude within a single turn. Painting each delta as it lands makes the size of those lumps visible, and that is what reads as jagged.
So arrival sets a target and the reveal rate is derived from the backlog instead. A burst makes the text catch up faster; it does not make the text jump. Shrinking the coalescing window does not fix this — it makes the lumps smaller and more frequent, at the cost of message rate on a daemon loop that already contends with terminal frames and per-message relay encryption.
Invariants
- The coalescer is leading + trailing. The first delta after an idle window flushes synchronously; only the rest of the burst waits for the trailing timer. Reverting to trailing-only adds a full window to the first character of every turn. Same shape and the same reason as
TerminalOutputCoalescer. - The leading flush adds a canonical row, and that is fine. A burst's first chunk lands as its own timeline row.
mergeAssistantChunks/mergeReasoningChunksintimeline-projection.tsjoin contiguous same-turn rows, and clients read the projected timeline, so history is unaffected. Tests that assert on raw rows have to account for the extra row; tests that assert on what a client sees do not. - The store holds the full text; only the rendered slice is paced. Copy, selection, the chat outline, and scroll geometry all read the same string the user can see. Pacing the store instead would leave the bottom anchor chasing a content height that is ahead of the reveal.
- Markdown blocks belong to presentation, and every assistant message is a block group.
agent-stream/presentation.tssplits a message into one display row per Markdown block, the same way whether it arrived streaming or as fetched history. Splitting in the reducer instead would discard paragraph separators and expose fragments to plugin callbacks, which need the whole source text. Live-head work stays cheap by parsing only the growing last block during append, retaining completed blocks by object identity, and caching the split per source item so a tail change does not re-split history. History was once left whole to keep cross-block Markdown context; that gave one message two shapes depending on how it arrived, and message-addressed features only worked on one of them. - A row id is never a message id. Block rows are
${messageId}:block:${n}withblockGroupId = messageId, including single-block messages, so nothing can come to rely on the two being equal.getStreamItemMessageIdinpresentation.tsis the only way to go from a row to its message, and web rows carry it asdata-message-idalongsidedata-history-row-id. Anything addressing a message — chat find, scroll-to-message, history reveal, the find expansion that lifts the render cap — uses the message id and must expect several rows to answer to it. Row ids stay for React keys, virtualizer measurement, scroll anchors, and per-row caches like assistant image occurrence keys. - First sight of a text is revealed whole. Only growth is paced. This is what makes history hydration, timeline replay, a virtualized row remounting on scroll, and an already-finished message all render complete on first paint without a special case for each.
- Leaving
phase: "streaming"snaps the reveal. A completed turn must never be left holding characters.layoutStreamsets the phase, so anything outside the live head with an active turn is already complete. - The reducer queue commits on a frame, with a timer as the ceiling. A frame callback never fires in a hidden tab, so a timer races it and wins when nothing is painting — the store has to keep advancing either way.
- A history row re-renders only when its item or layout item identity changes. The inverted
FlatList hands every mounted cell a new
indexandrefwhenever a row is prepended, so without a memo boundary each coalesced tick re-rendered every mounted row (about 50 on a phone, 100–250 ms of JS per tick).layoutStreamkeeps a layout item's identity when nothing about it changed,useRevisedHistoryRowshands a fresh item identity to rows whose tool-call group, expanded state, or breakpoint changed, andHistoryStreamRowmemoizes on both. Every viewport runs its history through that hook; the web viewport once skipped it and history hosts of a live tool group went stale. A new field onStreamLayoutItemmust be added toareLayoutItemsEquivalent, or sharing silently stops.
Measuring
- Smoothness (user-perceived):
packages/app/e2e/browser/agent-stream-smoothness.spec.ts, gated behindPASEO_AGENT_STREAM_PERF_E2E=1. Drives the mock provider'sbursty-streammodel and reports coefficient of variation of characters painted per frame (smoothness) plus p95 gap between visible updates (stalls). Both numbers are needed: a stalled stream is perfectly smooth. - Reproducing bursty arrival: the
bursty-streammodel inmock-load-test-agent.tsemits uneven runs of tokens separated by idle gaps. Burst sizes come from a seeded generator, so a run repeats exactly. - Rate policy in isolation:
computeRevealStepinpackages/app/src/agent-stream/text-reveal.tsis pure;text-reveal.test.tscovers convergence and burst flattening without a renderer.
Healthy numbers (2026-08, Expo web against a local dev daemon, real Claude Haiku agent, ~8.5s samples during active streaming). Setting TEXT_REVEAL_HORIZON_MS to 0 makes the reveal paint on arrival, which is how the baseline column was taken:
| paint on arrival | paced | |
|---|---|---|
| frames that advanced the text | 6% | 87% |
| chars-per-frame CV | 4.11 | 1.86 |
| gap between visible updates, p50 | 317ms | 17ms |
| gap between visible updates, p95 | 383ms | 17ms |
Total characters painted is roughly the same either way — the reveal changes when they land, not how many arrive.
Measure the total length across every assistant-message element, not the last one. A turn emits many assistant messages, so the tail element keeps changing identity and its length is not monotonic; sampling only the tail reads those handovers as resets and reports almost no growth. sampleStreamFrames does this correctly.