mirror of
https://github.com/getpaseo/paseo.git
synced 2026-10-21 16:52:40 +00:00
e22ad117fd
* Stream assistant text at a steady rate instead of in arrival lumps Assistant text arrived in lumps and was painted that way. The daemon coalesces one message per 60ms window, and that window carries however many characters the model produced in it, which swings by an order of magnitude within a turn. Painting each delta as it landed made the size of those lumps visible. Three changes: - The agent stream coalescer is now leading + trailing. The first delta after an idle window flushes synchronously instead of waiting a full window, matching the invariant TerminalOutputCoalescer already documents. The window stays at 60ms; shrinking it would raise message rate on a loop that already contends with terminal frames and relay encryption, without making the stream smoother. - The client reducer queue commits on a frame instead of an arbitrary 48ms timer, with the timer kept as a ceiling so a hidden tab still advances. - Assistant and reasoning text are revealed at a rate derived from the backlog rather than from arrival timing. Arrival changes the target, not the rate. The store keeps the full text and only the rendered slice is paced, so copy, selection, the chat outline, and scroll geometry all stay consistent with what is on screen. The leading flush means a burst's first chunk lands as its own timeline row. The projection already merges contiguous same-turn rows and clients read the projected timeline, so history is unchanged; the server tests that assert on raw rows were updated for the extra row. Measured on Expo web against a local dev daemon. Mock bursty-stream provider, 6s samples, paint-on-arrival vs paced: frames that advanced the text 55 -> 244, chars-per-frame CV 2.66 -> 1.44, gap p50 100ms -> 17ms. Real Claude Haiku agent over 52s: 84% of frames advance the text, CV 1.29, gap p50 and p95 both 17ms (one frame at 60Hz). Adds a bursty-stream mock model with seeded burst sizes, a gated browser perf spec, and docs/agent-stream-performance.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Address review: safe reveal boundaries and a testable seam Two findings from the PR review. Unicode boundaries. Slicing the revealed text at an arbitrary index could strand a lone surrogate, which paints as a replacement glyph, or split an emoji ZWJ sequence so a family briefly rendered as its members. Both were visible for a frame or two mid-stream — the exact flicker the paced reveal exists to remove. The rendered cut is now pulled back off surrogate pairs, combining marks, variation selectors, skin tone modifiers, and ZWJ joins. The reveal counter stays monotonic and only the painted boundary moves, so a long cluster cannot stall the reveal. Test placement. The hook test used JSDOM, @testing-library renderHook, and monkey-patched requestAnimationFrame, all of which docs/testing.md rules out; it prescribes fixing the missing seam instead. The reveal policy is now a pure state machine in text-reveal.ts (begin/retarget/advance/ complete/visible) covered by plain unit tests with no environment, and the hook is reduced to the frame-clock binding. Rendered behavior is left to the browser spec, which no longer carries frame sampling and statistics in its body — those moved to e2e/support/helpers/stream-smoothness.ts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Measure total painted text, not the tail message The smoothness sampler read the length of the last assistant-message element. A real turn emits many assistant messages, so that element keeps changing identity and its length is not monotonic — handovers read as resets, and a stream that was plainly moving on screen reported almost no growth at all. Sampling the total across every assistant message fixes it. This invalidated the numbers previously recorded for the real-agent run, so the doc table is replaced with a fresh A/B taken with the corrected sampler against a Claude Haiku agent: frames that advanced the text 6% -> 87%, chars-per-frame CV 4.11 -> 1.86, gap p50 317ms -> 17ms, p95 383ms -> 17ms. The paced column also reported zero Unicode defects across a stream containing emoji, which exercises the reveal boundary clamp end to end. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf(agent-stream): harden paced text reveal Use standards-compliant grapheme segmentation when available and preserve full arrivals on older runtimes. Cap reveal commits at 60Hz and measure the real browser rendering path so high-refresh displays stay smooth without rendering at hardware frame rate. * Hold back trailing fragments split by arrival boundaries The reveal boundary is not the only place a grapheme gets cut. The daemon's coalescing window ends a delta wherever the window closes, so a caught-up reveal can be handed half a cluster and paint it. Measured on a mixed-script Haiku stream: five frames in ~350 ending in a single regional indicator, i.e. half a flag on screen for a frame. While the turn is streaming, a trailing fragment that cannot stand alone — a dangling high surrogate, a trailing joiner, an odd regional indicator — now waits for the rest of its cluster. Once the turn ends everything received is painted, because nothing more is coming. Complements the grapheme-boundary clamp, which handles the reveal cut; this handles the arrival cut. The same stream now measures one such frame in ~400 rather than five. The remainder is a flush boundary that coincides with the end of an assistant message, putting the two halves in different messages, which the client cannot rejoin — that one belongs in the coalescer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: Mohamed Boudra <boudra.moha@gmail.com>
5.3 KiB
5.3 KiB
Agent stream performance
How assistant text gets from a provider to the screen, and why it is paced on the way. Read this before changing packages/server/src/server/agent/agent-stream-coalescer.ts, the reducer queue in packages/app/src/timeline/session-stream-reducers.ts, or the reveal in packages/app/src/hooks/use-revealed-text.ts.
For terminal output, which is a separate pipeline with separate budgets, see terminal-performance.md.
The pipeline
provider deltas (every provider streams incrementally)
→ AgentStreamCoalescer (daemon, leading + trailing, ≤1 message per 60ms per agent)
→ recordTimeline: one canonical row per flushed item
→ agent_stream ws message
→ reducer queue (app, one commit per frame) → session store
→ paced reveal (app, per assistant/reasoning item) → markdown blocks → paint
Every provider delivers incremental text, so there is no provider that needs special handling: Claude via includePartialMessages, Codex via agent_message_delta, ACP agents via agent_message_chunk, Pi and OMP via text_delta.
Why the reveal is paced
Arrival is lumpy and there is no fixing that at the source. A 60ms coalescing window carries however many characters the model produced in those 60ms, which swings by an order of magnitude within a single turn. Painting each delta as it lands makes the size of those lumps visible, and that is what reads as jagged.
So arrival sets a target and the reveal rate is derived from the backlog instead. A burst makes the text catch up faster; it does not make the text jump. Shrinking the coalescing window does not fix this — it makes the lumps smaller and more frequent, at the cost of message rate on a daemon loop that already contends with terminal frames and per-message relay encryption.
Invariants
- The coalescer is leading + trailing. The first delta after an idle window flushes synchronously; only the rest of the burst waits for the trailing timer. Reverting to trailing-only adds a full window to the first character of every turn. Same shape and the same reason as
TerminalOutputCoalescer. - The leading flush adds a canonical row, and that is fine. A burst's first chunk lands as its own timeline row.
mergeAssistantChunks/mergeReasoningChunksintimeline-projection.tsjoin contiguous same-turn rows, and clients read the projected timeline, so history is unaffected. Tests that assert on raw rows have to account for the extra row; tests that assert on what a client sees do not. - The store holds the full text; only the rendered slice is paced. Copy, selection, the chat outline, and scroll geometry all read the same string the user can see. Pacing the store instead would leave the bottom anchor chasing a content height that is ahead of the reveal.
- First sight of a text is revealed whole. Only growth is paced. This is what makes history hydration, timeline replay, a virtualized row remounting on scroll, and an already-finished message all render complete on first paint without a special case for each.
- Leaving
phase: "streaming"snaps the reveal. A completed turn must never be left holding characters.layoutStreamsets the phase, so anything outside the live head with an active turn is already complete. - The reducer queue commits on a frame, with a timer as the ceiling. A frame callback never fires in a hidden tab, so a timer races it and wins when nothing is painting — the store has to keep advancing either way.
Measuring
- Smoothness (user-perceived):
packages/app/e2e/browser/agent-stream-smoothness.spec.ts, gated behindPASEO_AGENT_STREAM_PERF_E2E=1. Drives the mock provider'sbursty-streammodel and reports coefficient of variation of characters painted per frame (smoothness) plus p95 gap between visible updates (stalls). Both numbers are needed: a stalled stream is perfectly smooth. - Reproducing bursty arrival: the
bursty-streammodel inmock-load-test-agent.tsemits uneven runs of tokens separated by idle gaps. Burst sizes come from a seeded generator, so a run repeats exactly. - Rate policy in isolation:
computeRevealStepinpackages/app/src/agent-stream/text-reveal.tsis pure;text-reveal.test.tscovers convergence and burst flattening without a renderer.
Healthy numbers (2026-08, Expo web against a local dev daemon, real Claude Haiku agent, ~8.5s samples during active streaming). Setting TEXT_REVEAL_HORIZON_MS to 0 makes the reveal paint on arrival, which is how the baseline column was taken:
| paint on arrival | paced | |
|---|---|---|
| frames that advanced the text | 6% | 87% |
| chars-per-frame CV | 4.11 | 1.86 |
| gap between visible updates, p50 | 317ms | 17ms |
| gap between visible updates, p95 | 383ms | 17ms |
Total characters painted is roughly the same either way — the reveal changes when they land, not how many arrive.
Measure the total length across every assistant-message element, not the last one. A turn emits many assistant messages, so the tail element keeps changing identity and its length is not monotonic; sampling only the tail reads those handovers as resets and reports almost no growth. sampleStreamFrames does this correctly.