mirror of
https://github.com/getpaseo/paseo.git
synced 2026-10-23 17:52:49 +00:00
perf(markdown): keep native streaming updates in the unfinished tail
Native markdown reparses each updated view in full. Keep parser-closed blocks in a memoized head during paced reveal, then reunite completed prose for cross-paragraph selection. Preserve trailing spacing and source indentation at the split.
This commit is contained in:
Mohamed Boudra
@@ -29,7 +29,7 @@ So arrival sets a _target_ and the reveal rate is derived from the backlog inste
|
||||
- **The coalescer is leading + trailing.** The first delta after an idle window flushes synchronously; only the rest of the burst waits for the trailing timer. Reverting to trailing-only adds a full window to the first character of every turn. Same shape and the same reason as `TerminalOutputCoalescer`.
|
||||
- **The leading flush adds a canonical row, and that is fine.** A burst's first chunk lands as its own timeline row. `mergeAssistantChunks` / `mergeReasoningChunks` in `timeline-projection.ts` join contiguous same-turn rows, and clients read the projected timeline, so history is unaffected. Tests that assert on raw rows have to account for the extra row; tests that assert on what a client sees do not.
|
||||
- **The store holds the full text; only the rendered slice is paced.** Copy, selection, the chat outline, and scroll geometry all read the same string the user can see. Pacing the store instead would leave the bottom anchor chasing a content height that is ahead of the reveal.
|
||||
- **One source assistant message is one display row.** Presentation projects plugin output and caches unchanged history. The shared [Markdown renderer](markdown-renderer.md) owns rendering on every platform. Chat paces the visible source and applies a 32,000-character render cap on web; native keeps the complete message. It does not split Markdown or promote completed blocks into history.
|
||||
- **One source assistant message is one display row.** Presentation projects plugin output and caches unchanged history. The shared [Markdown renderer](markdown-renderer.md) owns rendering on every platform. Chat paces the visible source and applies a 32,000-character render cap on web; native keeps the complete message. The native renderer keeps closed blocks in a stable head and sends the paced suffix to a separate tail view during streaming; completion reunites them for selection. This stays inside the renderer, without promoting blocks into timeline history.
|
||||
- **Message identity survives completion.** Assistant row ids are source message ids. Plugin rows can replace a source, so find, scroll-to-message and history reveal still use `getStreamItemMessageId`. Width-specific measurements describe whole rendered messages.
|
||||
- **First sight of a text is revealed whole.** Only growth is paced. This is what makes history hydration, timeline replay, a virtualized row remounting on scroll, and an already-finished message all render complete on first paint without a special case for each.
|
||||
- **Leaving `phase: "streaming"` snaps the reveal.** A completed turn must never be left holding characters. `layoutStream` sets the phase, so anything outside the live head with an active turn is already complete.
|
||||
@@ -44,6 +44,12 @@ So arrival sets a _target_ and the reveal rate is derived from the backlog inste
|
||||
stale. A new field on `StreamLayoutItem` must be added to `areLayoutItemsEquivalent`, or sharing
|
||||
silently stops.
|
||||
|
||||
## Native parsing cost
|
||||
|
||||
The native library reparses and measures its entire `markdown` prop on each update. During streaming, the renderer splits source at an mdast block boundary and memoizes the head, so per-frame updates parse only the tail. Boundary discovery runs when a source line completes, rather than on every reveal frame. The unfinished line and final parsed block stay in the tail, preserving list continuation, setext headings, tables and open fences. Reference-dependent source and HTML retain a larger tail because later source can change their meaning. A single long block therefore still carries its full parsing cost.
|
||||
|
||||
Completion restores one native text view per Mermaid-delimited segment, so selection crosses prose paragraphs again. The head retains its trailing block margin while split. JS reveal remains the pacing source; native opacity animation stays disabled.
|
||||
|
||||
## Measuring
|
||||
|
||||
- **Smoothness (user-perceived):** `packages/app/e2e/browser/agent-stream-smoothness.spec.ts`, gated behind `PASEO_AGENT_STREAM_PERF_E2E=1`. Drives the mock provider's `bursty-stream` model and reports coefficient of variation of characters painted per frame (smoothness) plus p95 gap between visible updates (stalls). Both numbers are needed: a stalled stream is perfectly smooth.
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Use `MarkdownRenderer` for chat replies, file previews, plans, pull requests and changelog content. It uses `react-native-enriched-markdown` on iOS, Android and web. Keep source adaptation and renderer configuration in `components/markdown`; consumers supply Markdown and, when needed, its file location.
|
||||
|
||||
Native selection belongs to the library's text view. Keep an assistant message in one row so selection can cross paragraphs, headings and lists. Code blocks and tables use the library's native views and their selection behavior. Web uses browser selection over semantic HTML. Once its WASM parser is initialized, the library parses synchronously for the first commit of each mounted row; an empty interim row would corrupt virtualized measurements and move the reader. Chat owns paced reveal, the web render cap, message identity and list measurement. Native receives complete messages.
|
||||
Native selection belongs to the library's text view. Keep an assistant message in one row so selection can cross paragraphs, headings and lists. Code blocks and tables use the library's native views and their selection behavior. Web uses browser selection over semantic HTML. Once its WASM parser is initialized, the library parses synchronously for the first commit of each mounted row; an empty interim row would corrupt virtualized measurements and move the reader. Chat owns paced reveal, the web render cap, message identity and list measurement. Native renders completed messages together. While streaming, it keeps closed blocks in one head view and updates a separate tail, then reunites the text on completion. See [native parsing cost](agent-stream-performance.md#native-parsing-cost) for the boundary and pacing constraints.
|
||||
|
||||
## Links and images
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Native text selection
|
||||
|
||||
Keep assistant prose in one native text view so selection crosses paragraphs, headings and lists. The shared [Markdown renderer](markdown-renderer.md) owns that view.
|
||||
Keep completed assistant prose in one native text view so selection crosses paragraphs, headings and lists. The shared [Markdown renderer](markdown-renderer.md) owns that view.
|
||||
|
||||
## Android inverted chat
|
||||
|
||||
|
||||
@@ -13,6 +13,7 @@ import { MarkdownRendererFrame } from "./frame";
|
||||
import { WEB_RENDERERS } from "./renderers";
|
||||
import { createMarkdownStyles } from "./styles";
|
||||
import { PENDING_MARKDOWN_IMAGE } from "./source";
|
||||
import { splitStreamingMarkdown } from "./streaming";
|
||||
import { useMarkdownSource, type MarkdownImageContext } from "./images";
|
||||
|
||||
import { MermaidFence, MermaidStreamingContext, splitMermaidFences } from "./fence/mermaid";
|
||||
@@ -32,13 +33,31 @@ export interface MarkdownRendererProps extends MarkdownImageContext {
|
||||
}
|
||||
|
||||
/** All Markdown surfaces share the same engine and source adaptation. */
|
||||
export const MarkdownRenderer = memo(function MarkdownRenderer({
|
||||
export const MarkdownRenderer = memo(function MarkdownRenderer(props: MarkdownRendererProps) {
|
||||
const streaming = !isWeb && !!props.streaming;
|
||||
// Only completed lines affect block closure. Do not run mdast on every reveal frame.
|
||||
const completeLines = streaming ? props.text.slice(0, props.text.lastIndexOf("\n") + 1) : "";
|
||||
const boundary = useMemo(
|
||||
() => splitStreamingMarkdown(completeLines, true).head.length,
|
||||
[completeLines],
|
||||
);
|
||||
if (!streaming || boundary === 0) return <MarkdownPart {...props} />;
|
||||
return (
|
||||
<>
|
||||
<MarkdownPart {...props} text={props.text.slice(0, boundary)} continues />
|
||||
<MarkdownPart {...props} text={props.text.slice(boundary)} />
|
||||
</>
|
||||
);
|
||||
});
|
||||
|
||||
const MarkdownPart = memo(function MarkdownPart({
|
||||
text,
|
||||
continues = false,
|
||||
compact = false,
|
||||
streaming = false,
|
||||
onLinkPress,
|
||||
...imageContext
|
||||
}: MarkdownRendererProps) {
|
||||
}: MarkdownRendererProps & { continues?: boolean }) {
|
||||
const environment = useAssistantFileLinkEnvironment();
|
||||
const actions = useAssistantFileLinkActions();
|
||||
const client =
|
||||
@@ -80,11 +99,13 @@ export const MarkdownRenderer = memo(function MarkdownRenderer({
|
||||
uniProps={compact ? compactStyle : regularStyle}
|
||||
flavor="github"
|
||||
selectable
|
||||
allowTrailingMargin={continues && segment === segments[segments.length - 1]}
|
||||
md4cFlags={MD4C_FLAGS}
|
||||
linkRecognition={linkRecognition}
|
||||
enableTaskListItemToggle={false}
|
||||
enableLinkPreview={false}
|
||||
enableBlockContextMenu={false}
|
||||
// JS reveal owns pacing; native fade animation is a separate experiment.
|
||||
streamingAnimation={false}
|
||||
onLinkPress={pressLink}
|
||||
onImagePress={pressImage}
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# Streaming feel experiment
|
||||
|
||||
The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content.
|
||||
|
||||
## Plan
|
||||
|
||||
- Read the current state and identify the relevant boundary.
|
||||
- Keep the implementation small and verify the visible result.
|
||||
- Compare the recordings at their original playback speed.
|
||||
|
||||
The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content.
|
||||
|
||||
## Example
|
||||
|
||||
```typescript
|
||||
const value1 = calculateTotal(items.slice(0, 1)); // update the next result
|
||||
const value2 = calculateTotal(items.slice(0, 2)); // update the next result
|
||||
const value3 = calculateTotal(items.slice(0, 3)); // update the next result
|
||||
const value4 = calculateTotal(items.slice(0, 4)); // update the next result
|
||||
const value5 = calculateTotal(items.slice(0, 5)); // update the next result
|
||||
const value6 = calculateTotal(items.slice(0, 6)); // update the next result
|
||||
const value7 = calculateTotal(items.slice(0, 7)); // update the next result
|
||||
const value8 = calculateTotal(items.slice(0, 8)); // update the next result
|
||||
const value9 = calculateTotal(items.slice(0, 9)); // update the next result
|
||||
const value10 = calculateTotal(items.slice(0, 10)); // update the next result
|
||||
const value11 = calculateTotal(items.slice(0, 11)); // update the next result
|
||||
const value12 = calculateTotal(items.slice(0, 12)); // update the next result
|
||||
```
|
||||
|
||||
## Results
|
||||
|
||||
| Stage | Observation | Result |
|
||||
| ------ | ---------------------------------------- | ------ |
|
||||
| Step 1 | The next group of words becomes readable | Ready |
|
||||
| Step 2 | The next group of words becomes readable | Ready |
|
||||
| Step 3 | The next group of words becomes readable | Ready |
|
||||
| Step 4 | The next group of words becomes readable | Ready |
|
||||
| Step 5 | The next group of words becomes readable | Ready |
|
||||
| Step 6 | The next group of words becomes readable | Ready |
|
||||
| Step 7 | The next group of words becomes readable | Ready |
|
||||
| Step 8 | The next group of words becomes readable | Ready |
|
||||
|
||||
## Explanation
|
||||
|
||||
The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content. The stream should stay easy to follow while new text arrives. Small groups of words explain the next step without distracting from what is already readable. We keep the same fixture and delivery rate so differences come from rendering rather than the content.
|
||||
|
||||
END STREAMING FEEL
|
||||
@@ -0,0 +1,71 @@
|
||||
import { readFileSync } from "node:fs";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { splitStreamingMarkdown } from "./streaming";
|
||||
|
||||
describe("native streaming Markdown", () => {
|
||||
it("keeps completed messages together for selection", () => {
|
||||
const text = "First.\n\nSecond.\n\nThird.";
|
||||
expect(splitStreamingMarkdown(text, false)).toEqual({ head: text, tail: "" });
|
||||
});
|
||||
it("commits earlier blocks only after the following source line is complete", () => {
|
||||
expect(splitStreamingMarkdown("First.\n\nSecond", true).head).toBe("");
|
||||
expect(splitStreamingMarkdown("First.\n\nSecond\n", true)).toEqual({
|
||||
head: "First.\n\n",
|
||||
tail: "Second\n",
|
||||
});
|
||||
});
|
||||
it.each([
|
||||
"- one\n\n- two\n",
|
||||
"> first\n> second\n",
|
||||
"Title\n---\n",
|
||||
"```mermaid\ngraph LR\nA --> B\n```\n",
|
||||
"- one\n - nested\n\n paragraph\n",
|
||||
" indented code\n\n still code\n",
|
||||
"| A | B |\n| --- | --- |\n| x | y |\n",
|
||||
"```ts\nconst value = 1;\n```\n",
|
||||
"<div>\nbody\n\nmore\n</div>\n",
|
||||
"[label]\n\n[label]: /target\n",
|
||||
"[label]: /target\n\n[label]\n",
|
||||
])("keeps an unfinished construct together: %s", (block) => {
|
||||
const prefix = "Introduction.\n\n";
|
||||
const source = prefix + block;
|
||||
const split = splitStreamingMarkdown(source, true);
|
||||
expect(split.head.length).toBeLessThanOrEqual(prefix.length);
|
||||
expect(split.head + split.tail).toBe(source);
|
||||
let previous = 0;
|
||||
for (let end = 0; end <= source.length; end++) {
|
||||
const prefixSplit = splitStreamingMarkdown(source.slice(0, end), true);
|
||||
expect(prefixSplit.head.length, `prefix ${end} of ${source}`).toBeGreaterThanOrEqual(
|
||||
previous,
|
||||
);
|
||||
previous = prefixSplit.head.length;
|
||||
}
|
||||
});
|
||||
it("preserves the indentation that controls fence content and list continuation", () => {
|
||||
for (const tail of [" ```ts\n const n = 1;\n", " - first\n continued\n"]) {
|
||||
expect(splitStreamingMarkdown(`Intro.\n\n${tail}`, true)).toEqual({
|
||||
head: "Intro.\n\n",
|
||||
tail,
|
||||
});
|
||||
}
|
||||
});
|
||||
it("keeps unresolved references and HTML in the tail even after later blocks", () => {
|
||||
for (const block of ["[label]", "[label]: /target", "<div>body</div>"]) {
|
||||
expect(splitStreamingMarkdown(`Intro.\n\n${block}\n\nNext.\n\nEnd.\n`, true).head).toBe(
|
||||
"Intro.\n\n",
|
||||
);
|
||||
}
|
||||
});
|
||||
it("never retreats or loses source across every prefix of mixed Markdown", () => {
|
||||
const fixture = readFileSync(new URL("./streaming.fixture.md", import.meta.url), "utf8");
|
||||
let previous = 0;
|
||||
for (let end = 0; end <= fixture.length; end++) {
|
||||
const text = fixture.slice(0, end);
|
||||
const { head, tail } = splitStreamingMarkdown(text, true);
|
||||
expect(head.length, `prefix ${end}`).toBeGreaterThanOrEqual(previous);
|
||||
expect(head + tail).toBe(text);
|
||||
previous = head.length;
|
||||
}
|
||||
expect(previous).toBeGreaterThan(1000);
|
||||
}, 15000);
|
||||
});
|
||||
@@ -0,0 +1,32 @@
|
||||
import { fromMarkdown } from "mdast-util-from-markdown";
|
||||
import { gfmTable } from "micromark-extension-gfm-table";
|
||||
import { gfmTableFromMarkdown } from "mdast-util-gfm-table";
|
||||
import type { Nodes } from "mdast";
|
||||
|
||||
// A later definition can resolve even a currently plain-text shortcut reference.
|
||||
// Keep its entire containing block, and all definitions, in the shared tail.
|
||||
// HTML normalization likewise needs the complete enclosing source.
|
||||
function needsDocumentContext(node: Nodes): boolean {
|
||||
if (node.type === "html" || node.type === "definition") return true;
|
||||
if (node.type === "linkReference" || node.type === "imageReference") return true;
|
||||
if (node.type === "text" && node.value.includes("[")) return true;
|
||||
return "children" in node && node.children.some(needsDocumentContext);
|
||||
}
|
||||
|
||||
/** The last parsed block and unfinished line are mutable; earlier blocks are closed. */
|
||||
export function splitStreamingMarkdown(text: string, streaming: boolean) {
|
||||
if (!streaming) return { head: text, tail: "" };
|
||||
const completeLines = text.slice(0, text.lastIndexOf("\n") + 1);
|
||||
const tree = fromMarkdown(completeLines, {
|
||||
extensions: [gfmTable()],
|
||||
mdastExtensions: [gfmTableFromMarkdown()],
|
||||
});
|
||||
let boundary = 0;
|
||||
for (let index = 0; index < tree.children.length - 1; index++) {
|
||||
if (needsDocumentContext(tree.children[index])) break;
|
||||
// mdast starts some nodes after their indentation; the next parser needs it too.
|
||||
const nextStart = tree.children[index + 1].position!.start.offset!;
|
||||
boundary = text.lastIndexOf("\n", nextStart - 1) + 1;
|
||||
}
|
||||
return { head: text.slice(0, boundary), tail: text.slice(boundary) };
|
||||
}
|
||||
Reference in New Issue
Block a user