mirror of
https://github.com/getpaseo/paseo.git
synced 2026-10-31 05:32:58 +00:00
refactor(server): stream terminal replay from the worker
Move ANSI generation and replay/live ordering to the worker while preserving existing ANSI clients through the daemon's single-frame restore adapter. Keep 1,000 rows of retention and hard cut the unreleased cell transport. Pace replay writes so a healthy large restore cannot fill the shared socket. If the bounded live tail overflows, end every viewer of that terminal with an explicit error; a fresh attach restores current state and the PTY survives. Credit Manav Dhindsa (DrakenKor), #6139, and Lennart Schoch, #6743, for the prior worker-side ANSI and scrollback work informing this change.
This commit is contained in:
@@ -6,9 +6,9 @@ How terminal output stays low-latency, what the invariants are, and how to measu
|
||||
|
||||
```
|
||||
pty (node-pty, forked worker process)
|
||||
→ headless xterm parse (worker, snapshot fidelity)
|
||||
→ TerminalOutputCoalescer (worker, ≤1 IPC message per 5ms per terminal)
|
||||
→ process.send IPC → daemon main process
|
||||
→ headless xterm parse (worker, ANSI replay fidelity)
|
||||
→ TerminalOutputCoalescer (worker, ≤1 socket frame per 5ms per terminal)
|
||||
→ framed local socket → daemon main process
|
||||
→ TerminalOutputCoalescer (per client stream, terminal-session-controller.ts)
|
||||
→ binary ws frame (2-byte header + raw bytes)
|
||||
→ client decode (daemon-client.ts) → stream router → emulator runtime
|
||||
@@ -20,9 +20,9 @@ Terminal frames share the daemon main event loop with all agent traffic. The `ev
|
||||
## Invariants (the easy-to-break ones)
|
||||
|
||||
- **Coalescers are leading+trailing throttles.** The first chunk after an idle window flushes immediately (synchronously); only sustained bursts wait for the trailing timer. Reverting to trailing-only adds a full window (~5ms) to every keystroke echo.
|
||||
- **Output coalescing happens in the worker, before IPC.** One `process.send` per pty chunk was a main-loop flood under build output. Non-output messages (snapshot/snapshotReady/titleChange/exit) must flush the coalescer first so ordering is preserved.
|
||||
- **Output coalescing happens in the worker, before IPC.** One socket frame per pty chunk was a main-loop flood under build output. Non-output messages (snapshot/snapshotReady/titleChange/exit) must flush the coalescer first so ordering is preserved.
|
||||
- **Coalesced output carries the LAST chunk's revision.** Snapshot replay dedup (`replayTerminalOutputAfterSnapshot`) skips buffered output with `revision <= replayRevision`; a merged batch with a lower revision would be wrongly skipped (lost output).
|
||||
- **The input-mode tracker runs once per process boundary, not per hop.** The worker owns the authoritative tracker; the daemon caches the replay preamble from `getTerminalState` responses and `snapshotReady` messages. Do not reintroduce a per-chunk `feed()` on the daemon main loop.
|
||||
- **The input-mode tracker runs once per process boundary, not per hop.** The worker owns the authoritative tracker; the daemon caches the replay preamble from replay responses and `snapshotReady` messages. Do not reintroduce a per-chunk `feed()` on the daemon main loop.
|
||||
- **Snapshot catch-up is backpressure-gated.** A stream falls back to a full snapshot only when `outputBytesSinceSnapshot > MAX_TERMINAL_OUTPUT_FRAME_BYTES` (256KB) **and** the client transport reports `bufferedAmount > MAX_CLIENT_BUFFERED_BYTES` (4MB). A client that keeps draining streams continuously, no matter how much output is produced. Before this gate existed, every 256KB of build output dropped a frame and forced a full JSON cell-grid snapshot (~200k objects across IPC) — the historical source of spiky lag and GC hitches.
|
||||
- **Plugin daemon sessions report IPC queue bytes.** Their virtual socket increments `bufferedAmount` before `process.send` and decrements it only in the send callback. Text and binary frames share that ordered queue, so the normal snapshot catch-up and physical high-water gates remain valid for server-side plugin SDK traffic.
|
||||
- **Client output writes are not serialized per frame.** The emulator runtime drains contiguous plain writes straight into xterm (which buffers internally). Only barrier ops (`clear`, `snapshot`, `suppressInput` writes) wait — behind a zero-length sentinel write — so resets can't interleave with in-flight output.
|
||||
|
||||
@@ -40,10 +40,16 @@ The worker writes `$PASEO_HOME/terminal-worker/<pid>.json` (owner-only) with its
|
||||
|
||||
Never more than two workers run. A new worker launches only when none is current, none is incompatible and fewer than two run. The count comes from the records, so an unreachable or unidentified worker counts, and so does one a cleanup could not confirm gone. When two run, the daemon starts with no current worker, and the first terminal created after one of them exits launches it.
|
||||
|
||||
**The frozen envelope never changes across versions:** the frame format (4-byte length + `v8.serialize`), `hello`/`helloAck`, the `listTerminals` info fields and `shutdown`. That is what lets any daemon close any worker: a kill stop sends `shutdown` to a newer worker it cannot otherwise drive. When you bump the version, keep a previous-version adapter so the draining worker can still be driven, and drop the adapter for the version before that.
|
||||
**The frozen envelope never changes across versions:** the frame format (4-byte length + `v8.serialize`), `hello`/`helloAck`, the `listTerminals` info fields and `shutdown`. That is what lets any daemon close any worker: a kill stop sends `shutdown` to a newer worker it cannot otherwise drive. Protocol 2 hard cuts the unreleased protocol 1 cell transport: protocol 1 workers are obsolete and close. For future versions, keep a previous-version adapter so the draining worker can still be driven, and drop the adapter for the version before that.
|
||||
|
||||
Tests stand up other versions with `PASEO_TERMINAL_WORKER_PROTOCOL_OVERRIDE`, read by the worker. It is test-only.
|
||||
|
||||
## Display replay
|
||||
|
||||
The worker keeps the headless xterm model and generates ANSI. A replay request emits begin metadata (dimensions, retained history, revision and input modes), at most 64 KiB per data chunk, and end before later live output. Capture is synchronous with PTY callbacks; flush the output coalescer before the cut so a later batch never includes bytes already in replay. Retained-history metadata counts the worker history before the request limit.
|
||||
|
||||
Replay waits for each socket write to complete before sending the next chunk. PTYs keep parsing while writes wait. The replaying terminal holds at most 4 MiB of live output until replay ends; exceeding that bound fails the replay and ends every display stream of that terminal with an error requiring a fresh attach. Existing viewers share the worker live stream, so they must also stop before bytes resume after a discarded tail. Sibling terminals continue streaming. Live traffic that fills the shared socket beyond 4 MiB disconnects the stalled controller; the daemon reconnects and requests fresh replay. Existing ANSI clients receive one collected Restore frame from the daemon. Capture stays a worker request; bootstrap readiness uses `hasOutput`, not a transported grid.
|
||||
|
||||
## Running from its own copy
|
||||
|
||||
A worker outlives the installation that started it. An npm or desktop update replaces that installation's files, and the next terminal the old worker creates would load node-pty's `spawn-helper` and the zsh integration from paths that no longer exist. So a built daemon does not launch the worker from its install:
|
||||
|
||||
Reference in New Issue
Block a user