mirror of
https://github.com/getpaseo/paseo.git
synced 2026-11-10 10:25:03 +00:00
Mohamed Boudra
48a1497fc0
* fix(server): keep cold diffs out from behind background git fan-out Add a background lane to the git process scheduler and run reconciliation checkout reads, observation-driven workspace refreshes and background fetches in it. Bound reconciliation checkout reads to four at a time. Stop the Linux file observer's startup scan from reporting existing files as creates, which made every cold diff compute and discard its first revision. * fix(server): pick the git job when its start token fires Background jobs that already reserved a start-rate token no longer start ahead of normal work submitted after them. * fix(server): re-read the working tree once its watcher covers it The Linux observer no longer reports pre-existing files at startup, so an edit made between a workspace's first snapshot and its watcher subscribing reached no consumer. WorkspaceGit now runs one background working-tree read once observation setup completes with a live watcher. Demand refreshes keep the caller's Git lane instead of forcing normal, so user actions running at high priority are not demoted. Scope the populated-tree observer test to Linux (FSEvents replays recent writes) and update the observer benchmark test, which expected the removed startup batch. * fix(server): re-read checkout metadata once observation covers it A branch switch made before the Git metadata watcher subscribed left the cached branch stale. The post-coverage read now refreshes structure as well as the working tree, matching the refresh the Linux startup events used to trigger.
22 lines
5.5 KiB
Markdown
22 lines
5.5 KiB
Markdown
# File observation
|
|
|
|
Use `packages/server/src/server/file-observer` for recursive filesystem observation. Create one observer service at the owning service boundary, subscribe roots through it, and close it during owner shutdown. Keep platform watchers, directory discovery, exclusion reconciliation, event batching, failure cleanup, diagnostics, and teardown inside the module. A consumer supplies a root and excluded subtrees; it must not branch on the platform or manage child watchers.
|
|
|
|
Do not replace the Linux implementation with `fs.watch({ recursive: true })`. Node 22 implements Linux recursion in JavaScript by walking the tree and watching every file and directory. Paseo's directory-only Linux watcher already exhausted resources before ignored-tree pruning and a watcher cap were added in [#794](https://github.com/getpaseo/paseo/pull/794).
|
|
|
|
Closing an observation must remain safe after its root is renamed or removed. Workspace archive removes owned worktrees while releasing observation references, so teardown cannot assume the watched path still exists.
|
|
|
|
`unsubscribe()` is an awaited barrier. Once it resolves, no scan, reconciliation, queued event, native handle, or callback may remain. `updateIgnore()` is the equivalent barrier for excluded paths: events queued below a newly excluded root cannot be delivered after it resolves. Treat path disappearance while scanning, attaching, or receiving a watcher error as rename/archive churn. Surface permission and resource errors so WorkspaceGit can enter bounded polling.
|
|
|
|
Linux topology repair is trailing-debounced and scoped to the renamed path. A file rename needs one classification stat; a new or moved directory scans only its subtree. Full-tree scans are reserved for startup and ignore-set replacement. A subscription reports changes after `subscribe()` resolves, not the files that already existed. macOS FSEvents can still replay writes made moments before the stream started. Consumers read their initial state after `subscribe()` resolves; WorkspaceGit re-reads its checkout once observation setup completes, because its first refresh runs in parallel with setup. The Linux startup scan therefore emits nothing; only a scoped reconciliation of new topology reports files found before their directory's watcher was installed. Startup creates made every cold diff compute and discard a first revision. Benchmark startup cost with pre-populated trees.
|
|
|
|
Windows and macOS use one native recursive watcher plus an ignored-pruned, directory-indexed inventory. Native events deliver immediately. Named events trigger serialized shallow scans of their parent directories; known directory events also reconcile that subtree. This recovers concrete create and delete paths omitted by coalescing without walking the whole root for each edit. Classification is admission-controlled to 32 concurrent `fs.stat` calls, since `fs.stat` runs on the libuv threadpool the whole daemon shares. Startup and exclusion changes use an awaited full inventory barrier. Activity arms a full safety audit 30 seconds after it settles, with a 30-second minimum interval and a five-minute starvation bound; pathless native events also trigger an immediate shallow root scan. Observation fails into polling rather than retaining more than 250,000 file and directory entries per root. A repository that large is also expensive to poll, so degraded polling doubles its gap from 5 seconds to a 60-second ceiling while the workspace snapshot comes back unchanged, and resets to 5 seconds on the first change. Explicit reads still force a refresh; the ceiling only bounds how long an edit made outside Paseo stays undiscovered.
|
|
|
|
Read aggregate health from the owning observer service. Runtime metrics include active observations, native handles, tracked native files, pending events and immediate reconciliation work, scoped and full reconciliation activity and latency, and failure counts. `nativeTrackedFileCount` and other per-file counters are native-backend diagnostics only — the Linux backend keeps no per-file inventory and reports zero — so a cross-platform assertion belongs on delivered events, not those counters. Delayed safety audits do not count as pending work. Closing the service releases every subscription and clears its diagnostics. Do not add path lists, watcher handles, or platform-specific controls to this interface.
|
|
|
|
Git owns Git-ignore evaluation. The observer accepts absolute excluded roots and applies updates without replacing the observation or exposing its watcher topology. This keeps tracked files inside otherwise ignored directories observable and keeps Git policy out of the filesystem module.
|
|
|
|
Workspace Git verifies each repository metadata subscription with a one-shot canary inside the Git directory. If the event does not round-trip through the subscription callback, treat the watcher as unavailable and enter degraded polling. Refresh working-tree Git-ignore exclusions from ignore-file events, newly observed directories, and watcher recovery, never from a healthy-watcher timer. `git ls-files` reports an ignored directory only once it exists, so new directory topology is a trigger in its own right; the refresh is edge-triggered per directory and debounced so a dependency install collapses into a couple of Git runs.
|
|
|
|
The real-filesystem contracts and daemon auto-archive lifecycle run in the normal server test suite. Use the scripts only for manual performance and soak work: `npm run measure:file-observer --workspace=@getpaseo/server` measures burst and sustained-create behavior, and `npm run repro:file-observer-teardown --workspace=@getpaseo/server` runs the teardown soak.
|