Mohamed Boudra
e998e0a082
* fix(server): back off degraded git polling on an unchanged workspace When the recursive watcher declines a repository (over 250,000 entries on macOS/Windows, or any watcher setup failure), WorkspaceGit falls back to polling. That fallback rescheduled itself every 5 seconds for as long as the workspace existed, with no client or agent involved. On a repository large enough to defeat the watcher each tick is an expensive `git status`, so the daemon held about a core indefinitely. Double the gap after every tick whose workspace snapshot came back identical, up to a 60-second ceiling, and reset to the 5-second base on the first change. Both degraded loops — working tree and repository metadata — share the rule. Explicit reads still force a refresh, so the ceiling only bounds how long an edit made outside Paseo stays undiscovered. Fixes #5102 * test(server): cover repository-metadata degraded backoff, document failed-tick semantics * test(server): drive the recovery-refresh assertion off the fake clock, not waitFor * test(server): declare metadata poll counter as a function
5.1 KiB
File observation
Use packages/server/src/server/file-observer for recursive filesystem observation. Create one observer service at the owning service boundary, subscribe roots through it, and close it during owner shutdown. Keep platform watchers, directory discovery, exclusion reconciliation, event batching, failure cleanup, diagnostics, and teardown inside the module. A consumer supplies a root and excluded subtrees; it must not branch on the platform or manage child watchers.
Do not replace the Linux implementation with fs.watch({ recursive: true }). Node 22 implements Linux recursion in JavaScript by walking the tree and watching every file and directory. Paseo's directory-only Linux watcher already exhausted resources before ignored-tree pruning and a watcher cap were added in #794.
Closing an observation must remain safe after its root is renamed or removed. Workspace archive removes owned worktrees while releasing observation references, so teardown cannot assume the watched path still exists.
unsubscribe() is an awaited barrier. Once it resolves, no scan, reconciliation, queued event, native handle, or callback may remain. updateIgnore() is the equivalent barrier for excluded paths: events queued below a newly excluded root cannot be delivered after it resolves. Treat path disappearance while scanning, attaching, or receiving a watcher error as rename/archive churn. Surface permission and resource errors so WorkspaceGit can enter bounded polling.
Linux topology repair is trailing-debounced and scoped to the renamed path. A file rename needs one classification stat; a new or moved directory scans only its subtree. Full-tree scans are reserved for startup and ignore-set replacement. The initial Linux scan conservatively emits existing files discovered before their directory watcher was installed; WorkspaceGit batches these into one refresh. Benchmark this startup cost with pre-populated trees.
Windows and macOS use one native recursive watcher plus an ignored-pruned, directory-indexed inventory. Native events deliver immediately. Named events trigger serialized shallow scans of their parent directories; known directory events also reconcile that subtree. This recovers concrete create and delete paths omitted by coalescing without walking the whole root for each edit. Classification is admission-controlled to 32 concurrent fs.stat calls, since fs.stat runs on the libuv threadpool the whole daemon shares. Startup and exclusion changes use an awaited full inventory barrier. Activity arms a full safety audit 30 seconds after it settles, with a 30-second minimum interval and a five-minute starvation bound; pathless native events also trigger an immediate shallow root scan. Observation fails into polling rather than retaining more than 250,000 file and directory entries per root. A repository that large is also expensive to poll, so degraded polling doubles its gap from 5 seconds to a 60-second ceiling while the workspace snapshot comes back unchanged, and resets to 5 seconds on the first change. Explicit reads still force a refresh; the ceiling only bounds how long an edit made outside Paseo stays undiscovered.
Read aggregate health from the owning observer service. Runtime metrics include active observations, native handles, tracked native files, pending events and immediate reconciliation work, scoped and full reconciliation activity and latency, and failure counts. nativeTrackedFileCount and other per-file counters are native-backend diagnostics only — the Linux backend keeps no per-file inventory and reports zero — so a cross-platform assertion belongs on delivered events, not those counters. Delayed safety audits do not count as pending work. Closing the service releases every subscription and clears its diagnostics. Do not add path lists, watcher handles, or platform-specific controls to this interface.
Git owns Git-ignore evaluation. The observer accepts absolute excluded roots and applies updates without replacing the observation or exposing its watcher topology. This keeps tracked files inside otherwise ignored directories observable and keeps Git policy out of the filesystem module.
Workspace Git verifies each repository metadata subscription with a one-shot canary inside the Git directory. If the event does not round-trip through the subscription callback, treat the watcher as unavailable and enter degraded polling. Refresh working-tree Git-ignore exclusions from ignore-file events, newly observed directories, and watcher recovery, never from a healthy-watcher timer. git ls-files reports an ignored directory only once it exists, so new directory topology is a trigger in its own right; the refresh is edge-triggered per directory and debounced so a dependency install collapses into a couple of Git runs.
The real-filesystem contracts and daemon auto-archive lifecycle run in the normal server test suite. Use the scripts only for manual performance and soak work: npm run measure:file-observer --workspace=@getpaseo/server measures burst and sustained-create behavior, and npm run repro:file-observer-teardown --workspace=@getpaseo/server runs the teardown soak.