nix: shrink daemon install with @vercel/nft tracing (#966)

* nix: share npmDeps FOD between paseo and paseo-desktop

The daemon and desktop derivations share package-lock.json, so their
npmDeps FODs contain byte-identical content. But Nix names the FOD
store path with the consuming pname prefix, so paseo-<v>-npm-deps and
paseo-desktop-<v>-npm-deps resolve to different store paths despite
identical bytes. Building both runs prefetch-npm-deps twice in
parallel; on a clean store this downloads the entire registry
(~1-2 GB of tarballs) twice over the wire.

Wire paseo through as a callPackage arg of the desktop drv and
inherit (paseo) npmDeps. One FOD, one fetch, one store path.

Override path becomes paseo.override { npmDepsHash = "..."; } — the
desktop drv picks up the overridden npmDeps transitively. Downstream
flakes that previously did paseo-desktop.override { npmDepsHash }
need to either drop the desktop-side override (recommended) or pass
the overridden daemon through as paseo-desktop.override { paseo }.

* nix: trace daemon runtime closure with @vercel/nft

The daemon Nix installPhase used to `cp -a node_modules $out/lib/paseo/`,
shipping every package in the hoisted root — Expo, React Native, Metro,
Electron, ML stacks, ~1700 third-party deps the daemon never `require()`s
at runtime. Result: ~2.6 GB store path for a daemon whose actual runtime
closure is a tiny fraction of that.

Replace it with static module-graph tracing via @vercel/nft (the same
library Vercel and Next.js use for serverless bundling). The new
installPhase runs `node scripts/trace-daemon.mjs`, which:

1. Calls nodeFileTrace on three entry points — cli/dist/index.js,
   server/dist/scripts/supervisor-entrypoint.js, and the forked
   server/dist/server/terminal/terminal-worker-process.js. The worker
   needs to be a separate entry because nft does not follow fork()
   process boundaries.
2. Unions the trace result with an explicit list of non-JS runtime
   inputs nft does not detect: shell-integration assets read via
   readFileSync, the Silero VAD ONNX model loaded by the sherpa
   provider, .env.example, the CLI shebang script, and node-pty's
   compiled prebuilt binary for the host platform.
3. Ignores cross-platform native variants we explicitly do not ship
   (sherpa-onnx-*, @mariozechner/clipboard-*) and node-fetch's
   optional `encoding` peer.

installPhase consumes the file list, copies each entry into
$out/lib/paseo/ preserving its repo-relative path, and wraps two bin
entries (paseo, paseo-server) via makeWrapper.

Verified end-to-end:
- daemon $out drops from ~2.6 GB to ~131 MB (≈95% reduction)
- paseo --version, paseo --help work from the new $out
- paseo-server boots, HTTP server reaches "Server listening" within
  ~40 ms, no missing-module errors in the bootstrap log
- speech features degrade gracefully when sherpa-onnx-* / model files
  are absent (the documented behaviour for the Nix build, unchanged)

@vercel/nft added to root devDependencies (1.5.0); used only at
build time by the Nix derivation.

* fix(nix): gitignore result

* fix(nix): refresh npm deps hash post-rebase

The conflict resolution during rebase kept the trace-daemon hash, but
the merged package-lock.json combines upstream's lockfile updates with
the @vercel/nft addition, so the hash needed to be recomputed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Matan Bendix Shenhav
2026-05-15 14:08:06 +02:00
committed by GitHub
parent 5ea68dcdfa
commit e309d82b41
8 changed files with 477 additions and 70 deletions

99
scripts/trace-daemon.mjs Normal file
View File

@@ -0,0 +1,99 @@
#!/usr/bin/env node
// Emit the set of files the daemon and CLI need at runtime, computed by
// static module-graph tracing (@vercel/nft) from the three entry points.
// Used by nix/package.nix's installPhase to materialize $out/lib/paseo
// with only the bytes the daemon actually loads — no Expo, RN, Metro,
// Electron, ML stacks, or other non-daemon workspace bloat.
//
// Output: newline-separated repo-relative file paths on stdout. The Nix
// installPhase copies each path to $out/lib/paseo/<path>, preserving the
// directory structure node's module resolution expects.
//
// Run from the repo root, after `npm run build:daemon`. Requires
// node_modules populated (the Nix build invokes this post-configHook).
import { nodeFileTrace } from "@vercel/nft";
import { glob } from "node:fs/promises";
import path from "node:path";
const REPO_ROOT = path.resolve(import.meta.dirname, "..");
// Three entry points. The terminal worker is forked into its own Node
// process and has its own require tree (node-pty, etc.) — nft does not
// follow fork boundaries, so it must be traced separately.
const entries = [
"packages/cli/dist/index.js",
"packages/server/dist/scripts/supervisor-entrypoint.js",
"packages/server/dist/server/terminal/terminal-worker-process.js",
];
// Files read at runtime via fs APIs rather than `require`. nft only
// traces the module graph; data files have to be listed explicitly.
const additionalInputs = [
// Shell integration scripts loaded by the terminal manager
"packages/server/dist/server/terminal/shell-integration/**",
// Silero VAD ONNX model (sherpa speech provider)
"packages/server/dist/server/server/speech/providers/local/sherpa/assets/silero_vad.onnx",
// Server runtime config files (read by path, not require)
"packages/server/.env.example",
// CLI shebang script wrapping dist/index.js
"packages/cli/bin/paseo",
// node-pty's compiled native addon. nft can't trace it because
// node-pty loads it via `require(path.join(__dirname, 'prebuilds/<plat>/pty.node'))`
// with a runtime-computed platform suffix. Pin to the host platform —
// the Nix derivation builds for one platform at a time and ships only
// its own binaries.
`node_modules/node-pty/prebuilds/${process.platform}-${process.arch}/**`,
];
// Trace.
const { fileList, warnings } = await nodeFileTrace(entries, {
base: REPO_ROOT,
// Tolerate the conditional / dynamic patterns we already audited:
// sherpa-onnx-${platform}-${arch} package resolution (sherpa is
// intentionally not built in the Nix sandbox; voice features degrade
// gracefully when unavailable), and a handful of test-only requires
// that get tree-shaken out by tsc.
ignore: [
// Cross-platform native packages for the sherpa speech runtime;
// unsupported in the Nix build, lazily loaded when present.
"sherpa-onnx-*/**",
// Platform-specific clipboard variants; only the host's variant
// is needed at runtime, and the package's index.js resolver picks
// the right one dynamically.
"@mariozechner/clipboard-*/**",
// node-fetch optional peer for non-UTF-8 charset decoding; not
// loaded in our usage.
"encoding/**",
// Tests are stripped during the daemon build; nft sometimes still
// tries to walk into them via index files. Belt and suspenders.
"**/*.test.js",
"**/*.e2e.test.js",
],
});
// Surface non-trivial trace warnings so the Nix build log captures them.
for (const w of warnings) {
// Drop the "Failed to resolve dependency" noise for things we
// explicitly ignore above.
const msg = w.message ?? String(w);
if (/sherpa-onnx-/.test(msg)) continue;
console.error("trace warning:", msg);
}
// Expand globs in additionalInputs.
const expanded = new Set(fileList);
for (const pattern of additionalInputs) {
if (pattern.includes("*")) {
for await (const file of glob(pattern, { cwd: REPO_ROOT })) {
expanded.add(file);
}
} else {
expanded.add(pattern);
}
}
// Emit sorted, deduplicated.
for (const p of [...expanded].sort()) {
console.log(p);
}