mirror of
https://github.com/getpaseo/paseo.git
synced 2026-07-29 12:01:31 +00:00
nix: shrink daemon install with @vercel/nft tracing (#966)
* nix: share npmDeps FOD between paseo and paseo-desktop
The daemon and desktop derivations share package-lock.json, so their
npmDeps FODs contain byte-identical content. But Nix names the FOD
store path with the consuming pname prefix, so paseo-<v>-npm-deps and
paseo-desktop-<v>-npm-deps resolve to different store paths despite
identical bytes. Building both runs prefetch-npm-deps twice in
parallel; on a clean store this downloads the entire registry
(~1-2 GB of tarballs) twice over the wire.
Wire paseo through as a callPackage arg of the desktop drv and
inherit (paseo) npmDeps. One FOD, one fetch, one store path.
Override path becomes paseo.override { npmDepsHash = "..."; } — the
desktop drv picks up the overridden npmDeps transitively. Downstream
flakes that previously did paseo-desktop.override { npmDepsHash }
need to either drop the desktop-side override (recommended) or pass
the overridden daemon through as paseo-desktop.override { paseo }.
* nix: trace daemon runtime closure with @vercel/nft
The daemon Nix installPhase used to `cp -a node_modules $out/lib/paseo/`,
shipping every package in the hoisted root — Expo, React Native, Metro,
Electron, ML stacks, ~1700 third-party deps the daemon never `require()`s
at runtime. Result: ~2.6 GB store path for a daemon whose actual runtime
closure is a tiny fraction of that.
Replace it with static module-graph tracing via @vercel/nft (the same
library Vercel and Next.js use for serverless bundling). The new
installPhase runs `node scripts/trace-daemon.mjs`, which:
1. Calls nodeFileTrace on three entry points — cli/dist/index.js,
server/dist/scripts/supervisor-entrypoint.js, and the forked
server/dist/server/terminal/terminal-worker-process.js. The worker
needs to be a separate entry because nft does not follow fork()
process boundaries.
2. Unions the trace result with an explicit list of non-JS runtime
inputs nft does not detect: shell-integration assets read via
readFileSync, the Silero VAD ONNX model loaded by the sherpa
provider, .env.example, the CLI shebang script, and node-pty's
compiled prebuilt binary for the host platform.
3. Ignores cross-platform native variants we explicitly do not ship
(sherpa-onnx-*, @mariozechner/clipboard-*) and node-fetch's
optional `encoding` peer.
installPhase consumes the file list, copies each entry into
$out/lib/paseo/ preserving its repo-relative path, and wraps two bin
entries (paseo, paseo-server) via makeWrapper.
Verified end-to-end:
- daemon $out drops from ~2.6 GB to ~131 MB (≈95% reduction)
- paseo --version, paseo --help work from the new $out
- paseo-server boots, HTTP server reaches "Server listening" within
~40 ms, no missing-module errors in the bootstrap log
- speech features degrade gracefully when sherpa-onnx-* / model files
are absent (the documented behaviour for the Nix build, unchanged)
@vercel/nft added to root devDependencies (1.5.0); used only at
build time by the Nix derivation.
* fix(nix): gitignore result
* fix(nix): refresh npm deps hash post-rebase
The conflict resolution during rebase kept the trace-daemon hash, but
the merged package-lock.json combines upstream's lockfile updates with
the @vercel/nft addition, so the hash needed to be recomputed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
committed by
GitHub
parent
5ea68dcdfa
commit
e309d82b41
99
scripts/trace-daemon.mjs
Normal file
99
scripts/trace-daemon.mjs
Normal file
@@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env node
|
||||
// Emit the set of files the daemon and CLI need at runtime, computed by
|
||||
// static module-graph tracing (@vercel/nft) from the three entry points.
|
||||
// Used by nix/package.nix's installPhase to materialize $out/lib/paseo
|
||||
// with only the bytes the daemon actually loads — no Expo, RN, Metro,
|
||||
// Electron, ML stacks, or other non-daemon workspace bloat.
|
||||
//
|
||||
// Output: newline-separated repo-relative file paths on stdout. The Nix
|
||||
// installPhase copies each path to $out/lib/paseo/<path>, preserving the
|
||||
// directory structure node's module resolution expects.
|
||||
//
|
||||
// Run from the repo root, after `npm run build:daemon`. Requires
|
||||
// node_modules populated (the Nix build invokes this post-configHook).
|
||||
|
||||
import { nodeFileTrace } from "@vercel/nft";
|
||||
import { glob } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
|
||||
const REPO_ROOT = path.resolve(import.meta.dirname, "..");
|
||||
|
||||
// Three entry points. The terminal worker is forked into its own Node
|
||||
// process and has its own require tree (node-pty, etc.) — nft does not
|
||||
// follow fork boundaries, so it must be traced separately.
|
||||
const entries = [
|
||||
"packages/cli/dist/index.js",
|
||||
"packages/server/dist/scripts/supervisor-entrypoint.js",
|
||||
"packages/server/dist/server/terminal/terminal-worker-process.js",
|
||||
];
|
||||
|
||||
// Files read at runtime via fs APIs rather than `require`. nft only
|
||||
// traces the module graph; data files have to be listed explicitly.
|
||||
const additionalInputs = [
|
||||
// Shell integration scripts loaded by the terminal manager
|
||||
"packages/server/dist/server/terminal/shell-integration/**",
|
||||
// Silero VAD ONNX model (sherpa speech provider)
|
||||
"packages/server/dist/server/server/speech/providers/local/sherpa/assets/silero_vad.onnx",
|
||||
// Server runtime config files (read by path, not require)
|
||||
"packages/server/.env.example",
|
||||
// CLI shebang script wrapping dist/index.js
|
||||
"packages/cli/bin/paseo",
|
||||
// node-pty's compiled native addon. nft can't trace it because
|
||||
// node-pty loads it via `require(path.join(__dirname, 'prebuilds/<plat>/pty.node'))`
|
||||
// with a runtime-computed platform suffix. Pin to the host platform —
|
||||
// the Nix derivation builds for one platform at a time and ships only
|
||||
// its own binaries.
|
||||
`node_modules/node-pty/prebuilds/${process.platform}-${process.arch}/**`,
|
||||
];
|
||||
|
||||
// Trace.
|
||||
const { fileList, warnings } = await nodeFileTrace(entries, {
|
||||
base: REPO_ROOT,
|
||||
// Tolerate the conditional / dynamic patterns we already audited:
|
||||
// sherpa-onnx-${platform}-${arch} package resolution (sherpa is
|
||||
// intentionally not built in the Nix sandbox; voice features degrade
|
||||
// gracefully when unavailable), and a handful of test-only requires
|
||||
// that get tree-shaken out by tsc.
|
||||
ignore: [
|
||||
// Cross-platform native packages for the sherpa speech runtime;
|
||||
// unsupported in the Nix build, lazily loaded when present.
|
||||
"sherpa-onnx-*/**",
|
||||
// Platform-specific clipboard variants; only the host's variant
|
||||
// is needed at runtime, and the package's index.js resolver picks
|
||||
// the right one dynamically.
|
||||
"@mariozechner/clipboard-*/**",
|
||||
// node-fetch optional peer for non-UTF-8 charset decoding; not
|
||||
// loaded in our usage.
|
||||
"encoding/**",
|
||||
// Tests are stripped during the daemon build; nft sometimes still
|
||||
// tries to walk into them via index files. Belt and suspenders.
|
||||
"**/*.test.js",
|
||||
"**/*.e2e.test.js",
|
||||
],
|
||||
});
|
||||
|
||||
// Surface non-trivial trace warnings so the Nix build log captures them.
|
||||
for (const w of warnings) {
|
||||
// Drop the "Failed to resolve dependency" noise for things we
|
||||
// explicitly ignore above.
|
||||
const msg = w.message ?? String(w);
|
||||
if (/sherpa-onnx-/.test(msg)) continue;
|
||||
console.error("trace warning:", msg);
|
||||
}
|
||||
|
||||
// Expand globs in additionalInputs.
|
||||
const expanded = new Set(fileList);
|
||||
for (const pattern of additionalInputs) {
|
||||
if (pattern.includes("*")) {
|
||||
for await (const file of glob(pattern, { cwd: REPO_ROOT })) {
|
||||
expanded.add(file);
|
||||
}
|
||||
} else {
|
||||
expanded.add(pattern);
|
||||
}
|
||||
}
|
||||
|
||||
// Emit sorted, deduplicated.
|
||||
for (const p of [...expanded].sort()) {
|
||||
console.log(p);
|
||||
}
|
||||
Reference in New Issue
Block a user