From cbbf6c1684fb0415b7949e684d152f5f7453e769 Mon Sep 17 00:00:00 2001 From: Michael Wu Date: Mon, 27 Jul 2026 20:18:40 -0500 Subject: [PATCH] Carry local files and shared state into new worktrees (#2419) * feat(server): support symlink worktree includes * refactor(server): simplify worktree include handling * fix(server): constrain worktree include traversal * fix(server): clean up failed worktree branches * fix(server): skip missing worktree include entries * fix(server): make worktree includes best effort * fix(server): skip failed worktree includes * fix(server): harden worktree include planning * fix(server): preserve reused worktree result shape Materialization reports describe one creation attempt, not durable worktree identity. Carry them beside the worktree so newly created and reused results keep the same stable shape. * fix(server): harden worktree include boundaries Replace directory snapshots exactly, keep canonical Git metadata protected, report recovery skips, and only roll back branches owned before worktree creation. * fix(server): follow safe include aliases Traverse canonical in-checkout directory links during glob planning and treat coded revalidation failures as per-entry skips. * fix(server): make include preflight race safe Create fetched checkout refs atomically, retain overlapping copy entries for independent fallback, and protect the full managed-worktree base. * fix(server): preserve staged recovery state Validate completed directory snapshots, retain backups after failed restoration, and derive atomic ref guards from the repository object ID width. * fix(server): preserve partial include progress Keep safe glob matches, enforce recursive-directory types, validate staged roots, and retain OID guards through rollback. --------- Co-authored-by: Mohamed Boudra --- SECURITY.md | 6 + docs/development.md | 1 + .../src/server/paseo-worktree-service.ts | 3 + packages/server/src/server/session.ts | 1 + .../workspace-recovery-service.test.ts | 14 + .../workspace-recovery-service.ts | 12 + packages/server/src/server/worktree-core.ts | 20 +- .../server/src/server/worktree-session.ts | 11 + .../server/src/utils/worktree-include.test.ts | 635 ++++++++ packages/server/src/utils/worktree-include.ts | 1359 +++++++++++++++++ .../server/src/utils/worktree.posix.test.ts | 298 ++++ packages/server/src/utils/worktree.ts | 462 ++++-- public-docs/worktrees.md | 40 + 13 files changed, 2721 insertions(+), 141 deletions(-) create mode 100644 packages/server/src/utils/worktree-include.test.ts create mode 100644 packages/server/src/utils/worktree-include.ts diff --git a/SECURITY.md b/SECURITY.md index dacb14c41..4acb376b4 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -50,6 +50,12 @@ The daemon also supports an optional shared-secret password (set via `auth.passw Connected clients are trusted operators of the daemon user. File previews follow that authority: a preview request may read any regular file the daemon process can read, while keeping path normalization and symlink checks in the daemon file service. Workspace-relative paths remain a UI convenience, not a security boundary. +An explicit `symlink ` entry in a repository's .worktreeinclude intentionally gives a +Paseo-created worktree live access to that source-checkout file or directory. It is useful for +local dependencies and caches, but it weakens the usual worktree isolation: agents and lifecycle +scripts can modify the source through the link. Paseo validates entries and refuses traversal or +destination-link escapes, but the linked source is a deliberate shared-data boundary. + If you expose the daemon beyond loopback, such as by binding to `0.0.0.0`, forwarding it through a tunnel or reverse proxy, or publishing it from a Docker container, you are responsible for restricting and securing that access. Setting a password is strongly recommended in that case. In Docker, the official image runs the daemon and agents as the non-root diff --git a/docs/development.md b/docs/development.md index cd96239ba..31b35d012 100644 --- a/docs/development.md +++ b/docs/development.md @@ -29,6 +29,7 @@ Root checkout dev is intentionally split across terminals: - **Repo dev scripts** default to `$ROOT/.dev/paseo-home`, where `$ROOT` is the current checkout or worktree root. This keeps all dev state scoped to the checkout instead of the packaged desktop app. - **`npm run cli -- ...`** runs through the same dev-home wrapper as the dev scripts, so the in-repo CLI automatically targets the current checkout's `.dev/paseo-home` and configured dev daemon endpoint. - **Paseo-created worktrees** seed `$PASEO_WORKTREE_PATH/.dev/paseo-home` from `$PASEO_SOURCE_CHECKOUT_PATH/.dev/paseo-home` by copying durable JSON metadata. Runtime files like pid files, sockets, and logs are not copied. +- **Paseo-created worktrees** read `.worktreeinclude` from the live source checkout before creation. Bare paths and `copy ` copy a snapshot into the new worktree; `symlink ` creates a live source link. Missing paths, malformed entries, unsafe paths, incompatible include overlaps, destination conflicts, unavailable platform links, and ordinary read/write failures are skipped individually and reported in the daemon log, so the rest of the plan still runs. A source symlink is allowed only when its resolved target remains inside the active source checkout. If that checkout is itself Paseo-managed, its own paths remain eligible while other managed worktree paths stay protected; `copy` snapshots that resolved target, while `symlink` links directly to it. Hard links are ordinary files. Each include is staged before it is committed; Paseo aborts creation only if it cannot safely clean up partial materialization state (or Git/worktree setup itself fails). Materialization finishes before `worktree.setup` runs. - **This repo's worktree setup** also best-effort seeds `packages/app/ios` and the newest `.dev/ios-build` entry from the source checkout so iOS simulator services can reuse native project and Xcode cache state when it is safe enough to do so. Override knobs: diff --git a/packages/server/src/server/paseo-worktree-service.ts b/packages/server/src/server/paseo-worktree-service.ts index f09a81338..fa2e61cf0 100644 --- a/packages/server/src/server/paseo-worktree-service.ts +++ b/packages/server/src/server/paseo-worktree-service.ts @@ -28,6 +28,7 @@ import type { WorktreeCreationIntent } from "./resolve-worktree-creation-intent. import { resolveFirstAgentPromptTitle } from "./agent/create-agent-title.js"; import { buildAgentBranchNameSeed } from "./agent/prompt-attachments.js"; import type { FirstAgentContext } from "@getpaseo/protocol/messages"; +import type { WorktreeIncludeSummary } from "../utils/worktree-include.js"; export interface CreatePaseoWorktreeInput extends CreateWorktreeCoreInput { projectId?: string; @@ -36,6 +37,7 @@ export interface CreatePaseoWorktreeInput extends CreateWorktreeCoreInput { export interface CreatePaseoWorktreeResult { worktree: WorktreeConfig; + worktreeIncludeSummary?: WorktreeIncludeSummary; intent: WorktreeCreationIntent; workspace: PersistedWorkspaceRecord; repoRoot: string; @@ -98,6 +100,7 @@ export async function createPaseoWorktree( return { worktree: createdWorktree.worktree, + worktreeIncludeSummary: createdWorktree.worktreeIncludeSummary, intent: createdWorktree.intent, workspace, repoRoot: createdWorktree.repoRoot, diff --git a/packages/server/src/server/session.ts b/packages/server/src/server/session.ts index 7164c6de6..51059637a 100644 --- a/packages/server/src/server/session.ts +++ b/packages/server/src/server/session.ts @@ -745,6 +745,7 @@ export class Session { logger: this.sessionLogger, }); this.workspaceRecovery = createWorkspaceRecoveryService({ + logger: this.sessionLogger, paseoHome: this.paseoHome, worktreesRoot: this.worktreesRoot, getWorkspace: (workspaceId) => this.workspaceRegistry.get(workspaceId), diff --git a/packages/server/src/server/session/workspace-recovery/workspace-recovery-service.test.ts b/packages/server/src/server/session/workspace-recovery/workspace-recovery-service.test.ts index 91631a563..2203e3072 100644 --- a/packages/server/src/server/session/workspace-recovery/workspace-recovery-service.test.ts +++ b/packages/server/src/server/session/workspace-recovery/workspace-recovery-service.test.ts @@ -76,6 +76,7 @@ function createHarness(input?: { const directories = new Set(input?.directories ?? ["/repo"]); const unarchived: string[] = []; const service = createWorkspaceRecoveryService({ + logger: { warn: () => undefined } as never, paseoHome: input?.paseoHome ?? "/paseo-home", worktreesRoot: input?.worktreesRoot ?? "/worktrees", getWorkspace: async (workspaceId) => @@ -135,6 +136,7 @@ describe("workspace recovery", () => { const sourceSubdirectory = join(repoDir, "packages", "app"); mkdirSync(sourceSubdirectory, { recursive: true }); writeFileSync(join(sourceSubdirectory, "README.md"), "app\n"); + writeFileSync(join(repoDir, ".worktreeinclude"), "missing.local\n"); execFileSync("git", ["add", "."], { cwd: repoDir, stdio: "pipe" }); execFileSync("git", ["commit", "-m", "add app"], { cwd: repoDir, stdio: "pipe" }); execFileSync("git", ["branch", branch], { cwd: repoDir, stdio: "pipe" }); @@ -170,7 +172,9 @@ describe("workspace recovery", () => { mainRepoRoot: repoDir, }); const unarchived: string[] = []; + const warnings: unknown[][] = []; const service = createWorkspaceRecoveryService({ + logger: { warn: (...args: unknown[]) => warnings.push(args) } as never, paseoHome, worktreesRoot, getWorkspace: async (workspaceId) => @@ -189,6 +193,15 @@ describe("workspace recovery", () => { expect(existsSync(worktreeRoot)).toBe(true); expect(existsSync(workspaceCwd)).toBe(true); expect(unarchived).toEqual([workspace.workspaceId]); + expect(warnings).toEqual([ + [ + expect.objectContaining({ + materialized: 0, + skipped: [expect.objectContaining({ raw: "missing.local", reason: "missing" })], + }), + "Worktree include completed with skipped entries during workspace recovery", + ], + ]); }); test("keeps an exact-subdirectory workspace archived when its branch lacks that directory", async () => { @@ -220,6 +233,7 @@ describe("workspace recovery", () => { }); const unarchived: string[] = []; const service = createWorkspaceRecoveryService({ + logger: { warn: () => undefined } as never, paseoHome, worktreesRoot, getWorkspace: async (workspaceId) => diff --git a/packages/server/src/server/session/workspace-recovery/workspace-recovery-service.ts b/packages/server/src/server/session/workspace-recovery/workspace-recovery-service.ts index 1d98a9128..539dc218f 100644 --- a/packages/server/src/server/session/workspace-recovery/workspace-recovery-service.ts +++ b/packages/server/src/server/session/workspace-recovery/workspace-recovery-service.ts @@ -1,4 +1,5 @@ import { basename } from "node:path"; +import type { Logger } from "pino"; import { createRealpathAwarePathMatcher } from "../../../utils/path.js"; import { runGitCommand } from "../../../utils/run-git-command.js"; @@ -59,6 +60,7 @@ type RecoveryPlan = type UnavailableRecoveryState = Extract; export function createWorkspaceRecoveryService(deps: { + logger: Logger; paseoHome: string; worktreesRoot?: string; getWorkspace: (workspaceId: string) => Promise; @@ -195,6 +197,16 @@ export function createWorkspaceRecoveryService(deps: { worktreesRoot: deps.worktreesRoot, }); recreatedWorktreePath = result.worktreePath; + if (result.worktreeIncludeSummary.skipped.length > 0) { + deps.logger.warn( + { + materialized: result.worktreeIncludeSummary.materialized, + skipped: result.worktreeIncludeSummary.skipped, + worktreePath: result.worktreePath, + }, + "Worktree include completed with skipped entries during workspace recovery", + ); + } } catch (error) { throw toWorktreeRequestError(error); } diff --git a/packages/server/src/server/worktree-core.ts b/packages/server/src/server/worktree-core.ts index 90d078890..c83fdf7b6 100644 --- a/packages/server/src/server/worktree-core.ts +++ b/packages/server/src/server/worktree-core.ts @@ -8,6 +8,7 @@ import { validateBranchSlug, type WorktreeConfig, } from "../utils/worktree.js"; +import type { WorktreeIncludeSummary } from "../utils/worktree-include.js"; import { resolveWorktreeCreationIntent, type ResolveWorktreeCreationIntentInput, @@ -42,6 +43,7 @@ export interface CreateWorktreeCoreDeps { export interface CreateWorktreeCoreResult { worktree: WorktreeConfig; + worktreeIncludeSummary?: WorktreeIncludeSummary; intent: WorktreeCreationIntent; repoRoot: string; created: boolean; @@ -120,15 +122,17 @@ export async function createWorktreeCore( return { worktree: existingWorktree, intent, repoRoot, created: false }; } + const { worktreeIncludeSummary, ...worktree } = await createWorktree({ + cwd: repoRoot, + worktreeSlug: normalizedSlug, + source: intent, + runSetup: input.runSetup ?? true, + paseoHome: input.paseoHome, + worktreesRoot: input.worktreesRoot, + }); return { - worktree: await createWorktree({ - cwd: repoRoot, - worktreeSlug: normalizedSlug, - source: intent, - runSetup: input.runSetup ?? true, - paseoHome: input.paseoHome, - worktreesRoot: input.worktreesRoot, - }), + worktree, + worktreeIncludeSummary, intent, repoRoot, created: true, diff --git a/packages/server/src/server/worktree-session.ts b/packages/server/src/server/worktree-session.ts index 5573a945f..3b558affc 100644 --- a/packages/server/src/server/worktree-session.ts +++ b/packages/server/src/server/worktree-session.ts @@ -605,6 +605,17 @@ export async function createPaseoWorktreeWorkflow( const workspace = createdWorktree.workspace; const setupContinuation = options?.setupContinuation ?? { kind: "workspace" }; + if (createdWorktree.created && createdWorktree.worktreeIncludeSummary?.skipped.length) { + dependencies.sessionLogger.warn( + { + materialized: createdWorktree.worktreeIncludeSummary.materialized, + skipped: createdWorktree.worktreeIncludeSummary.skipped, + worktreePath: createdWorktree.worktree.worktreePath, + }, + "Worktree include completed with skipped entries", + ); + } + setTimeout(() => { if (input.firstAgentContext) { dependencies.autoNameWorkspaceBranchForFirstAgent({ diff --git a/packages/server/src/utils/worktree-include.test.ts b/packages/server/src/utils/worktree-include.test.ts new file mode 100644 index 000000000..b846b9b7c --- /dev/null +++ b/packages/server/src/utils/worktree-include.test.ts @@ -0,0 +1,635 @@ +import { afterEach, beforeEach, describe, expect, it } from "vitest"; +import { + existsSync, + chmodSync, + lstatSync, + linkSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + readlinkSync, + realpathSync, + rmSync, + symlinkSync, + writeFileSync, +} from "fs"; +import { tmpdir } from "os"; +import { dirname, join, relative } from "path"; +import { isPlatform } from "../test-utils/platform.js"; +import { materializeWorktreeIncludePlan, readWorktreeIncludePlan } from "./worktree-include.js"; + +describe("worktree include planning", () => { + let tempDir: string; + let sourceRoot: string; + + beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), "worktree-include-test-")); + sourceRoot = join(tempDir, "source"); + mkdirSync(sourceRoot); + }); + + afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); + }); + + it("copies bare entries and accepts explicit copy and symlink modes", async () => { + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + [".env.local", ".cache/**", "symlink shared-state", "copy packages/*/.runtime.env", ""].join( + "\n", + ), + ); + writeFileSync(join(sourceRoot, ".env.local"), "source\n"); + mkdirSync(join(sourceRoot, ".cache"), { recursive: true }); + writeFileSync(join(sourceRoot, ".cache", "state.txt"), "cache\n"); + mkdirSync(join(sourceRoot, "shared-state"), { recursive: true }); + writeFileSync(join(sourceRoot, "shared-state", "state.txt"), "shared\n"); + mkdirSync(join(sourceRoot, "packages", "api"), { recursive: true }); + writeFileSync(join(sourceRoot, "packages", "api", ".runtime.env"), "api\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toHaveLength(4); + expect(plan.materializations).toEqual( + expect.arrayContaining([ + expect.objectContaining({ mode: "copy", relativePath: ".env.local", sourceKind: "file" }), + expect.objectContaining({ mode: "copy", relativePath: ".cache", sourceKind: "directory" }), + expect.objectContaining({ + mode: "symlink", + relativePath: "shared-state", + sourceKind: "directory", + }), + expect.objectContaining({ + mode: "copy", + relativePath: "packages/api/.runtime.env", + sourceKind: "file", + }), + ]), + ); + }); + + it("skips invalid and conflicting entries while retaining safe entries", async () => { + writeFileSync(join(sourceRoot, "shared"), "source\n"); + writeFileSync(join(sourceRoot, ".env"), "source\n"); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ["../outside", "symlink", "shared", "symlink shared", ".env", ""].join("\n"), + ); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ mode: "copy", relativePath: ".env", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual( + expect.arrayContaining([ + expect.objectContaining({ lineNumber: 1, raw: "../outside", reason: "invalid" }), + expect.objectContaining({ lineNumber: 2, raw: "symlink", reason: "invalid" }), + expect.objectContaining({ lineNumber: 3, raw: "shared", reason: "conflict" }), + expect.objectContaining({ lineNumber: 4, raw: "symlink shared", reason: "conflict" }), + ]), + ); + }); + + it("skips missing entries while retaining existing literal and glob matches", async () => { + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + [".env", ".env.local", "config/*.env", "missing/**", ""].join("\n"), + ); + writeFileSync(join(sourceRoot, ".env"), "source\n"); + mkdirSync(join(sourceRoot, "config"), { recursive: true }); + writeFileSync(join(sourceRoot, "config", "runtime.env"), "runtime\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ mode: "copy", relativePath: ".env", sourceKind: "file" }), + expect.objectContaining({ + mode: "copy", + relativePath: "config/runtime.env", + sourceKind: "file", + }), + ]); + expect(plan.skipped).toEqual( + expect.arrayContaining([ + expect.objectContaining({ raw: ".env.local", reason: "missing" }), + expect.objectContaining({ raw: "missing/**", reason: "missing" }), + ]), + ); + }); + + it("requires the optimized trailing recursive form to resolve to a directory", async () => { + writeFileSync(join(sourceRoot, "cache"), "not-a-directory\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "cache/**\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([expect.objectContaining({ raw: "cache/**", reason: "unsafe" })]); + }); + + it("skips directory copies that overlap protected worktree paths", async () => { + const protectedWorktreeRoot = join(sourceRoot, ".dev", "paseo-home", "worktrees", "project"); + mkdirSync(protectedWorktreeRoot, { recursive: true }); + writeFileSync(join(sourceRoot, ".worktreeinclude"), ".dev/**\n"); + + const plan = await readWorktreeIncludePlan({ + sourceRoot, + excludedSourceRoots: [protectedWorktreeRoot], + }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([expect.objectContaining({ raw: ".dev/**", reason: "unsafe" })]); + }); + + it("protects every managed project beneath checkout-local worktree storage", async () => { + const managedWorktreesRoot = join(sourceRoot, ".dev", "paseo-home", "worktrees"); + const siblingWorktree = join(managedWorktreesRoot, "other-project", "sibling"); + mkdirSync(siblingWorktree, { recursive: true }); + writeFileSync(join(siblingWorktree, ".env"), "secret\n"); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ".dev/paseo-home/worktrees/other-project/sibling/.env\n", + ); + + const plan = await readWorktreeIncludePlan({ + sourceRoot, + excludedSourceRoots: [managedWorktreesRoot], + }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([expect.objectContaining({ reason: "unsafe" })]); + }); + + it("allows includes from a source worktree inside the protected worktree root", async () => { + const managedWorktreesRoot = join(tempDir, "paseo-home", "worktrees", "project"); + sourceRoot = join(managedWorktreesRoot, "source-worktree"); + mkdirSync(sourceRoot, { recursive: true }); + writeFileSync(join(sourceRoot, ".worktreeinclude"), ".env\n"); + writeFileSync(join(sourceRoot, ".env"), "source\n"); + + const plan = await readWorktreeIncludePlan({ + sourceRoot, + excludedSourceRoots: [managedWorktreesRoot], + }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: ".env", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual([]); + }); + + it.skipIf(isPlatform("win32"))( + "skips protected paths reached through a symlink alias", + async () => { + const sourceAlias = join(tempDir, "source-alias"); + symlinkSync(sourceRoot, sourceAlias, "dir"); + mkdirSync(join(sourceRoot, ".dev", "paseo-home", "worktrees", "project"), { + recursive: true, + }); + writeFileSync(join(sourceRoot, ".worktreeinclude"), ".dev/**\n"); + + const plan = await readWorktreeIncludePlan({ + sourceRoot, + excludedSourceRoots: [join(sourceAlias, ".dev", "paseo-home", "worktrees", "project")], + }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([expect.objectContaining({ raw: ".dev/**", reason: "unsafe" })]); + }, + ); + + it.skipIf(isPlatform("win32"))( + "skips external source links while retaining safe entries", + async () => { + const outsidePath = join(tempDir, "outside.txt"); + writeFileSync(outsidePath, "outside\n"); + writeFileSync(join(sourceRoot, "safe.txt"), "safe\n"); + symlinkSync(outsidePath, join(sourceRoot, "linked.txt")); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ["safe.txt", "linked.txt", ""].join("\n"), + ); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: "safe.txt", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual([ + expect.objectContaining({ raw: "linked.txt", reason: "unsafe" }), + ]); + }, + ); + + it.skipIf(isPlatform("win32"))("matches globs beneath a safe symlinked directory", async () => { + mkdirSync(join(sourceRoot, "actual-config")); + writeFileSync(join(sourceRoot, "actual-config", "runtime.env"), "runtime\n"); + symlinkSync(join(sourceRoot, "actual-config"), join(sourceRoot, "config"), "dir"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "config/*.env\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: "config/runtime.env", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual([]); + }); + + it.skipIf(isPlatform("win32"))( + "does not scan unrelated directories for bounded globs", + async () => { + writeFileSync(join(sourceRoot, ".worktreeinclude"), "packages/*/.runtime.env\n"); + mkdirSync(join(sourceRoot, "packages", "api"), { recursive: true }); + writeFileSync(join(sourceRoot, "packages", "api", ".runtime.env"), "api\n"); + const unreadableDirectory = join(sourceRoot, "unrelated"); + mkdirSync(unreadableDirectory); + chmodSync(unreadableDirectory, 0o000); + + try { + const plan = await readWorktreeIncludePlan({ sourceRoot }); + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: "packages/api/.runtime.env" }), + ]); + } finally { + chmodSync(unreadableDirectory, 0o700); + } + }, + ); + + it.skipIf(isPlatform("win32") || process.getuid?.() === 0)( + "skips inaccessible entries while retaining safe paths", + async () => { + const inaccessibleDirectory = join(sourceRoot, "private"); + mkdirSync(inaccessibleDirectory); + writeFileSync(join(inaccessibleDirectory, "secret.txt"), "secret\n"); + writeFileSync(join(sourceRoot, "safe.txt"), "safe\n"); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ["private/**", "private/*", "safe.txt", ""].join("\n"), + ); + chmodSync(inaccessibleDirectory, 0o000); + + try { + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: "safe.txt", sourceKind: "file" }), + ]); + expect(plan.skipped).toEqual( + expect.arrayContaining([ + expect.objectContaining({ raw: "private/**", reason: "materialization" }), + expect.objectContaining({ raw: "private/*", reason: "materialization" }), + ]), + ); + } finally { + chmodSync(inaccessibleDirectory, 0o700); + } + }, + ); + + it.skipIf(isPlatform("win32") || process.getuid?.() === 0)( + "retains valid glob matches when a viable sibling is unreadable", + async () => { + mkdirSync(join(sourceRoot, "packages", "good"), { recursive: true }); + mkdirSync(join(sourceRoot, "packages", "private"), { recursive: true }); + writeFileSync(join(sourceRoot, "packages", "good", ".runtime.env"), "good\n"); + writeFileSync(join(sourceRoot, "packages", "private", ".runtime.env"), "private\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "packages/*/.runtime.env\n"); + chmodSync(join(sourceRoot, "packages", "private"), 0o000); + + try { + const plan = await readWorktreeIncludePlan({ sourceRoot }); + expect(plan.materializations).toEqual([ + expect.objectContaining({ relativePath: "packages/good/.runtime.env" }), + ]); + expect(plan.skipped).toEqual([ + expect.objectContaining({ raw: "packages/*/.runtime.env", reason: "materialization" }), + ]); + } finally { + chmodSync(join(sourceRoot, "packages", "private"), 0o700); + } + }, + ); +}); + +describe.skipIf(isPlatform("win32"))("worktree include materialization", () => { + let tempDir: string; + let sourceRoot: string; + let worktreeRoot: string; + + beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), "worktree-include-materialize-test-")); + sourceRoot = join(tempDir, "source"); + worktreeRoot = join(tempDir, "worktree"); + mkdirSync(sourceRoot); + mkdirSync(worktreeRoot); + }); + + afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); + }); + + it("copies snapshots, links shared paths, and is idempotent", async () => { + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + ["copy.txt", "copy-dir/**", "symlink linked.txt", "symlink linked-dir"].join("\n"), + ); + writeFileSync(join(sourceRoot, "copy.txt"), "copy-v1\n"); + mkdirSync(join(sourceRoot, "copy-dir"), { recursive: true }); + writeFileSync(join(sourceRoot, "copy-dir", "state.txt"), "copy-dir-v1\n"); + writeFileSync(join(sourceRoot, "linked.txt"), "linked-v1\n"); + mkdirSync(join(sourceRoot, "linked-dir"), { recursive: true }); + writeFileSync(join(sourceRoot, "linked-dir", "state.txt"), "linked-dir-v1\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(lstatSync(join(worktreeRoot, "copy.txt")).isSymbolicLink()).toBe(false); + expect(lstatSync(join(worktreeRoot, "copy-dir")).isSymbolicLink()).toBe(false); + expect(lstatSync(join(worktreeRoot, "linked.txt")).isSymbolicLink()).toBe(true); + expect(lstatSync(join(worktreeRoot, "linked-dir")).isSymbolicLink()).toBe(true); + + writeFileSync(join(sourceRoot, "copy.txt"), "copy-v2\n"); + writeFileSync(join(sourceRoot, "copy-dir", "state.txt"), "copy-dir-v2\n"); + writeFileSync(join(sourceRoot, "linked.txt"), "linked-v2\n"); + writeFileSync(join(sourceRoot, "linked-dir", "state.txt"), "linked-dir-v2\n"); + + expect(readFileSync(join(worktreeRoot, "copy.txt"), "utf8")).toBe("copy-v1\n"); + expect(readFileSync(join(worktreeRoot, "copy-dir", "state.txt"), "utf8")).toBe("copy-dir-v1\n"); + expect(readFileSync(join(worktreeRoot, "linked.txt"), "utf8")).toBe("linked-v2\n"); + expect(readFileSync(join(worktreeRoot, "linked-dir", "state.txt"), "utf8")).toBe( + "linked-dir-v2\n", + ); + + await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + expect(readFileSync(join(worktreeRoot, "copy.txt"), "utf8")).toBe("copy-v2\n"); + expect(readFileSync(join(worktreeRoot, "copy-dir", "state.txt"), "utf8")).toBe("copy-dir-v2\n"); + }); + + it("replaces an existing directory snapshot without retaining destination-only files", async () => { + mkdirSync(join(sourceRoot, "cache")); + writeFileSync(join(sourceRoot, "cache", "current.txt"), "current\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "cache/**\n"); + mkdirSync(join(worktreeRoot, "cache")); + writeFileSync(join(worktreeRoot, "cache", "stale.txt"), "stale\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(result).toMatchObject({ materialized: 1, skipped: [] }); + expect(readFileSync(join(worktreeRoot, "cache", "current.txt"), "utf8")).toBe("current\n"); + expect(existsSync(join(worktreeRoot, "cache", "stale.txt"))).toBe(false); + }); + + it("retains an explicit descendant when an overlapping directory copy is skipped", async () => { + mkdirSync(join(sourceRoot, "config", "nested"), { recursive: true }); + writeFileSync(join(sourceRoot, "config", "local.env"), "local\n"); + writeFileSync(join(sourceRoot, "config", "nested", "state.txt"), "state\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "config/**\nconfig/local.env\n"); + mkdirSync(join(worktreeRoot, "config")); + writeFileSync(join(worktreeRoot, "config", "nested"), "conflict\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(result).toMatchObject({ materialized: 1 }); + expect(result.skipped).toEqual([ + expect.objectContaining({ raw: "config/**", reason: "conflict" }), + ]); + expect(readFileSync(join(worktreeRoot, "config", "local.env"), "utf8")).toBe("local\n"); + }); + + it("skips a destination parent symlink without writing through it", async () => { + mkdirSync(join(sourceRoot, "config"), { recursive: true }); + writeFileSync(join(sourceRoot, "config", "local.json"), "{}\n"); + writeFileSync(join(sourceRoot, "safe.txt"), "safe\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "config/local.json\nsafe.txt\n"); + + const outsideRoot = join(tempDir, "outside"); + mkdirSync(outsideRoot); + symlinkSync(outsideRoot, join(worktreeRoot, "config")); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(existsSync(join(outsideRoot, "local.json"))).toBe(false); + expect(readFileSync(join(worktreeRoot, "safe.txt"), "utf8")).toBe("safe\n"); + expect(result).toMatchObject({ materialized: 1 }); + expect(result.skipped).toEqual([ + expect.objectContaining({ raw: "config/local.json", reason: "conflict" }), + ]); + expect(readdirSync(worktreeRoot)).not.toContain( + expect.stringMatching(/^\.paseo-worktreeinclude-/), + ); + }); + + it("copies resolved source links and creates direct live links", async () => { + const targetPath = join(sourceRoot, "target.txt"); + const targetDirectoryPath = join(sourceRoot, "target-directory"); + writeFileSync(targetPath, "v1\n"); + mkdirSync(targetDirectoryPath); + writeFileSync(join(targetDirectoryPath, "state.txt"), "v1\n"); + symlinkSync(targetPath, join(sourceRoot, "copy-link.txt")); + symlinkSync(targetPath, join(sourceRoot, "live-link.txt")); + symlinkSync(targetDirectoryPath, join(sourceRoot, "copy-directory-link"), "dir"); + symlinkSync(targetDirectoryPath, join(sourceRoot, "live-directory-link"), "dir"); + writeFileSync( + join(sourceRoot, ".worktreeinclude"), + [ + "copy-link.txt", + "symlink live-link.txt", + "copy-directory-link", + "symlink live-directory-link", + "", + ].join("\n"), + ); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + const copiedPath = join(worktreeRoot, "copy-link.txt"); + const linkedPath = join(worktreeRoot, "live-link.txt"); + const copiedDirectoryPath = join(worktreeRoot, "copy-directory-link"); + const linkedDirectoryPath = join(worktreeRoot, "live-directory-link"); + const canonicalWorktreeRoot = realpathSync(worktreeRoot); + expect(result).toMatchObject({ materialized: 4, skipped: [] }); + expect(lstatSync(copiedPath).isSymbolicLink()).toBe(false); + expect(lstatSync(linkedPath).isSymbolicLink()).toBe(true); + expect(lstatSync(copiedDirectoryPath).isSymbolicLink()).toBe(false); + expect(lstatSync(linkedDirectoryPath).isSymbolicLink()).toBe(true); + expect(readlinkSync(linkedPath)).toBe( + relative(dirname(join(canonicalWorktreeRoot, "live-link.txt")), realpathSync(targetPath)), + ); + expect(readlinkSync(linkedDirectoryPath)).toBe( + relative( + dirname(join(canonicalWorktreeRoot, "live-directory-link")), + realpathSync(targetDirectoryPath), + ), + ); + expect(realpathSync(linkedPath)).toBe(realpathSync(targetPath)); + expect(realpathSync(linkedDirectoryPath)).toBe(realpathSync(targetDirectoryPath)); + + writeFileSync(targetPath, "v2\n"); + writeFileSync(join(targetDirectoryPath, "state.txt"), "v2\n"); + expect(readFileSync(copiedPath, "utf8")).toBe("v1\n"); + expect(readFileSync(linkedPath, "utf8")).toBe("v2\n"); + expect(readFileSync(join(copiedDirectoryPath, "state.txt"), "utf8")).toBe("v1\n"); + expect(readFileSync(join(linkedDirectoryPath, "state.txt"), "utf8")).toBe("v2\n"); + }); + + it("rejects source links that alias Git metadata", async () => { + mkdirSync(join(sourceRoot, ".git")); + writeFileSync(join(sourceRoot, ".git", "config"), "secret\n"); + symlinkSync(join(sourceRoot, ".git"), join(sourceRoot, "metadata"), "dir"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "metadata\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([expect.objectContaining({ raw: "metadata", reason: "unsafe" })]); + }); + + it("treats hard links as ordinary files", async () => { + const targetPath = join(sourceRoot, "target.txt"); + const hardLinkPath = join(sourceRoot, "hard-link.txt"); + writeFileSync(targetPath, "v1\n"); + linkSync(targetPath, hardLinkPath); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "hard-link.txt\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + const copiedPath = join(worktreeRoot, "hard-link.txt"); + expect(lstatSync(copiedPath).isSymbolicLink()).toBe(false); + expect(readFileSync(copiedPath, "utf8")).toBe("v1\n"); + }); + + it("skips a source link retargeted outside the checkout after planning", async () => { + const insidePath = join(sourceRoot, "inside.txt"); + const linkedPath = join(sourceRoot, "linked.txt"); + const outsidePath = join(tempDir, "outside.txt"); + writeFileSync(insidePath, "inside\n"); + writeFileSync(outsidePath, "outside\n"); + symlinkSync(insidePath, linkedPath); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "symlink linked.txt\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + rmSync(linkedPath); + symlinkSync(outsidePath, linkedPath); + + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(existsSync(join(worktreeRoot, "linked.txt"))).toBe(false); + expect(result.skipped).toEqual([ + expect.objectContaining({ raw: "symlink linked.txt", reason: "unsafe" }), + ]); + }); + + it("skips a linked directory that exposes an external nested link", async () => { + const sharedPath = join(sourceRoot, "shared"); + const outsidePath = join(tempDir, "outside.txt"); + mkdirSync(sharedPath); + writeFileSync(outsidePath, "outside\n"); + symlinkSync(outsidePath, join(sharedPath, "outside.txt")); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "symlink shared\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + + expect(plan.materializations).toEqual([]); + expect(plan.skipped).toEqual([ + expect.objectContaining({ raw: "symlink shared", reason: "unsafe" }), + ]); + }); + + it("allows a linked directory with internal nested links", async () => { + const sharedPath = join(sourceRoot, "shared"); + const targetPath = join(sourceRoot, "shared-target"); + mkdirSync(sharedPath); + mkdirSync(targetPath); + writeFileSync(join(targetPath, "state.txt"), "source\n"); + symlinkSync(targetPath, join(sharedPath, "target"), "dir"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "symlink shared\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(result).toMatchObject({ materialized: 1, skipped: [] }); + expect(readFileSync(join(worktreeRoot, "shared", "target", "state.txt"), "utf8")).toBe( + "source\n", + ); + }); + + it("skips a source removed after planning", async () => { + writeFileSync(join(sourceRoot, ".worktreeinclude"), "runtime.env\n"); + const sourcePath = join(sourceRoot, "runtime.env"); + writeFileSync(sourcePath, "source\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + rmSync(sourcePath); + + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + expect(existsSync(join(worktreeRoot, "runtime.env"))).toBe(false); + expect(result.skipped).toEqual([ + expect.objectContaining({ raw: "runtime.env", reason: "missing" }), + ]); + }); + + it.skipIf(process.getuid?.() === 0)( + "skips ordinary filesystem failures during materialization revalidation", + async () => { + mkdirSync(join(sourceRoot, "private")); + writeFileSync(join(sourceRoot, "private", "state.txt"), "private\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "private/**\n"); + const plan = await readWorktreeIncludePlan({ sourceRoot }); + chmodSync(join(sourceRoot, "private"), 0o000); + + try { + const result = await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + expect(result).toMatchObject({ materialized: 0 }); + expect(result.skipped).toEqual([ + expect.objectContaining({ raw: "private/**", reason: "materialization" }), + ]); + } finally { + chmodSync(join(sourceRoot, "private"), 0o700); + } + }, + ); +}); + +describe.skipIf(!isPlatform("win32"))("worktree include Windows directory links", () => { + let tempDir: string; + let sourceRoot: string; + let worktreeRoot: string; + + beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), "worktree-include-windows-test-")); + sourceRoot = join(tempDir, "source"); + worktreeRoot = join(tempDir, "worktree"); + mkdirSync(sourceRoot); + mkdirSync(worktreeRoot); + }); + + afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); + }); + + it("uses a live directory link and removes it without touching the source", async () => { + mkdirSync(join(sourceRoot, "shared-state")); + writeFileSync(join(sourceRoot, "shared-state", "state.txt"), "source-v1\n"); + writeFileSync(join(sourceRoot, ".worktreeinclude"), "symlink shared-state\n"); + + const plan = await readWorktreeIncludePlan({ sourceRoot }); + await materializeWorktreeIncludePlan({ plan, worktreeRoot }); + + writeFileSync(join(sourceRoot, "shared-state", "state.txt"), "source-v2\n"); + expect(readFileSync(join(worktreeRoot, "shared-state", "state.txt"), "utf8")).toBe( + "source-v2\n", + ); + + rmSync(worktreeRoot, { recursive: true, force: true }); + expect(readFileSync(join(sourceRoot, "shared-state", "state.txt"), "utf8")).toBe("source-v2\n"); + }); +}); diff --git a/packages/server/src/utils/worktree-include.ts b/packages/server/src/utils/worktree-include.ts new file mode 100644 index 000000000..32c00abbc --- /dev/null +++ b/packages/server/src/utils/worktree-include.ts @@ -0,0 +1,1359 @@ +import { type Dirent, type Stats } from "fs"; +import { + copyFile, + cp, + lstat, + mkdir, + mkdtemp, + readFile, + readdir, + realpath, + rename, + rm, + rmdir, + symlink, +} from "fs/promises"; +import { + basename as pathBasename, + dirname, + isAbsolute, + join, + relative, + resolve, + win32, +} from "path"; +import { areEquivalentPaths, isPathInsideRoot } from "./path.js"; + +const WORKTREE_INCLUDE_FILE_NAME = ".worktreeinclude"; + +export type WorktreeIncludeMode = "copy" | "symlink"; +type WorktreeIncludeSourceKind = "file" | "directory"; +type WorktreeIncludeErrorCode = + | "conflict" + | "invalid_entry" + | "missing_source" + | "source_changed" + | "unsupported_source" + | "windows_symlink_unavailable"; + +export type WorktreeIncludeSkipReason = + | "conflict" + | "invalid" + | "materialization" + | "missing" + | "source_changed" + | "unsafe"; + +interface WorktreeIncludeEntry { + lineNumber: number; + mode: WorktreeIncludeMode; + raw: string; + relativePath: string; +} + +export interface WorktreeIncludeMaterialization { + lineNumber: number; + mode: WorktreeIncludeMode; + raw: string; + relativePath: string; + sourceKind: WorktreeIncludeSourceKind; +} + +export interface WorktreeIncludeSkippedEntry { + lineNumber: number; + message: string; + raw: string; + reason: WorktreeIncludeSkipReason; +} + +export interface WorktreeIncludeSummary { + materialized: number; + skipped: WorktreeIncludeSkippedEntry[]; +} + +export interface WorktreeIncludePlan { + excludedSourceRoots: string[]; + materializations: WorktreeIncludeMaterialization[]; + skipped: WorktreeIncludeSkippedEntry[]; + sourceRoot: string; +} + +export interface ReadWorktreeIncludePlanOptions { + excludedSourceRoots?: string[]; + sourceRoot: string; +} + +export interface MaterializeWorktreeIncludePlanOptions { + plan: WorktreeIncludePlan; + worktreeRoot: string; +} + +interface ResolvedWorktreeIncludeMaterialization { + materialization: WorktreeIncludeMaterialization; + sourcePath: string; +} + +interface StagedWorktreeIncludeMaterialization { + directoryPath: string; + entryPath: string; +} + +interface ParsedWorktreeIncludeEntries { + entries: WorktreeIncludeEntry[]; + skipped: WorktreeIncludeSkippedEntry[]; +} + +interface NormalizedWorktreeIncludeMaterializations { + materializations: WorktreeIncludeMaterialization[]; + skipped: WorktreeIncludeSkippedEntry[]; +} + +interface WorktreeIncludeCandidateCollection { + candidates: string[]; + errorsByPattern: Map; +} + +export class WorktreeIncludeError extends Error { + constructor( + public readonly code: WorktreeIncludeErrorCode, + message: string, + ) { + super(message); + this.name = "WorktreeIncludeError"; + } +} + +class WorktreeIncludeCleanupError extends Error { + constructor( + message: string, + public readonly cleanupError: unknown, + ) { + super(message); + this.name = "WorktreeIncludeCleanupError"; + } +} + +export async function readWorktreeIncludePlan( + options: ReadWorktreeIncludePlanOptions, +): Promise { + const sourceRoot = await realpath(options.sourceRoot); + const { entries, skipped } = await readWorktreeIncludeEntries(sourceRoot); + if (entries.length === 0) { + return { + sourceRoot, + excludedSourceRoots: [], + materializations: [], + skipped, + }; + } + + const excludedSourceRoots = ( + await Promise.all((options.excludedSourceRoots ?? []).map(canonicalizeExistingPathPrefix)) + ).filter((candidate) => !isPathInsideRoot(candidate, sourceRoot)); + const candidatePatterns = entries + .filter( + (entry) => entry.relativePath.includes("*") && getRecursiveDirectoryPath(entry) === null, + ) + .map((entry) => entry.relativePath); + const candidateCollection = + candidatePatterns.length > 0 + ? await collectWorktreeIncludeCandidates({ + sourceRoot, + excludedSourceRoots, + patterns: candidatePatterns, + }) + : { candidates: [], errorsByPattern: new Map() }; + const materializations: WorktreeIncludeMaterialization[] = []; + + for (const entry of entries) { + const matchedPaths = resolveEntryMatches({ entry, candidates: candidateCollection.candidates }); + const candidateError = candidateCollection.errorsByPattern.get(entry.relativePath); + if (candidateError !== undefined) { + skipped.push(toSkippedEntry(entry, candidateError)); + if (matchedPaths.length === 0) { + continue; + } + } + if (matchedPaths.length === 0) { + skipped.push(toSkippedEntry(entry, noMatchError(entry))); + continue; + } + + for (const relativePath of matchedPaths) { + try { + const resolved = await resolveSourceMaterialization({ + entry, + excludedSourceRoots, + relativePath, + sourceRoot, + }); + materializations.push(resolved.materialization); + } catch (error) { + if (!isWorktreeIncludeMaterializationError(error)) { + throw error; + } + skipped.push(toSkippedEntry(entry, error)); + } + } + } + + const normalized = normalizeMaterializations(materializations); + + return { + excludedSourceRoots, + materializations: normalized.materializations, + skipped: [...skipped, ...normalized.skipped], + sourceRoot, + }; +} + +async function canonicalizeExistingPathPrefix(path: string): Promise { + const absolutePath = resolve(path); + const missingSegments: string[] = []; + let existingPath = absolutePath; + + while (true) { + try { + return join(await realpath(existingPath), ...missingSegments); + } catch (error) { + if (getErrorCode(error) !== "ENOENT" && getErrorCode(error) !== "ENOTDIR") { + throw error; + } + + const parentPath = dirname(existingPath); + if (parentPath === existingPath) { + return absolutePath; + } + missingSegments.unshift(pathBasename(existingPath)); + existingPath = parentPath; + } + } +} + +export async function materializeWorktreeIncludePlan( + options: MaterializeWorktreeIncludePlanOptions, +): Promise { + const skipped: WorktreeIncludeSkippedEntry[] = []; + let materialized = 0; + if (options.plan.materializations.length === 0) { + return { materialized, skipped }; + } + + const worktreeRoot = await realpath(options.worktreeRoot); + for (const materialization of options.plan.materializations) { + const entry = { + lineNumber: materialization.lineNumber, + mode: materialization.mode, + raw: materialization.raw, + relativePath: materialization.relativePath, + }; + let resolved: ResolvedWorktreeIncludeMaterialization; + try { + resolved = await resolveSourceMaterialization({ + entry, + excludedSourceRoots: options.plan.excludedSourceRoots, + relativePath: materialization.relativePath, + sourceRoot: options.plan.sourceRoot, + }); + } catch (error) { + if (isSkippableWorktreeIncludeError(error) || getErrorCode(error) !== null) { + skipped.push(toSkippedEntry(entry, error)); + continue; + } + throw error; + } + if (resolved.materialization.sourceKind !== materialization.sourceKind) { + skipped.push( + toSkippedEntry( + entry, + new WorktreeIncludeError( + "source_changed", + `Source for .worktreeinclude entry '${materialization.raw}' changed type before it could be materialized`, + ), + ), + ); + continue; + } + + let staged: StagedWorktreeIncludeMaterialization | null = null; + let createdDestinationParents: string[] = []; + try { + const destinationPath = getDestinationPath({ + worktreeRoot, + relativePath: resolved.materialization.relativePath, + }); + if ( + !(await preflightDestination({ + worktreeRoot, + resolved, + })) + ) { + materialized++; + continue; + } + + staged = await stageMaterialization({ + destinationPath, + resolved, + worktreeRoot, + }); + createdDestinationParents = await ensureDestinationParent({ + worktreeRoot, + relativePath: resolved.materialization.relativePath, + }); + if ( + !(await preflightDestination({ + worktreeRoot, + resolved, + })) + ) { + await cleanupStagingDirectory(staged.directoryPath); + staged = null; + await cleanupCreatedDestinationParents(createdDestinationParents); + materialized++; + continue; + } + + const destinationStats = await lstatIfExists(destinationPath); + if (resolved.materialization.mode === "copy" && destinationStats !== null) { + await copyStagedMaterializationToExistingDestination({ + destinationPath, + resolved, + staged, + }); + } else { + await rename(staged.entryPath, destinationPath); + } + + await cleanupStagingDirectory(staged.directoryPath); + staged = null; + materialized++; + } catch (error) { + if (error instanceof WorktreeIncludeCleanupError) { + throw error; + } + if (staged !== null) { + await cleanupStagingDirectory(staged.directoryPath); + } + await cleanupCreatedDestinationParents(createdDestinationParents); + if (isWorktreeIncludeMaterializationError(error)) { + skipped.push(toSkippedEntry(entry, error)); + continue; + } + throw error; + } + } + + return { materialized, skipped }; +} + +async function stageMaterialization(options: { + destinationPath: string; + resolved: ResolvedWorktreeIncludeMaterialization; + worktreeRoot: string; +}): Promise { + const directoryPath = await mkdtemp(join(options.worktreeRoot, ".paseo-worktreeinclude-")); + const entryPath = join(directoryPath, "entry"); + try { + if (options.resolved.materialization.mode === "copy") { + await copyMaterializationPath({ + destinationPath: entryPath, + sourceKind: options.resolved.materialization.sourceKind, + sourcePath: options.resolved.sourcePath, + }); + if (options.resolved.materialization.sourceKind === "directory") { + const stagedStats = await lstat(entryPath); + if (!stagedStats.isDirectory()) { + throw new WorktreeIncludeError( + "source_changed", + `.worktreeinclude entry '${options.resolved.materialization.raw}' changed type while it was staged`, + ); + } + await assertCopyDirectorySafe({ + entry: options.resolved.materialization, + sourcePath: entryPath, + }); + } + } else { + await createMaterializationSymlink({ + destinationPath: entryPath, + linkDestinationPath: options.destinationPath, + resolved: options.resolved, + }); + } + return { directoryPath, entryPath }; + } catch (error) { + await cleanupStagingDirectory(directoryPath); + throw error; + } +} + +async function copyStagedMaterializationToExistingDestination(options: { + destinationPath: string; + resolved: ResolvedWorktreeIncludeMaterialization; + staged: StagedWorktreeIncludeMaterialization; +}): Promise { + const backupPath = join(options.staged.directoryPath, "backup"); + await rename(options.destinationPath, backupPath); + try { + await rename(options.staged.entryPath, options.destinationPath); + } catch (error) { + await restoreCopiedDestination({ + backupPath, + destinationPath: options.destinationPath, + }); + throw error; + } +} + +async function copyMaterializationPath(options: { + destinationPath: string; + sourceKind: WorktreeIncludeSourceKind; + sourcePath: string; +}): Promise { + if (options.sourceKind === "file") { + await copyFile(options.sourcePath, options.destinationPath); + return; + } + await cp(options.sourcePath, options.destinationPath, { + recursive: true, + force: true, + dereference: false, + }); +} + +async function restoreCopiedDestination(options: { + backupPath: string; + destinationPath: string; +}): Promise { + try { + const destinationStats = await lstatIfExists(options.destinationPath); + if (destinationStats !== null) { + if (destinationStats.isSymbolicLink()) { + throw new Error("destination changed while restoring a failed materialization"); + } + await rm(options.destinationPath, { + recursive: destinationStats.isDirectory(), + force: true, + }); + } + await rename(options.backupPath, options.destinationPath); + } catch (error) { + throw new WorktreeIncludeCleanupError( + `Unable to restore .worktreeinclude destination '${options.destinationPath}' after a failed materialization`, + error, + ); + } +} + +async function cleanupStagingDirectory(directoryPath: string): Promise { + try { + await rm(directoryPath, { recursive: true, force: true }); + } catch (error) { + throw new WorktreeIncludeCleanupError( + `Unable to clean up .worktreeinclude staging directory '${directoryPath}'`, + error, + ); + } +} + +async function cleanupCreatedDestinationParents(createdPaths: string[]): Promise { + for (const path of createdPaths.toReversed()) { + try { + await rmdir(path); + } catch (error) { + const code = getErrorCode(error); + if (code === "ENOENT" || code === "ENOTEMPTY" || code === "EEXIST") { + continue; + } + throw new WorktreeIncludeCleanupError( + `Unable to clean up .worktreeinclude destination directory '${path}'`, + error, + ); + } + } +} + +async function readWorktreeIncludeEntries( + sourceRoot: string, +): Promise { + let contents: string; + try { + contents = await readFile(join(sourceRoot, WORKTREE_INCLUDE_FILE_NAME), "utf8"); + } catch (error) { + if (getErrorCode(error) === "ENOENT") { + return { entries: [], skipped: [] }; + } + throw error; + } + + const entries: WorktreeIncludeEntry[] = []; + const skipped: WorktreeIncludeSkippedEntry[] = []; + + for (const [index, sourceLine] of contents.split(/\r?\n/).entries()) { + const lineNumber = index + 1; + const line = sourceLine.trim(); + if (line.length === 0) { + continue; + } + + if (line.startsWith("#")) { + continue; + } + + try { + entries.push(parseWorktreeIncludeEntry({ line, lineNumber })); + } catch (error) { + if (!isSkippableWorktreeIncludeError(error)) { + throw error; + } + skipped.push(toSkippedEntry({ lineNumber, raw: line }, error)); + } + } + + return { entries, skipped }; +} + +function parseWorktreeIncludeEntry(options: { + line: string; + lineNumber: number; +}): WorktreeIncludeEntry { + let mode: WorktreeIncludeMode = "copy"; + let path = options.line; + const separatorIndex = options.line.search(/\s/); + + if (separatorIndex === -1) { + if (options.line === "copy" || options.line === "symlink") { + throw new WorktreeIncludeError( + "invalid_entry", + `.worktreeinclude ${options.line} entry on line ${options.lineNumber} requires a path`, + ); + } + } else { + const verb = options.line.slice(0, separatorIndex); + if (verb === "copy" || verb === "symlink") { + mode = verb; + path = options.line.slice(separatorIndex).trim(); + } + } + + return { + lineNumber: options.lineNumber, + mode, + raw: options.line, + relativePath: normalizeRelativePath({ entry: path, lineNumber: options.lineNumber }), + }; +} + +function normalizeRelativePath(options: { entry: string; lineNumber: number }): string { + const fail = (reason: string): never => { + throw new WorktreeIncludeError( + "invalid_entry", + `Invalid .worktreeinclude entry '${options.entry}' on line ${options.lineNumber}: ${reason}`, + ); + }; + + if ( + options.entry.includes("\0") || + options.entry.startsWith("/") || + options.entry.startsWith("\\") || + isAbsolute(options.entry) || + win32.isAbsolute(options.entry) || + /^[A-Za-z]:/.test(options.entry) + ) { + fail("absolute paths are not allowed"); + } + + const segments = options.entry + .split(/[\\/]+/) + .filter((segment) => segment.length > 0 && segment !== "."); + if (segments.length === 0) { + fail("path must not be empty"); + } + + for (const segment of segments) { + if (segment === "..") { + fail("parent-directory segments are not allowed"); + } + if (segment.toLowerCase() === ".git") { + fail("git metadata cannot be materialized"); + } + if (segment.includes(":") || /[. ]$/.test(segment) || isWindowsReservedSegment(segment)) { + fail("path is not portable to Windows"); + } + } + + return segments.join("/"); +} + +function isWindowsReservedSegment(segment: string): boolean { + const basename = segment.split(".", 1)[0]?.toLowerCase() ?? ""; + return /^(con|prn|aux|nul|com[1-9]|lpt[1-9])$/.test(basename); +} + +function getRecursiveDirectoryPath(entry: WorktreeIncludeEntry): string | null { + const segments = entry.relativePath.split("/"); + if ( + segments.length < 2 || + segments.at(-1) !== "**" || + segments.slice(0, -1).some((segment) => segment.includes("*")) + ) { + return null; + } + return segments.slice(0, -1).join("/"); +} + +async function collectWorktreeIncludeCandidates(options: { + excludedSourceRoots: string[]; + patterns: string[]; + sourceRoot: string; +}): Promise { + const candidates: string[] = []; + const errorsByPattern = new Map(); + + async function visit( + directoryPath: string, + directorySegments: string[], + patterns: string[], + ancestorCanonicalDirectories: ReadonlySet, + ): Promise { + let entries: Dirent[]; + try { + entries = await readdir(directoryPath, { withFileTypes: true }); + } catch (error) { + if (getErrorCode(error) === null) { + throw error; + } + for (const pattern of patterns) { + errorsByPattern.set(pattern, error); + } + return; + } + for (const entry of entries) { + if (entry.name.toLowerCase() === ".git") { + continue; + } + + const pathSegments = [...directorySegments, entry.name]; + const sourcePath = join(options.sourceRoot, ...pathSegments); + if ( + options.excludedSourceRoots.some((excludedRoot) => + isPathInsideRoot(excludedRoot, sourcePath), + ) + ) { + continue; + } + + const relativePath = pathSegments.join("/"); + if (patterns.some((pattern) => worktreeIncludeGlobMatches(pattern, relativePath))) { + candidates.push(relativePath); + } + const descendantPatterns = patterns.filter((pattern) => + canGlobMatchDescendant(pattern, pathSegments), + ); + if ((entry.isDirectory() || entry.isSymbolicLink()) && descendantPatterns.length > 0) { + try { + const canonicalDirectory = await realpath(sourcePath); + const canonicalStats = await lstat(canonicalDirectory); + const canonicalRelativePath = relative(options.sourceRoot, canonicalDirectory); + const isGitMetadata = canonicalRelativePath + .split(/[\\/]/) + .some((segment) => segment.toLowerCase() === ".git"); + const overlapsProtectedRoot = options.excludedSourceRoots.some( + (excludedRoot) => + isPathInsideRoot(excludedRoot, canonicalDirectory) || + isPathInsideRoot(canonicalDirectory, excludedRoot), + ); + if ( + !canonicalStats.isDirectory() || + !isPathInsideRoot(options.sourceRoot, canonicalDirectory) || + isGitMetadata || + overlapsProtectedRoot || + ancestorCanonicalDirectories.has(canonicalDirectory) + ) { + continue; + } + await visit( + sourcePath, + pathSegments, + descendantPatterns, + new Set([...ancestorCanonicalDirectories, canonicalDirectory]), + ); + } catch (error) { + if (getErrorCode(error) === null) { + throw error; + } + for (const pattern of descendantPatterns) { + errorsByPattern.set(pattern, error); + } + } + } + } + } + + await visit( + options.sourceRoot, + [], + options.patterns, + new Set([await realpath(options.sourceRoot)]), + ); + return { candidates: candidates.sort(), errorsByPattern }; +} + +function canGlobMatchDescendant(pattern: string, directorySegments: string[]): boolean { + const patternSegments = pattern.split("/"); + const cache = new Map(); + + function match(patternIndex: number, directoryIndex: number): boolean { + const cacheKey = `${patternIndex}:${directoryIndex}`; + const cached = cache.get(cacheKey); + if (cached !== undefined) { + return cached; + } + + const patternSegment = patternSegments[patternIndex]; + let result: boolean; + if (directoryIndex === directorySegments.length) { + result = patternSegment !== undefined; + } else if (patternSegment === "**") { + result = match(patternIndex + 1, directoryIndex) || match(patternIndex, directoryIndex + 1); + } else { + const directorySegment = directorySegments[directoryIndex]; + result = + patternSegment !== undefined && + segmentGlobMatches(patternSegment, directorySegment) && + match(patternIndex + 1, directoryIndex + 1); + } + + cache.set(cacheKey, result); + return result; + } + + return match(0, 0); +} + +function resolveEntryMatches(options: { + candidates: string[]; + entry: WorktreeIncludeEntry; +}): string[] { + if (!options.entry.relativePath.includes("*")) { + return [options.entry.relativePath]; + } + + const recursiveDirectoryPath = getRecursiveDirectoryPath(options.entry); + if (recursiveDirectoryPath !== null) { + return [recursiveDirectoryPath]; + } + + return options.candidates.filter((candidate) => + worktreeIncludeGlobMatches(options.entry.relativePath, candidate), + ); +} + +function worktreeIncludeGlobMatches(pattern: string, candidate: string): boolean { + const patternSegments = pattern.split("/"); + const candidateSegments = candidate.split("/"); + const cache = new Map(); + + function match(patternIndex: number, candidateIndex: number): boolean { + const cacheKey = `${patternIndex}:${candidateIndex}`; + const cached = cache.get(cacheKey); + if (cached !== undefined) { + return cached; + } + + const patternSegment = patternSegments[patternIndex]; + let result: boolean; + if (patternSegment === undefined) { + result = candidateIndex === candidateSegments.length; + } else if (patternSegment === "**") { + result = + match(patternIndex + 1, candidateIndex) || + (candidateIndex < candidateSegments.length && match(patternIndex, candidateIndex + 1)); + } else { + const candidateSegment = candidateSegments[candidateIndex]; + result = + candidateSegment !== undefined && + segmentGlobMatches(patternSegment, candidateSegment) && + match(patternIndex + 1, candidateIndex + 1); + } + + cache.set(cacheKey, result); + return result; + } + + return match(0, 0); +} + +function segmentGlobMatches(pattern: string, value: string): boolean { + const escaped = pattern.replace(/[|\\{}()[\]^$+?.]/g, "\\$&").replace(/\*/g, "[^/]*"); + return new RegExp(`^${escaped}$`).test(value); +} + +async function resolveSourceMaterialization(options: { + entry: WorktreeIncludeEntry; + excludedSourceRoots: string[]; + relativePath: string; + sourceRoot: string; +}): Promise { + const requestedSourcePath = join(options.sourceRoot, ...options.relativePath.split("/")); + const sourcePath = await realpathSourcePath(requestedSourcePath, options.entry); + assertSourcePathIsSafe({ + entry: options.entry, + excludedSourceRoots: options.excludedSourceRoots, + sourcePath, + sourceRoot: options.sourceRoot, + }); + + const sourceKind = await getCanonicalSourceKind({ + entry: options.entry, + sourcePath, + }); + if ( + getRecursiveDirectoryPath(options.entry) === options.relativePath && + sourceKind !== "directory" + ) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} requires a directory`, + ); + } + if (sourceKind === "directory") { + if (options.entry.mode === "copy") { + await assertCopyDirectorySafe({ + entry: options.entry, + sourcePath, + }); + } else { + await assertSymlinkDirectorySafe({ + entry: options.entry, + excludedSourceRoots: options.excludedSourceRoots, + sourcePath, + sourceRoot: options.sourceRoot, + }); + } + } + + return { + materialization: { + lineNumber: options.entry.lineNumber, + mode: options.entry.mode, + raw: options.entry.raw, + relativePath: options.relativePath, + sourceKind, + }, + sourcePath, + }; +} + +function assertSourcePathIsSafe(options: { + entry: WorktreeIncludeEntry; + excludedSourceRoots: string[]; + sourcePath: string; + sourceRoot: string; +}): void { + if (!isPathInsideRoot(options.sourceRoot, options.sourcePath)) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} resolves outside the source checkout`, + ); + } + + const canonicalRelativePath = relative(options.sourceRoot, options.sourcePath); + if (canonicalRelativePath.split(/[\\/]/).some((segment) => segment.toLowerCase() === ".git")) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} resolves into Git metadata`, + ); + } + + const excludedRoot = options.excludedSourceRoots.find( + (candidate) => + isPathInsideRoot(options.sourcePath, candidate) || + isPathInsideRoot(candidate, options.sourcePath), + ); + if (excludedRoot === undefined) { + return; + } + + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} overlaps with a protected worktree path`, + ); +} + +async function getCanonicalSourceKind(options: { + entry: WorktreeIncludeEntry; + sourcePath: string; +}): Promise { + const stats = await lstatSourcePath(options.sourcePath, options.entry); + if (stats.isSymbolicLink()) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} changed while its source path was resolved`, + ); + } + if (!stats.isFile() && !stats.isDirectory()) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} must match a regular file or directory`, + ); + } + + return stats.isDirectory() ? "directory" : "file"; +} + +async function realpathSourcePath( + sourcePath: string, + entry: WorktreeIncludeEntry, +): Promise { + try { + return await realpath(sourcePath); + } catch (error) { + const code = getErrorCode(error); + if (code === "ENOENT" || code === "ENOTDIR") { + throw noMatchError(entry); + } + if (code === "ELOOP") { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${entry.raw}' on line ${entry.lineNumber} contains a symbolic-link loop`, + ); + } + throw error; + } +} + +async function lstatSourcePath(sourcePath: string, entry: WorktreeIncludeEntry): Promise { + try { + return await lstat(sourcePath); + } catch (error) { + if (getErrorCode(error) === "ENOENT" || getErrorCode(error) === "ENOTDIR") { + throw noMatchError(entry); + } + throw error; + } +} + +async function assertCopyDirectorySafe(options: { + entry: WorktreeIncludeEntry; + sourcePath: string; +}): Promise { + for (const name of await readdir(options.sourcePath)) { + if (name.toLowerCase() === ".git") { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} contains git metadata`, + ); + } + + const sourcePath = join(options.sourcePath, name); + const stats = await lstatSourcePath(sourcePath, options.entry); + if (stats.isSymbolicLink() || (!stats.isFile() && !stats.isDirectory())) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} contains an unsupported file type or symbolic link`, + ); + } + if (stats.isDirectory()) { + await assertCopyDirectorySafe({ sourcePath, entry: options.entry }); + } + } +} + +async function assertSymlinkDirectorySafe(options: { + entry: WorktreeIncludeEntry; + excludedSourceRoots: string[]; + sourcePath: string; + sourceRoot: string; +}): Promise { + const visitedDirectories = new Set(); + + async function visit(directoryPath: string): Promise { + if (visitedDirectories.has(directoryPath)) { + return; + } + visitedDirectories.add(directoryPath); + + for (const name of await readdir(directoryPath)) { + if (name.toLowerCase() === ".git") { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} contains git metadata`, + ); + } + + const childPath = join(directoryPath, name); + const childStats = await lstatSourcePath(childPath, options.entry); + if (childStats.isSymbolicLink()) { + const linkedPath = await realpathSourcePath(childPath, options.entry); + assertSourcePathIsSafe({ + entry: options.entry, + excludedSourceRoots: options.excludedSourceRoots, + sourcePath: linkedPath, + sourceRoot: options.sourceRoot, + }); + const linkedKind = await getCanonicalSourceKind({ + entry: options.entry, + sourcePath: linkedPath, + }); + if (linkedKind === "directory") { + await visit(linkedPath); + } + continue; + } + + if (childStats.isDirectory()) { + await visit(childPath); + continue; + } + if (!childStats.isFile()) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.entry.raw}' on line ${options.entry.lineNumber} contains an unsupported file type`, + ); + } + } + } + + await visit(options.sourcePath); +} + +function normalizeMaterializations( + materializations: WorktreeIncludeMaterialization[], +): NormalizedWorktreeIncludeMaterializations { + const byPath = new Map(); + for (const materialization of materializations) { + const matches = byPath.get(materialization.relativePath) ?? []; + matches.push(materialization); + byPath.set(materialization.relativePath, matches); + } + + const skipped: WorktreeIncludeSkippedEntry[] = []; + const candidates: WorktreeIncludeMaterialization[] = []; + for (const [relativePath, matches] of byPath) { + if (new Set(matches.map((materialization) => materialization.mode)).size === 1) { + candidates.push(matches[0]!); + continue; + } + + const message = `.worktreeinclude entries for '${relativePath}' use both copy and symlink modes`; + skipped.push( + ...matches.map((materialization) => + toSkippedEntry(materialization, new WorktreeIncludeError("conflict", message)), + ), + ); + } + + const conflicted = new Set(); + const skippedConflictEntries = new Set(); + const skipConflict = (materialization: WorktreeIncludeMaterialization, message: string): void => { + conflicted.add(materialization); + if (skippedConflictEntries.has(materialization)) { + return; + } + skippedConflictEntries.add(materialization); + skipped.push(toSkippedEntry(materialization, new WorktreeIncludeError("conflict", message))); + }; + + for (let leftIndex = 0; leftIndex < candidates.length; leftIndex++) { + const left = candidates[leftIndex]!; + for (let rightIndex = leftIndex + 1; rightIndex < candidates.length; rightIndex++) { + const right = candidates[rightIndex]!; + let ancestor: WorktreeIncludeMaterialization | null = null; + if (isRelativePathAncestor(left.relativePath, right.relativePath)) { + ancestor = left; + } else if (isRelativePathAncestor(right.relativePath, left.relativePath)) { + ancestor = right; + } + if (ancestor === null || (left.mode === "copy" && right.mode === "copy")) { + continue; + } + + const descendant = ancestor === left ? right : left; + const message = `.worktreeinclude entries for '${ancestor.relativePath}' and '${descendant.relativePath}' overlap with a symlink`; + skipConflict(left, message); + skipConflict(right, message); + } + } + + const sorted = candidates + .filter((materialization) => !conflicted.has(materialization)) + .sort((left, right) => { + const depthDifference = + left.relativePath.split("/").length - right.relativePath.split("/").length; + return depthDifference === 0 + ? left.relativePath.localeCompare(right.relativePath) + : depthDifference; + }); + return { materializations: sorted, skipped }; +} + +function isRelativePathAncestor(ancestor: string, candidate: string): boolean { + return ancestor !== candidate && candidate.startsWith(`${ancestor}/`); +} + +async function preflightDestination(options: { + resolved: ResolvedWorktreeIncludeMaterialization; + worktreeRoot: string; +}): Promise { + const { materialization } = options.resolved; + const destinationPath = getDestinationPath({ + worktreeRoot: options.worktreeRoot, + relativePath: materialization.relativePath, + }); + const destinationStats = await lstatIfExists(destinationPath); + if (destinationStats === null) { + return true; + } + + if (destinationStats.isSymbolicLink()) { + if ( + materialization.mode === "symlink" && + (await isExpectedSymlink({ + destinationPath, + sourcePath: options.resolved.sourcePath, + })) + ) { + return false; + } + throw destinationConflict(materialization); + } + + if (materialization.mode === "symlink") { + throw destinationConflict(materialization); + } + + if ( + (materialization.sourceKind === "file" && !destinationStats.isFile()) || + (materialization.sourceKind === "directory" && !destinationStats.isDirectory()) + ) { + throw destinationConflict(materialization); + } + + if (materialization.sourceKind === "directory") { + await assertCopyDestinationTreeSafe({ + sourcePath: options.resolved.sourcePath, + destinationPath, + materialization, + }); + } + return true; +} + +function getDestinationPath(options: { relativePath: string; worktreeRoot: string }): string { + const destinationPath = join(options.worktreeRoot, ...options.relativePath.split("/")); + if (!isPathInsideRoot(options.worktreeRoot, destinationPath)) { + throw new WorktreeIncludeError( + "invalid_entry", + `.worktreeinclude entry '${options.relativePath}' resolves outside the worktree`, + ); + } + return destinationPath; +} + +async function ensureDestinationParent(options: { + relativePath: string; + worktreeRoot: string; +}): Promise { + const parentSegments = options.relativePath.split("/").slice(0, -1); + let currentPath = options.worktreeRoot; + const createdPaths: string[] = []; + try { + for (const segment of parentSegments) { + currentPath = join(currentPath, segment); + const stats = await lstatIfExists(currentPath); + if (stats === null) { + await mkdir(currentPath); + createdPaths.push(currentPath); + continue; + } + if (stats.isSymbolicLink() || !stats.isDirectory()) { + throw new WorktreeIncludeError( + "conflict", + `Refusing to materialize .worktreeinclude entry '${options.relativePath}' through '${currentPath}'`, + ); + } + } + } catch (error) { + await cleanupCreatedDestinationParents(createdPaths); + throw error; + } + return createdPaths; +} + +async function assertCopyDestinationTreeSafe(options: { + destinationPath: string; + materialization: WorktreeIncludeMaterialization; + sourcePath: string; +}): Promise { + for (const name of await readdir(options.sourcePath)) { + const sourceChildPath = join(options.sourcePath, name); + const sourceStats = await lstat(sourceChildPath); + if (sourceStats.isSymbolicLink()) { + throw new WorktreeIncludeError( + "unsupported_source", + `.worktreeinclude entry '${options.materialization.relativePath}' contains a symbolic link`, + ); + } + + const destinationChildPath = join(options.destinationPath, name); + const destinationStats = await lstatIfExists(destinationChildPath); + if (destinationStats === null) { + continue; + } + if (destinationStats.isSymbolicLink()) { + throw destinationConflict(options.materialization); + } + if ( + (sourceStats.isFile() && !destinationStats.isFile()) || + (sourceStats.isDirectory() && !destinationStats.isDirectory()) + ) { + throw destinationConflict(options.materialization); + } + if (sourceStats.isDirectory()) { + await assertCopyDestinationTreeSafe({ + sourcePath: sourceChildPath, + destinationPath: destinationChildPath, + materialization: options.materialization, + }); + } + } +} + +async function createMaterializationSymlink(options: { + destinationPath: string; + linkDestinationPath?: string; + resolved: ResolvedWorktreeIncludeMaterialization; +}): Promise { + if (process.platform !== "win32") { + const target = relative( + dirname(options.linkDestinationPath ?? options.destinationPath), + options.resolved.sourcePath, + ); + await symlink(target, options.destinationPath); + return; + } + + let type: "dir" | "file" | "junction" = "file"; + if (options.resolved.materialization.sourceKind === "directory") { + type = isWindowsNetworkPath(options.resolved.sourcePath) ? "dir" : "junction"; + } + try { + await symlink(options.resolved.sourcePath, options.destinationPath, type); + } catch (error) { + throw toWindowsSymlinkError({ error, entry: options.resolved.materialization }); + } +} + +function isWindowsSymlinkPrivilegeError(error: unknown): boolean { + const code = getErrorCode(error); + return code === "EACCES" || code === "EPERM" || code === "ENOTSUP"; +} + +function isWindowsNetworkPath(path: string): boolean { + return path.startsWith("\\\\"); +} + +function toWindowsSymlinkError(options: { + entry: WorktreeIncludeMaterialization; + error: unknown; +}): Error { + if (!isWindowsSymlinkPrivilegeError(options.error)) { + return options.error instanceof Error ? options.error : new Error(String(options.error)); + } + return new WorktreeIncludeError( + "windows_symlink_unavailable", + `Unable to create a Windows symlink for .worktreeinclude entry '${options.entry.relativePath}'. Enable Developer Mode or use copy ${options.entry.relativePath}.`, + ); +} + +async function isExpectedSymlink(options: { + destinationPath: string; + sourcePath: string; +}): Promise { + try { + const [destinationTarget, sourceTarget] = await Promise.all([ + realpath(options.destinationPath), + realpath(options.sourcePath), + ]); + return areEquivalentPaths(destinationTarget, sourceTarget); + } catch { + return false; + } +} + +async function lstatIfExists(path: string): Promise { + try { + return await lstat(path); + } catch (error) { + if (getErrorCode(error) === "ENOENT") { + return null; + } + throw error; + } +} + +function destinationConflict( + materialization: WorktreeIncludeMaterialization, +): WorktreeIncludeError { + return new WorktreeIncludeError( + "conflict", + `.worktreeinclude entry '${materialization.relativePath}' on line ${materialization.lineNumber} conflicts with the new worktree`, + ); +} + +function noMatchError(entry: WorktreeIncludeEntry): WorktreeIncludeError { + return new WorktreeIncludeError( + "missing_source", + `No paths matched .worktreeinclude entry '${entry.raw}' on line ${entry.lineNumber}`, + ); +} + +function toSkippedEntry( + entry: Pick, + error: unknown, +): WorktreeIncludeSkippedEntry { + return { + lineNumber: entry.lineNumber, + message: error instanceof Error ? error.message : String(error), + raw: entry.raw, + reason: getSkipReason(error), + }; +} + +function getSkipReason(error: unknown): WorktreeIncludeSkipReason { + if (!(error instanceof WorktreeIncludeError)) { + return "materialization"; + } + switch (error.code) { + case "conflict": + return "conflict"; + case "invalid_entry": + return "invalid"; + case "windows_symlink_unavailable": + return "materialization"; + case "missing_source": + return "missing"; + case "source_changed": + return "source_changed"; + case "unsupported_source": + return "unsafe"; + } +} + +function isSkippableWorktreeIncludeError(error: unknown): error is WorktreeIncludeError { + return error instanceof WorktreeIncludeError && error.code !== "windows_symlink_unavailable"; +} + +function isWorktreeIncludeMaterializationError(error: unknown): boolean { + return error instanceof WorktreeIncludeError || getErrorCode(error) !== null; +} + +function getErrorCode(error: unknown): string | null { + if (typeof error !== "object" || error === null || !("code" in error)) { + return null; + } + const { code } = error; + return typeof code === "string" ? code : null; +} diff --git a/packages/server/src/utils/worktree.posix.test.ts b/packages/server/src/utils/worktree.posix.test.ts index d895df539..53a555cf4 100644 --- a/packages/server/src/utils/worktree.posix.test.ts +++ b/packages/server/src/utils/worktree.posix.test.ts @@ -34,6 +34,7 @@ import { writeFileSync, readFileSync, chmodSync, + lstatSync, } from "fs"; import { delimiter, dirname, join } from "path"; import { tmpdir } from "os"; @@ -362,6 +363,68 @@ describe.skipIf(isPlatform("win32"))("worktree POSIX-only", () => { expect(metadata).toMatchObject({ baseRefName: "main" }); }); + it("removes fetched branches when include planning fails", async () => { + const remoteDir = join(tempDir, "remote.git"); + const remoteCloneDir = join(tempDir, "remote-clone"); + execFileSync("git", ["clone", "--bare", repoDir, remoteDir]); + execFileSync("git", ["remote", "add", "origin", remoteDir], { cwd: repoDir }); + + execFileSync("git", ["clone", remoteDir, remoteCloneDir]); + execFileSync("git", ["config", "user.email", "test@test.com"], { cwd: remoteCloneDir }); + execFileSync("git", ["config", "user.name", "Test"], { cwd: remoteCloneDir }); + execFileSync("git", ["checkout", "-b", "contributor/cleanup"], { cwd: remoteCloneDir }); + writeFileSync(join(remoteCloneDir, "file.txt"), "from-pr\n"); + execFileSync("git", ["add", "file.txt"], { cwd: remoteCloneDir }); + execFileSync("git", ["-c", "commit.gpgsign=false", "commit", "-m", "pr branch"], { + cwd: remoteCloneDir, + }); + const prHead = execFileSync("git", ["rev-parse", "HEAD"], { cwd: remoteCloneDir }) + .toString() + .trim(); + execFileSync("git", ["push", "origin", "contributor/cleanup"], { cwd: remoteCloneDir }); + execFileSync("git", [`--git-dir=${remoteDir}`, "update-ref", "refs/pull/44/head", prHead]); + mkdirSync(join(repoDir, ".worktreeinclude")); + + await expect( + createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "pr-44-cleanup", + source: { + kind: "checkout-github-pr", + githubPrNumber: 44, + headRef: "contributor/cleanup", + baseRefName: "main", + }, + runSetup: false, + paseoHome, + }), + ).rejects.toMatchObject({ code: "EISDIR" }); + + expect(() => + execFileSync("git", ["show-ref", "--verify", "--quiet", "refs/heads/contributor/cleanup"], { + cwd: repoDir, + stdio: "pipe", + }), + ).toThrow(); + + await expect( + createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "branch-cleanup", + source: { kind: "checkout-branch", branchName: "contributor/cleanup" }, + runSetup: false, + paseoHome, + }), + ).rejects.toMatchObject({ code: "EISDIR" }); + + expect(() => + execFileSync("git", ["show-ref", "--verify", "--quiet", "refs/heads/contributor/cleanup"], { + cwd: repoDir, + stdio: "pipe", + }), + ).toThrow(); + }); + it("fetches a GitHub PR branch when the head ref contains uppercase letters and dots", async () => { const remoteDir = join(tempDir, "remote.git"); const remoteCloneDir = join(tempDir, "remote-clone"); @@ -1062,6 +1125,241 @@ describe.skipIf(isPlatform("win32"))("worktree POSIX-only", () => { }); }); + it("materializes copies and symlinks before setup, then removes only the new worktree links", async () => { + writeFileSync( + join(repoDir, ".gitignore"), + [".copy.env", "copy-cache/", "linked-file.txt", "linked-state", "setup.log", ""].join("\n"), + ); + writeFileSync( + join(repoDir, "paseo.json"), + JSON.stringify({ + worktree: { + setup: [ + "test -f .copy.env", + "test -L linked-file.txt", + "test -L linked-state", + "cat linked-state/state.txt > setup.log", + ], + }, + }), + ); + execFileSync("git", ["add", ".gitignore", "paseo.json"], { cwd: repoDir }); + execFileSync("git", ["-c", "commit.gpgsign=false", "commit", "-m", "add include fixture"], { + cwd: repoDir, + }); + + writeFileSync( + join(repoDir, ".worktreeinclude"), + [".copy.env", "copy-cache/**", "symlink linked-file.txt", "symlink linked-state", ""].join( + "\n", + ), + ); + writeFileSync(join(repoDir, ".copy.env"), "copy-v1\n"); + mkdirSync(join(repoDir, "copy-cache"), { recursive: true }); + writeFileSync(join(repoDir, "copy-cache", "state.txt"), "copy-cache-v1\n"); + writeFileSync(join(repoDir, "linked-file.txt"), "linked-file-v1\n"); + mkdirSync(join(repoDir, "linked-state"), { recursive: true }); + writeFileSync(join(repoDir, "linked-state", "state.txt"), "linked-state-v1\n"); + + const result = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "include-links", + source: { kind: "branch-off", baseBranch: "main", branchName: "feature/include-links" }, + runSetup: true, + paseoHome, + }); + + expect(readFileSync(join(result.worktreePath, "setup.log"), "utf8")).toBe( + "linked-state-v1\n", + ); + expect(lstatSync(join(result.worktreePath, ".copy.env")).isSymbolicLink()).toBe(false); + expect(lstatSync(join(result.worktreePath, "copy-cache")).isSymbolicLink()).toBe(false); + expect(lstatSync(join(result.worktreePath, "linked-file.txt")).isSymbolicLink()).toBe(true); + expect(lstatSync(join(result.worktreePath, "linked-state")).isSymbolicLink()).toBe(true); + expect( + execFileSync("git", ["status", "--porcelain"], { + cwd: result.worktreePath, + encoding: "utf8", + }), + ).toBe(""); + + writeFileSync(join(repoDir, ".copy.env"), "copy-v2\n"); + writeFileSync(join(repoDir, "copy-cache", "state.txt"), "copy-cache-v2\n"); + writeFileSync(join(repoDir, "linked-file.txt"), "linked-file-v2\n"); + writeFileSync(join(repoDir, "linked-state", "state.txt"), "linked-state-v2\n"); + + expect(readFileSync(join(result.worktreePath, ".copy.env"), "utf8")).toBe("copy-v1\n"); + expect(readFileSync(join(result.worktreePath, "copy-cache", "state.txt"), "utf8")).toBe( + "copy-cache-v1\n", + ); + expect(readFileSync(join(result.worktreePath, "linked-file.txt"), "utf8")).toBe( + "linked-file-v2\n", + ); + expect(readFileSync(join(result.worktreePath, "linked-state", "state.txt"), "utf8")).toBe( + "linked-state-v2\n", + ); + + await deletePaseoWorktree({ + cwd: repoDir, + worktreePath: result.worktreePath, + paseoHome, + }); + + expect(existsSync(result.worktreePath)).toBe(false); + expect(readFileSync(join(repoDir, "linked-file.txt"), "utf8")).toBe("linked-file-v2\n"); + expect(readFileSync(join(repoDir, "linked-state", "state.txt"), "utf8")).toBe( + "linked-state-v2\n", + ); + }); + + it("skips missing includes and materializes paths that exist", async () => { + const projectHash = await deriveWorktreeProjectHash(repoDir); + const expectedWorktreePath = join(paseoHome, "worktrees", projectHash, "missing-include"); + writeFileSync( + join(repoDir, ".worktreeinclude"), + [".env", ".env.local", ".sops.yaml", ""].join("\n"), + ); + writeFileSync(join(repoDir, ".env"), "present\n"); + + const result = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "missing-include", + source: { kind: "branch-off", baseBranch: "main", branchName: "feature/missing-include" }, + runSetup: false, + paseoHome, + }); + + expect(result.worktreePath).toBe(expectedWorktreePath); + expect(readFileSync(join(result.worktreePath, ".env"), "utf8")).toBe("present\n"); + expect(existsSync(join(result.worktreePath, ".env.local"))).toBe(false); + expect(existsSync(join(result.worktreePath, ".sops.yaml"))).toBe(false); + expect( + execFileSync("git", ["worktree", "list", "--porcelain"], { + cwd: repoDir, + encoding: "utf8", + }), + ).toContain(expectedWorktreePath); + }); + + it("skips checkout-local worktree storage and creates the remaining includes", async () => { + const checkoutLocalPaseoHome = join(repoDir, ".dev", "paseo-home"); + const projectHash = await deriveWorktreeProjectHash(repoDir); + const expectedWorktreePath = join( + checkoutLocalPaseoHome, + "worktrees", + projectHash, + "protected-include", + ); + mkdirSync(join(checkoutLocalPaseoHome, "worktrees", projectHash), { recursive: true }); + writeFileSync(join(repoDir, ".worktreeinclude"), [".dev/**", ".env", ""].join("\n")); + writeFileSync(join(repoDir, ".env"), "present\n"); + + const result = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "protected-include", + source: { + kind: "branch-off", + baseBranch: "main", + branchName: "feature/protected-include", + }, + runSetup: false, + paseoHome: checkoutLocalPaseoHome, + }); + + expect(result.worktreePath).toBe(expectedWorktreePath); + expect(readFileSync(join(result.worktreePath, ".env"), "utf8")).toBe("present\n"); + expect(result.worktreeIncludeSummary?.skipped).toEqual([ + expect.objectContaining({ raw: ".dev/**", reason: "unsafe" }), + ]); + expect( + execFileSync("git", ["worktree", "list", "--porcelain"], { + cwd: repoDir, + encoding: "utf8", + }), + ).toContain(expectedWorktreePath); + expect( + execFileSync("git", ["branch", "--list", "feature/protected-include"], { + cwd: repoDir, + encoding: "utf8", + }).trim(), + ).toContain("feature/protected-include"); + }); + + it("keeps includes when branching from a Paseo-managed worktree", async () => { + writeFileSync(join(repoDir, ".gitignore"), ".env\n"); + writeFileSync(join(repoDir, ".worktreeinclude"), ".env\n"); + execFileSync("git", ["add", ".gitignore", ".worktreeinclude"], { cwd: repoDir }); + execFileSync("git", ["-c", "commit.gpgsign=false", "commit", "-m", "add include config"], { + cwd: repoDir, + }); + writeFileSync(join(repoDir, ".env"), "source\n"); + + const sourceWorktree = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "include-source", + source: { + kind: "branch-off", + baseBranch: "main", + branchName: "feature/include-source", + }, + runSetup: false, + paseoHome, + }); + const nestedWorktree = await createLegacyWorktreeForTest({ + cwd: sourceWorktree.worktreePath, + worktreeSlug: "include-nested", + source: { + kind: "branch-off", + baseBranch: "feature/include-source", + branchName: "feature/include-nested", + }, + runSetup: false, + paseoHome, + }); + + expect(readFileSync(join(sourceWorktree.worktreePath, ".env"), "utf8")).toBe("source\n"); + expect(readFileSync(join(nestedWorktree.worktreePath, ".env"), "utf8")).toBe("source\n"); + expect(nestedWorktree.worktreeIncludeSummary?.skipped).toEqual([]); + }); + + it("skips a symlink include conflict and keeps the new worktree", async () => { + const projectHash = await deriveWorktreeProjectHash(repoDir); + const expectedWorktreePath = join(paseoHome, "worktrees", projectHash, "include-conflict"); + writeFileSync(join(repoDir, "paseo.json"), JSON.stringify({ scripts: {} })); + writeFileSync(join(repoDir, ".worktreeinclude"), "symlink paseo.json\n"); + + const result = await createLegacyWorktreeForTest({ + cwd: repoDir, + worktreeSlug: "include-conflict", + source: { + kind: "branch-off", + baseBranch: "main", + branchName: "feature/include-conflict", + }, + runSetup: false, + paseoHome, + }); + + expect(result.worktreePath).toBe(expectedWorktreePath); + expect(existsSync(expectedWorktreePath)).toBe(true); + expect(lstatSync(join(expectedWorktreePath, "paseo.json")).isSymbolicLink()).toBe(false); + expect(result.worktreeIncludeSummary?.skipped).toEqual([ + expect.objectContaining({ raw: "symlink paseo.json", reason: "conflict" }), + ]); + expect( + execFileSync("git", ["worktree", "list", "--porcelain"], { + cwd: repoDir, + encoding: "utf8", + }), + ).toContain(expectedWorktreePath); + expect( + execFileSync("git", ["branch", "--list", "feature/include-conflict"], { + cwd: repoDir, + encoding: "utf8", + }).trim(), + ).toContain("feature/include-conflict"); + }); + it("creates a worktree without error when no paseo.json exists in the main repo", async () => { const result = await createLegacyWorktreeForTest({ cwd: repoDir, diff --git a/packages/server/src/utils/worktree.ts b/packages/server/src/utils/worktree.ts index 4ab641806..f46ae6c54 100644 --- a/packages/server/src/utils/worktree.ts +++ b/packages/server/src/utils/worktree.ts @@ -4,7 +4,7 @@ import { existsSync, mkdirSync, realpathSync, rmSync, statSync } from "fs"; import { copyFile, rm, stat } from "fs/promises"; import { join, basename, dirname, isAbsolute, resolve, sep } from "path"; import net from "node:net"; -import { createHash } from "node:crypto"; +import { createHash, randomUUID } from "node:crypto"; import stripAnsi from "strip-ansi"; import { buildStringCommandShellInvocation, @@ -36,6 +36,11 @@ import { createExternalProcessEnv } from "../server/paseo-env.js"; import { parseGitRevParsePath, resolveGitRevParsePath } from "./git-rev-parse-path.js"; import { validateBranchSlug } from "@getpaseo/protocol/branch-slug"; import { expandTilde, getRealpathAwareRelativePath, isPathInsideRoot } from "./path.js"; +import { + materializeWorktreeIncludePlan, + readWorktreeIncludePlan, + type WorktreeIncludeSummary, +} from "./worktree-include.js"; export { slugify, validateBranchSlug } from "@getpaseo/protocol/branch-slug"; @@ -49,6 +54,10 @@ export interface WorktreeConfig { worktreePath: string; } +export interface CreatedWorktreeConfig extends WorktreeConfig { + worktreeIncludeSummary: WorktreeIncludeSummary; +} + export interface WorktreeRuntimeEnv { [key: string]: string; PASEO_SOURCE_CHECKOUT_PATH: string; @@ -1161,13 +1170,67 @@ export async function deletePaseoWorktree({ } } +export interface RollbackCreatedPaseoWorktreeOptions extends DeletePaseoWorktreeOptions { + createdBranchName?: string; + expectedOid?: string; +} + +async function removeCreatedWorktreeBranch(options: { + createdBranchName?: string; + expectedOid?: string; + cwd?: string | null; +}): Promise { + if (!options.createdBranchName || !options.cwd) { + return; + } + if (!(await localBranchExists(options.cwd, options.createdBranchName))) { + return; + } + if (options.expectedOid) { + await runGitCommand( + ["update-ref", "-d", `refs/heads/${options.createdBranchName}`, options.expectedOid], + { cwd: options.cwd }, + ); + } else { + await runGitCommand(["branch", "--delete", "--force", options.createdBranchName], { + cwd: options.cwd, + }); + } +} + +async function rollbackCreatedWorktreeBranch( + options: { + createdBranchName?: string; + expectedOid?: string; + cwd: string; + }, + cause: unknown, +): Promise { + let cleanupError: unknown; + try { + await removeCreatedWorktreeBranch(options); + } catch (error) { + cleanupError = error; + } + if (cleanupError) { + const failure = new Error( + `${cause instanceof Error ? cause.message : "Worktree workflow failed"}; rollback also failed: ${cleanupError instanceof Error ? cleanupError.message : String(cleanupError)}`, + { cause }, + ); + Object.assign(failure, { cleanupError }); + throw failure; + } + throw cause; +} + export async function rollbackCreatedPaseoWorktree( - options: DeletePaseoWorktreeOptions, + options: RollbackCreatedPaseoWorktreeOptions, cause: unknown, ): Promise { let cleanupError: unknown; try { await deletePaseoWorktree(options); + await removeCreatedWorktreeBranch(options); } catch (error) { cleanupError = error; } @@ -1233,50 +1296,100 @@ export const createWorktree = async ({ runSetup, paseoHome, worktreesRoot, -}: CreateWorktreeOptions): Promise => { +}: CreateWorktreeOptions): Promise => { const sourcePlan = await resolveWorktreeSourcePlan({ cwd, source, desiredSlug: worktreeSlug }); - let worktreePath = join(await getPaseoWorktreesRoot(cwd, paseoHome, worktreesRoot), worktreeSlug); - mkdirSync(dirname(worktreePath), { recursive: true }); + const { worktreeIncludePlan, worktreePath } = await (async () => { + try { + const paseoWorktreesBaseRoot = resolvePaseoWorktreesBaseRoot({ paseoHome, worktreesRoot }); + const paseoWorktreesRoot = await getPaseoWorktreesRoot(cwd, paseoHome, worktreesRoot); + const includePlan = await readWorktreeIncludePlan({ + sourceRoot: cwd, + excludedSourceRoots: [paseoWorktreesBaseRoot], + }); + const requestedWorktreePath = join(paseoWorktreesRoot, worktreeSlug); + mkdirSync(dirname(requestedWorktreePath), { recursive: true }); - // Also handle worktree path collision - let finalWorktreePath = worktreePath; - let pathSuffix = 1; - while (existsSync(finalWorktreePath)) { - finalWorktreePath = `${worktreePath}-${pathSuffix}`; - pathSuffix++; - } + // Also handle worktree path collision + let finalWorktreePath = requestedWorktreePath; + let pathSuffix = 1; + while (existsSync(finalWorktreePath)) { + finalWorktreePath = `${requestedWorktreePath}-${pathSuffix}`; + pathSuffix++; + } - // Primitive owner for `git worktree add`; callers route through createWorktreeCore. - await runGitCommand(["worktree", "add", finalWorktreePath, ...sourcePlan.addArguments], { - cwd, - timeout: 120_000, - }); - worktreePath = normalizePathForOwnership(finalWorktreePath); + // Primitive owner for `git worktree add`; callers route through createWorktreeCore. + await runGitCommand(["worktree", "add", finalWorktreePath, ...sourcePlan.addArguments], { + cwd, + timeout: 120_000, + }); - if (sourcePlan.pushRemote) { - await configureWorktreePushRemote({ - cwd, - branchName: sourcePlan.branchName, - remote: sourcePlan.pushRemote, + return { + worktreeIncludePlan: includePlan, + worktreePath: normalizePathForOwnership(finalWorktreePath), + }; + } catch (error) { + return rollbackCreatedWorktreeBranch( + { + cwd, + createdBranchName: sourcePlan.createdBranchNameBeforeWorktreeAdd, + expectedOid: sourcePlan.createdBranchOidBeforeWorktreeAdd, + }, + error, + ); + } + })(); + + let worktreeIncludeSummary: WorktreeIncludeSummary = { + materialized: 0, + skipped: [...worktreeIncludePlan.skipped], + }; + try { + if (sourcePlan.pushRemote) { + await configureWorktreePushRemote({ + cwd, + branchName: sourcePlan.branchName, + remote: sourcePlan.pushRemote, + }); + } + if (sourcePlan.trackingRemote) { + await configureWorktreeTrackingRemote({ + cwd, + branchName: sourcePlan.branchName, + remote: sourcePlan.trackingRemote, + }); + } + + writePaseoWorktreeMetadata(worktreePath, { + baseRefName: sourcePlan.metadataBaseRefName, + ...(sourcePlan.changeRequestLookupTarget + ? { changeRequestLookupTarget: sourcePlan.changeRequestLookupTarget } + : {}), }); - } - if (sourcePlan.trackingRemote) { - await configureWorktreeTrackingRemote({ - cwd, - branchName: sourcePlan.branchName, - remote: sourcePlan.trackingRemote, + + await seedPaseoConfigFile({ sourceCwd: cwd, targetCwd: worktreePath }); + const materialization = await materializeWorktreeIncludePlan({ + plan: worktreeIncludePlan, + worktreeRoot: worktreePath, }); + worktreeIncludeSummary = { + materialized: materialization.materialized, + skipped: [...worktreeIncludePlan.skipped, ...materialization.skipped], + }; + } catch (error) { + await rollbackCreatedPaseoWorktree( + { + cwd, + worktreePath, + teardownCwds: [], + paseoHome, + worktreesBaseRoot: worktreesRoot, + createdBranchName: sourcePlan.createdBranchName, + expectedOid: sourcePlan.createdBranchOidBeforeWorktreeAdd, + }, + error, + ); } - writePaseoWorktreeMetadata(worktreePath, { - baseRefName: sourcePlan.metadataBaseRefName, - ...(sourcePlan.changeRequestLookupTarget - ? { changeRequestLookupTarget: sourcePlan.changeRequestLookupTarget } - : {}), - }); - - await seedPaseoConfigFile({ sourceCwd: cwd, targetCwd: worktreePath }); - if (runSetup) { await runWorktreeSetupCommands({ worktreePath, @@ -1287,6 +1400,7 @@ export const createWorktree = async ({ return { branchName: sourcePlan.branchName, + worktreeIncludeSummary, worktreePath, }; }; @@ -1299,6 +1413,9 @@ interface ResolveWorktreeSourcePlanOptions { interface WorktreeSourcePlan { branchName: string; + createdBranchName?: string; + createdBranchNameBeforeWorktreeAdd?: string; + createdBranchOidBeforeWorktreeAdd?: string; metadataBaseRefName: string; changeRequestLookupTarget?: PaseoWorktreeChangeRequestLookupTarget; addArguments: string[]; @@ -1314,6 +1431,11 @@ interface WorktreeSourcePlan { }; } +type ChangeRequestWorktreeSource = Extract< + WorktreeSource, + { kind: "checkout-change-request" | "checkout-github-pr" } +>; + async function resolveWorktreeSourcePlan({ cwd, source, @@ -1332,94 +1454,144 @@ async function resolveWorktreeSourcePlan({ return { branchName: newBranchName, + createdBranchName: newBranchName, metadataBaseRefName: normalizedBaseBranch, addArguments: ["-b", newBranchName, "--no-track", base], }; } - case "checkout-branch": { - await validateExistingWorktreeBranchName(cwd, source.branchName); - if (!(await localBranchExists(cwd, source.branchName))) { - try { - await runGitCommand(["fetch", "origin", `${source.branchName}:${source.branchName}`], { - cwd, - timeout: 120_000, - }); - } catch { - throw new UnknownBranchError({ branchName: source.branchName, cwd }); - } - } - if (await isBranchCheckedOut(cwd, source.branchName)) { - throw new BranchAlreadyCheckedOutError(source.branchName); - } - - return { - branchName: source.branchName, - metadataBaseRefName: source.branchName, - addArguments: [source.branchName], - }; - } + case "checkout-branch": + return resolveCheckoutBranchWorktreeSourcePlan({ cwd, branchName: source.branchName }); case "checkout-change-request": - case "checkout-github-pr": { - const localBranchCandidate = source.localBranchName ?? source.headRef; - await validateExistingWorktreeBranchName(cwd, localBranchCandidate); - const localBranchName = await resolveUniqueLocalBranchName(cwd, localBranchCandidate); - const normalizedBaseRefName = normalizeRequiredBaseBranch(source.baseRefName); - const changeRequestNumber = - source.kind === "checkout-github-pr" ? source.githubPrNumber : source.changeRequestNumber; - await fetchWorktreeCheckoutRefs({ - cwd, - localBranchName, - checkoutRefs: source.checkoutRefs ?? [ - { remoteName: "origin", remoteRef: `refs/pull/${changeRequestNumber}/head` }, - ], - }); - const shouldTrackOriginHead = source.trackOriginHead === true; - const trackingRemote = shouldTrackOriginHead - ? await tryFetchWorktreeTrackingRemote({ - cwd, - remoteName: "origin", - headRef: source.headRef, - }) - : undefined; - const remotePlan: Pick = {}; - if (source.pushRemoteUrl) { - const remoteName = `paseo-pr-${changeRequestNumber}`; - remotePlan.pushRemote = { - name: remoteName, - url: source.pushRemoteUrl, - headRef: source.headRef, - track: true, - }; - } else if (shouldTrackOriginHead && localBranchName !== source.headRef) { - const originUrl = await getWorktreeRemotePushUrl(cwd, "origin"); - if (originUrl) { - remotePlan.pushRemote = { - name: `paseo-pr-${changeRequestNumber}`, - url: originUrl, - headRef: source.headRef, - track: false, - }; - } - } - if (trackingRemote) { - remotePlan.trackingRemote = trackingRemote; - } + case "checkout-github-pr": + return resolveChangeRequestWorktreeSourcePlan({ cwd, source }); + } +} - return { - branchName: localBranchName, - metadataBaseRefName: normalizedBaseRefName, - changeRequestLookupTarget: { - headRef: source.headRef, - ...(source.headRepositoryOwner - ? { headRepositoryOwner: source.headRepositoryOwner } - : {}), - changeRequestNumber, - }, - addArguments: [localBranchName], - ...remotePlan, - }; +async function resolveCheckoutBranchWorktreeSourcePlan(options: { + branchName: string; + cwd: string; +}): Promise { + await validateExistingWorktreeBranchName(options.cwd, options.branchName); + const needsFetch = !(await localBranchExists(options.cwd, options.branchName)); + let createdBranchOid: string | undefined; + if (needsFetch) { + try { + createdBranchOid = await fetchNewLocalBranchAtomically({ + cwd: options.cwd, + localBranchName: options.branchName, + remoteName: "origin", + remoteRef: `refs/heads/${options.branchName}`, + }); + } catch { + throw new UnknownBranchError({ branchName: options.branchName, cwd: options.cwd }); } } + + try { + if (await isBranchCheckedOut(options.cwd, options.branchName)) { + throw new BranchAlreadyCheckedOutError(options.branchName); + } + } catch (error) { + if (!needsFetch) { + throw error; + } + return rollbackCreatedWorktreeBranch( + { + cwd: options.cwd, + createdBranchName: options.branchName, + expectedOid: createdBranchOid, + }, + error, + ); + } + + return { + branchName: options.branchName, + ...(needsFetch + ? { + createdBranchName: options.branchName, + createdBranchNameBeforeWorktreeAdd: options.branchName, + createdBranchOidBeforeWorktreeAdd: createdBranchOid, + } + : {}), + metadataBaseRefName: options.branchName, + addArguments: [options.branchName], + }; +} + +async function resolveChangeRequestWorktreeSourcePlan(options: { + cwd: string; + source: ChangeRequestWorktreeSource; +}): Promise { + const { cwd, source } = options; + const localBranchCandidate = source.localBranchName ?? source.headRef; + await validateExistingWorktreeBranchName(cwd, localBranchCandidate); + const localBranchName = await resolveUniqueLocalBranchName(cwd, localBranchCandidate); + const normalizedBaseRefName = normalizeRequiredBaseBranch(source.baseRefName); + const changeRequestNumber = + source.kind === "checkout-github-pr" ? source.githubPrNumber : source.changeRequestNumber; + + let createdBranchOid: string | undefined; + try { + createdBranchOid = await fetchWorktreeCheckoutRefs({ + cwd, + localBranchName, + checkoutRefs: source.checkoutRefs ?? [ + { remoteName: "origin", remoteRef: `refs/pull/${changeRequestNumber}/head` }, + ], + }); + const shouldTrackOriginHead = source.trackOriginHead === true; + const trackingRemote = shouldTrackOriginHead + ? await tryFetchWorktreeTrackingRemote({ + cwd, + remoteName: "origin", + headRef: source.headRef, + }) + : undefined; + const remotePlan: Pick = {}; + if (source.pushRemoteUrl) { + const remoteName = `paseo-pr-${changeRequestNumber}`; + remotePlan.pushRemote = { + name: remoteName, + url: source.pushRemoteUrl, + headRef: source.headRef, + track: true, + }; + } else if (shouldTrackOriginHead && localBranchName !== source.headRef) { + const originUrl = await getWorktreeRemotePushUrl(cwd, "origin"); + if (originUrl) { + remotePlan.pushRemote = { + name: `paseo-pr-${changeRequestNumber}`, + url: originUrl, + headRef: source.headRef, + track: false, + }; + } + } + if (trackingRemote) { + remotePlan.trackingRemote = trackingRemote; + } + + return { + branchName: localBranchName, + createdBranchName: localBranchName, + createdBranchNameBeforeWorktreeAdd: localBranchName, + createdBranchOidBeforeWorktreeAdd: createdBranchOid, + metadataBaseRefName: normalizedBaseRefName, + changeRequestLookupTarget: { + headRef: source.headRef, + ...(source.headRepositoryOwner ? { headRepositoryOwner: source.headRepositoryOwner } : {}), + changeRequestNumber, + }, + addArguments: [localBranchName], + ...remotePlan, + }; + } catch (error) { + return rollbackCreatedWorktreeBranch( + { cwd, createdBranchName: localBranchName, expectedOid: createdBranchOid }, + error, + ); + } } async function configureWorktreePushRemote(options: { @@ -1471,27 +1643,22 @@ async function fetchWorktreeCheckoutRefs(options: { cwd: string; localBranchName: string; checkoutRefs: WorktreeCheckoutRef[]; -}): Promise { +}): Promise { let lastResult: | Awaited> | { stderr: string; stdout: string; exitCode: number | null } | null = null; for (const checkoutRef of options.checkoutRefs) { - lastResult = await runGitCommand( - [ - "fetch", - checkoutRef.remoteName ?? "origin", - `+${checkoutRef.remoteRef}:refs/heads/${options.localBranchName}`, - "--force", - ], - { + try { + return await fetchNewLocalBranchAtomically({ cwd: options.cwd, - timeout: 120_000, - acceptExitCodes: [0, 1, 128], - }, - ); - if (lastResult.exitCode === 0) { - return; + localBranchName: options.localBranchName, + remoteName: checkoutRef.remoteName ?? "origin", + remoteRef: checkoutRef.remoteRef, + }); + } catch (error) { + lastResult = + error instanceof Error ? { stderr: error.message, stdout: "", exitCode: 1 } : null; } } const attemptedRefs = options.checkoutRefs @@ -1502,6 +1669,35 @@ async function fetchWorktreeCheckoutRefs(options: { ); } +async function fetchNewLocalBranchAtomically(options: { + cwd: string; + localBranchName: string; + remoteName: string; + remoteRef: string; +}): Promise { + const temporaryRef = `refs/paseo/worktree-fetch/${randomUUID()}`; + try { + await runGitCommand( + ["fetch", options.remoteName, `${options.remoteRef}:${temporaryRef}`, "--force"], + { cwd: options.cwd, timeout: 120_000 }, + ); + const { stdout } = await runGitCommand(["rev-parse", "--verify", temporaryRef], { + cwd: options.cwd, + }); + const oid = stdout.trim(); + const nullOid = "0".repeat(oid.length); + await runGitCommand(["update-ref", `refs/heads/${options.localBranchName}`, oid, nullOid], { + cwd: options.cwd, + }); + return oid; + } finally { + await runGitCommand(["update-ref", "-d", temporaryRef], { + cwd: options.cwd, + acceptExitCodes: [0, 1, 128], + }); + } +} + async function tryFetchWorktreeTrackingRemote(options: { cwd: string; remoteName: string; diff --git a/public-docs/worktrees.md b/public-docs/worktrees.md index bdc8a24f2..90b49b4b1 100644 --- a/public-docs/worktrees.md +++ b/public-docs/worktrees.md @@ -110,6 +110,46 @@ Both fields accept a multiline shell script or an array of commands; commands ru Commands run with the worktree as `cwd`. Use `$PASEO_SOURCE_CHECKOUT_PATH` to reach files in the original checkout (untracked config, local caches, etc). +## .worktreeinclude + +Use a root-level .worktreeinclude to materialize local source-checkout files before +worktree.setup runs. Each path is relative to the source checkout. + + # Copy is the default. + .env.local + .cache/** + + # Modes can also be explicit. + symlink node_modules + copy .tool-state/** + +Each line is `[copy|symlink] `; the mode is optional and defaults to `copy`. Blank lines +and whole-line comments are ignored. A single star matches within a path segment and a double +star matches recursively; a directory ending in /\*\* materializes that directory as one +recursive entry. Absolute paths, parent-directory paths, and .git paths are rejected. + +Copy entries are independent snapshots: a copied file or directory replaces an existing path on +a later materialization. Symlink entries point directly at +the live source file or directory, so changes through either path affect the same data. Paseo +does not replace an existing file, directory, or different link with a symlink. + +Entries must resolve to regular files or directories. A top-level source symbolic link is +dereferenced only when its canonical target remains inside the source checkout: `copy` snapshots +that target and `symlink` links directly to it. Directory snapshots reject nested symbolic links +so Paseo never writes through an unexpected path. A symlinked directory intentionally exposes its +live source contents. + +Prefer paths ignored by the target branch. For a symlinked directory, use an ignore rule without +a trailing slash (for example, node_modules, not node_modules/), because Git treats the link +itself as a file. Unignored materialized paths appear in git status. + +On Windows, Paseo uses junctions for local directories. File links and network-directory links +require Windows symbolic-link support. It never silently copies an explicit `symlink ` +entry; enable Developer Mode or switch that entry to `copy ` if link creation fails. + +Archiving removes only the worktree's links, not their source targets. If the source path is +later moved or deleted, a symlink becomes broken; Paseo does not repair it automatically. + ## Scripts and services `scripts` are named commands you can run inside a worktree on demand. Mark one as a _service_ and Paseo supervises it as a long-running process, assigns it a port, and routes HTTP traffic to it through the daemon's reverse proxy.