Compare commits

..
Author SHA1 Message Date
paseo-bot[bot]andRyan Swift 7ebc7fff8d fix(usage): show Copilot premium request quota
The usage source discarded quota snapshots, leaving the card with only a
plan and reset date. Publish finite premium usage through the existing
window interface and retain its remaining request count.

Co-authored-by: Ryan Swift <ryan@mlh.io>
2026-11-01 19:31:51 +00:00
paseo-bot[bot] 04fe4be40f fix(claude): retain configured model metadata before a turn (#8465)
Rebuilding an idle session cleared its observed model and published null despite retaining an explicit model in its configuration. Fall back to that configuration until the next turn supplies an observed model.

Co-authored-by: paseo-bot[bot] <266920839+paseo-bot[bot]@users.noreply.github.com>
2026-11-01 18:59:04 +00:00
paseo-bot[bot] 2d75cc80b2 test(server): cover routed permissions in one host journey (#8486)
Co-authored-by: paseo-bot[bot] <266920839+paseo-bot[bot]@users.noreply.github.com>
2026-11-01 18:34:57 +00:00
paseo-bot[bot]andDominic Anaeto bfac132920 Stop Git polling while workspace folders are missing (#8462)
* fix(server): stop polling git for a workspace whose folder is gone

When a workspace's folder disappears together with its project (the
repository deleted, a volume unmounted), reconciliation leaves the
workspace registered, and the daemon kept polling it: a failed git
command and a warning about once a minute, and a working-tree watcher
retry every minute or two, for as long as the daemon ran.

The degraded poll now checks that the workspace folder exists before
refreshing. A workspace whose folder is gone is suspended once: it is
detached from its repository and working-tree targets, its timers stop,
and one warning is logged. A worktree that shared a repository target
hands the repository's polling and background fetch to a surviving
worktree. Nothing is archived. A working-tree watch that an open diff
still holds skips its consumer notifications and watcher recovery while
its folder is missing.

While suspended, the workspace runs no git. A stat every 60 seconds, a
new subscription, a forced snapshot or a refresh request checks whether
the folder is back; when it is, the workspace reloads its git facts,
refreshes, and rebuilds its observation.

Fixes #7856

* fix(server): keep missing workspaces suspended during refresh

Admit Git refreshes only after the folder has returned. Cover missing-folder polling, explicit refreshes, and restoration with real Git and the Linux observer.

Co-authored-by: Dominic Anaeto <dominic.anaeto@gmail.com>

* fix(server): preserve workspace snapshots through suspension

Drop queued refreshes and withhold publication when a folder disappears during an active read. Wait for that read before restoring observation so its stale result cannot overwrite the recovered snapshot.

---------

Co-authored-by: Dominic Anaeto <dominic.anaeto@gmail.com>
Co-authored-by: paseo-bot[bot] <266920839+paseo-bot[bot]@users.noreply.github.com>
2026-11-01 18:02:26 +00:00
paseo-bot[bot]andDominic Anaeto 34f996b32a Preserve pending questions and approvals after peer messages (#8453)
* fix(server): keep a recipient's pending permission when another agent messages it

A message from another agent (send_agent_message with sourceAgentId) was
enqueued with clearPendingPermissions: true, so it superseded the
recipient's pending permission request and the request was denied before
a human saw it. Agent-sourced messages now leave pending permissions
alone. Typed messages and sourceless `paseo send` still clear them.

Fixes #7602

* test(server): verify peer messages preserve answerable questions

Exercise message delivery through the daemon and provider interfaces so the regression checks that the question survives and its answer reaches the provider. Retain the human-steer cancellation behavior.

Co-authored-by: Dominic Anaeto <dominic.anaeto@gmail.com>

* test(terminal): make runtime copy fixture independent of umask

Set the source helper permissions after creation so the copy test measures preservation instead of inheriting the shell file-creation mask. The existing test fails under umask 077 without this setup correction.

Fixes #8175

---------

Co-authored-by: Dominic Anaeto <dominic.anaeto@gmail.com>
Co-authored-by: paseo-bot[bot] <266920839+paseo-bot[bot]@users.noreply.github.com>
2026-11-01 17:11:42 +00:00
13 changed files with 1208 additions and 293 deletions
@@ -1,190 +0,0 @@
import { expect, test } from "vitest";
import { createTestLogger } from "../../../../test-utils/test-logger.js";
import { collectSessionTurnEvents } from "../test-utils/session-stream-adapter.js";
import { ClaudeAgentClient } from "./agent.js";
import type { ClaudeQueryFactory } from "./query.js";
const LIMIT = "You've hit your session limit · resets 5:20pm (Asia/Singapore)";
interface ScriptedTurn {
// An API error reply as Claude Code writes it: a synthetic assistant message with `error`.
errorReply?: string;
// Assistant text the model wrote before or after it.
before?: string;
after?: string;
isError: boolean;
usage?: {
inputTokens: number;
cachedInputTokens: number;
outputTokens: number;
totalCostUsd: number;
};
}
// A query dependency that speaks Claude Code's stream interface, one scripted turn per prompt.
function scriptedQuery(turns: ScriptedTurn[]): ClaudeQueryFactory {
return ({ prompt }) => {
let index = 0;
const assistant = (id: string, text: string, error?: string) => ({
type: "assistant",
session_id: "error-session",
parent_tool_use_id: null,
uuid: id,
message: {
id,
role: "assistant",
model: error ? "<synthetic>" : "claude-opus-5-5",
content: [{ type: "text", text }],
stop_reason: "stop_sequence",
usage: { input_tokens: 0, output_tokens: 0 },
},
...(error ? { error } : {}),
});
const messages = (async function* () {
if (typeof prompt === "string") throw new Error("Expected streaming input");
for await (const _message of prompt) {
const turn = turns[index];
index += 1;
yield {
type: "system",
subtype: "init",
session_id: "error-session",
permissionMode: "default",
model: "claude-opus-5-5",
};
if (turn.before) yield assistant(`before-${index}`, turn.before);
if (turn.errorReply) yield assistant(`error-${index}`, turn.errorReply, "rate_limit");
if (turn.after) yield assistant(`after-${index}`, turn.after);
yield {
type: "result",
subtype: "success",
session_id: "error-session",
duration_ms: 1,
duration_api_ms: 0,
is_error: turn.isError,
num_turns: 1,
result: turn.isError ? (turn.errorReply ?? "") : (turn.after ?? turn.errorReply ?? ""),
stop_reason: "stop_sequence",
total_cost_usd: turn.usage?.totalCostUsd ?? 0,
usage: {
input_tokens: turn.usage?.inputTokens ?? 0,
output_tokens: turn.usage?.outputTokens ?? 0,
cache_read_input_tokens: turn.usage?.cachedInputTokens ?? 0,
},
modelUsage: {},
permission_denials: [],
uuid: `result-${index}`,
};
}
})();
return Object.assign(messages, {
close: () => {
void messages.return();
},
interrupt: async () => undefined,
setModel: async () => undefined,
setPermissionMode: async () => undefined,
supportedModels: async () => [],
supportedCommands: async () => [],
}) as ReturnType<ClaudeQueryFactory>;
};
}
async function runTurns(turns: ScriptedTurn[]) {
const client = new ClaudeAgentClient({
logger: createTestLogger(),
queryFactory: scriptedQuery(turns),
resolveBinary: async () => process.execPath,
});
const session = await client.createSession({
provider: "claude",
cwd: process.cwd(),
});
try {
const results = [];
for (const [index] of turns.entries()) {
results.push(await collectSessionTurnEvents(session, `turn ${index + 1}`));
}
return results;
} finally {
await session.close();
}
}
function assistantTexts(events: Awaited<ReturnType<typeof runTurns>>[number]): string[] {
return events.flatMap((event) =>
event.type === "timeline" && event.item.type === "assistant_message" ? [event.item.text] : [],
);
}
function terminal(events: Awaited<ReturnType<typeof runTurns>>[number]) {
return events.find(
(event) =>
event.type === "turn_completed" ||
event.type === "turn_failed" ||
event.type === "turn_canceled",
);
}
test("a turn Claude Code ends on an API error fails with the error, shown once", async () => {
const [limited, overloaded, next] = await runTurns([
{ errorReply: LIMIT, isError: true },
{
before: "Checking the deploy doc.",
errorReply: "API Error: 529 Overloaded",
isError: true,
},
{ after: "Done.", isError: false },
]);
expect(terminal(limited)).toMatchObject({
type: "turn_failed",
error: LIMIT,
});
expect(assistantTexts(limited)).toEqual([]);
expect(terminal(overloaded)).toMatchObject({
type: "turn_failed",
error: "API Error: 529 Overloaded",
});
expect(assistantTexts(overloaded)).toEqual(["Checking the deploy doc."]);
// The session keeps working after a failed turn.
expect(terminal(next)).toMatchObject({ type: "turn_completed" });
expect(assistantTexts(next)).toEqual(["Done."]);
});
test("a failed turn retains its final token counts and cost", async () => {
const usage = {
inputTokens: 120,
cachedInputTokens: 80,
outputTokens: 30,
totalCostUsd: 0.012,
};
const [failed] = await runTurns([
{ before: "Checking the deploy doc.", errorReply: LIMIT, isError: true, usage },
]);
expect(failed).toContainEqual(
expect.objectContaining({ type: "usage_updated", usage: expect.objectContaining(usage) }),
);
expect(terminal(failed)).toMatchObject({ type: "turn_failed", error: LIMIT });
expect(assistantTexts(failed)).toEqual(["Checking the deploy doc."]);
});
test("an error reply the turn moves past is still shown", async () => {
const [recovered, completed] = await runTurns([
{
errorReply: "API Error: 529 Overloaded",
after: "Recovered.",
isError: false,
},
{ errorReply: LIMIT, isError: false },
]);
expect(terminal(recovered)).toMatchObject({ type: "turn_completed" });
expect(assistantTexts(recovered)).toEqual(["API Error: 529 Overloaded", "Recovered."]);
expect(terminal(completed)).toMatchObject({ type: "turn_completed" });
expect(assistantTexts(completed)).toEqual([LIMIT]);
});
@@ -0,0 +1,48 @@
import pino from "pino";
import { describe, expect, test } from "vitest";
import { ClaudeAgentClient } from "./agent.js";
describe("Claude runtime info before a turn", () => {
const client = new ClaudeAgentClient({ logger: pino({ level: "silent" }) });
test("reports the explicitly configured model before the first turn", async () => {
const session = await client.createSession({
provider: "claude",
cwd: process.cwd(),
model: "claude-haiku-5-5",
});
try {
expect(await session.getRuntimeInfo()).toMatchObject({ model: "claude-haiku-5-5" });
} finally {
await session.close();
}
});
test("reports the persisted configured model after rebuilding an idle session", async () => {
const session = await client.resumeSession({
provider: "claude",
sessionId: "runtime-model-before-turn",
metadata: { cwd: process.cwd(), model: "claude-haiku-5-5" },
});
try {
expect(await session.getRuntimeInfo()).toMatchObject({
sessionId: "runtime-model-before-turn",
model: "claude-haiku-5-5",
});
expect((await session.getRuntimeInfo()).model).toBe("claude-haiku-5-5");
expect((await session.getRuntimeInfo()).extra?.runtimeModel).toBeUndefined();
} finally {
await session.close();
}
});
test("leaves the model unknown when Claude will choose its default", async () => {
const session = await client.createSession({ provider: "claude", cwd: process.cwd() });
try {
expect((await session.getRuntimeInfo()).model).toBeNull();
} finally {
await session.close();
}
});
});
@@ -2389,12 +2389,6 @@ class ClaudeAgentSession implements AgentSession {
private mainTurnInFlight = false;
private foregroundHasVisibleActivity = false;
private activeTurnHasAssistantText = false;
/**
* The reply Claude Code writes when a turn stops on an API error ("You've hit your session
* limit", "API Error: 529 ..."), held back until the result says whether the turn failed: a
* failed turn shows the same text as its error, so showing both would repeat it.
*/
private withheldErrorReply: AgentTimelineItem[] = [];
private readonly contextUsage: ClaudeContextUsageState;
private userMessageIds: string[] = [];
private readonly emittedUserMessageIds = new Set<string>();
@@ -2466,7 +2460,7 @@ class ClaudeAgentSession implements AgentSession {
const info: AgentRuntimeInfo = {
provider: "claude",
sessionId: this.claudeSessionId,
model: this.lastOptionsModel,
model: this.lastOptionsModel ?? this.config.model ?? null,
modeId: this.currentMode ?? null,
...(this.lastRuntimeModel
? {
@@ -2537,7 +2531,6 @@ class ClaudeAgentSession implements AgentSession {
this.activeForegroundTurnId = turnId;
this.foregroundHasVisibleActivity = false;
this.activeTurnHasAssistantText = false;
this.withheldErrorReply = [];
this.contextUsage.beginTurn();
this.transitionTurnState("foreground", "foreground turn started");
this.clearRecentStderr();
@@ -4426,21 +4419,6 @@ class ClaudeAgentSession implements AgentSession {
}) satisfies AgentStreamEvent,
);
if (!readClaudeParentToolUseId(message)) {
if (message.type === "assistant" && message.error) {
this.withheldErrorReply.push(...assistantTimelineEvents.map((event) => event.item));
return messageEvents;
}
if (
message.type === "assistant" ||
message.type === "user" ||
message.type === "stream_event"
) {
const released: AgentStreamEvent[] = [];
this.releaseWithheldErrorReply(released);
return [...released, ...messageEvents, ...assistantTimelineEvents];
}
}
return [...messageEvents, ...assistantTimelineEvents];
}
@@ -4988,23 +4966,6 @@ class ClaudeAgentSession implements AgentSession {
events: AgentStreamEvent[],
): void {
const usage = this.convertUsage(message);
// Claude Code reports a turn that stopped on an API error (a usage limit, an overloaded or
// rejected request) as `success` with `is_error`, carrying the error text in `result`.
if (message.subtype === "success" && message.is_error) {
const withheld = this.withheldErrorReply;
this.withheldErrorReply = [];
const resultText = typeof message.result === "string" ? message.result.trim() : "";
const withheldText = withheld
.map((item) => (item.type === "assistant_message" ? item.text : ""))
.join("\n")
.trim();
if (usage) events.push({ type: "usage_updated", provider: "claude", usage });
events.push(...this.sidechainTracker.finishAll("failed"));
events.push(this.buildTurnFailedEvent(resultText || withheldText || "Claude run failed"));
return;
}
const releasedErrorReply = this.withheldErrorReply.length > 0;
this.releaseWithheldErrorReply(events);
if (message.subtype === "success") {
events.push(...this.sidechainTracker.finishAll("completed"));
// Built-in slash commands (e.g. /voice, /usage, "Unknown command: …")
@@ -5014,12 +4975,7 @@ class ClaudeAgentSession implements AgentSession {
// accounting from provider gateways does not duplicate streamed output.
const resultText = typeof message.result === "string" ? message.result.trim() : "";
const outputTokens = message.usage?.output_tokens;
if (
resultText.length > 0 &&
outputTokens === 0 &&
!this.activeTurnHasAssistantText &&
!releasedErrorReply
) {
if (resultText.length > 0 && outputTokens === 0 && !this.activeTurnHasAssistantText) {
events.push({
type: "timeline",
provider: "claude",
@@ -5041,14 +4997,6 @@ class ClaudeAgentSession implements AgentSession {
events.push(this.buildTurnFailedEvent(errorMessage));
}
private releaseWithheldErrorReply(events: AgentStreamEvent[]): void {
if (this.withheldErrorReply.length === 0) return;
for (const item of this.withheldErrorReply) {
events.push({ type: "timeline", item, provider: "claude" });
}
this.withheldErrorReply = [];
}
private createClaudeSessionChangedNotice(
oldSessionId: string,
newSessionId: string,
@@ -183,7 +183,7 @@ test("ordinary routed APIs isolate scopes and enforce one-hop, disable and origi
await paseo.dispose();
});
test("routed authority is bounded by the caller, with isolated simultaneous sessions", async () => {
test("routed authority isolates simultaneous sessions and preserves ordinary permission RPC errors", async () => {
const b = await daemon();
const a = await daemon();
const owner = await api(a);
@@ -210,33 +210,16 @@ test("routed authority is bounded by the caller, with isolated simultaneous sess
code: "permission_denied",
});
expect((await owner.paseo.hosts.list()).hosts).toHaveLength(1);
await remote.dispose();
});
test("SDK host clients preserve ordinary permission RPC errors", async () => {
const b = await daemon();
const a = await daemon();
const owner = await api(a);
await owner.paseo.hosts.add({ connection: `127.0.0.1:${b.port}` });
const serverId = (await online(owner.paseo)).hosts[0]!.serverId!;
const clients = [a, b].map((host) =>
createHostClient(`127.0.0.1:${host.port}`, {
clientId: crypto.randomUUID(),
sessionPermissions: ["daemon.read", "workspace.read"],
reconnect: { enabled: false },
}),
);
for (const client of clients) {
cleanup.push(() => client.close());
await client.connect();
}
const origin = createPaseoApi(clients[0]!);
const direct = createPaseoApi(clients[1]!);
const remote = await origin.hosts.client(serverId);
const directClient = createHostClient(`127.0.0.1:${b.port}`, {
clientId: crypto.randomUUID(),
sessionPermissions: ["daemon.read", "workspace.read"],
reconnect: { enabled: false },
});
await directClient.connect();
const direct = createPaseoApi(directClient);
cleanup.push(async () => {
await remote.dispose();
await direct.dispose();
await origin.dispose();
await directClient.close();
});
const input = { source: { kind: "directory" as const, path: b.staticDir } };
const directError = (await direct.workspaces.create(input).catch((error: unknown) => error)) as {
@@ -247,6 +230,7 @@ test("SDK host clients preserve ordinary permission RPC errors", async () => {
};
expect(directError.code).toBe("access_denied");
expect(remoteError.code).toBe(directError.code);
await remote.dispose();
});
test("distinct origin sessions cannot collide through a caller-selected route scope, including terminal binary output", async () => {
@@ -0,0 +1,129 @@
import { mkdir, mkdtemp, rm } from "node:fs/promises";
import path from "node:path";
import type { FormInfo } from "@opencode/client";
import { expect, test } from "vitest";
import { createTestLogger } from "../../test-utils/test-logger.js";
import { OpenCodeV2AgentClient } from "../agent/providers/opencode/v2/agent.js";
import { V2Harness } from "../agent/providers/opencode/test-utils/v2-harness.js";
import { createTestAgentClients } from "../test-utils/fake-agent-client.js";
import { createTestPaseoDaemon } from "../test-utils/paseo-daemon.js";
import { DaemonClient } from "../test-utils/daemon-client.js";
async function createQuestionHarness() {
const artifacts = path.resolve(".artifacts");
await mkdir(artifacts, { recursive: true });
const root = await mkdtemp(path.join(artifacts, "peer-permissions-"));
const runtime = new V2Harness();
runtime.autoComplete = false;
runtime.api.model.list = async () => ({ location: runtime.info.location, data: [] });
runtime.api.provider.list = async () => ({
location: runtime.info.location,
data: [{ id: "test", name: "Test", activation: "enabled", package: "test" }],
});
let form: FormInfo | null = null;
const answers: Array<Record<string, unknown>> = [];
runtime.api.session.form.list = async () => (form ? [form] : []);
runtime.api.session.form.cancel = async () => {
form = null;
runtime.finishExecution("interrupted");
};
runtime.api.session.form.reply = async ({ answer }) => {
answers.push(answer);
form = null;
runtime.finishExecution();
};
// A steer is admitted to the existing execution, rather than starting another one.
runtime.prompt = async (input) => {
runtime.prompts.push(input.text);
if (runtime.prompts.length === 1) runtime.startExecution();
};
const provider = new OpenCodeV2AgentClient({
logger: createTestLogger(),
runtime: runtime.runtime,
});
const daemon = await createTestPaseoDaemon({
agentClients: { ...createTestAgentClients(), opencode: provider },
paseoHomeRoot: path.join(root, "daemon"),
staticDir: path.join(root, "static"),
cleanup: false,
});
const client = new DaemonClient({ url: `ws://127.0.0.1:${daemon.port}/ws` });
try {
await client.connect();
await client.fetchAgents({ subscribe: {} });
const recipient = await client.createAgent({
provider: "opencode",
cwd: root,
title: "Waiting for a human answer",
});
const sender = await client.createAgent({ provider: "claude", cwd: root });
await client.sendMessage(recipient.id, "Ask which color to use");
await expect.poll(() => runtime.prompts).toHaveLength(1);
form = {
id: "color-question",
sessionID: "session",
title: "Color",
fields: [{ key: "color", title: "Color", type: "string" }],
};
runtime.push({ id: "question-created", created: 2, type: "form.created", data: { form } });
await client.waitForAgentUpsert(recipient.id, (agent) =>
(agent.pendingPermissions ?? []).some((request) => request.id === "color-question"),
);
return {
client,
recipient,
sender,
answers,
async close() {
await client.close();
await daemon.close();
await rm(root, { recursive: true, force: true });
},
};
} catch (error) {
await client.close();
await daemon.close();
await rm(root, { recursive: true, force: true });
throw error;
}
}
test("an agent-sourced steer leaves the question pending and answerable", async () => {
const { client, recipient, sender, answers, close } = await createQuestionHarness();
try {
await client.sendAgentMessage(recipient.id, "Peer status check", {
sourceAgentId: sender.id,
});
// The public wait includes admitted message delivery before reporting permission status.
const waiting = await client.waitForFinish(recipient.id);
expect(waiting.status).toBe("permission");
expect(waiting.final?.pendingPermissions).toMatchObject([
{ id: "color-question", kind: "question" },
]);
await client.respondToPermission(recipient.id, "color-question", {
behavior: "allow",
updatedInput: { answers: { Color: "Blue" } },
});
await client.waitForAgentUpsert(
recipient.id,
(agent) => (agent.pendingPermissions ?? []).length === 0 && agent.status === "idle",
);
expect(answers).toEqual([{ color: "Blue" }]);
} finally {
await close();
}
});
test("a human steer still dismisses the question", async () => {
const { client, recipient, answers, close } = await createQuestionHarness();
try {
await client.sendAgentMessage(recipient.id, "Use red instead", { activeTurnBehavior: "steer" });
const finished = await client.waitForFinish(recipient.id);
expect(finished.status).toBe("idle");
expect(finished.final?.pendingPermissions ?? []).toEqual([]);
expect(answers).toEqual([]);
} finally {
await close();
}
});
+1 -1
View File
@@ -8318,7 +8318,7 @@ export class Session {
},
activeTurnBehavior: msg.activeTurnBehavior ?? "steer",
sessionMode: msg.sessionMode,
clearPendingPermissions: true,
clearPendingPermissions: false,
});
deliveryStatus = "queued";
queued();
@@ -1,3 +1,4 @@
import { flushGitCommandTrace } from "../utils/git-command-trace.js";
import { createWorkspaceRuntimeStore } from "./workspace-runtime-store/index.js";
import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync } from "node:fs";
import { execFileSync } from "node:child_process";
@@ -730,3 +731,103 @@ test("Git event identity retains a symlink's literal target before canonicalizin
rmSync(root, { recursive: true, force: true });
}
});
test.skipIf(process.platform !== "linux")(
"missing folders stop degraded Git polling and explicit refreshes, then recover",
async () => {
const artifacts = path.resolve(".artifacts");
mkdirSync(artifacts, { recursive: true });
const root = mkdtempSync(path.join(artifacts, "workspace-git-missing-"));
const cwd = path.join(root, "repo");
const initializeRepo = (branch: string) => {
mkdirSync(cwd);
execFileSync("git", ["init", "-b", branch], { cwd, stdio: "pipe" });
writeFileSync(path.join(cwd, "README.md"), "fixture\n");
execFileSync("git", ["add", "."], { cwd, stdio: "pipe" });
execFileSync(
"git",
[
"-c",
"user.name=Test",
"-c",
"user.email=test@example.invalid",
"-c",
"commit.gpgsign=false",
"commit",
"-m",
"fixture",
],
{ cwd, stdio: "pipe" },
);
};
initializeRepo("main");
// Exceed the real Linux observer's directory limit to enter degraded polling.
// No watcher errors or Git responses are injected.
for (let index = 0; index < 5100; index++) {
mkdirSync(path.join(cwd, `generated-${index}`));
}
const trace = path.join(root, "git.jsonl");
writeFileSync(trace, "");
const previousTrace = process.env.PASEO_GIT_TRACE_FILE;
process.env.PASEO_GIT_TRACE_FILE = trace;
let degraded = false;
let suspended = false;
const logger = pino(
{ level: "warn" },
{
write(line) {
const message = JSON.parse(line).msg;
if (message === "Failed to start working tree watcher; using degraded polling")
degraded = true;
if (
message ===
"Workspace folder no longer exists; suspended git observation until it is back"
)
suspended = true;
},
},
);
const service = new WorkspaceGitServiceImpl({ logger, paseoHome: path.join(root, "home") });
const subscription = service.registerWorkspace({ cwd }, () => {});
const commandCount = async () => {
await flushGitCommandTrace();
return readFileSync(trace, "utf8").trim().split("\n").filter(Boolean).length;
};
try {
await expect
.poll(() => degraded && service.getMetrics().workspaceRefreshInFlightCount === 0, {
timeout: 15_000,
})
.toBe(true);
rmSync(cwd, { recursive: true, force: true });
await expect.poll(() => suspended, { timeout: 15_000 }).toBe(true);
await expect.poll(() => service.getMetrics().workspaceRefreshInFlightCount).toBe(0);
const commandsWhileMissing = await commandCount();
await service.getSnapshot(cwd, {
force: true,
includeForge: false,
reason: "refresh-button",
});
await service.refresh(cwd);
expect(await commandCount()).toBe(commandsWhileMissing);
await new Promise((resolve) => setTimeout(resolve, 16_000));
expect(await commandCount()).toBe(commandsWhileMissing);
initializeRepo("restored");
const snapshot = await service.getSnapshot(cwd, {
force: true,
includeForge: false,
reason: "refresh-button",
});
expect(snapshot.git).toMatchObject({ isGit: true, currentBranch: "restored" });
await expect.poll(() => service.getMetrics().repositoryWorkspaceLinkCount).toBe(1);
} finally {
subscription.unsubscribe();
await service.dispose();
if (previousTrace === undefined) delete process.env.PASEO_GIT_TRACE_FILE;
else process.env.PASEO_GIT_TRACE_FILE = previousTrace;
rmSync(root, { recursive: true, force: true });
}
},
60_000,
);
@@ -244,6 +244,7 @@ function createService(
filterEvents: (events) => events,
verify: vi.fn(async () => {}),
})),
workspaceFolderExists: vi.fn(async () => true),
...overrides,
} as never,
});
@@ -1,5 +1,7 @@
import { createWorkspaceRuntimeStore } from "./workspace-runtime-store/index.js";
import { afterEach, beforeEach, describe, expect, test, vi } from "vitest";
import { existsSync } from "node:fs";
import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises";
import os from "node:os";
import path, { join } from "node:path";
import type pino from "pino";
@@ -10,11 +12,14 @@ import type {
PullRequestStatusResult,
} from "../utils/checkout-git.js";
import {
BACKGROUND_GIT_FETCH_INTERVAL_MS,
WORKSPACE_GIT_OBSERVATION_REENSURE_INTERVAL_MS,
WORKSPACE_GIT_OBSERVATION_SETUP_CONCURRENCY,
WORKSPACE_GIT_REFRESH_CONCURRENCY,
WORKSPACE_GIT_WATCHER_SUBSCRIBE_TIMEOUT_MS,
WorkspaceGitServiceImpl,
type WorkspaceGitRuntimeSnapshot,
workspaceFolderExists,
} from "./workspace-git-service.js";
const REPO_CWD = path.resolve("/tmp/repo");
@@ -23,6 +28,7 @@ function createLogger() {
const logger = {
child: () => logger,
debug: vi.fn(),
info: vi.fn(),
warn: vi.fn(),
};
return logger;
@@ -251,7 +257,7 @@ function createGitHubServiceStub(): ForgeService {
interface CreateServiceTestOptions {
subscribe?: ReturnType<typeof vi.fn>;
getCheckoutStatus?: ReturnType<typeof vi.fn>;
getCheckoutSnapshotFacts?: ReturnType<typeof vi.fn>;
getCheckoutSnapshotFacts?: (cwd: string) => Promise<CheckoutSnapshotFacts>;
getCheckoutShortstat?: ReturnType<typeof vi.fn>;
getCheckoutWorktreeState?: ReturnType<typeof vi.fn>;
getPullRequestStatus?: ReturnType<typeof vi.fn>;
@@ -261,6 +267,7 @@ interface CreateServiceTestOptions {
runGitFetch?: ReturnType<typeof vi.fn>;
runGitCommand?: ReturnType<typeof vi.fn>;
getWorkspaceGitSelfHealPhaseMs?: (cwd: string) => number;
workspaceFolderExists?: (cwd: string) => Promise<boolean>;
now?: () => Date;
}
@@ -294,6 +301,7 @@ function buildDefaultTestServiceDeps() {
signal: null,
})),
getWorkspaceGitSelfHealPhaseMs: vi.fn(() => 30_000),
workspaceFolderExists: vi.fn(async () => true),
createWatcherLivenessCanary: vi.fn(() => ({
path: "",
filterEvents: (events) => events,
@@ -324,6 +332,145 @@ function createService(options?: CreateServiceTestOptions) {
});
}
type WatchCallback = (error: Error | null, events: Array<{ path: string; type: "update" }>) => void;
interface FolderBackedGitOptions {
facts?: (cwd: string) => CheckoutSnapshotFacts;
unwatchableDirectories?: string[];
heldWatch?: { directory: string; release: Promise<void> };
}
function createFolderBackedGit(options?: FolderBackedGitOptions) {
const watchCallbacks = new Map<string, WatchCallback>();
const subscribe = vi.fn(async (directory: string, callback: WatchCallback) => {
if (!existsSync(directory) || options?.unwatchableDirectories?.includes(directory)) {
throw new Error(`Cannot watch ${directory}`);
}
if (options?.heldWatch?.directory === directory) {
await options.heldWatch.release;
}
watchCallbacks.set(directory, callback);
return createAsyncSubscription();
});
const getCheckoutSnapshotFacts = vi.fn(
async (cwd: string): Promise<CheckoutSnapshotFacts> =>
existsSync(cwd) ? (options?.facts ?? createCheckoutSnapshotFacts)(cwd) : { isGit: false },
);
const getCheckoutStatus = vi.fn(async (cwd: string) =>
existsSync(cwd) ? createCheckoutStatus(cwd) : ({ isGit: false } as const),
);
const getCheckoutShortstat = vi.fn(async (_cwd: string) => ({ additions: 1, deletions: 0 }));
const getCheckoutWorktreeState = vi.fn(async (_cwd: string) => ({
isDirty: false,
diffStat: { additions: 1, deletions: 0 },
}));
const runGitCommand = vi.fn(async (args: string[], runOptions: { cwd: string }) => ({
stdout: args[0] === "rev-parse" ? `${runOptions.cwd}\n` : "",
stderr: "",
truncated: false,
exitCode: 0,
signal: null,
}));
const runGitFetch = vi.fn(async (_cwd: string) => ({ changes: [], error: null }));
const callsIn = (fake: ReturnType<typeof vi.fn>, cwd: string) =>
fake.mock.calls.filter(([calledCwd]) => calledCwd === cwd).length;
return {
deps: {
subscribe,
getCheckoutSnapshotFacts,
getCheckoutStatus,
getCheckoutShortstat,
getCheckoutWorktreeState,
runGitCommand,
runGitFetch,
workspaceFolderExists: async (cwd: string) => existsSync(cwd),
now: () => new Date(),
},
watchCallbackFor(directory: string): WatchCallback {
const callback = watchCallbacks.get(directory);
if (!callback) {
throw new Error(`No watcher was started for ${directory}`);
}
return callback;
},
watchAttemptsFor: (directory: string) => callsIn(subscribe, directory),
pollsOf: (cwd: string) => callsIn(getCheckoutShortstat, cwd),
fetchesIn: (cwd: string) => callsIn(runGitFetch, cwd),
gitCommandsIn(cwd: string): number {
const cwdCommands = [
getCheckoutSnapshotFacts,
getCheckoutStatus,
getCheckoutShortstat,
getCheckoutWorktreeState,
runGitFetch,
].reduce((count, fake) => count + callsIn(fake, cwd), 0);
const commandsRunInCwd = runGitCommand.mock.calls.filter(
([, runOptions]) => runOptions.cwd === cwd,
).length;
return cwdCommands + commandsRunInCwd;
},
};
}
function linkedWorktreeFacts(gitCommonDir: string) {
return (cwd: string): CheckoutSnapshotFacts => ({
...createCheckoutSnapshotFacts(cwd),
absoluteGitDir: join(gitCommonDir, "worktrees", path.basename(cwd)),
gitCommonDir,
});
}
const SETTLE_ATTEMPTS = 1_000;
async function settle(assertion: () => void): Promise<void> {
let lastFailure: unknown = null;
for (let attempt = 0; attempt < SETTLE_ATTEMPTS; attempt += 1) {
try {
assertion();
return;
} catch (failure) {
lastFailure = failure;
}
await vi.advanceTimersByTimeAsync(0);
}
throw lastFailure;
}
async function waitForRefreshesToSettle(service: WorkspaceGitServiceImpl): Promise<void> {
await settle(() => {
expect(service.getMetrics().workspaceRefreshInFlightCount).toBe(0);
});
}
async function waitForObservationToSettle(
service: WorkspaceGitServiceImpl,
repositoryWorkspaceLinkCount: number,
): Promise<void> {
await settle(() => {
const metrics = service.getMetrics();
expect(metrics.repositoryWorkspaceLinkCount).toBe(repositoryWorkspaceLinkCount);
expect(metrics.workspaceObservationSetupInFlightCount).toBe(0);
expect(metrics.workspaceRefreshInFlightCount).toBe(0);
});
}
async function advanceThroughDegradedPolls(service: WorkspaceGitServiceImpl): Promise<void> {
for (const delayMs of [5_000, 10_000, 20_000, 40_000, 60_000, 60_000]) {
await vi.advanceTimersByTimeAsync(delayMs);
await waitForRefreshesToSettle(service);
}
}
async function advanceThroughWorkingTreeRecoveryAttempts(
service: WorkspaceGitServiceImpl,
): Promise<void> {
for (const delayMs of [30_000, 60_000, 120_000]) {
await vi.advanceTimersByTimeAsync(delayMs);
await waitForRefreshesToSettle(service);
}
}
describe("WorkspaceGitServiceImpl", () => {
beforeEach(() => {
vi.useFakeTimers();
@@ -1553,3 +1700,460 @@ describe("WorkspaceGitServiceImpl", () => {
service.dispose();
});
});
describe("WorkspaceGitServiceImpl with workspace folders that no longer exist", () => {
const temporaryFolders: string[] = [];
async function createTemporaryFolder(): Promise<string> {
const folder = await mkdtemp(join(os.tmpdir(), "paseo-workspace-git-"));
temporaryFolders.push(folder);
return folder;
}
beforeEach(() => {
vi.useFakeTimers();
});
afterEach(async () => {
vi.useRealTimers();
await Promise.all(
temporaryFolders.splice(0).map((folder) => rm(folder, { recursive: true, force: true })),
);
});
test("polling stops for a workspace whose folder no longer exists", async () => {
const workspace = await createTemporaryFolder();
const git = createFolderBackedGit();
const service = createService(git.deps);
const subscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
await rm(workspace, { recursive: true, force: true });
const gitCommandsAtDeletion = git.gitCommandsIn(workspace);
await advanceThroughDegradedPolls(service);
expect(git.gitCommandsIn(workspace)).toBe(gitCommandsAtDeletion);
expect(service.getMetrics()).toMatchObject({
repositoryTargetCount: 0,
workingTreeWatchTargetCount: 0,
});
expect(vi.getTimerCount()).toBe(1);
subscription.unsubscribe();
await service.dispose();
});
test("a deleted worktree leaves its shared repository polling and fetch to the surviving worktree", async () => {
const gitCommonDir = join(await createTemporaryFolder(), ".git");
const deletedWorktree = await createTemporaryFolder();
const survivingWorktree = await createTemporaryFolder();
const git = createFolderBackedGit({ facts: linkedWorktreeFacts(gitCommonDir) });
let survivingWorktreeEdits = 0;
git.deps.getCheckoutShortstat.mockImplementation(async (cwd: string) => ({
additions: cwd === survivingWorktree ? ++survivingWorktreeEdits : 1,
deletions: 0,
}));
const service = createService(git.deps);
const deletedSubscription = service.registerWorkspace({ cwd: deletedWorktree }, vi.fn());
await waitForObservationToSettle(service, 1);
const survivingSubscription = service.registerWorkspace({ cwd: survivingWorktree }, vi.fn());
await waitForObservationToSettle(service, 2);
await rm(deletedWorktree, { recursive: true, force: true });
await vi.advanceTimersByTimeAsync(60_000);
await waitForRefreshesToSettle(service);
const deletedWorktreeCommands = git.gitCommandsIn(deletedWorktree);
for (let tick = 0; tick < 3; tick += 1) {
const survivingWorktreePolls = git.pollsOf(survivingWorktree);
await vi.advanceTimersByTimeAsync(5_000);
await settle(() => {
expect(git.pollsOf(survivingWorktree)).toBe(survivingWorktreePolls + 1);
});
}
expect(git.gitCommandsIn(deletedWorktree)).toBe(deletedWorktreeCommands);
const fetchesBefore = git.deps.runGitFetch.mock.calls.length;
await vi.advanceTimersByTimeAsync(2 * BACKGROUND_GIT_FETCH_INTERVAL_MS);
expect(git.deps.runGitFetch.mock.calls.slice(fetchesBefore).map(([cwd]) => cwd)).toEqual([
survivingWorktree,
survivingWorktree,
]);
expect(git.gitCommandsIn(deletedWorktree)).toBe(deletedWorktreeCommands);
deletedSubscription.unsubscribe();
survivingSubscription.unsubscribe();
await service.dispose();
});
test("deleting every worktree of a repository stops its polling and background fetch", async () => {
const gitCommonDir = join(await createTemporaryFolder(), ".git");
const firstWorktree = await createTemporaryFolder();
const secondWorktree = await createTemporaryFolder();
const git = createFolderBackedGit({ facts: linkedWorktreeFacts(gitCommonDir) });
const service = createService(git.deps);
const firstSubscription = service.registerWorkspace({ cwd: firstWorktree }, vi.fn());
await waitForObservationToSettle(service, 1);
const secondSubscription = service.registerWorkspace({ cwd: secondWorktree }, vi.fn());
await waitForObservationToSettle(service, 2);
await rm(firstWorktree, { recursive: true, force: true });
await rm(secondWorktree, { recursive: true, force: true });
const commandsAtDeletion = [
git.gitCommandsIn(firstWorktree),
git.gitCommandsIn(secondWorktree),
];
await advanceThroughDegradedPolls(service);
expect([git.gitCommandsIn(firstWorktree), git.gitCommandsIn(secondWorktree)]).toEqual(
commandsAtDeletion,
);
expect(service.getMetrics()).toMatchObject({
repositoryTargetCount: 0,
workingTreeWatchTargetCount: 0,
});
expect(vi.getTimerCount()).toBe(2);
firstSubscription.unsubscribe();
secondSubscription.unsubscribe();
await service.dispose();
});
test("working tree fallback polling and watcher recovery stop for a workspace whose folder no longer exists", async () => {
const workspace = await createTemporaryFolder();
await mkdir(join(workspace, ".git"));
const git = createFolderBackedGit();
const service = createService(git.deps);
const subscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
await rm(workspace, { recursive: true, force: true });
git.watchCallbackFor(workspace)(new Error("watched directory was deleted"), []);
await vi.advanceTimersByTimeAsync(1_000);
await waitForRefreshesToSettle(service);
const commandsAfterWatcherError = git.gitCommandsIn(workspace);
const watchAttemptsAfterWatcherError = git.watchAttemptsFor(workspace);
await advanceThroughDegradedPolls(service);
expect(git.gitCommandsIn(workspace)).toBe(commandsAfterWatcherError);
expect(git.watchAttemptsFor(workspace)).toBe(watchAttemptsAfterWatcherError);
expect(vi.getTimerCount()).toBe(1);
subscription.unsubscribe();
await service.dispose();
});
test("a workspace whose folder comes back resumes polling on its next subscription", async () => {
const workspace = await createTemporaryFolder();
const git = createFolderBackedGit();
const service = createService(git.deps);
const firstSubscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
await rm(workspace, { recursive: true, force: true });
const commandsWhileMissing = git.gitCommandsIn(workspace);
await advanceThroughDegradedPolls(service);
expect(git.gitCommandsIn(workspace)).toBe(commandsWhileMissing);
await mkdir(workspace);
const secondSubscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
const pollsAfterResume = git.pollsOf(workspace);
await vi.advanceTimersByTimeAsync(5_000);
await waitForRefreshesToSettle(service);
expect(git.pollsOf(workspace)).toBe(pollsAfterResume + 1);
firstSubscription.unsubscribe();
secondSubscription.unsubscribe();
await service.dispose();
});
test("a suspended workspace costs no git while its folder is missing, even when a refresh is requested", async () => {
const workspace = await createTemporaryFolder();
const git = createFolderBackedGit();
const service = createService(git.deps);
const subscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
await rm(workspace, { recursive: true, force: true });
await advanceThroughDegradedPolls(service);
const commandsWhileMissing = git.gitCommandsIn(workspace);
for (let probe = 0; probe < 10; probe += 1) {
await vi.advanceTimersByTimeAsync(WORKSPACE_GIT_OBSERVATION_REENSURE_INTERVAL_MS);
await waitForRefreshesToSettle(service);
}
service.scheduleRefreshForCwd(workspace);
service.onWorkspaceStateMayHaveChanged(workspace);
await vi.advanceTimersByTimeAsync(5_000);
await waitForRefreshesToSettle(service);
expect(git.gitCommandsIn(workspace)).toBe(commandsWhileMissing);
expect(vi.getTimerCount()).toBe(1);
subscription.unsubscribe();
await service.dispose();
});
test("a suspended worktree resumes polling and its repository fetch once its folder is back, without a new subscription", async () => {
const gitCommonDir = join(await createTemporaryFolder(), ".git");
const worktree = await createTemporaryFolder();
const git = createFolderBackedGit({ facts: linkedWorktreeFacts(gitCommonDir) });
const service = createService(git.deps);
const subscription = service.registerWorkspace({ cwd: worktree }, vi.fn());
await waitForObservationToSettle(service, 1);
await rm(worktree, { recursive: true, force: true });
await advanceThroughDegradedPolls(service);
await waitForObservationToSettle(service, 0);
await mkdir(worktree);
const fetchesBeforeRestore = git.fetchesIn(worktree);
await vi.advanceTimersByTimeAsync(WORKSPACE_GIT_OBSERVATION_REENSURE_INTERVAL_MS);
await waitForObservationToSettle(service, 1);
expect(git.fetchesIn(worktree)).toBe(fetchesBeforeRestore + 1);
const pollsAfterRestore = git.pollsOf(worktree);
await vi.advanceTimersByTimeAsync(5_000);
await waitForRefreshesToSettle(service);
expect(git.pollsOf(worktree)).toBe(pollsAfterRestore + 1);
subscription.unsubscribe();
await service.dispose();
});
test("a forced snapshot resumes a suspended workspace whose folder is back", async () => {
const workspace = await createTemporaryFolder();
const git = createFolderBackedGit();
const service = createService(git.deps);
const subscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
await rm(workspace, { recursive: true, force: true });
await advanceThroughDegradedPolls(service);
await waitForObservationToSettle(service, 0);
await mkdir(workspace);
const snapshot = await service.getSnapshot(workspace, {
force: true,
reason: "refresh-button",
});
await waitForObservationToSettle(service, 1);
expect(snapshot.git.isGit).toBe(true);
subscription.unsubscribe();
await service.dispose();
});
test("a resumed workspace rebuilds observation from fresh facts, not the non-git facts cached while its folder was gone", async () => {
const workspace = await createTemporaryFolder();
const git = createFolderBackedGit();
const service = createService(git.deps);
const firstSubscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
await rm(workspace, { recursive: true, force: true });
service.onWorkspaceStateMayHaveChanged(workspace);
await vi.advanceTimersByTimeAsync(1_000);
await waitForRefreshesToSettle(service);
expect((await service.getSnapshot(workspace)).git.isGit).toBe(false);
await advanceThroughDegradedPolls(service);
await waitForObservationToSettle(service, 0);
await mkdir(workspace);
const secondSubscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
expect((await service.getSnapshot(workspace)).git.isGit).toBe(true);
firstSubscription.unsubscribe();
secondSubscription.unsubscribe();
await service.dispose();
});
test("resuming waits for an observation setup that was still running when the folder vanished", async () => {
const workspace = await createTemporaryFolder();
const gitDir = join(workspace, ".git");
await mkdir(gitDir);
const metadataWatchRelease = createDeferred<void>();
const git = createFolderBackedGit({
unwatchableDirectories: [workspace],
heldWatch: { directory: gitDir, release: metadataWatchRelease.promise },
});
const service = createService(git.deps);
const firstSubscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await settle(() => {
expect(git.watchAttemptsFor(gitDir)).toBe(1);
});
await rm(workspace, { recursive: true, force: true });
await vi.advanceTimersByTimeAsync(5_000);
await settle(() => {
expect(service.getMetrics()).toMatchObject({
repositoryTargetCount: 0,
workingTreeWatchTargetCount: 0,
});
});
await mkdir(workspace);
const secondSubscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
metadataWatchRelease.resolve();
await waitForObservationToSettle(service, 1);
expect(service.getMetrics().workingTreeWatchTargetCount).toBe(1);
firstSubscription.unsubscribe();
secondSubscription.unsubscribe();
await service.dispose();
});
test("an open working tree watch neither notifies nor recovers while its folder is missing, and picks up once it is back", async () => {
const workspace = await createTemporaryFolder();
await mkdir(join(workspace, ".git"));
const git = createFolderBackedGit();
const service = createService(git.deps);
const workspaceSubscription = service.registerWorkspace({ cwd: workspace }, vi.fn());
await waitForObservationToSettle(service, 1);
const diffListener = vi.fn();
const diffWatch = await service.requestWorkingTreeWatch(workspace, diffListener);
await rm(workspace, { recursive: true, force: true });
git.watchCallbackFor(workspace)(new Error("watched directory was deleted"), []);
await vi.advanceTimersByTimeAsync(1_000);
await waitForRefreshesToSettle(service);
const commandsAfterWatcherError = git.gitCommandsIn(workspace);
const notificationsAfterWatcherError = diffListener.mock.calls.length;
const watchAttemptsAfterWatcherError = git.watchAttemptsFor(workspace);
await advanceThroughWorkingTreeRecoveryAttempts(service);
expect(git.gitCommandsIn(workspace)).toBe(commandsAfterWatcherError);
expect(diffListener.mock.calls.length).toBe(notificationsAfterWatcherError);
expect(git.watchAttemptsFor(workspace)).toBe(watchAttemptsAfterWatcherError);
expect(service.getMetrics().workingTreeWatchTargetCount).toBe(1);
await mkdir(join(workspace, ".git"), { recursive: true });
await vi.advanceTimersByTimeAsync(60_000);
await waitForRefreshesToSettle(service);
expect(diffListener.mock.calls.length).toBe(notificationsAfterWatcherError + 1);
await vi.advanceTimersByTimeAsync(240_000);
await waitForRefreshesToSettle(service);
expect(git.watchAttemptsFor(workspace)).toBe(watchAttemptsAfterWatcherError + 1);
const notificationsAfterResubscribe = diffListener.mock.calls.length;
await vi.advanceTimersByTimeAsync(120_000);
await waitForRefreshesToSettle(service);
expect(diffListener.mock.calls.length).toBe(notificationsAfterResubscribe);
diffWatch.unsubscribe();
workspaceSubscription.unsubscribe();
await service.dispose();
});
test("a refresh crossing suspension preserves the cached snapshot and drops queued work", async () => {
const workspace = await createTemporaryFolder();
const git = createFolderBackedGit();
const started = createDeferred<void>();
const release = createDeferred<void>();
let holdNextRead = false;
let factsReads = 0;
const service = createService({
...git.deps,
getCheckoutSnapshotFacts: async (cwd: string) => {
factsReads++;
if (holdNextRead) {
holdNextRead = false;
started.resolve();
await release.promise;
}
return git.deps.getCheckoutSnapshotFacts(cwd);
},
});
const updates: WorkspaceGitRuntimeSnapshot[] = [];
const subscription = service.registerWorkspace({ cwd: workspace }, (snapshot) =>
updates.push(snapshot),
);
await waitForObservationToSettle(service, 1);
const cached = await service.getSnapshot(workspace);
holdNextRead = true;
const active = service.getSnapshot(workspace, {
force: true,
includeForge: false,
reason: "overlapping-refresh",
});
await started.promise;
const queued = service.getSnapshot(workspace, {
force: true,
includeForge: true,
reason: "queued-refresh",
});
await rm(workspace, { recursive: true, force: true });
await vi.advanceTimersByTimeAsync(5_000);
await settle(() => expect(service.getMetrics().repositoryWorkspaceLinkCount).toBe(0));
const readsAtSuspension = factsReads;
const updatesAtSuspension = updates.length;
release.resolve();
expect(await active).toEqual(cached);
expect(await queued).toEqual(cached);
expect(await service.getSnapshot(workspace)).toEqual(cached);
expect(updates).toHaveLength(updatesAtSuspension);
expect(factsReads).toBe(readsAtSuspension);
subscription.unsubscribe();
await service.dispose();
});
test("restoration settles a suspended refresh before reloading facts", async () => {
const workspace = await createTemporaryFolder();
const git = createFolderBackedGit();
const staleFacts = createDeferred<CheckoutSnapshotFacts>();
const started = createDeferred<void>();
let holdNextRead = false;
const service = createService({
...git.deps,
getCheckoutSnapshotFacts: async (cwd: string) => {
if (holdNextRead) {
holdNextRead = false;
started.resolve();
return staleFacts.promise;
}
return git.deps.getCheckoutSnapshotFacts(cwd);
},
});
const updates: WorkspaceGitRuntimeSnapshot[] = [];
const subscription = service.registerWorkspace({ cwd: workspace }, (snapshot) =>
updates.push(snapshot),
);
await waitForObservationToSettle(service, 1);
const cached = await service.getSnapshot(workspace);
holdNextRead = true;
const active = service.getSnapshot(workspace, {
force: true,
includeForge: false,
reason: "overlapping-refresh",
});
await started.promise;
await rm(workspace, { recursive: true, force: true });
await vi.advanceTimersByTimeAsync(5_000);
await settle(() => expect(service.getMetrics().repositoryWorkspaceLinkCount).toBe(0));
const updatesAtSuspension = updates.length;
await mkdir(workspace);
const restored = service.getSnapshot(workspace, {
force: true,
includeForge: false,
reason: "folder-returned",
});
await vi.advanceTimersByTimeAsync(0);
staleFacts.resolve({ isGit: false });
expect(await active).toEqual(cached);
expect((await restored).git.isGit).toBe(true);
expect(updates.slice(updatesAtSuspension).every((snapshot) => snapshot.git.isGit)).toBe(true);
await waitForObservationToSettle(service, 1);
subscription.unsubscribe();
await service.dispose();
});
test("workspaceFolderExists reports a missing folder and a path under a file as missing", async () => {
const folder = await createTemporaryFolder();
const file = join(folder, "file.txt");
await writeFile(file, "");
expect(await workspaceFolderExists(folder)).toBe(true);
expect(await workspaceFolderExists(join(folder, "missing"))).toBe(false);
expect(await workspaceFolderExists(join(file, "nested"))).toBe(false);
});
});
@@ -1,5 +1,5 @@
import { createHash } from "node:crypto";
import { readFile } from "node:fs/promises";
import { readFile, stat } from "node:fs/promises";
import { basename, dirname, join, resolve } from "node:path";
import { LRUCache } from "lru-cache";
import { CheckoutDiffCache } from "./checkout-diff-cache.js";
@@ -74,7 +74,7 @@ import { checkoutLiteFromGitSnapshot } from "./workspace-registry-model.js";
import { createWatcherLivenessCanary } from "./watcher-liveness-canary.js";
const WORKSPACE_GIT_WATCH_DEBOUNCE_MS = 1_000;
const BACKGROUND_GIT_FETCH_INTERVAL_MS = 180_000;
export const BACKGROUND_GIT_FETCH_INTERVAL_MS = 180_000;
const FETCH_METADATA_ECHO_TTL_MS = 5_000;
export const WORKSPACE_GIT_OBSERVATION_REENSURE_INTERVAL_MS = 60_000;
const FORGE_PR_STATUS_POLL_FAST_INTERVAL_MS = 20_000;
@@ -384,6 +384,7 @@ interface WorkspaceGitServiceDependencies {
runGitCommand: typeof runGitCommand;
getWorkspaceGitSelfHealPhaseMs: typeof getWorkspaceGitObservationReensurePhaseMs;
createWatcherLivenessCanary: typeof createWatcherLivenessCanary;
workspaceFolderExists: (cwd: string) => Promise<boolean>;
now: () => Date;
}
@@ -435,6 +436,8 @@ interface WorkspaceGitTarget {
repoGitRoot: string | null;
observationSetupPromise: Promise<void> | null;
observationSetupComplete: boolean;
folderMissing: boolean;
folderRestorePromise: Promise<void> | null;
closed: boolean;
}
@@ -571,10 +574,21 @@ function buildDefaultWorkspaceGitServiceDeps(
runGitCommand,
getWorkspaceGitSelfHealPhaseMs: getWorkspaceGitObservationReensurePhaseMs,
createWatcherLivenessCanary,
workspaceFolderExists,
now: () => new Date(),
};
}
export async function workspaceFolderExists(cwd: string): Promise<boolean> {
try {
await stat(cwd);
return true;
} catch (error) {
const code = (error as NodeJS.ErrnoException).code;
return code !== "ENOENT" && code !== "ENOTDIR";
}
}
function resolveWorkspaceGitServiceDeps(
subscribe: SubscribeToFileChanges,
deps: Partial<WorkspaceGitServiceDependencies> | undefined,
@@ -660,6 +674,9 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
const cwd = resolve(params.cwd);
const target = this.ensureWorkspaceTarget(cwd);
target.listeners.add(listener);
if (target.folderMissing) {
void this.resumeWorkspaceIfFolderRestored(target);
}
if (target.listeners.size === 1) {
this.startWorkspaceSubscriptionTimers(target);
}
@@ -751,6 +768,9 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
if (!request.force && target.latestSnapshot) {
return target.latestSnapshot;
}
if (target.folderMissing) {
await this.resumeWorkspaceIfFolderRestored(target);
}
return this.requestWorkspaceSnapshot(target, request);
}
@@ -972,6 +992,9 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
this.assertNotDisposed();
cwd = resolve(cwd);
const target = this.ensureWorkspaceTarget(cwd);
if (target.folderMissing) {
await this.resumeWorkspaceIfFolderRestored(target);
}
await this.refreshWorkspaceTarget(target, {
force: false,
refreshStructure: true,
@@ -1005,9 +1028,14 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
this.assertNotDisposed();
cwd = resolve(cwd);
const target = this.workspaceTargets.get(cwd);
if (target) {
this.scheduleWorkspaceRefresh(target, { queueIfBusy: false });
if (!target) {
return;
}
if (target.folderMissing) {
void this.resumeWorkspaceIfFolderRestored(target);
return;
}
this.scheduleWorkspaceRefresh(target, { queueIfBusy: false });
}
onWorkspaceStateMayHaveChanged(cwd: string): void {
@@ -1018,6 +1046,10 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
return;
}
this.invalidateForge(normalizedCwd);
if (target.folderMissing) {
void this.resumeWorkspaceIfFolderRestored(target);
return;
}
this.scheduleWorkspaceRefresh(target, {
force: true,
includeForge: true,
@@ -1227,6 +1259,8 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
repoGitRoot: null,
observationSetupPromise: null,
observationSetupComplete: false,
folderMissing: false,
folderRestorePromise: null,
closed: false,
};
@@ -1358,6 +1392,7 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
private isActiveObservedWorkspaceTarget(target: WorkspaceGitTarget): boolean {
return (
!target.closed &&
!target.folderMissing &&
target.listeners.size > 0 &&
this.workspaceTargets.get(target.cwd) === target
);
@@ -1612,11 +1647,110 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
workspaceTarget: WorkspaceGitTarget,
request: WorkspaceGitRefreshRequest,
): Promise<boolean> {
if (await this.suspendWorkspaceIfFolderMissing(workspaceTarget)) {
return false;
}
const fingerprintBefore = workspaceTarget.latestFingerprint;
await this.refreshWorkspaceTarget(workspaceTarget, request);
return workspaceTarget.latestFingerprint !== fingerprintBefore;
}
private async suspendWorkspaceIfFolderMissing(target: WorkspaceGitTarget): Promise<boolean> {
if (target.folderMissing) {
return true;
}
if (await this.deps.workspaceFolderExists(target.cwd)) {
return false;
}
this.suspendWorkspaceWithMissingFolder(target);
return true;
}
private suspendWorkspaceWithMissingFolder(target: WorkspaceGitTarget): void {
if (target.folderMissing || target.closed || this.workspaceTargets.get(target.cwd) !== target) {
return;
}
target.folderMissing = true;
if (target.refreshState.status === "in-flight") {
target.refreshState.queued = null;
}
this.detachWorkspaceFromRepoTarget(target);
target.repoGitRoot = null;
this.stopWorkspaceObservation(target);
target.observationSetupComplete = false;
this.scheduleMissingFolderProbe(target);
this.logger.warn(
{ cwd: target.cwd },
"Workspace folder no longer exists; suspended git observation until it is back",
);
}
private isSuspendedWorkspaceTarget(target: WorkspaceGitTarget): boolean {
return (
target.folderMissing && !target.closed && this.workspaceTargets.get(target.cwd) === target
);
}
private scheduleMissingFolderProbe(target: WorkspaceGitTarget): void {
if (
!this.isSuspendedWorkspaceTarget(target) ||
target.observationReensureTimer ||
target.folderRestorePromise
) {
return;
}
target.observationReensureTimer = setTimeout(() => {
target.observationReensureTimer = null;
void this.resumeWorkspaceIfFolderRestored(target);
}, WORKSPACE_GIT_OBSERVATION_REENSURE_INTERVAL_MS);
}
private resumeWorkspaceIfFolderRestored(target: WorkspaceGitTarget): Promise<void> {
if (!target.folderMissing) {
return Promise.resolve();
}
target.folderRestorePromise ??= this.resumeWorkspaceWithRestoredFolder(target).finally(() => {
target.folderRestorePromise = null;
this.scheduleMissingFolderProbe(target);
});
return target.folderRestorePromise;
}
private async resumeWorkspaceWithRestoredFolder(target: WorkspaceGitTarget): Promise<void> {
if (!(await this.deps.workspaceFolderExists(target.cwd))) {
return;
}
await target.observationSetupPromise;
if (target.refreshState.status === "in-flight") {
await target.refreshState.promise.catch(() => undefined);
}
if (!this.isSuspendedWorkspaceTarget(target)) {
return;
}
target.latestFacts = null;
target.folderMissing = false;
await this.refreshWorkspaceTarget(target, {
force: false,
refreshStructure: true,
refreshWorktree: true,
includeForge: false,
reason: "workspace-folder-restored",
queueIfBusy: true,
movedRemoteRefs: new Set(),
});
if (target.closed || this.workspaceTargets.get(target.cwd) !== target) {
return;
}
target.observationSetupComplete = false;
if (target.observationReensureTimer) {
clearTimeout(target.observationReensureTimer);
target.observationReensureTimer = null;
}
this.logger.info({ cwd: target.cwd }, "Workspace folder is back; resumed git observation");
this.startWorkspaceSubscriptionTimers(target);
this.scheduleWorkspaceObservationSetup(target);
}
private startWorkingTreeWatchFallback(
target: WorkingTreeWatchTarget,
reason: WorkingTreeWatchFallbackReason,
@@ -1654,7 +1788,9 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
return changed;
}),
);
this.notifyWorkingTreeConsumers(target);
if (await this.deps.workspaceFolderExists(target.watchPath)) {
this.notifyWorkingTreeConsumers(target);
}
if (!target.closed && (target.subscription === null || target.repoRoot === null)) {
target.fallbackPollQuietTickCount = changes.some(Boolean)
? 0
@@ -1731,6 +1867,10 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
if (target.closed || target.subscription) {
return;
}
if (!(await this.deps.workspaceFolderExists(target.watchPath))) {
this.scheduleWorkingTreeWatchRecovery(target);
return;
}
await this.refreshWorkingTreeIgnoredDirectories(target);
if (target.closed || target.subscription) {
return;
@@ -2786,7 +2926,7 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
}
private updateForgePrStatusPollForTarget(target: WorkspaceGitTarget): void {
if (target.listeners.size === 0) {
if (!this.isActiveObservedWorkspaceTarget(target)) {
this.stopForgePrStatusPollForTarget(target);
return;
}
@@ -3049,6 +3189,9 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
target: WorkspaceGitTarget,
request: WorkspaceGitRefreshRequest,
): Promise<WorkspaceGitRuntimeSnapshot> {
if (target.folderMissing) {
return Promise.resolve(target.latestSnapshot ?? buildNotGitSnapshot(target.cwd));
}
if (target.refreshState.status === "in-flight") {
const active = target.refreshState.request;
const addsWork =
@@ -3191,11 +3334,19 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
try {
const runRefreshGitCommand = createRunGitCommand(`workspace-refresh:${request.reason}`);
const admittedSnapshot = await this.workspaceRefreshLimit(() => {
if (target.closed || this.workspaceTargets.get(target.cwd) !== target) {
if (
target.closed ||
target.folderMissing ||
this.workspaceTargets.get(target.cwd) !== target
) {
return null;
}
return this.refreshSnapshot(target, request, runRefreshGitCommand);
});
if (target.folderMissing) {
failure = null;
break;
}
if (!admittedSnapshot) {
break;
}
@@ -3209,6 +3360,10 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
failure = { error };
}
if (target.folderMissing) {
failure = null;
break;
}
if (this.disposed || target.closed || this.workspaceTargets.get(target.cwd) !== target) {
break;
}
@@ -3504,7 +3659,7 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
snapshot: WorkspaceGitRuntimeSnapshot,
options?: { forceEmit?: boolean },
): void {
if (target.closed || this.workspaceTargets.get(target.cwd) !== target) {
if (target.closed || target.folderMissing || this.workspaceTargets.get(target.cwd) !== target) {
return;
}
target.latestSnapshot = snapshot;
@@ -3660,21 +3815,25 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
}
private removeWorkspaceTarget(target: WorkspaceGitTarget): void {
if (target.repoGitRoot) {
const repoTarget = this.repoTargets.get(target.repoGitRoot);
repoTarget?.workspaceKeys.delete(target.cwd);
if (repoTarget && repoTarget.workspaceKeys.size === 0) {
this.closeRepoTarget(repoTarget);
this.repoTargets.delete(target.repoGitRoot);
} else if (repoTarget?.cwd === target.cwd) {
repoTarget.cwd = repoTarget.workspaceKeys.values().next().value ?? repoTarget.cwd;
}
}
this.detachWorkspaceFromRepoTarget(target);
this.closeWorkspaceTarget(target);
this.workspaceTargets.delete(target.cwd);
}
private detachWorkspaceFromRepoTarget(target: WorkspaceGitTarget): void {
if (!target.repoGitRoot) {
return;
}
const repoTarget = this.repoTargets.get(target.repoGitRoot);
repoTarget?.workspaceKeys.delete(target.cwd);
if (repoTarget && repoTarget.workspaceKeys.size === 0) {
this.closeRepoTarget(repoTarget);
this.repoTargets.delete(target.repoGitRoot);
} else if (repoTarget?.cwd === target.cwd) {
repoTarget.cwd = repoTarget.workspaceKeys.values().next().value ?? repoTarget.cwd;
}
}
private removeWorkingTreeWatchListener(cwd: string, listener: () => void): void {
const target = this.workingTreeWatchTargets.get(cwd);
if (!target) {
@@ -3719,6 +3878,11 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
private closeWorkspaceTarget(target: WorkspaceGitTarget): void {
target.closed = true;
this.stopWorkspaceObservation(target);
target.listeners.clear();
}
private stopWorkspaceObservation(target: WorkspaceGitTarget): void {
if (target.workingTreeWatchTarget) {
this.removeWorkspaceWorkingTreeLink(target.workingTreeWatchTarget, target.cwd);
target.workingTreeWatchTarget = null;
@@ -3727,12 +3891,12 @@ export class WorkspaceGitServiceImpl implements WorkspaceGitService {
clearTimeout(target.debounceTimer);
target.debounceTimer = null;
}
target.pendingDebounceRequest = null;
if (target.observationReensureTimer) {
clearTimeout(target.observationReensureTimer);
target.observationReensureTimer = null;
}
this.stopForgePrStatusPollForTarget(target);
target.listeners.clear();
}
private closeWorkingTreeWatchTarget(target: WorkingTreeWatchTarget): void {
@@ -1,5 +1,6 @@
import { afterEach, describe, expect, it } from "vitest";
import {
chmodSync,
existsSync,
mkdirSync,
mkdtempSync,
@@ -54,6 +55,7 @@ function makeSources(bundleSource = "export {};\n"): {
mkdirSync(native, { recursive: true });
writeFileSync(path.join(native, "pty.node"), "native");
writeFileSync(path.join(native, "spawn-helper"), "#!/bin/sh\n", { mode: 0o755 });
chmodSync(path.join(native, "spawn-helper"), 0o755);
// Another platform's build is not copied.
mkdirSync(path.join(nodePtyDirectory, "prebuilds", "other-platform"));
return { bundleDirectory, nodePtyDirectory };
@@ -171,3 +171,90 @@ it.each([401, 403])("reports an existing login rejected with HTTP %i", async (st
else process.env["COPILOT_TOKEN"] = previous;
}
});
it.each([
{ remaining: 1500, percentRemaining: 100, usedPct: 0 },
{ remaining: 1125, percentRemaining: 75, usedPct: 25 },
{ remaining: 0, percentRemaining: 0, usedPct: 100 },
])(
"reports premium quota with $remaining requests remaining",
async ({ remaining, percentRemaining, usedPct }) => {
const report = await fetchPremiumQuota({
entitlement: 1500,
remaining,
percent_remaining: percentRemaining,
unlimited: false,
});
expect(report).toMatchObject({
status: "available",
planLabel: "individual",
windows: [
{
id: "premium_interactions",
label: "Premium requests",
usedPct,
remainingPct: percentRemaining,
resetsAt: "2026-12-01T00:00:00.000Z",
},
],
details: expect.arrayContaining([
{
id: "premium_remaining",
label: "Premium requests remaining",
value: `${remaining} / 1500`,
},
]),
});
},
);
it.each([
undefined,
null,
{ unlimited: true, percent_remaining: 100, remaining: 1500, entitlement: 1500 },
])(
"does not invent a finite quota for absent or unlimited premium snapshots (%j)",
async (premium) => {
const report = await fetchPremiumQuota(premium);
expect(report).toMatchObject({
status: "available",
windows: [],
details: [{ id: "reset", label: "Quota reset", value: "2026-12-01" }],
});
},
);
it("reports a percentage without inventing missing counts and falls back to the legacy reset", async () => {
const report = await fetchPremiumQuota({ percent_remaining: 60, unlimited: false }, null);
expect(report).toMatchObject({
status: "available",
windows: [{ usedPct: 40, remainingPct: 60, resetsAt: "2026-12-01" }],
details: [{ id: "reset", label: "Quota reset", value: "2026-12-01" }],
});
});
async function fetchPremiumQuota(
premium: unknown,
utcReset: string | null = "2026-12-01T00:00:00.000Z",
) {
const { writeFile } = await import("node:fs/promises");
const directory = mkdtempSync(join(tmpdir(), "copilot-premium-"));
const locator = join(directory, "hosts.yml");
try {
await writeFile(locator, "github.com:\n oauth_token: fixture-copilot\n");
return await fetchUsage({ store: "file", locator }, async () =>
jsonResponse({
copilot_plan: "individual",
quota_reset_date: "2026-12-01",
quota_reset_date_utc: utcReset,
quota_snapshots: {
premium_interactions: premium,
chat: { unlimited: true, percent_remaining: 100 },
completions: { unlimited: true, percent_remaining: 100 },
},
}),
);
} finally {
rmSync(directory, { recursive: true, force: true });
}
}
+38 -1
View File
@@ -5,6 +5,8 @@ import { join } from "node:path";
import { z } from "zod";
import {
unavailable,
windowFromUsedPct,
type UsageWindow,
type UsageAccount,
type UsageReport,
type UsageDetail,
@@ -18,6 +20,19 @@ const ApiOptionalStringSchema = z.preprocess(
const CopilotUsageResponseSchema = z.object({
copilot_plan: ApiOptionalStringSchema,
quota_reset_date: ApiOptionalStringSchema,
quota_reset_date_utc: ApiOptionalStringSchema,
quota_snapshots: z
.object({
premium_interactions: z
.object({
entitlement: z.number().nonnegative().nullish(),
remaining: z.number().nonnegative().nullish(),
percent_remaining: z.number().min(0).max(100).nullish(),
unlimited: z.boolean().optional(),
})
.nullish(),
})
.nullish(),
});
async function readToken(input: UsageInput): Promise<string | undefined> {
@@ -72,10 +87,32 @@ export async function fetchUsage(
? [{ id: "reset", label: "Quota reset", value: resp.quota_reset_date }]
: [];
const windows: UsageWindow[] = [];
const premium = resp.quota_snapshots?.premium_interactions;
if (premium && !premium.unlimited) {
if (premium.percent_remaining != null) {
windows.push(
windowFromUsedPct({
id: "premium_interactions",
label: "Premium requests",
utilizationPct: 100 - premium.percent_remaining,
resetsAt: resp.quota_reset_date_utc || resp.quota_reset_date,
}),
);
}
if (premium.remaining != null && premium.entitlement != null) {
details.push({
id: "premium_remaining",
label: "Premium requests remaining",
value: `${premium.remaining} / ${premium.entitlement}`,
});
}
}
return {
status: "available",
planLabel: resp.copilot_plan || undefined,
windows: [],
windows,
balances: [],
details,
};