mirror of
https://github.com/getpaseo/paseo.git
synced 2026-11-06 08:24:46 +00:00
* feat(app): swipe open collapsed tablet navigation Reuse the retained sidebar and mobile panel motion for a partial-width navigation overlay when the wide dock is hidden. Preserve the dock preference, project gestures through the sidebar width, and dismiss the overlay when revealing workspace content. Reset the animated frame width explicitly across dock transitions and keep the resize origin on the UI thread. Validated on Duo portrait and landscape, iPad portrait and landscape, iPhone full-width navigation, and browser sidebar interactions. Typecheck, lint, targeted formatting, and 31 relevant tests passed. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): refine touch navigation overlays Enable navigation swipes only on touch-capable runtimes and reject mouse and pen input per gesture event. Allow swiping from anywhere in the workspace instead of requiring an edge start. Cover the full application window with the backdrop, including safe-area padding, and remove the wide overlay's inline close button. Preserve swipe, backdrop, and destination-selection dismissal. Validated interior swipes on Duo and iPad, mouse exclusion in a browser, and full-window dimming and dismissal in Duo portrait and landscape. Typecheck, lint, and targeted formatting passed. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * refactor(app): let the panel store drop the overlay target on dock Docking the sidebar, or any layout that no longer presents it as an overlay, now drops the transient agent-list target inside the panel store. App chrome publishes one left sidebar presentation (overlay, dock, hidden) instead of an effect in SidebarChrome watching flags. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * refactor(app): answer fine pointer and touch screen from one module Merge use-fine-pointer and use-has-touch-screen into hooks/pointer-capabilities, which reads both answers from the media queries it subscribes to. The touch answer now comes from (any-pointer: coarse) instead of navigator.maxTouchPoints, which that query's change events never reported. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): keep the phone web drawer on its edge swipe Restore the 32px left-edge start for the compact web navigation drawer, so interior rightward swipes over terminals, code blocks, and diffs stay with their horizontal scrollers. The wide navigation overlay still opens from anywhere. The edge rule moves into resolveMobilePanelGestureIntent so both drawers share it and it has a test. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * refactor(app): one sidebar handler per destination Settings, host settings, add host, add project, and every navigation row now use one handler that closes the navigation overlay before it navigates. Closing is idempotent, so the docked sidebar shares it and the overlay ? mobile : desktop handler pairs go away. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * refactor(app): resolve the left panel width in one place MobilePanelsProvider now resolves the left sidebar width (full window on compact, the saved width clamped to the safe viewport on wide) and exposes it through the panel runtime. The open and close gestures, the overlay frame, and the docked sidebar all read that value, so gesture travel and panel width cannot disagree. MobileGestureWrapper no longer recomputes it, and the panelWidth/width parameters go away. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * docs: fold the wide navigation overlay into the mobile panel rules Move the facts the overlay PR appended into the sections that own them: the destinations intro, store ownership of the agent-list target, the projection widths, gesture pointer and edge rules, and the sidebar presentation bullets. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): keep the sidebar pans live across dock and overlay switches On an Android tablet, opening the dock with the header toggle and collapsing it again left the touch swipe unable to open the overlay until the app reloaded. Swipe-to-close from the sidebar also stopped working after the same toggle cycle. Both pans took the dock/overlay state through RNGH's `.enabled()`. On Android, RNGH 2.32 cancels the native handler when `isEnabled` changes and resets its config on every update. After a false -> true flip the handler stayed inert. `.enabled()` now carries only the device capability (touch screen present), which does not change at runtime. The dock/overlay state moves to a UI-thread derived value that fails the gesture in onTouchesDown, next to the existing non-touch pointer check. The wide sidebar's workspace list now always receives the close gesture ref, as the compact sidebar already does. Toggling it between undefined and the ref rewired the list's simultaneous handlers on every dock switch, and that also left the close pan dead after a toggle cycle. This is a native RNGH handler behavior, so there is no unit repro. It was verified on the Android fold emulator: toggle open, toggle closed, swipe opens the overlay, swipe on an empty sidebar strip closes it, no reload. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): keep the workspace full width after collapsing the dock On an Android tablet, opening the dock with the header toggle and collapsing it again could leave the workspace at the docked width, with a blank strip on the right, until the overlay was next opened. Elsa's original head showed the same. The sidebar's panel host flips between an in-flow dock and an absolute overlay. Its width came from a Reanimated style that read the React `overlay` flag, so the width reached the native view on the UI thread after React had already committed the new presentation. Layout logs on the fold emulator show the lag on every toggle-open: the workspace laid out at width 0 while the docked host still had the overlay's full window width, then a second pass laid out the 320dp dock. When the late update was lost on collapse, the workspace kept its dock width. The host no longer has a width of its own: - The dock sizes to the inner sidebar view, which already carries the animated resize width. - The overlay host is absoluteFill, which covers the whole row including the safe-area padding. With no animated layout prop on the host, a presentation switch is one React commit, and the host and the workspace lay out in the same pass. Device evidence on the fold emulator: - Toggle open and collapse cycles: the host and the workspace lay out together (320/521dp, then 841/841dp). - A swipe opens the 320dp overlay with a full-window backdrop, and a swipe on the sidebar closes it. - Dragging the dock edge still resizes the dock and the workspace together. There is no unit repro: the defect is the ordering of Reanimated's UI-thread layout updates against React commits. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): cancel pending sidebar swipes when docking An opening swipe still has the agent target until release. Leaving overlay presentation must advance its revision even then, so motion ownership and queued commits reject the obsolete swipe. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): keep mouse selection free of the touch-only pan's capture On a touch-capable browser the navigation pan registers pointer listeners over the whole workspace. Gesture Handler's web pointer manager captured the pressed element before the handler saw the pointer, and captured it again on every move, so a mouse drag that the pan rejects stayed captured and the browser's selection stopped at the starting paragraph. The patched manager now captures after pointer-down, and only a pointer the handler still tracks. The resize handle and touch swipes keep capturing. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): drop the compact Explorer target when the layout widens Opening the Explorer on a compact layout selects the `file-explorer` mobile panel target. Widening the window (unfolding a foldable, rotating a tablet) kept that target, because the store only heard the left sidebar presentation, which is `overlay` on both compact layouts and wide layouts with a collapsed dock. The navigation swipe begins only from `agent`, so on the wide layout every swipe was rejected until something else reset the target. App chrome now publishes the whole panel layout (compact or wide, plus the left sidebar presentation) and the store drops any target that layout does not present. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): keep wide overlay sidebar content inside the left safe area The overlay starts at the window's left edge, over the workspace's safe-area padding, so its content now takes the left inset itself. The dock already sits inside app chrome's padding and stays unchanged. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): return an interrupted swipe to its anchor when docking cancels it Docking mid-swipe bumps the panel revision while keeping the agent target. The command treated the unchanged target as motion already in progress and skipped the animation, leaving the dragged position stranded between anchors. A command that interrupts a drag now always animates to its target. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): synchronize startup panel selection and gate pending swipes A child layout effect can publish panel layout before the motion provider subscribes. Establish the current selection when subscribing, then observe later commands. Recheck presentation eligibility before gesture activation so docking also cancels touches that have not started dragging. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * test(app): cover panel subscription through its public contract Verify initial snapshot delivery, later commands, and unsubscription directly in the existing panel-store suite instead of mounting test-only React components. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * test(app): reopen compact navigation after leaving settings Destination navigation clears the transient overlay selection. Reopen the compact drawer before asserting its configured footer rather than relying on a stale open target surviving a wide layout. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * test(app): open navigation by keyboard in composer focus checks Navigation pans intentionally reject mouse drags. Use the keyboard sidebar command to exercise focus and fullscreen Escape behavior without relying on the former mouse swipe helper. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * fix(app): complete sidebar integration after main rebase Retain the resize width resolver and format the shared division-width and hybrid touch-sizing integration. Co-authored-by: Elsa Granger <zeyugao@outlook.com> * test(desktop): give split browser panes room for tab rails After #8740, narrow panes replace tab rails with compact dropdowns. The desktop browser journey counts and selects rail tabs, so widen the real Electron content window before splitting instead of treating missing narrow-pane rails as missing panes. Co-authored-by: Elsa Granger <zeyugao@outlook.com> --------- Co-authored-by: Elsa <zeyugao@outlook.com>
1318 lines
48 KiB
JavaScript
1318 lines
48 KiB
JavaScript
#!/usr/bin/env node
|
||
|
||
import { spawn } from "node:child_process";
|
||
import fs from "node:fs";
|
||
import { createServer } from "node:http";
|
||
import net from "node:net";
|
||
import os from "node:os";
|
||
import path from "node:path";
|
||
import process from "node:process";
|
||
import { setTimeout as delay } from "node:timers/promises";
|
||
import { fileURLToPath } from "node:url";
|
||
import { experimental_createMCPClient } from "ai";
|
||
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
|
||
import { chromium } from "playwright";
|
||
import { expect } from "@playwright/test";
|
||
import { hideDeveloperToasts } from "../../app/e2e/support/developer-ui.mjs";
|
||
import { runAppearanceFontSizeRegression } from "./appearance-font-size.electron.mjs";
|
||
import { runAdaptiveThemeRegression } from "./appearance-theme.electron.mjs";
|
||
import { runSettingsScrollRegression } from "./settings-scroll.electron.mjs";
|
||
import { runSettingsMemoryRegression } from "./settings-memory.electron.mjs";
|
||
|
||
import { seedPluginLinks, runPluginLinksRegression } from "./plugin-links.electron.mjs";
|
||
|
||
const scriptDir = path.dirname(fileURLToPath(import.meta.url));
|
||
const desktopDir = path.resolve(scriptDir, "..");
|
||
const rootDir = path.resolve(desktopDir, "../..");
|
||
const devRunner = path.join(desktopDir, "scripts", "dev-runner.mjs");
|
||
const workspaceIds = [
|
||
"desktop-browser-original",
|
||
"desktop-browser-evict-one",
|
||
"desktop-browser-evict-two",
|
||
"desktop-browser-evict-three",
|
||
];
|
||
const timeoutMs = 90_000;
|
||
|
||
function assert(condition, message) {
|
||
if (!condition) throw new Error(message);
|
||
}
|
||
|
||
async function reservePort() {
|
||
return await new Promise((resolve, reject) => {
|
||
const server = net.createServer();
|
||
server.unref();
|
||
server.once("error", reject);
|
||
server.listen(0, "127.0.0.1", () => {
|
||
const address = server.address();
|
||
server.close((error) => {
|
||
if (error) return reject(error);
|
||
if (!address || typeof address === "string") {
|
||
return reject(new Error("Failed to reserve a local port"));
|
||
}
|
||
resolve(address.port);
|
||
});
|
||
});
|
||
});
|
||
}
|
||
|
||
async function waitForPort(port, label, processInfo) {
|
||
const deadline = Date.now() + timeoutMs;
|
||
while (Date.now() < deadline) {
|
||
if (
|
||
processInfo &&
|
||
(processInfo.child.exitCode !== null || processInfo.child.signalCode !== null)
|
||
) {
|
||
throw new Error(
|
||
`${label} process exited before opening its port; see ${processInfo.logPath}`,
|
||
);
|
||
}
|
||
const connected = await new Promise((resolve) => {
|
||
const socket = net.createConnection({ host: "127.0.0.1", port });
|
||
socket.setTimeout(500);
|
||
socket.once("connect", () => {
|
||
socket.destroy();
|
||
resolve(true);
|
||
});
|
||
socket.once("timeout", () => {
|
||
socket.destroy();
|
||
resolve(false);
|
||
});
|
||
socket.once("error", () => resolve(false));
|
||
});
|
||
if (connected) return;
|
||
await delay(200);
|
||
}
|
||
throw new Error(`Timed out waiting for ${label} on port ${port}`);
|
||
}
|
||
|
||
function writeJson(filePath, value) {
|
||
fs.mkdirSync(path.dirname(filePath), { recursive: true });
|
||
fs.writeFileSync(filePath, `${JSON.stringify(value, null, 2)}\n`);
|
||
}
|
||
|
||
function seedPaseoHome(paseoHome, listen, workspaceRoot) {
|
||
const timestamp = "2026-01-01T00:00:00.000Z";
|
||
const projects = workspaceIds.map((workspaceId, index) => {
|
||
const cwd = path.join(workspaceRoot, `workspace-${index + 1}`);
|
||
fs.mkdirSync(cwd, { recursive: true });
|
||
return {
|
||
projectId: `project-${workspaceId}`,
|
||
rootPath: cwd,
|
||
kind: "non_git",
|
||
displayName: `Desktop browser project ${index + 1}`,
|
||
customName: null,
|
||
createdAt: timestamp,
|
||
updatedAt: timestamp,
|
||
archivedAt: null,
|
||
};
|
||
});
|
||
const workspaces = workspaceIds.map((workspaceId, index) => ({
|
||
workspaceId,
|
||
projectId: projects[index].projectId,
|
||
cwd: projects[index].rootPath,
|
||
kind: "directory",
|
||
displayName: `Desktop browser workspace ${index + 1}`,
|
||
title: `Desktop browser workspace ${index + 1}`,
|
||
branch: null,
|
||
baseBranch: null,
|
||
createdAt: timestamp,
|
||
updatedAt: timestamp,
|
||
archivedAt: null,
|
||
pinnedAt: null,
|
||
}));
|
||
|
||
writeJson(path.join(paseoHome, "config.json"), {
|
||
version: 1,
|
||
daemon: {
|
||
listen,
|
||
relay: { enabled: false },
|
||
mcp: { enabled: true, injectIntoAgents: true },
|
||
browserTools: { enabled: true },
|
||
cors: { allowedOrigins: ["*"] },
|
||
},
|
||
});
|
||
writeJson(path.join(paseoHome, "projects", "projects.json"), projects);
|
||
writeJson(path.join(paseoHome, "projects", "workspaces.json"), workspaces);
|
||
}
|
||
|
||
function spawnLogged(name, command, args, options, logDir) {
|
||
const logPath = path.join(logDir, `${name}.log`);
|
||
const log = fs.createWriteStream(logPath, { flags: "a" });
|
||
const child = spawn(command, args, {
|
||
...options,
|
||
detached: process.platform !== "win32",
|
||
stdio: ["ignore", "pipe", "pipe"],
|
||
});
|
||
let openStreams = 2;
|
||
const closeLogStream = () => {
|
||
openStreams -= 1;
|
||
if (openStreams === 0) log.end();
|
||
};
|
||
child.stdout.pipe(log, { end: false });
|
||
child.stderr.pipe(log, { end: false });
|
||
child.stdout.once("end", closeLogStream);
|
||
child.stderr.once("end", closeLogStream);
|
||
return { child, logPath };
|
||
}
|
||
|
||
function stopProcess(child) {
|
||
if (!child?.pid || child.exitCode !== null) return;
|
||
try {
|
||
if (process.platform === "win32") child.kill("SIGTERM");
|
||
else process.kill(-child.pid, "SIGTERM");
|
||
} catch {
|
||
// The process may exit between the liveness check and signal delivery.
|
||
}
|
||
}
|
||
|
||
async function waitForAppPage(browser, expoPort) {
|
||
const deadline = Date.now() + timeoutMs;
|
||
while (Date.now() < deadline) {
|
||
for (const context of browser.contexts()) {
|
||
for (const page of context.pages()) {
|
||
if (page.url().includes(`localhost:${expoPort}`)) return page;
|
||
}
|
||
}
|
||
await delay(250);
|
||
}
|
||
throw new Error("Timed out waiting for the real Electron app renderer");
|
||
}
|
||
|
||
async function waitForDesktopStatus(page) {
|
||
const deadline = Date.now() + timeoutMs;
|
||
let lastError;
|
||
while (Date.now() < deadline) {
|
||
try {
|
||
const status = await page.evaluate(async () => {
|
||
if (typeof window.paseoDesktop?.invoke !== "function") return null;
|
||
return await window.paseoDesktop.invoke("desktop_daemon_status");
|
||
});
|
||
if (typeof status?.serverId === "string") return status;
|
||
} catch (error) {
|
||
// Metro may replace the renderer execution context during its initial load.
|
||
lastError = error;
|
||
}
|
||
await delay(250);
|
||
}
|
||
throw new Error(
|
||
`Timed out waiting for the Electron desktop bridge${lastError ? `: ${String(lastError)}` : ""}`,
|
||
);
|
||
}
|
||
|
||
async function settleDesktopStartup(page) {
|
||
await expect(
|
||
page.getByRole("button", { name: "Desktop browser workspace 1", exact: true }),
|
||
).toBeVisible({ timeout: timeoutMs });
|
||
// The dev launcher supplies its package directory as an open-project request.
|
||
// Before the local host connects, that request opens the connection prompt.
|
||
// Finish startup before exercising the unrelated Settings/browser workflows.
|
||
const connectionPrompt = page.getByTestId("add-host-method-modal");
|
||
if (await connectionPrompt.isVisible()) {
|
||
await connectionPrompt.getByRole("button", { name: "Close", exact: true }).click();
|
||
await expect(connectionPrompt).toBeHidden();
|
||
}
|
||
}
|
||
|
||
async function startTargetPage() {
|
||
const server = createServer((_request, response) => {
|
||
response.writeHead(200, { "content-type": "text/html; charset=utf-8" });
|
||
response.end(`<!doctype html>
|
||
<html>
|
||
<head><title>Desktop browser target</title>
|
||
<style>body { min-height: 100vh; background: rgb(255, 0, 0); }</style>
|
||
<script>
|
||
window.frameCount = 0;
|
||
function tick() { window.frameCount++; requestAnimationFrame(tick); }
|
||
requestAnimationFrame(tick);
|
||
</script>
|
||
</head>
|
||
<body>
|
||
<button id="bridge-target" onclick="this.textContent = 'Clicked'">Bridge target</button>
|
||
<label for="typing-target">Typing target</label>
|
||
<input id="typing-target" />
|
||
</body>
|
||
</html>`);
|
||
});
|
||
await new Promise((resolve, reject) => {
|
||
server.once("error", reject);
|
||
server.listen(0, "127.0.0.1", resolve);
|
||
});
|
||
const address = server.address();
|
||
assert(address && typeof address !== "string", "Target page did not bind a TCP port");
|
||
return { server, url: `http://127.0.0.1:${address.port}/` };
|
||
}
|
||
|
||
async function closeServer(server) {
|
||
await new Promise((resolve) => server.close(resolve));
|
||
}
|
||
|
||
function mcpPayload(result, command) {
|
||
const payload = result.structuredContent;
|
||
assert(
|
||
payload && typeof payload === "object",
|
||
`${command} returned no structured payload: ${JSON.stringify(result)}`,
|
||
);
|
||
assert(payload.ok === true, `${command} failed: ${JSON.stringify(payload)}`);
|
||
return payload.result;
|
||
}
|
||
|
||
async function callBrowserTool(client, name, args = {}) {
|
||
return mcpPayload(await client.callTool({ name, args }), name);
|
||
}
|
||
|
||
async function waitForGuestSelector(client, browserId) {
|
||
const deadline = Date.now() + 5_000;
|
||
while (Date.now() < deadline) {
|
||
const evaluated = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => Boolean(globalThis.__paseoSelector)",
|
||
});
|
||
if (JSON.parse(evaluated.resultJson) === true) {
|
||
return true;
|
||
}
|
||
await delay(50);
|
||
}
|
||
return false;
|
||
}
|
||
|
||
async function waitForGuestActiveElement(client, browserId, elementId) {
|
||
const deadline = Date.now() + 5_000;
|
||
while (Date.now() < deadline) {
|
||
const evaluated = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => document.activeElement?.id ?? null",
|
||
});
|
||
if (JSON.parse(evaluated.resultJson) === elementId) {
|
||
return true;
|
||
}
|
||
await delay(50);
|
||
}
|
||
return false;
|
||
}
|
||
|
||
async function createCallerAgent(daemonPort, workspaceId = workspaceIds[0]) {
|
||
const transport = new StreamableHTTPClientTransport(
|
||
new URL(`http://127.0.0.1:${daemonPort}/mcp/agents`),
|
||
);
|
||
const client = await experimental_createMCPClient({ transport });
|
||
try {
|
||
const response = await client.callTool({
|
||
name: "create_agent",
|
||
args: {
|
||
relationship: { kind: "detached" },
|
||
workspace: { kind: "existing", workspaceId },
|
||
title: "Browser desktop browser E2E caller",
|
||
provider: "mock/ten-second-stream",
|
||
settings: { modeId: "load-test" },
|
||
initialPrompt: "Remain available while the browser bridge regression runs.",
|
||
},
|
||
});
|
||
const result = response.structuredContent;
|
||
assert(
|
||
result && typeof result === "object",
|
||
`create_agent returned no structured payload: ${JSON.stringify(response)}`,
|
||
);
|
||
assert(typeof result.agentId === "string", "create_agent returned no caller agent id");
|
||
assert(
|
||
result.workspaceId === workspaceId,
|
||
`MCP caller attached to unexpected workspace ${result.workspaceId}`,
|
||
);
|
||
return result.agentId;
|
||
} finally {
|
||
await client.close();
|
||
}
|
||
}
|
||
|
||
async function readGuest(page, browserId) {
|
||
return await page.evaluate((id) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
if (!(webview instanceof HTMLElement) || typeof webview.getWebContentsId !== "function") {
|
||
return null;
|
||
}
|
||
return {
|
||
webContentsId: webview.getWebContentsId(),
|
||
parentId: webview.parentElement?.id ?? null,
|
||
width: Math.round(webview.getBoundingClientRect().width),
|
||
height: Math.round(webview.getBoundingClientRect().height),
|
||
};
|
||
}, browserId);
|
||
}
|
||
|
||
async function readPresentation(page, browserId) {
|
||
return await page.evaluate((id) => {
|
||
const surface = document.querySelector(`[data-paseo-browser-surface="${id}"]`);
|
||
const clip = document.querySelector(`[data-testid="browser-webview-clip-${id}"]`);
|
||
if (!(surface instanceof HTMLElement) || !(clip instanceof HTMLElement)) return null;
|
||
const surfaceRect = surface.getBoundingClientRect();
|
||
const clipRect = clip.getBoundingClientRect();
|
||
const webview = surface.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
if (!(webview instanceof HTMLElement)) return null;
|
||
const webviewRect = webview.getBoundingClientRect();
|
||
const outsidePoint = {
|
||
x: Math.max(0, Math.round(clipRect.left - 1)),
|
||
y: Math.round(clipRect.top + clipRect.height / 2),
|
||
};
|
||
const outsideTarget = document.elementFromPoint(outsidePoint.x, outsidePoint.y);
|
||
return {
|
||
surface: {
|
||
left: Math.round(surfaceRect.left),
|
||
top: Math.round(surfaceRect.top),
|
||
right: Math.round(surfaceRect.right),
|
||
bottom: Math.round(surfaceRect.bottom),
|
||
},
|
||
clip: {
|
||
left: Math.round(clipRect.left),
|
||
top: Math.round(clipRect.top),
|
||
right: Math.round(clipRect.right),
|
||
bottom: Math.round(clipRect.bottom),
|
||
},
|
||
webview: {
|
||
left: Math.round(webviewRect.left),
|
||
top: Math.round(webviewRect.top),
|
||
},
|
||
capturesOutsideInput: surface.contains(outsideTarget),
|
||
};
|
||
}, browserId);
|
||
}
|
||
|
||
async function readViewport(client, browserId) {
|
||
const evaluated = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function:
|
||
"() => ({ width: window.innerWidth, height: window.innerHeight, scale: window.devicePixelRatio })",
|
||
});
|
||
return JSON.parse(evaluated.resultJson);
|
||
}
|
||
|
||
async function clickGuestElement(page, client, browserId, selector) {
|
||
const evaluated = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: `() => {
|
||
const element = document.querySelector(${JSON.stringify(selector)});
|
||
if (!(element instanceof HTMLElement)) return null;
|
||
const rect = element.getBoundingClientRect();
|
||
return { x: rect.x, y: rect.y, width: rect.width, height: rect.height };
|
||
}`,
|
||
});
|
||
const elementRect = JSON.parse(evaluated.resultJson);
|
||
assert(elementRect, `Guest element ${selector} was unavailable`);
|
||
await page.locator(`[data-paseo-browser-id="${browserId}"]`).click({
|
||
position: {
|
||
x: elementRect.x + elementRect.width / 2,
|
||
y: elementRect.y + elementRect.height / 2,
|
||
},
|
||
});
|
||
}
|
||
|
||
async function selectDeviceSize(page, label) {
|
||
await page.locator('[aria-label="Device size"]').click();
|
||
const item = page.getByText(label, { exact: true });
|
||
await item.waitFor({ state: "visible", timeout: timeoutMs });
|
||
const rect = await item.boundingBox();
|
||
assert(rect, `Device size menu item ${label} had no bounds`);
|
||
const clip = {
|
||
x: Math.max(0, rect.x),
|
||
y: Math.max(0, rect.y),
|
||
width: rect.width,
|
||
height: rect.height,
|
||
};
|
||
const openPixels = await page.screenshot({ clip });
|
||
await page.keyboard.press("Escape");
|
||
await item.waitFor({ state: "hidden", timeout: timeoutMs });
|
||
const closedPixels = await page.screenshot({ clip });
|
||
|
||
await page.locator('[aria-label="Device size"]').click();
|
||
await item.waitFor({ state: "visible", timeout: timeoutMs });
|
||
await page.mouse.click(rect.x + rect.width / 2, rect.y + rect.height / 2);
|
||
await page.keyboard.press("Escape");
|
||
return !openPixels.equals(closedPixels);
|
||
}
|
||
|
||
async function selectElementAndReadAnnotationPaint({ page, client, browserId, artifactDir }) {
|
||
await clickGuestElement(page, client, browserId, "#bridge-target");
|
||
const comment = page.getByRole("textbox", {
|
||
name: "Message to the agent about this element…",
|
||
});
|
||
await comment.waitFor({ state: "visible", timeout: timeoutMs });
|
||
const bounds = await comment.boundingBox();
|
||
assert(bounds, "Element annotation comment box had no bounds");
|
||
const receivesInput = await comment.evaluate((element) => {
|
||
const elementBounds = element.getBoundingClientRect();
|
||
const target = document.elementFromPoint(
|
||
elementBounds.left + elementBounds.width / 2,
|
||
elementBounds.top + elementBounds.height / 2,
|
||
);
|
||
return target === element || element.contains(target);
|
||
});
|
||
const clip = {
|
||
x: Math.max(0, bounds.x),
|
||
y: Math.max(0, bounds.y),
|
||
width: bounds.width,
|
||
height: bounds.height,
|
||
};
|
||
const openPixels = await page.screenshot({
|
||
clip,
|
||
path: path.join(artifactDir, "element-annotation-open.png"),
|
||
});
|
||
await page.keyboard.press("Escape");
|
||
await comment.waitFor({ state: "hidden", timeout: timeoutMs });
|
||
const closedPixels = await page.screenshot({
|
||
clip,
|
||
path: path.join(artifactDir, "element-annotation-closed.png"),
|
||
});
|
||
return receivesInput && !openPixels.equals(closedPixels);
|
||
}
|
||
|
||
function recordViewportMismatch(failures, label, actual, expected) {
|
||
if (actual.width === expected.width && actual.height === expected.height) {
|
||
return;
|
||
}
|
||
failures.push(
|
||
`${label}: expected ${expected.width}x${expected.height}, received ${actual.width}x${actual.height}`,
|
||
);
|
||
}
|
||
|
||
async function evaluateDesktopWindow(inspectorPort, expression) {
|
||
const [target] = await (await fetch(`http://127.0.0.1:${inspectorPort}/json/list`)).json();
|
||
const socket = new WebSocket(target.webSocketDebuggerUrl);
|
||
try {
|
||
return await new Promise((resolve, reject) => {
|
||
socket.addEventListener("error", reject, { once: true });
|
||
socket.addEventListener("message", ({ data }) => {
|
||
const response = JSON.parse(data);
|
||
if (response.id !== 1) return;
|
||
if (response.error || response.result?.exceptionDetails) reject(new Error(data));
|
||
else resolve(response.result.result.value);
|
||
});
|
||
socket.addEventListener(
|
||
"open",
|
||
() =>
|
||
socket.send(
|
||
JSON.stringify({
|
||
id: 1,
|
||
method: "Runtime.evaluate",
|
||
params: {
|
||
expression: `(() => { const win = process.mainModule.require('electron').BrowserWindow.getAllWindows().find(win => win.webContents.getURL().includes('localhost:')); ${expression} })()`,
|
||
returnByValue: true,
|
||
},
|
||
}),
|
||
),
|
||
{ once: true },
|
||
);
|
||
});
|
||
} finally {
|
||
socket.close();
|
||
}
|
||
}
|
||
|
||
async function setWindowHidden(inspectorPort, hidden) {
|
||
await evaluateDesktopWindow(
|
||
inspectorPort,
|
||
`win.${hidden ? "hide" : "show"}(); if (win.isVisible() !== ${!hidden}) throw new Error('Window visibility did not change');`,
|
||
);
|
||
}
|
||
|
||
async function resizeDesktopWindow(inspectorPort, { width, height }) {
|
||
return evaluateDesktopWindow(
|
||
inspectorPort,
|
||
`const [width, height] = win.getContentSize(); win.setContentSize(${width}, ${height}); return { width, height };`,
|
||
);
|
||
}
|
||
|
||
async function verifyHiddenBrowserScreenshots({
|
||
page,
|
||
client,
|
||
browserId,
|
||
artifactDir,
|
||
inspectorPort,
|
||
}) {
|
||
const readFrames = async () => {
|
||
const result = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => window.frameCount",
|
||
});
|
||
return JSON.parse(result.resultJson);
|
||
};
|
||
const expectAnimation = async () => {
|
||
const start = await readFrames();
|
||
assert(typeof start === "number", "Animation fixture was not loaded");
|
||
const deadline = Date.now() + 5_000;
|
||
while (Date.now() < deadline) {
|
||
if ((await readFrames()) > start + 2) return;
|
||
await delay(50);
|
||
}
|
||
throw new Error("Browser did not animate in the restored window");
|
||
};
|
||
const measurements = [];
|
||
const expectIdle = async (label) => {
|
||
await delay(500); // Let Chromium consume the visibility change before sampling.
|
||
const start = await readFrames();
|
||
await delay(1_000); // A fixed sampling interval is the behavior under test.
|
||
const frames = (await readFrames()) - start;
|
||
measurements.push({ label, frames });
|
||
writeJson(path.join(artifactDir, "hidden-browser-frames.json"), measurements);
|
||
assert(frames === 0, `${label}: hidden browser produced ${frames} animation frames`);
|
||
};
|
||
try {
|
||
await expectAnimation();
|
||
await setWindowHidden(inspectorPort, true);
|
||
await expectIdle("before-capture");
|
||
await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => { document.body.style.background = 'rgb(0,255,0)'; }",
|
||
});
|
||
const response = await client.callTool({ name: "browser_screenshot", args: { browserId } });
|
||
mcpPayload(response, "browser_screenshot");
|
||
const screenshot = response.content.find((item) => item.type === "image");
|
||
assert(screenshot, "browser_screenshot returned no image");
|
||
fs.writeFileSync(
|
||
path.join(artifactDir, "hidden-browser-viewport.png"),
|
||
Buffer.from(screenshot.data, "base64"),
|
||
);
|
||
const pixel = await page.evaluate(async (base64) => {
|
||
const image = new Image();
|
||
image.src = `data:image/png;base64,${base64}`;
|
||
await image.decode();
|
||
const canvas = document.createElement("canvas");
|
||
canvas.width = image.width;
|
||
canvas.height = image.height;
|
||
const context = canvas.getContext("2d");
|
||
context.drawImage(image, 0, 0);
|
||
return [
|
||
...context.getImageData(Math.floor(image.width / 2), Math.floor(image.height / 2), 1, 1)
|
||
.data,
|
||
];
|
||
}, screenshot.data);
|
||
assert(
|
||
JSON.stringify(pixel) === "[0,255,0,255]",
|
||
`Hidden screenshot returned stale pixels: ${pixel}`,
|
||
);
|
||
await expectIdle("after-capture");
|
||
} finally {
|
||
await setWindowHidden(inspectorPort, false);
|
||
}
|
||
await expectAnimation();
|
||
}
|
||
|
||
async function verifyBackgroundTypingStaysInGuest({ page, client, serverId, browserId, agent }) {
|
||
await page
|
||
.getByTestId(`workspace-deck-entry-${serverId}:${agent.workspaceId}`)
|
||
.getByTestId(`workspace-tab-agent_${agent.agentId}`)
|
||
.click({ timeout: timeoutMs });
|
||
const composer = page
|
||
.getByRole("textbox", { name: "Message agent..." })
|
||
.filter({ visible: true })
|
||
.first();
|
||
const draft = "draft the user is typing";
|
||
await composer.click();
|
||
await composer.pressSequentially(draft);
|
||
assert(
|
||
await composer.evaluate((element) => element === document.activeElement),
|
||
"Composer did not keep focus while the user typed",
|
||
);
|
||
|
||
const snapshot = await callBrowserTool(client, "browser_snapshot", { browserId });
|
||
const ref = snapshot.snapshot.match(/textbox "Typing target" \[ref=(@e\d+)\]/)?.[1];
|
||
assert(ref, `browser_snapshot did not expose the typing target: ${snapshot.snapshot}`);
|
||
await callBrowserTool(client, "browser_type", { browserId, ref, text: "agent text" });
|
||
await callBrowserTool(client, "browser_keypress", { browserId, ref, key: "!" });
|
||
|
||
// The user keeps typing in the composer while the agent types into the tab's focused field.
|
||
await composer.click();
|
||
await composer.press("End");
|
||
assert(
|
||
await composer.evaluate((element) => element === document.activeElement),
|
||
"Composer did not take focus back from the browser tab",
|
||
);
|
||
await callBrowserTool(client, "browser_type", { browserId, text: " more" });
|
||
await callBrowserTool(client, "browser_keypress", { browserId, key: "?" });
|
||
|
||
const guestValue = JSON.parse(
|
||
(
|
||
await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => document.getElementById('typing-target').value",
|
||
})
|
||
).resultJson,
|
||
);
|
||
const composerValue = await composer.inputValue();
|
||
assert(
|
||
composerValue === draft,
|
||
`browser_type and browser_keypress in a background tab changed the focused composer: ${JSON.stringify(composerValue)}`,
|
||
);
|
||
assert(
|
||
guestValue === "agent text! more?",
|
||
`browser_type and browser_keypress did not reach the browser tab: ${JSON.stringify(guestValue)}`,
|
||
);
|
||
await composer.fill("");
|
||
}
|
||
|
||
async function runRegression({
|
||
page,
|
||
client,
|
||
serverId,
|
||
targetUrl,
|
||
callerAgentId,
|
||
typingAgent,
|
||
artifactDir,
|
||
inspectorPort,
|
||
}) {
|
||
const failures = [];
|
||
const originalWorkspaceId = workspaceIds[0];
|
||
const originalWorkspaceRow = page.getByTestId(
|
||
`sidebar-workspace-row-${serverId}:${originalWorkspaceId}`,
|
||
);
|
||
await originalWorkspaceRow.waitFor({ state: "visible", timeout: timeoutMs });
|
||
await originalWorkspaceRow.click();
|
||
|
||
await page.evaluate(() => {
|
||
if (document.getElementById("overlay-root")) return;
|
||
const overlayRoot = document.createElement("div");
|
||
overlayRoot.id = "overlay-root";
|
||
overlayRoot.style.position = "fixed";
|
||
overlayRoot.style.inset = "0";
|
||
overlayRoot.style.pointerEvents = "none";
|
||
document.body.appendChild(overlayRoot);
|
||
});
|
||
|
||
const created = await callBrowserTool(client, "browser_new_tab", { url: targetUrl });
|
||
const browserId = created.browserId;
|
||
assert(typeof browserId === "string", "browser_new_tab returned no browserId");
|
||
|
||
const originalDeck = page.getByTestId(`workspace-deck-entry-${serverId}:${originalWorkspaceId}`);
|
||
await originalDeck.getByTestId(`workspace-tab-browser_${browserId}`).click();
|
||
await page.waitForFunction(
|
||
(id) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
const surface = webview?.parentElement;
|
||
return (
|
||
surface?.getAttribute("aria-hidden") === "false" && surface.style.pointerEvents === "auto"
|
||
);
|
||
},
|
||
browserId,
|
||
{ timeout: timeoutMs },
|
||
);
|
||
const firstGuest = await readGuest(page, browserId);
|
||
assert(firstGuest, "Original browser guest was not attached to its workspace pane");
|
||
recordViewportMismatch(
|
||
failures,
|
||
"Responsive viewport follows the visible browser pane",
|
||
await readViewport(client, browserId),
|
||
{ width: firstGuest.width, height: firstGuest.height },
|
||
);
|
||
|
||
await clickGuestElement(page, client, browserId, "#typing-target");
|
||
assert(
|
||
await waitForGuestActiveElement(client, browserId, "typing-target"),
|
||
"Physical browser click did not focus the guest input",
|
||
);
|
||
const focusedGuest = await page.evaluate(
|
||
(id) => window.paseoDesktop?.browser?.focus?.(id),
|
||
browserId,
|
||
);
|
||
assert(focusedGuest === true, "Electron did not focus the registered browser guest");
|
||
|
||
const deviceSizeMenuPainted = await selectDeviceSize(page, "iPhone SE · 375×667");
|
||
assert(deviceSizeMenuPainted, "Device size menu did not paint above the browser surface");
|
||
const deviceViewport = await readViewport(client, browserId);
|
||
recordViewportMismatch(
|
||
failures,
|
||
"device size menu paints and receives input above the browser surface",
|
||
deviceViewport,
|
||
{ width: 375, height: 667 },
|
||
);
|
||
|
||
await callBrowserTool(client, "browser_wait", {
|
||
browserId,
|
||
text: "Bridge target",
|
||
timeoutMs: 5_000,
|
||
});
|
||
const resizedScreenshot = await callBrowserTool(client, "browser_screenshot", { browserId });
|
||
assert(
|
||
resizedScreenshot.width === Math.round(375 * deviceViewport.scale) &&
|
||
resizedScreenshot.height === Math.round(667 * deviceViewport.scale),
|
||
`Screenshot after resize returned ${resizedScreenshot.width}×${resizedScreenshot.height}`,
|
||
);
|
||
const requestedViewport = { width: 640, height: 480 };
|
||
await callBrowserTool(client, "browser_resize", { browserId, ...requestedViewport });
|
||
recordViewportMismatch(
|
||
failures,
|
||
"browser_resize updates the visible shared viewport",
|
||
await readViewport(client, browserId),
|
||
requestedViewport,
|
||
);
|
||
|
||
const oversizedViewport = { width: 2560, height: 1440 };
|
||
await callBrowserTool(client, "browser_resize", { browserId, ...oversizedViewport });
|
||
recordViewportMismatch(
|
||
failures,
|
||
"oversized preset preserves the requested guest viewport",
|
||
await readViewport(client, browserId),
|
||
oversizedViewport,
|
||
);
|
||
await page.waitForFunction(
|
||
({ id, width, height }) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
return (
|
||
webview instanceof HTMLElement &&
|
||
Math.round(webview.getBoundingClientRect().width) === width &&
|
||
Math.round(webview.getBoundingClientRect().height) === height
|
||
);
|
||
},
|
||
{ id: browserId, ...oversizedViewport },
|
||
{ timeout: timeoutMs },
|
||
);
|
||
const presentation = await readPresentation(page, browserId);
|
||
assert(presentation, "Oversized browser presentation geometry was unavailable");
|
||
if (
|
||
presentation.surface.left < presentation.clip.left ||
|
||
presentation.surface.top < presentation.clip.top ||
|
||
presentation.surface.right > presentation.clip.right ||
|
||
presentation.surface.bottom > presentation.clip.bottom
|
||
) {
|
||
failures.push(
|
||
`oversized browser surface stays inside its pane: ${JSON.stringify(presentation)}`,
|
||
);
|
||
}
|
||
if (presentation.capturesOutsideInput) {
|
||
failures.push("oversized browser surface captures input outside its pane");
|
||
}
|
||
await callBrowserTool(client, "browser_resize", { browserId, ...oversizedViewport });
|
||
const repeatedResizePresentation = await readPresentation(page, browserId);
|
||
assert(repeatedResizePresentation, "Repeated resize presentation geometry was unavailable");
|
||
if (
|
||
repeatedResizePresentation.webview.left !== presentation.webview.left ||
|
||
repeatedResizePresentation.webview.top !== presentation.webview.top
|
||
) {
|
||
failures.push(
|
||
`repeating an oversized resize preserves the guest offset: before ${JSON.stringify(presentation.webview)}, after ${JSON.stringify(repeatedResizePresentation.webview)}`,
|
||
);
|
||
}
|
||
await callBrowserTool(client, "browser_resize", { browserId, ...requestedViewport });
|
||
|
||
await selectDeviceSize(page, "Responsive");
|
||
const responsiveViewport = await readViewport(client, browserId);
|
||
|
||
await originalDeck.getByTestId(`workspace-tab-agent_${callerAgentId}`).click();
|
||
await page.waitForFunction(
|
||
({ id, webContentsId }) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
return (
|
||
webview?.parentElement?.getAttribute("data-paseo-browser-surface") === id &&
|
||
webview.parentElement.style.width === "1px" &&
|
||
webview.parentElement.style.pointerEvents === "none" &&
|
||
webview.getWebContentsId() === webContentsId
|
||
);
|
||
},
|
||
{ id: browserId, webContentsId: firstGuest.webContentsId },
|
||
{ timeout: timeoutMs },
|
||
);
|
||
try {
|
||
await callBrowserTool(client, "browser_screenshot", { browserId });
|
||
} catch (error) {
|
||
failures.push(`inactive browser remains captureable: ${String(error)}`);
|
||
}
|
||
recordViewportMismatch(
|
||
failures,
|
||
"inactive browser preserves the shared viewport",
|
||
await readViewport(client, browserId),
|
||
responsiveViewport,
|
||
);
|
||
const focusContinuitySentinel = "preserve-browser-document-across-focus";
|
||
await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: `() => {
|
||
globalThis.__paseoFocusContinuity = ${JSON.stringify(focusContinuitySentinel)};
|
||
globalThis.__paseoViewportTransitions = [{ width: innerWidth, height: innerHeight }];
|
||
addEventListener('resize', () => {
|
||
globalThis.__paseoViewportTransitions.push({ width: innerWidth, height: innerHeight });
|
||
});
|
||
return globalThis.__paseoFocusContinuity;
|
||
}`,
|
||
});
|
||
await page.evaluate((id) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
if (!webview) throw new Error(`Browser webview ${id} was unavailable`);
|
||
const events = [];
|
||
globalThis.__paseoBrowserReactivationEvents = events;
|
||
for (const name of [
|
||
"did-start-loading",
|
||
"did-navigate-in-page",
|
||
"load-commit",
|
||
"did-stop-loading",
|
||
]) {
|
||
webview.addEventListener(name, (event) => {
|
||
events.push({
|
||
name,
|
||
url: event.url ?? null,
|
||
isMainFrame: event.isMainFrame ?? null,
|
||
});
|
||
});
|
||
}
|
||
}, browserId);
|
||
|
||
await originalDeck.getByTestId(`workspace-tab-browser_${browserId}`).click();
|
||
await page.waitForFunction(
|
||
({ id, webContentsId }) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
return (
|
||
webview?.parentElement?.getAttribute("data-paseo-browser-surface") === id &&
|
||
webview.parentElement.style.pointerEvents === "auto" &&
|
||
webview.getWebContentsId() === webContentsId
|
||
);
|
||
},
|
||
{ id: browserId, webContentsId: firstGuest.webContentsId },
|
||
{ timeout: timeoutMs },
|
||
);
|
||
await page.waitForTimeout(1_500);
|
||
const reactivationEvents = await page.evaluate(
|
||
() => globalThis.__paseoBrowserReactivationEvents ?? [],
|
||
);
|
||
const unexpectedReactivationCommits = reactivationEvents.filter(
|
||
(event) => event.name === "did-navigate-in-page" || event.name === "load-commit",
|
||
);
|
||
assert(
|
||
unexpectedReactivationCommits.length === 0,
|
||
`responsive browser reactivation committed the current document: ${JSON.stringify(unexpectedReactivationCommits)}`,
|
||
);
|
||
const viewportTransitionsResult = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => globalThis.__paseoViewportTransitions ?? []",
|
||
});
|
||
const viewportTransitions = JSON.parse(viewportTransitionsResult.resultJson);
|
||
const collapsedViewport = viewportTransitions.find(
|
||
(viewport) => viewport.width <= 1 || viewport.height <= 1,
|
||
);
|
||
assert(
|
||
collapsedViewport === undefined,
|
||
`responsive browser reactivation collapsed the guest viewport: ${JSON.stringify(viewportTransitions)}`,
|
||
);
|
||
const continuityResult = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => globalThis.__paseoFocusContinuity ?? null",
|
||
});
|
||
const continuityValue = JSON.parse(continuityResult.resultJson);
|
||
if (continuityValue !== focusContinuitySentinel) {
|
||
failures.push(
|
||
`focusing the browser tab preserves the current document: expected ${JSON.stringify(focusContinuitySentinel)}, received ${JSON.stringify(continuityValue)}`,
|
||
);
|
||
}
|
||
|
||
for (const workspaceId of workspaceIds.slice(1)) {
|
||
await page.getByTestId(`sidebar-workspace-row-${serverId}:${workspaceId}`).click();
|
||
await page
|
||
.getByTestId(`workspace-deck-entry-${serverId}:${workspaceId}`)
|
||
.waitFor({ state: "visible" });
|
||
}
|
||
|
||
await page.waitForFunction(
|
||
({ id, previousWebContentsId }) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
return (
|
||
webview?.parentElement?.getAttribute("data-paseo-browser-surface") === id &&
|
||
webview.parentElement.style.width === "1px" &&
|
||
typeof webview.getWebContentsId === "function" &&
|
||
webview.getWebContentsId() === previousWebContentsId
|
||
);
|
||
},
|
||
{ id: browserId, previousWebContentsId: firstGuest.webContentsId },
|
||
{ timeout: timeoutMs },
|
||
);
|
||
const parkedGuest = await readGuest(page, browserId);
|
||
assert(parkedGuest, "Browser guest was not parked after workspace eviction");
|
||
|
||
await verifyHiddenBrowserScreenshots({ page, client, browserId, artifactDir, inspectorPort });
|
||
|
||
const listed = await callBrowserTool(client, "browser_list_tabs");
|
||
assert(
|
||
listed.tabs.some((tab) => tab.browserId === browserId),
|
||
"browser_list_tabs lost the original tab after workspace eviction",
|
||
);
|
||
|
||
const snapshot = await callBrowserTool(client, "browser_snapshot", { browserId });
|
||
const ref = snapshot.snapshot.match(/button "Bridge target" \[ref=(@e\d+)\]/)?.[1];
|
||
assert(ref, `browser_snapshot did not expose the target button: ${snapshot.snapshot}`);
|
||
|
||
const clicked = await callBrowserTool(client, "browser_click", { browserId, ref });
|
||
assert(
|
||
clicked.browserId === browserId && clicked.ref === ref,
|
||
"browser_click targeted another tab",
|
||
);
|
||
await callBrowserTool(client, "browser_wait", {
|
||
browserId,
|
||
text: "Clicked",
|
||
timeoutMs: 5_000,
|
||
});
|
||
await verifyBackgroundTypingStaysInGuest({
|
||
page,
|
||
client,
|
||
serverId,
|
||
browserId,
|
||
agent: typingAgent,
|
||
});
|
||
|
||
await originalWorkspaceRow.click();
|
||
await originalDeck.getByTestId(`workspace-tab-browser_${browserId}`).click();
|
||
await page.waitForFunction(
|
||
({ id, webContentsId }) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
return (
|
||
webview?.parentElement?.getAttribute("data-paseo-browser-surface") === id &&
|
||
webview.parentElement.style.pointerEvents === "auto" &&
|
||
webview.getWebContentsId() === webContentsId
|
||
);
|
||
},
|
||
{ id: browserId, webContentsId: firstGuest.webContentsId },
|
||
{ timeout: timeoutMs },
|
||
);
|
||
recordViewportMismatch(
|
||
failures,
|
||
"browser viewport survives workspace eviction and reattachment",
|
||
await readViewport(client, browserId),
|
||
responsiveViewport,
|
||
);
|
||
|
||
const annotateButton = originalDeck.getByRole("button", { name: "Annotate element" });
|
||
await page.evaluate((id) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
if (!(webview instanceof HTMLElement)) throw new Error(`Browser webview ${id} was unavailable`);
|
||
globalThis.__paseoOriginalIsLoadingDescriptor = Object.getOwnPropertyDescriptor(
|
||
webview,
|
||
"isLoading",
|
||
);
|
||
Object.defineProperty(webview, "isLoading", {
|
||
configurable: true,
|
||
value: () => true,
|
||
});
|
||
}, browserId);
|
||
await annotateButton.click();
|
||
await page.waitForTimeout(100);
|
||
assert(
|
||
!(await originalDeck.getByRole("button", { name: "Cancel element selector" }).isVisible()),
|
||
"Element selector started while the guest reported a genuine load",
|
||
);
|
||
const selectorDuringLoad = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => Boolean(globalThis.__paseoSelector)",
|
||
});
|
||
assert(
|
||
JSON.parse(selectorDuringLoad.resultJson) === false,
|
||
"Selector injected while the guest reported a genuine load",
|
||
);
|
||
assert(
|
||
await page.getByText("Wait for the page to finish loading").isVisible(),
|
||
"Element selector loading failure was not visible",
|
||
);
|
||
await page.evaluate((id) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
if (!(webview instanceof HTMLElement)) throw new Error(`Browser webview ${id} was unavailable`);
|
||
const descriptor = globalThis.__paseoOriginalIsLoadingDescriptor;
|
||
if (descriptor) Object.defineProperty(webview, "isLoading", descriptor);
|
||
else delete webview.isLoading;
|
||
delete globalThis.__paseoOriginalIsLoadingDescriptor;
|
||
}, browserId);
|
||
|
||
const readyStateResult = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => document.readyState",
|
||
});
|
||
assert(
|
||
JSON.parse(readyStateResult.resultJson) === "complete",
|
||
"Local browser document did not finish loading",
|
||
);
|
||
// Reproduce the report's mismatch: the guest is complete, but the pane's
|
||
// last loading signal says it is not ready.
|
||
await page.evaluate((id) => {
|
||
const webview = document.querySelector(`[data-paseo-browser-id="${id}"]`);
|
||
if (!(webview instanceof HTMLElement)) {
|
||
throw new Error(`Browser webview ${id} was unavailable`);
|
||
}
|
||
webview.dispatchEvent(new Event("did-start-loading"));
|
||
}, browserId);
|
||
|
||
await annotateButton.click();
|
||
const annotateSelectorActive = await waitForGuestSelector(client, browserId);
|
||
await page.screenshot({ path: path.join(artifactDir, "local-page-annotate-selector.png") });
|
||
if (!annotateSelectorActive) {
|
||
failures.push("loaded local browser remains ready for element annotation");
|
||
}
|
||
|
||
await originalDeck.getByRole("button", { name: "Cancel element selector" }).click();
|
||
await delay(10_000);
|
||
const screenshotButton = originalDeck.getByRole("button", { name: "Screenshot element" });
|
||
await screenshotButton.click();
|
||
const screenshotSelectorActive = await waitForGuestSelector(client, browserId);
|
||
if (!screenshotSelectorActive) {
|
||
failures.push("loaded local browser remains ready for element screenshots");
|
||
}
|
||
await delay(20_500);
|
||
const selectorAfterPriorTimeout = await callBrowserTool(client, "browser_evaluate", {
|
||
browserId,
|
||
function: "() => Boolean(globalThis.__paseoSelector)",
|
||
});
|
||
if (JSON.parse(selectorAfterPriorTimeout.resultJson) !== true) {
|
||
failures.push("a previous selector timeout does not destroy the current selector session");
|
||
}
|
||
await page.screenshot({ path: path.join(artifactDir, "local-page-screenshot-selector.png") });
|
||
await originalDeck.getByRole("button", { name: "Cancel element selector" }).click();
|
||
|
||
// This journey selects tabs from both pane rails. Keep both columns above the compact-header
|
||
// breakpoint; narrow panes deliberately replace their rail with a tab dropdown.
|
||
await resizeDesktopWindow(inspectorPort, { width: 1920, height: 1080 });
|
||
await page.waitForFunction(() => innerWidth >= 1920, null, { timeout: timeoutMs });
|
||
await originalDeck.getByTestId(`workspace-tab-agent_${callerAgentId}`).click();
|
||
await page.getByTestId("sidebar-search").click();
|
||
await page.getByTestId("command-center-input").fill("Split pane right");
|
||
await page.getByText("Split pane right", { exact: true }).click();
|
||
assert(
|
||
(await originalDeck.getByTestId("workspace-tabs-row").filter({ visible: true }).count()) === 2,
|
||
"Split pane command did not produce two visible panes",
|
||
);
|
||
await originalDeck.getByTestId(`workspace-tab-browser_${browserId}`).last().click();
|
||
await originalDeck
|
||
.getByTestId(`browser-webview-clip-${browserId}`)
|
||
.waitFor({ state: "visible", timeout: timeoutMs });
|
||
|
||
const splitAnnotateButton = originalDeck.getByRole("button", { name: "Annotate element" });
|
||
await splitAnnotateButton.click();
|
||
assert(
|
||
await waitForGuestSelector(client, browserId),
|
||
"Element selector did not start in the split browser pane",
|
||
);
|
||
assert(
|
||
await selectElementAndReadAnnotationPaint({
|
||
page,
|
||
client,
|
||
browserId,
|
||
artifactDir,
|
||
}),
|
||
"Element annotation comment box did not paint or receive input above the browser surface",
|
||
);
|
||
|
||
if (failures.length > 0) {
|
||
throw new Error(`Browser viewport regressions:\n- ${failures.join("\n- ")}`);
|
||
}
|
||
|
||
return {
|
||
browserId,
|
||
originalWebContentsId: firstGuest.webContentsId,
|
||
finalWebContentsId: parkedGuest.webContentsId,
|
||
viewport: "passed",
|
||
guestFocus: "passed",
|
||
overlayPlane: "passed",
|
||
inactiveCapture: "passed",
|
||
hiddenWindowCapture: "passed",
|
||
list: "passed",
|
||
snapshot: "passed",
|
||
click: "passed",
|
||
localPageSelectors: "passed",
|
||
};
|
||
}
|
||
|
||
async function main() {
|
||
const artifactDir =
|
||
process.env.PASEO_DESKTOP_BROWSER_E2E_ARTIFACT_DIR ??
|
||
fs.mkdtempSync(path.join(os.tmpdir(), "paseo-desktop-browser-e2e-artifacts-"));
|
||
const runtimeDir = fs.mkdtempSync(path.join(os.tmpdir(), "paseo-desktop-browser-e2e-"));
|
||
fs.mkdirSync(artifactDir, { recursive: true });
|
||
const paseoHome = path.join(runtimeDir, "paseo-home");
|
||
const userData = path.join(runtimeDir, "electron-user-data");
|
||
const workspaceRoot = path.join(runtimeDir, "workspaces");
|
||
fs.mkdirSync(paseoHome, { recursive: true });
|
||
|
||
const [daemonPort, expoPort, cdpPort, inspectorPort, remotePort] = await Promise.all([
|
||
reservePort(),
|
||
reservePort(),
|
||
reservePort(),
|
||
reservePort(),
|
||
reservePort(),
|
||
]);
|
||
const listen = `127.0.0.1:${daemonPort}`;
|
||
seedPaseoHome(paseoHome, listen, workspaceRoot);
|
||
const target = await startTargetPage();
|
||
seedPluginLinks(paseoHome, workspaceIds[0], target.url, workspaceIds[1]);
|
||
const remoteHome = path.join(runtimeDir, "remote-home");
|
||
seedPaseoHome(remoteHome, `127.0.0.1:${remotePort}`, path.join(runtimeDir, "remote-workspaces"));
|
||
const children = [];
|
||
let browser = null;
|
||
let client = null;
|
||
|
||
try {
|
||
// Observe the real Electron shell handoff without launching a user's browser.
|
||
const openerDirectory = path.join(runtimeDir, "url-handler");
|
||
const externalOpenLog = path.join(artifactDir, "external-opens.txt");
|
||
fs.mkdirSync(openerDirectory, { recursive: true });
|
||
fs.writeFileSync(externalOpenLog, "");
|
||
fs.writeFileSync(
|
||
path.join(openerDirectory, "xdg-open"),
|
||
'#!/bin/sh\nprintf "%s\\n" "$1" >> "$PASEO_TEST_EXTERNAL_OPEN_LOG"\n',
|
||
{ mode: 0o755 },
|
||
);
|
||
const commonEnv = {
|
||
...process.env,
|
||
PATH: `${openerDirectory}${path.delimiter}${process.env.PATH}`,
|
||
PASEO_TEST_EXTERNAL_OPEN_LOG: externalOpenLog,
|
||
PASEO_HOME: paseoHome,
|
||
PASEO_LISTEN: listen,
|
||
PASEO_DAEMON_ENDPOINT: `localhost:${daemonPort}`,
|
||
PASEO_CORS_ORIGINS: "*",
|
||
PASEO_LOCAL_SPEECH_AUTO_DOWNLOAD: "0",
|
||
PASEO_DICTATION_ENABLED: "0",
|
||
PASEO_VOICE_MODE_ENABLED: "0",
|
||
FORCE_COLOR: "0",
|
||
NO_COLOR: "1",
|
||
};
|
||
const daemon = spawnLogged(
|
||
"daemon",
|
||
process.execPath,
|
||
["--import", "tsx", path.join(rootDir, "packages/server/scripts/dev-runner.ts")],
|
||
{ cwd: rootDir, env: { ...commonEnv, PASEO_NODE_ENV: "development" } },
|
||
artifactDir,
|
||
);
|
||
children.push(daemon.child);
|
||
await waitForPort(daemonPort, "daemon", daemon);
|
||
|
||
const remoteDaemon = spawnLogged(
|
||
"remote-daemon",
|
||
process.execPath,
|
||
["--import", "tsx", path.join(rootDir, "packages/server/scripts/dev-runner.ts")],
|
||
{
|
||
cwd: rootDir,
|
||
env: {
|
||
...commonEnv,
|
||
PASEO_HOME: remoteHome,
|
||
PASEO_LISTEN: `127.0.0.1:${remotePort}`,
|
||
PASEO_SERVER_ID: "plugin-links-remote",
|
||
PASEO_NODE_ENV: "development",
|
||
},
|
||
},
|
||
artifactDir,
|
||
);
|
||
children.push(remoteDaemon.child);
|
||
await waitForPort(remotePort, "remote daemon", remoteDaemon);
|
||
|
||
const desktopArgs = [
|
||
process.execPath,
|
||
devRunner,
|
||
`--inspect=127.0.0.1:${inspectorPort}`,
|
||
...(process.platform === "linux" ? ["--no-sandbox"] : []),
|
||
];
|
||
const desktopCommand = process.platform === "linux" ? "xvfb-run" : desktopArgs.shift();
|
||
const desktopCommandArgs =
|
||
process.platform === "linux"
|
||
? ["-a", "--server-args=-screen 0 1280x800x24", ...desktopArgs]
|
||
: desktopArgs;
|
||
const desktop = spawnLogged(
|
||
"desktop",
|
||
desktopCommand,
|
||
desktopCommandArgs,
|
||
{
|
||
cwd: rootDir,
|
||
env: {
|
||
...commonEnv,
|
||
EXPO_UNSTABLE_HEADLESS: "1",
|
||
EXPO_PORT: String(expoPort),
|
||
EXPO_DEV_URL: `http://localhost:${expoPort}`,
|
||
PASEO_ELECTRON_REMOTE_DEBUGGING_PORT: String(cdpPort),
|
||
PASEO_ELECTRON_USER_DATA_DIR: userData,
|
||
PASEO_ELECTRON_FLAGS: `--remote-debugging-address=127.0.0.1 --remote-debugging-port=${cdpPort}`,
|
||
},
|
||
},
|
||
artifactDir,
|
||
);
|
||
children.push(desktop.child);
|
||
await waitForPort(cdpPort, "Electron CDP", desktop);
|
||
|
||
browser = await chromium.connectOverCDP(`http://127.0.0.1:${cdpPort}`);
|
||
const page = await waitForAppPage(browser, expoPort);
|
||
await hideDeveloperToasts(page);
|
||
const status = await waitForDesktopStatus(page);
|
||
await settleDesktopStartup(page);
|
||
|
||
const checkPluginLinks = () =>
|
||
runPluginLinksRegression({
|
||
page,
|
||
remotePort,
|
||
workspaceId: workspaceIds[0],
|
||
remoteWorkspaceId: workspaceIds[1],
|
||
url: target.url,
|
||
artifactDir,
|
||
externalOpenLog: process.platform === "linux" ? externalOpenLog : null,
|
||
});
|
||
if (process.env.PASEO_DESKTOP_PLUGIN_LINKS_ONLY === "1") {
|
||
const pluginLinks = await checkPluginLinks();
|
||
writeJson(path.join(artifactDir, "result.json"), { pluginLinks });
|
||
console.log("Plugin external links and workspace browser passed.");
|
||
return;
|
||
}
|
||
const settingsScroll = await runSettingsScrollRegression(page, (size) =>
|
||
resizeDesktopWindow(inspectorPort, size),
|
||
);
|
||
if (process.env.PASEO_DESKTOP_SETTINGS_SCROLL_ONLY === "1") {
|
||
writeJson(path.join(artifactDir, "result.json"), { settingsScroll });
|
||
console.log("Desktop Settings scroll regression passed.");
|
||
return;
|
||
}
|
||
const settingsMemory = await runSettingsMemoryRegression(page);
|
||
if (process.env.PASEO_DESKTOP_SETTINGS_MEMORY_ONLY === "1") {
|
||
writeJson(path.join(artifactDir, "result.json"), { settingsMemory });
|
||
console.log(
|
||
`Desktop Settings memory regression passed: ${settingsMemory.detachedAfterWarmRound} detached panes after warm and stress rotations.`,
|
||
);
|
||
return;
|
||
}
|
||
|
||
const adaptiveTheme = await runAdaptiveThemeRegression(artifactDir);
|
||
await runAppearanceFontSizeRegression(page);
|
||
|
||
const callerAgentId = await createCallerAgent(daemonPort);
|
||
const typingAgent = {
|
||
workspaceId: workspaceIds.at(-1),
|
||
agentId: await createCallerAgent(daemonPort, workspaceIds.at(-1)),
|
||
};
|
||
const transport = new StreamableHTTPClientTransport(
|
||
new URL(
|
||
`http://127.0.0.1:${daemonPort}/mcp/agents?callerAgentId=${encodeURIComponent(callerAgentId)}`,
|
||
),
|
||
);
|
||
client = await experimental_createMCPClient({ transport });
|
||
const report = await runRegression({
|
||
page,
|
||
client,
|
||
serverId: status.serverId,
|
||
targetUrl: target.url,
|
||
inspectorPort,
|
||
callerAgentId,
|
||
typingAgent,
|
||
artifactDir,
|
||
});
|
||
const pluginLinks = await checkPluginLinks();
|
||
writeJson(path.join(artifactDir, "result.json"), {
|
||
...report,
|
||
settingsMemory,
|
||
pluginLinks,
|
||
adaptiveTheme,
|
||
});
|
||
console.log(
|
||
`Browser desktop browser E2E passed: WebContents ${report.originalWebContentsId} remained ${report.finalWebContentsId}; viewport, inactive capture, focus continuity, list, snapshot, click, local-page selectors passed.`,
|
||
);
|
||
} catch (error) {
|
||
console.error(`Browser desktop browser E2E failed. Artifacts: ${artifactDir}`);
|
||
console.error(error);
|
||
throw error;
|
||
} finally {
|
||
await client?.close().catch(() => undefined);
|
||
await browser?.close().catch(() => undefined);
|
||
for (const child of children.toReversed()) stopProcess(child);
|
||
await closeServer(target.server);
|
||
await delay(1_000);
|
||
try {
|
||
fs.rmSync(runtimeDir, { recursive: true, force: true, maxRetries: 5, retryDelay: 100 });
|
||
} catch (error) {
|
||
console.warn(`Failed to remove isolated E2E state ${runtimeDir}`, error);
|
||
}
|
||
}
|
||
}
|
||
|
||
await main();
|