Plan: Design daemon client for E2E testing

Authored comprehensive design report covering 3 architectural approaches:
1. Simple WebSocket Wrapper (recommended)
2. Reactive Event Store
3. Hybrid approach

Recommendation: Approach 1 for ~300-400 lines, leveraging existing
messages.ts Zod schemas and test-utils/paseo-daemon.ts infrastructure.

Key files to create:
- daemon-client.ts - DaemonClient class
- daemon-test-context.ts - Test setup helpers
- daemon.e2e.test.ts - E2E test suite

See REPORT-daemon-client-design.md for full details.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Mohamed Boudra
2025-12-25 16:37:36 +07:00
parent 007b41e455
commit cf82296be9
2 changed files with 754 additions and 0 deletions

View File

@@ -0,0 +1,716 @@
# DaemonClient Design Report
## Objective
Design a `DaemonClient` class that enables full E2E testing of the Paseo daemon WITHOUT Playwright. Test the daemon directly via WebSocket, ensure API correctness, and UI correctness follows.
---
## Architectural Approaches
### Approach 1: Simple WebSocket Wrapper (Recommended)
**Description**: A thin, typed wrapper around WebSocket that directly mirrors the daemon's message protocol.
```typescript
// packages/server/src/server/test-utils/daemon-client.ts
import WebSocket from "ws";
import { z } from "zod";
import type {
SessionInboundMessage,
SessionOutboundMessage,
AgentSnapshotPayload,
AgentStreamEventPayload,
AgentPermissionRequest,
AgentPermissionResponse,
} from "../messages.js";
export interface DaemonClientConfig {
url: string; // ws://localhost:6767/ws
authHeader?: string;
}
export type AgentEventHandler = (event: {
type: "state" | "stream" | "permission_request" | "permission_resolved";
agentId: string;
payload: unknown;
}) => void;
export class DaemonClient {
private ws: WebSocket | null = null;
private pendingRequests: Map<string, { resolve: Function; reject: Function }> = new Map();
private eventHandlers: Set<AgentEventHandler> = new Set();
constructor(private config: DaemonClientConfig) {}
// --- Connection ---
async connect(): Promise<void>;
async close(): Promise<void>;
// --- Agent Lifecycle ---
async createAgent(config: CreateAgentConfig): Promise<AgentSnapshotPayload>;
async deleteAgent(agentId: string): Promise<void>;
async listAgents(): Promise<AgentSnapshotPayload[]>;
async listPersistedAgents(): Promise<PersistedAgentDescriptor[]>;
// --- Agent Interaction ---
async sendMessage(agentId: string, text: string, options?: SendMessageOptions): Promise<void>;
async cancelAgent(agentId: string): Promise<void>;
async setAgentMode(agentId: string, modeId: string): Promise<void>;
// --- Permissions ---
async respondToPermission(agentId: string, requestId: string, response: AgentPermissionResponse): Promise<void>;
// --- Streaming ---
waitForAgentIdle(agentId: string, options?: { timeout?: number }): Promise<AgentSnapshotPayload>;
waitForPermission(agentId: string, options?: { timeout?: number }): Promise<AgentPermissionRequest>;
// --- Event Subscription ---
on(handler: AgentEventHandler): () => void;
// --- Raw Message Access ---
send(message: SessionInboundMessage): Promise<void>;
receive(): AsyncIterable<SessionOutboundMessage>;
}
```
**Pros**:
- Simple, direct mapping to WebSocket protocol
- Full control over message timing and ordering
- Easy to debug - messages are visible
- No abstraction leaks
- Uses existing Zod schemas for type safety
**Cons**:
- Tests must handle async event streams manually
- No automatic retry/reconnect logic
**Files to create/modify**:
- `packages/server/src/server/test-utils/daemon-client.ts` (new)
- `packages/server/src/server/test-utils/index.ts` (export)
**Scope**: ~300-400 lines of code
---
### Approach 2: Reactive Event Store
**Description**: A more opinionated client that maintains reactive state, similar to how the UI works.
```typescript
export class DaemonClient {
// Reactive stores (like Zustand)
agents: Map<string, AgentSnapshotPayload>;
timelines: Map<string, AgentTimelineItem[]>;
permissions: Map<string, AgentPermissionRequest[]>;
// Subscribe to state changes
subscribe(selector: (state) => T, callback: (value: T) => void): () => void;
// Wait for state conditions
waitUntil<T>(selector: (state) => T, predicate: (value: T) => boolean): Promise<T>;
}
```
**Pros**:
- More declarative tests: `await client.waitUntil(s => s.agents.get(id)?.status, s => s === "idle")`
- Automatic state management
- Timeline history automatically tracked
**Cons**:
- More complex implementation
- State management adds overhead
- Harder to test low-level protocol edge cases
- Abstracts away message ordering which may hide bugs
**Files to create/modify**:
- `packages/server/src/server/test-utils/daemon-client.ts` (new, larger)
- Potentially need state management library
**Scope**: ~600-800 lines of code
---
### Approach 3: Hybrid - Simple Core + Helper Methods
**Description**: Simple WebSocket wrapper (Approach 1) with optional convenience helpers.
```typescript
export class DaemonClient {
// Core: direct WebSocket wrapper (Approach 1)
// ...
// Convenience: high-level test helpers
async createAgentAndWaitIdle(config, initialPrompt?): Promise<{ agent, timeline }>;
async sendMessageAndWaitIdle(agentId, text): Promise<{ timeline }>;
async approvePermissionAndWaitIdle(agentId, requestId): Promise<void>;
// Timeline helpers
getToolCalls(agentId): AgentTimelineItem[];
getLastMessage(agentId): string | null;
hasError(agentId): boolean;
}
```
**Pros**:
- Best of both: simple core, convenient helpers
- Helpers can be added incrementally
- Core stays testable and debuggable
- Helpers reduce test boilerplate
**Cons**:
- Helpers need maintenance as API evolves
- Risk of helpers hiding protocol bugs
**Scope**: ~500-600 lines of code
---
## Recommendation: Approach 1 (Simple WebSocket Wrapper)
**Rationale**:
1. **Matches the existing pattern** - The existing E2E tests (`agent-mcp.e2e.test.ts`) use direct MCP client calls. A WebSocket client should be similarly direct.
2. **Protocol transparency** - E2E tests should verify the actual protocol, not an abstracted version. A simple wrapper exposes the real messages.
3. **Debuggability** - When tests fail, you can see exactly what messages were sent/received. Abstractions hide this.
4. **Incremental** - Can add convenience methods later (Approach 3) if needed.
5. **Low maintenance** - Simple code means less to maintain as the daemon evolves.
---
## Implementation Design (Approach 1)
### Core Types
```typescript
// packages/server/src/server/test-utils/daemon-client.ts
import WebSocket from "ws";
import { nanoid } from "nanoid";
import { z } from "zod";
import type {
SessionInboundMessage,
SessionOutboundMessage,
AgentSnapshotPayload,
AgentStreamEventPayload,
AgentPermissionResponse,
} from "../messages.js";
export interface DaemonClientConfig {
url: string;
authHeader?: string;
}
export interface CreateAgentOptions {
provider: "claude" | "codex";
cwd: string;
title?: string;
model?: string;
modeId?: string;
initialPrompt?: string;
mcpServers?: Record<string, unknown>;
}
export interface SendMessageOptions {
messageId?: string;
images?: Array<{ data: string; mimeType: string }>;
}
export type DaemonEvent =
| { type: "agent_state"; agentId: string; payload: AgentSnapshotPayload }
| { type: "agent_stream"; agentId: string; event: AgentStreamEventPayload; timestamp: string }
| { type: "session_state"; agents: AgentSnapshotPayload[] }
| { type: "status"; payload: { status: string } }
| { type: "error"; message: string };
```
### Class Implementation
```typescript
export class DaemonClient {
private ws: WebSocket | null = null;
private messageQueue: SessionOutboundMessage[] = [];
private eventListeners: Set<(event: DaemonEvent) => void> = new Set();
private connectPromise: { resolve: () => void; reject: (e: Error) => void } | null = null;
constructor(private config: DaemonClientConfig) {}
// ============================================================================
// Connection
// ============================================================================
async connect(): Promise<void> {
return new Promise((resolve, reject) => {
this.ws = new WebSocket(this.config.url, {
headers: this.config.authHeader
? { Authorization: this.config.authHeader }
: undefined,
});
this.ws.on("open", () => {
this.connectPromise = null;
resolve();
});
this.ws.on("error", (err) => {
this.connectPromise?.reject(err);
this.connectPromise = null;
});
this.ws.on("message", (data) => {
const msg = JSON.parse(data.toString());
if (msg.type === "pong") return;
if (msg.type === "session") {
this.handleSessionMessage(msg.message);
}
});
this.connectPromise = { resolve, reject };
});
}
async close(): Promise<void> {
if (this.ws) {
this.ws.close();
this.ws = null;
}
}
// ============================================================================
// Agent Lifecycle
// ============================================================================
async createAgent(options: CreateAgentOptions): Promise<AgentSnapshotPayload> {
const requestId = nanoid();
this.send({
type: "create_agent_request",
requestId,
config: {
provider: options.provider,
cwd: options.cwd,
title: options.title,
model: options.model,
modeId: options.modeId,
mcpServers: options.mcpServers,
},
initialPrompt: options.initialPrompt,
});
// Wait for agent_state response
return this.waitFor((msg) => {
if (msg.type === "agent_state") {
return msg.payload;
}
return null;
});
}
async deleteAgent(agentId: string): Promise<void> {
this.send({ type: "delete_agent_request", agentId });
await this.waitFor((msg) => {
if (msg.type === "agent_deleted" && msg.payload.agentId === agentId) {
return true;
}
return null;
});
}
async listAgents(): Promise<AgentSnapshotPayload[]> {
// Trigger session_state by connecting or sending ping
// session_state is sent on connection
return this.waitFor((msg) => {
if (msg.type === "session_state") {
return msg.payload.agents;
}
return null;
});
}
// ============================================================================
// Agent Interaction
// ============================================================================
async sendMessage(agentId: string, text: string, options?: SendMessageOptions): Promise<void> {
this.send({
type: "send_agent_message",
agentId,
text,
messageId: options?.messageId,
images: options?.images,
});
}
async cancelAgent(agentId: string): Promise<void> {
this.send({ type: "cancel_agent_request", agentId });
}
async setAgentMode(agentId: string, modeId: string): Promise<void> {
this.send({ type: "set_agent_mode", agentId, modeId });
}
// ============================================================================
// Permissions
// ============================================================================
async respondToPermission(
agentId: string,
requestId: string,
response: AgentPermissionResponse
): Promise<void> {
this.send({
type: "agent_permission_response",
agentId,
requestId,
response,
});
}
// ============================================================================
// Waiting / Streaming
// ============================================================================
async waitForAgentIdle(agentId: string, timeout = 60000): Promise<AgentSnapshotPayload> {
return this.waitFor((msg) => {
if (msg.type === "agent_state" && msg.payload.id === agentId) {
if (msg.payload.status === "idle" || msg.payload.status === "error") {
return msg.payload;
}
}
return null;
}, timeout);
}
async waitForPermission(agentId: string, timeout = 30000): Promise<AgentPermissionRequest> {
return this.waitFor((msg) => {
if (msg.type === "agent_permission_request" && msg.payload.agentId === agentId) {
return msg.payload.request;
}
if (msg.type === "agent_stream" && msg.payload.agentId === agentId) {
if (msg.payload.event.type === "permission_requested") {
return msg.payload.event.request;
}
}
return null;
}, timeout);
}
// ============================================================================
// Event Subscription
// ============================================================================
on(handler: (event: DaemonEvent) => void): () => void {
this.eventListeners.add(handler);
return () => this.eventListeners.delete(handler);
}
// ============================================================================
// Internals
// ============================================================================
private send(message: SessionInboundMessage): void {
this.ws?.send(JSON.stringify({ type: "session", message }));
}
private handleSessionMessage(msg: SessionOutboundMessage): void {
this.messageQueue.push(msg);
// Notify event listeners
const event = this.toEvent(msg);
if (event) {
for (const handler of this.eventListeners) {
handler(event);
}
}
}
private toEvent(msg: SessionOutboundMessage): DaemonEvent | null {
switch (msg.type) {
case "agent_state":
return { type: "agent_state", agentId: msg.payload.id, payload: msg.payload };
case "agent_stream":
return { type: "agent_stream", agentId: msg.payload.agentId, event: msg.payload.event, timestamp: msg.payload.timestamp };
case "session_state":
return { type: "session_state", agents: msg.payload.agents };
case "status":
return { type: "status", payload: msg.payload };
default:
return null;
}
}
private async waitFor<T>(
predicate: (msg: SessionOutboundMessage) => T | null,
timeout = 30000
): Promise<T> {
const start = Date.now();
// Check queued messages first
for (const msg of this.messageQueue) {
const result = predicate(msg);
if (result !== null) return result;
}
// Wait for new messages
return new Promise((resolve, reject) => {
const check = (msg: SessionOutboundMessage) => {
const result = predicate(msg);
if (result !== null) {
resolve(result);
return true;
}
return false;
};
const handler = (data: WebSocket.Data) => {
const msg = JSON.parse(data.toString());
if (msg.type === "session" && check(msg.message)) {
this.ws?.off("message", handler);
clearTimeout(timer);
}
};
const timer = setTimeout(() => {
this.ws?.off("message", handler);
reject(new Error(`Timeout waiting for message (${timeout}ms)`));
}, timeout);
this.ws?.on("message", handler);
});
}
}
```
---
## Test Infrastructure
### Test Setup
```typescript
// packages/server/src/server/test-utils/daemon-test-context.ts
import { createTestPaseoDaemon, type TestPaseoDaemon } from "./paseo-daemon.js";
import { DaemonClient } from "./daemon-client.js";
export interface DaemonTestContext {
daemon: TestPaseoDaemon;
client: DaemonClient;
cleanup: () => Promise<void>;
}
export async function createDaemonTestContext(): Promise<DaemonTestContext> {
const daemon = await createTestPaseoDaemon();
const client = new DaemonClient({
url: `ws://127.0.0.1:${daemon.port}/ws`,
authHeader: daemon.agentMcpAuthHeader,
});
await client.connect();
return {
daemon,
client,
cleanup: async () => {
await client.close();
await daemon.close();
},
};
}
```
---
## Example E2E Tests
### Basic Flow
```typescript
// packages/server/src/server/daemon.e2e.test.ts
import { describe, test, expect, beforeEach, afterEach } from "vitest";
import { createDaemonTestContext, type DaemonTestContext } from "./test-utils/daemon-test-context.js";
describe("daemon E2E", () => {
let ctx: DaemonTestContext;
beforeEach(async () => {
ctx = await createDaemonTestContext();
});
afterEach(async () => {
await ctx.cleanup();
});
test("creates agent and receives response", async () => {
const agent = await ctx.client.createAgent({
provider: "codex",
cwd: "/tmp",
title: "Test Agent",
});
expect(agent.id).toBeTruthy();
expect(agent.provider).toBe("codex");
expect(agent.status).toBe("idle");
await ctx.client.sendMessage(agent.id, "Say hello");
const finalState = await ctx.client.waitForAgentIdle(agent.id);
expect(finalState.status).toBe("idle");
});
test("permission flow: approve", async () => {
const agent = await ctx.client.createAgent({
provider: "claude",
cwd: "/tmp",
});
await ctx.client.sendMessage(agent.id, "Run: rm -f test.txt");
const permission = await ctx.client.waitForPermission(agent.id);
expect(permission.name).toContain("Bash");
await ctx.client.respondToPermission(agent.id, permission.id, {
behavior: "allow",
});
const finalState = await ctx.client.waitForAgentIdle(agent.id);
expect(finalState.status).toBe("idle");
});
test("permission flow: deny", async () => {
const agent = await ctx.client.createAgent({
provider: "claude",
cwd: "/tmp",
});
await ctx.client.sendMessage(agent.id, "Run: rm -f test.txt");
const permission = await ctx.client.waitForPermission(agent.id);
await ctx.client.respondToPermission(agent.id, permission.id, {
behavior: "deny",
message: "Not allowed",
});
const finalState = await ctx.client.waitForAgentIdle(agent.id);
expect(finalState.status).toBe("idle");
});
test("agent persistence and resume", async () => {
const agent = await ctx.client.createAgent({
provider: "codex",
cwd: "/tmp",
title: "Persistent Agent",
});
await ctx.client.sendMessage(agent.id, "Remember: the password is 12345");
await ctx.client.waitForAgentIdle(agent.id);
// Delete active agent (keeps persistence)
await ctx.client.deleteAgent(agent.id);
// List persisted agents
const persisted = await ctx.client.listPersistedAgents();
const found = persisted.find((p) => p.title === "Persistent Agent");
expect(found).toBeTruthy();
// Resume
const resumed = await ctx.client.resumeAgent(found!.persistence!);
expect(resumed.id).toBeTruthy();
});
test("multi-agent: agent A launches agent B", async () => {
const agentA = await ctx.client.createAgent({
provider: "claude",
cwd: "/tmp",
title: "Parent Agent",
mcpServers: {
"agent-control": {
url: `http://127.0.0.1:${ctx.daemon.port}/mcp/agents`,
headers: { Authorization: ctx.daemon.agentMcpAuthHeader },
},
},
});
await ctx.client.sendMessage(
agentA.id,
"Create a new Codex agent titled 'Child Agent' and have it say hello"
);
// Wait for parent to finish (which includes child creation)
await ctx.client.waitForAgentIdle(agentA.id);
// Verify child agent was created
const agents = await ctx.client.listAgents();
const child = agents.find((a) => a.title === "Child Agent");
expect(child).toBeTruthy();
expect(child!.provider).toBe("codex");
});
});
```
---
## Files Summary
| File | Action | Description |
|------|--------|-------------|
| `packages/server/src/server/test-utils/daemon-client.ts` | Create | DaemonClient class (~300 lines) |
| `packages/server/src/server/test-utils/daemon-test-context.ts` | Create | Test setup helpers (~50 lines) |
| `packages/server/src/server/test-utils/index.ts` | Modify | Export new utilities |
| `packages/server/src/server/daemon.e2e.test.ts` | Create | E2E test suite (~200 lines) |
---
## Scope Estimate
- **DaemonClient implementation**: ~300-400 lines
- **Test context helpers**: ~50 lines
- **Example E2E tests**: ~200 lines
- **Total**: ~550-650 lines of new code
---
## Migration Path
1. **Phase 1**: Implement DaemonClient with core methods (connect, createAgent, sendMessage, waitForIdle)
2. **Phase 2**: Add permission methods (respondToPermission, waitForPermission)
3. **Phase 3**: Add persistence methods (listPersistedAgents, resumeAgent)
4. **Phase 4**: Add advanced features (event streaming, multi-agent tests)
5. **Phase 5**: Migrate existing E2E tests to use DaemonClient (optional)
Each phase is independently shippable and testable.
---
## Alternatives Considered
### Use MCP Client Instead
The existing `agent-mcp.e2e.test.ts` uses the MCP protocol to control agents. However:
- MCP is for agent-to-agent communication
- WebSocket is for client-to-daemon communication
- Testing WebSocket directly verifies the actual UI protocol
- MCP tests are complementary, not replacements
### Use HTTP API Only
The daemon has some HTTP endpoints, but:
- Agent streaming requires WebSocket
- Permission flow requires WebSocket
- Real-time state updates require WebSocket
- HTTP is only for health checks and static files
---
## Recommendation
Proceed with **Approach 1 (Simple WebSocket Wrapper)** for these reasons:
1. Lowest complexity, easiest to maintain
2. Full protocol transparency for debugging
3. Matches existing test patterns in the codebase
4. Can add convenience helpers later if needed
5. Estimated 2-3 hours to implement core functionality

38
plan.md
View File

@@ -906,3 +906,41 @@ Build a new Codex MCP provider sidebyside with the existing Codex SDK prov
- [x] **BUG**: Codex agent doesn't see agent-control MCP - Claude does.
- **Done (2025-12-25 22:45)**: WHAT: Fixed `buildCodexMcpConfig()` in `packages/server/src/server/agent/providers/codex-mcp-agent.ts:2337-2429` to include MCP servers in the Codex tool call. Added `CodexMcpServerConfig` and `CodexConfigPayload` types at lines 2324-2335. Added `managedAgentId` parameter to append caller agent ID to agent-control URL. Built MCP servers config including: (1) `agent-control` HTTP MCP with URL and `http_headers` (using Codex's field name, not `headers`), (2) `playwright` STDIO MCP server, (3) user-provided MCP servers from `config.mcpServers`. Added `managedAgentId` property to `CodexMcpAgentSession` class at line 2516. Updated `setManagedAgentId()` at lines 2907-2909 to store the ID. Updated all 3 call sites of `buildCodexMcpConfig()` at lines 2924, 2962, 2985 to pass `this.managedAgentId`. ROOT CAUSE: Claude provider at lines 672-699 builds MCP servers config and passes to Claude SDK. Codex MCP provider at line 2360 only passed `config.extra.codex` - completely ignoring `config.agentControlMcp` and `config.mcpServers`. Codex CLI expects MCP servers in `config.mcp_servers` field with `http_headers` (not `headers`) for HTTP servers. RESULT: Codex agents now receive agent-control and playwright MCP servers in tool call config. EVIDENCE: Typecheck passes (`npm run typecheck --workspace=@paseo/server`), unit test passes (`npm run test --workspace=@paseo/server -- codex-mcp-agent.test.ts -t "responds with text"`), quick verification script shows MCP config includes `agent-control` with URL and headers.
- [x] **Plan**: Design daemon client for E2E testing.
**Objective**: Create a daemon client that enables full E2E testing of the daemon WITHOUT Playwright. Test the daemon directly via WebSocket, ensure correctness, and UI correctness follows.
**Requirements**:
1. **DaemonClient class**:
- Constructor takes WebSocket URL (e.g., `ws://localhost:6767`)
- All WebSocket code isolated to this client - no WS code elsewhere in tests
- Methods for all daemon operations:
- `createAgent(config)` → returns agent ID
- `listAgents()` → returns agent list
- `sendMessage(agentId, message)` → streams response events
- `getAgentState(agentId)` → returns current state
- `deleteAgent(agentId)`
- `respondToPermission(agentId, callId, decision)`
- etc.
2. **Test infrastructure**:
- Launch daemon in-process for tests
- Use temp directory for `PASEO_HOME` (isolate from real agents.json)
- Clean up after tests
3. **E2E test examples to enable**:
- Create agent → send message → verify response
- Create agent → trigger tool call → approve permission → verify execution
- Create agent → persist → reload → verify state restored
- Multi-agent: create agent A → have A launch agent B → verify both work
**Output of this task**:
- Propose 2-3 architectural approaches for the DaemonClient
- Identify key files that need changes
- Estimate scope for each approach
- Recommend best approach with rationale
**DO NOT implement yet - just plan and propose options.**
- **Done (2025-12-25 23:15)**: WHAT: Authored comprehensive design report in `REPORT-daemon-client-design.md:1` covering 3 architectural approaches (Simple WebSocket Wrapper, Reactive Event Store, Hybrid), with full type definitions, implementation sketches, and example E2E tests. RESULT: Recommended Approach 1 (Simple WebSocket Wrapper) for ~300-400 lines of code, leveraging existing `messages.ts` Zod schemas and `test-utils/paseo-daemon.ts` infrastructure. Key files identified: `daemon-client.ts` (new), `daemon-test-context.ts` (new), `daemon.e2e.test.ts` (new E2E suite). EVIDENCE: `REPORT-daemon-client-design.md` contains: (1) 3 approaches with pros/cons, (2) full TypeScript interface for DaemonClient, (3) implementation code for core methods (connect, createAgent, sendMessage, waitForAgentIdle, respondToPermission), (4) test context helper, (5) 5 example E2E tests (basic flow, permission approve/deny, persistence/resume, multi-agent), (6) scope estimate of ~550-650 total lines, (7) 5-phase migration path.