Help agents diagnose Paseo provider problems (#3243)

* feat(skills): add Paseo help and provider diagnostics

Give support agents a topology-aware troubleshooting entry point backed by the deployed documentation index, and expose the daemon provider diagnostic through the CLI so agents can inspect the affected host directly. Agent-launching skills now use configured profiles instead of the removed orchestration preferences.

* test(cli): keep provider diagnostics on real boundaries

Remove the internal client mock and preserve its normalization and host-targeting assertions through the existing isolated-daemon coverage.
This commit is contained in:
Mohamed Boudra
2026-08-12 00:57:33 +02:00
committed by GitHub
parent 5f5c585e68
commit d012b6f071
10 changed files with 239 additions and 78 deletions
@@ -0,0 +1,45 @@
import type { Command } from "commander";
import { connectToDaemon } from "../../utils/client.js";
import type { CommandOptions, OutputSchema, SingleResult } from "../../output/index.js";
export interface ProviderDiagnosticResult {
provider: string;
diagnostic: string;
}
export const providerDiagnosticSchema: OutputSchema<ProviderDiagnosticResult> = {
idField: "provider",
columns: [
{ header: "PROVIDER", field: "provider" },
{ header: "DIAGNOSTIC", field: "diagnostic" },
],
renderHuman(result) {
if (result.type === "single") return result.data.diagnostic;
return result.data.map((entry) => entry.diagnostic).join("\n\n");
},
};
export interface ProviderDiagnosticOptions extends CommandOptions {
host?: string;
}
export async function runDiagnosticCommand(
provider: string,
options: ProviderDiagnosticOptions,
_command: Command,
): Promise<SingleResult<ProviderDiagnosticResult>> {
const client = await connectToDaemon({ host: options.host });
try {
const result = await client.getProviderDiagnostic(provider.trim().toLowerCase());
return {
type: "single",
data: {
provider: result.provider,
diagnostic: result.diagnostic,
},
schema: providerDiagnosticSchema,
};
} finally {
await client.close();
}
}
@@ -1,6 +1,7 @@
import { Command } from "commander";
import { runLsCommand } from "./ls.js";
import { runModelsCommand } from "./models.js";
import { runDiagnosticCommand } from "./diagnostic.js";
import { withOutput } from "../../output/index.js";
import { addJsonAndDaemonHostOptions } from "../../utils/command-options.js";
@@ -19,5 +20,12 @@ export function createProviderCommand(): Command {
.option("--thinking", "Include thinking option IDs for each model"),
).action(withOutput(runModelsCommand));
addJsonAndDaemonHostOptions(
provider
.command("diagnostic")
.description("Show provider installation, environment, and availability diagnostics")
.argument("<provider>", "Provider name"),
).action(withOutput(runDiagnosticCommand));
return provider;
}
+41
View File
@@ -17,6 +17,8 @@
* - provider models opencode lists opencode models
* - provider models unknown fails with error
* - provider models --json outputs valid JSON
* - provider diagnostic shows the daemon's provider diagnostic
* - provider diagnostic --json returns structured output
*/
import assert from "node:assert";
@@ -44,6 +46,11 @@ interface ProviderListRow {
enabled: string;
}
interface ProviderDiagnostic {
provider: string;
diagnostic: string;
}
const EXPECTED_CLAUDE_MODELS = [
{
id: "claude-opus-5",
@@ -190,6 +197,7 @@ try {
assert.strictEqual(result.exitCode, 0, "provider --help should exit 0");
assert(result.stdout.includes("ls"), "help should mention ls");
assert(result.stdout.includes("models"), "help should mention models");
assert(result.stdout.includes("diagnostic"), "help should mention diagnostic");
console.log("✓ provider --help shows subcommands\n");
}
@@ -424,6 +432,39 @@ try {
);
console.log("✓ provider models --quiet outputs model IDs only\n");
}
// Test 12: provider diagnostic shows the daemon's provider diagnostic
{
console.log("Test 12: provider diagnostic shows the daemon's provider diagnostic");
const result = await ctx.paseo([
"provider",
"diagnostic",
" Claude ",
"--host",
`127.0.0.1:${ctx.port}`,
]);
assert.strictEqual(result.exitCode, 0, "provider diagnostic should exit 0");
assert(result.stdout.includes("Claude Code"), "diagnostic should identify the provider");
assert(result.stdout.includes("Daemon PATH:"), "diagnostic should include the daemon PATH");
assert(result.stdout.includes("Resolved path:"), "diagnostic should include binary resolution");
assert(result.stdout.includes("Version:"), "diagnostic should include the provider version");
assert(result.stdout.includes("Status:"), "diagnostic should include provider status");
console.log("✓ provider diagnostic shows the daemon's provider diagnostic\n");
}
// Test 13: provider diagnostic --json returns structured output
{
console.log("Test 13: provider diagnostic --json returns structured output");
const result = await ctx.paseo(["provider", "diagnostic", "claude", "--json"]);
assert.strictEqual(result.exitCode, 0, "provider diagnostic --json should exit 0");
const data = JSON.parse(result.stdout.trim()) as ProviderDiagnostic;
assert.strictEqual(data.provider, "claude", "JSON should identify the provider");
assert(data.diagnostic.includes("Daemon PATH:"), "JSON should include the daemon PATH");
assert(data.diagnostic.includes("Resolved path:"), "JSON should include binary resolution");
assert(data.diagnostic.includes("Version:"), "JSON should include the provider version");
assert(data.diagnostic.includes("Status:"), "JSON should include provider status");
console.log("✓ provider diagnostic --json returns structured output\n");
}
} finally {
await ctx.stop();
}
+12
View File
@@ -23,6 +23,18 @@ paseo logs <id> # View agent timeline
paseo stop <id> # Stop an agent
```
## Provider diagnostics
Ask the daemon to inspect the provider environment it actually uses:
```bash
paseo provider diagnostic claude
paseo provider diagnostic codex --json
paseo provider diagnostic opencode --host devbox:6767
```
The diagnostic includes the configured command, daemon `PATH` and shell, matching binaries, resolved path, version, model count, and provider status. Use `--host` for a remote daemon. This is the same diagnostic shown under **Settings → your host → Providers → provider → Diagnostic**.
## Running agents
Use `paseo run` to start a new agent with a task:
+9
View File
@@ -24,6 +24,15 @@ Open **Settings → your host → Providers**, tap the provider, then tap **Diag
- **Daemon PATH** — the `PATH` Paseo is searching. Compare it to `echo $PATH` in a fresh terminal.
- **Version** — whether the binary actually runs.
From a terminal or agent, request the same diagnostic from the affected daemon:
```bash
paseo provider diagnostic <provider>
paseo provider diagnostic <provider> --host <host:port> --json
```
Use `--host` when the affected daemon is not the CLI's default local daemon.
`not found` together with a **Daemon PATH** that's missing your binary's directory is the common case: that directory is on your terminal's `PATH` but not on Paseo's.
### Fix it
+7 -8
View File
@@ -2,7 +2,7 @@
name: paseo-advisor
description: Spin up a single agent as an advisor — second opinion on the current task. Use when the user says "advisor", "second opinion", "what does X think", or wants an outside take without delegating the work itself.
user-invocable: true
argument-hint: "[--provider <name>] <question or topic>"
argument-hint: "[--profile <name>] <question or topic>"
---
# Paseo Advisor
@@ -13,16 +13,15 @@ Single agent. Reads the situation you're in. Gives a judgment. You decide what t
## Prerequisites
Read the **paseo** skill. Before choosing a provider, read `~/.paseo/orchestration-preferences.json` unless the user explicitly named a provider in this request. Do not create the advisor until you have read it.
Read the **paseo** skill. Call `list_profiles` before choosing the advisor. Do not create the advisor until you have read the configured profiles and their `notes`.
## Picking the advisor
1. **User named one** (`--provider claude/opus`) → use it.
2. **Otherwise** resolve from preferences — pick the category that matches the question:
- Design / approach question → `planning`
- "Did I miss something" review → `audit`
- "Is this even right" → `research`
3. **Contrast helps.** If your own provider matches what preferences would pick, swap to a different family on purpose — fresh perspective is the point.
1. **User named a profile** (`--profile UI Work`) → select it by name.
2. **Otherwise** choose the profile whose `notes` best fit the question. Match the actual work: design and approach, audit and review, or research and root-cause analysis.
3. **Contrast helps.** When several profiles fit, prefer a different provider family from your own so the second opinion is genuinely fresh.
Materialize the selected profile into `create_agent` as described by the **paseo** skill. If no profile fits, use Paseo's provider discovery fallback.
## The briefing
+7 -7
View File
@@ -6,7 +6,7 @@ user-invocable: true
# Committee Skill
Two agents from contrasting providers, fresh context, planning a solution in parallel. They stay alive for review after implementation.
Two agents from contrasting profiles, fresh context, planning a solution in parallel. They stay alive for review after implementation.
The purpose is to step back, not double down. The committee may propose a completely different approach.
@@ -14,18 +14,18 @@ The purpose is to step back, not double down. The committee may propose a comple
## Prerequisites
Read the **paseo** skill. Before choosing committee members, read `~/.paseo/orchestration-preferences.json` unless the user explicitly named providers in this request. Do not create committee agents until you have read it.
Read the **paseo** skill. Call `list_profiles` before choosing committee members. Do not create committee agents until you have read the configured profiles and their `notes`.
Contrast is the point of a committee, so pick across providers deliberately using the configured preferences rather than hardcoded defaults.
Contrast is the point of a committee, so pick profiles from different provider families when possible. Materialize each profile into `create_agent`.
## Composition
Two members with different reasoning styles, selected from orchestration preferences:
Two members with different reasoning styles, selected from configured Agent profiles:
- one planning/research-strength provider
- one contrasting high-reasoning provider
- one whose notes fit planning, research, or root-cause analysis
- one contrasting high-reasoning profile from another provider family
Override only when the user explicitly asks for different members.
If the user names profiles, use those. If fewer than two suitable profiles are configured, use Paseo's provider discovery fallback for the missing member and tell the user. Override the selection only when the user explicitly asks for different members.
## Hard rules
+2 -2
View File
@@ -12,11 +12,11 @@ Transfer the current task — context, decisions, failed attempts, constraints
## Prerequisites
Read the **paseo** skill. Before choosing a provider, read `~/.paseo/orchestration-preferences.json` unless the user explicitly named a provider in this request. Do not create the receiving agent until you have read it.
Read the **paseo** skill. Call `list_profiles` before choosing the receiving agent. Do not create it until you have read the configured profiles and their `notes`.
## Parsing arguments
1. **Provider** — explicit user request first; otherwise resolve from `impl` preference (or `ui` if the task is styling-only).
1. **Agent profile** — explicit profile name first; otherwise choose the profile whose `notes` best match the work. Materialize it into `create_agent` as described by the **paseo** skill. If no profile fits, use Paseo's provider discovery fallback.
2. **Isolation** — "in a worktree" / "worktree" → create a workspace with `isolation: "worktree"`, using a short branch name derived from the task.
3. **Task description** — anything else the user said.
+92
View File
@@ -0,0 +1,92 @@
---
name: paseo-help
description: Answer questions about the Paseo product and app, including setup, configuration, connectivity, providers, workspaces, updates, logs, and troubleshooting. Use when a user inside Paseo asks how Paseo works, how to configure it, or why something is broken; use the paseo skill instead to operate agents and workspaces through MCP or the CLI.
---
# Paseo Help
You are helping a user understand, configure, or troubleshoot Paseo itself. Answer their question directly, verify the answer against the current public documentation, and include the relevant documentation link. Do not send the user away to read the docs in place of helping them.
**User's question:** $ARGUMENTS
## Use current documentation
Fetch [https://paseo.sh/llms.txt](https://paseo.sh/llms.txt) first. It is the current index of Paseo documentation, with a description and Markdown URL for each page.
Use that index to select the page that owns the user's question, then fetch the linked `.md` page before answering. For troubleshooting, begin with [Common problems](https://paseo.sh/docs/troubleshooting.md) and follow its links when the issue belongs to a more specific page.
Prefer the deployed docs over memory. Answer the user directly, then link the relevant `.md` page as supporting documentation.
## Establish the topology first
Identify the daemon involved before diagnosing versions, paths, providers, logs, updates, or connectivity. Do not infer the daemon from the client: Paseo Desktop can manage its bundled local daemon and connect to other remote daemons at the same time.
Establish two facts:
1. **Where and how the daemon runs**
- **Desktop-managed:** Paseo Desktop bundles, starts, and updates a daemon on that computer. No separate daemon install is required.
- **Standalone:** the daemon was installed separately, commonly through the npm CLI, and runs independently of the desktop app.
- **Docker:** the daemon, its home, provider CLIs, credentials, and code mounts live in the container runtime.
2. **How the affected client reaches it**
- same-machine local connection
- relay connection
- direct LAN, VPN, or Tailscale connection
- daemon-served web UI
Use **Settings → About** to compare the app version with each connected host. For the affected host, open **Settings → your host → Overview → Full status**. On the daemon machine, `paseo daemon status --json` reports facts such as server ID, hostname, version, home, listen address, process owner, log path, and whether the daemon is desktop-managed.
Record which host the user is viewing and which machine or container runs it. A local `paseo daemon status` describes the daemon for that CLI's local `PASEO_HOME`; it may not be the remote host visible in the app.
Apply later checks to the daemon runtime, not automatically to the client device:
- Provider binaries, credentials, `PATH`, workspaces, config, and daemon logs live on the daemon machine or inside its container.
- App version and app logs live on the client device.
- A desktop-managed daemon follows the Desktop app lifecycle and update path.
- A standalone daemon follows its own CLI/npm lifecycle and may use a different `PASEO_HOME` or listen address.
- A Docker daemon uses container paths, volumes, user permissions, image versions, and container lifecycle commands.
## Diagnose before changing state
After identifying the affected host, compare that daemon's version with the client app version. Ask the user to update both through the correct topology-specific update path. Old versions and app/daemon version skew cause many apparent bugs, and fixes ship frequently. Use the Updates page and the installation-specific docs for current instructions.
Use the smallest relevant read-only checks:
```bash
paseo --version
paseo daemon status --json
paseo provider diagnostic <provider> --json
```
Use the status-reported home, listen address, and log path for further checks. Probe `http://127.0.0.1:6767/api/health` or read `~/.paseo/daemon.log` only when those values match the affected daemon. Do not restart the daemon, edit config, update software, or expose a network listener without the user's explicit permission. A daemon restart can interrupt the agent doing the diagnosis.
For a missing provider or `command not found`, run `paseo provider diagnostic <provider>` against the affected host, or open **Settings → your host → Providers → provider → Diagnostic**. Compare its resolved binary, daemon `PATH`, and provider version with a brand-new login shell. Shell aliases and functions are not executable paths.
## Logs and local files
Use these defaults on the machine where the daemon or Desktop app actually runs. Do not look for a remote daemon's files on the client device.
- Daemon config: `~/.paseo/config.json`
- Daemon log: `~/.paseo/daemon.log`
- Agent state directory: `~/.paseo/agents/`
- Default managed worktree root: `~/.paseo/worktrees/`
- macOS desktop log: `~/Library/Logs/Paseo/main.log`
- Linux desktop log: `~/.config/Paseo/logs/main.log`
- Windows desktop log: `%APPDATA%\Paseo\logs\main.log`
Substitute the status-reported `PASEO_HOME` for `~/.paseo`. In the official Docker image, the default is `/home/paseo/.paseo`; its host path depends on the volume mount, and container stdout is available through Docker. Desktop app logs describe the Desktop process; daemon logs describe the selected daemon. Read the narrowest useful slice and redact credentials, pairing offers, tokens, passwords, and user code before sharing logs.
If diagnosing the bundled daemon on a computer with Paseo Desktop installed, but `paseo` is not on `PATH`, the bundled CLI is at:
- macOS: `/Applications/Paseo.app/Contents/Resources/bin/paseo`
- Linux: `<install-dir>/resources/bin/paseo`
- Windows: `C:\Program Files\Paseo\resources\bin\paseo.cmd`
Offer to fix the PATH or symlink; do not change shell configuration silently.
## Escalate with evidence
If the current docs and diagnostics do not resolve the problem, collect the app and daemon versions, OS, install method, connection method, exact error, minimal reproduction, and a small redacted log excerpt.
- Bugs: [GitHub Issues](https://github.com/getpaseo/paseo/issues)
- Questions and quick help: [Paseo Discord](https://discord.gg/jz8T2uahpH)
- Product workflow discussions: [GitHub Discussions](https://github.com/getpaseo/paseo/discussions) or `#product` in Discord
+16 -61
View File
@@ -37,7 +37,7 @@ paseo script stop <name> [--cwd <path> | --workspace <workspace-id>]
**`create_agent`** — required: `title`, `provider` (`claude/opus`, `codex/gpt-5.4`, …), `initialPrompt`. Optional: `workspaceId`, `notifyOnFinish`, `settings`, `labels`. Returns `{ agentId, workspaceId, … }`.
Initial runtime settings live under `settings`: `modeId`, `thinkingOptionId`, and provider-specific `features`. For Codex fast mode, pass `settings: { features: { "fast_mode": true } }` when creating the agent.
Initial runtime settings live under `settings`: `modeId`, `thinkingOptionId`, and provider-specific `features`. Agent profiles are the preferred source for these values. For Codex fast mode, pass `settings: { features: { "fast_mode": true } }` when creating the agent.
Agent-scoped creation always creates your subagent. Omit `workspaceId` to use your current workspace; pass a workspace returned by `create_workspace` for isolated delegation. Placement never changes parentage.
@@ -53,7 +53,20 @@ Agent-scoped `create_agent` defaults `notifyOnFinish` to true. Set it to `false`
**`archive_agent`** — `{ agentId }`. Interrupts if running, removes from active list.
## Provider discovery
## Agent profiles and provider discovery
**`list_profiles`** — named launch bundles configured by the human. Before choosing how to launch a delegated agent, call this tool and read every profile's `notes`. Pick a named profile the user requested, or the profile whose notes best match the work.
There is no `profile` parameter on `create_agent`. Materialize the selected profile into the call:
- combine `provider` and `model` as the `provider/model` value for `create_agent.provider`
- copy `modeId` to `settings.modeId`
- copy `thinkingOptionId` to `settings.thinkingOptionId`
- copy `featureValues` to `settings.features`
Omit absent values. Do not remember a selected profile or infer drift later; a profile is only launch configuration.
If no profile fits, or no profiles are configured, use the provider discovery tools below rather than guessing. Tell the user when you fall back because no configured profile fits.
**`list_providers`** — compact provider availability and modes.
@@ -73,36 +86,6 @@ Only set feature IDs returned by `inspect_provider`. For Codex fast mode, look f
Schedules have the full list/inspect/update/pause/resume/run-once/log/delete surface. Heartbeats deliberately do not.
## Orchestration preferences
User-specific configuration at `~/.paseo/orchestration-preferences.json`. **Before any Paseo skill chooses a provider or creates an agent, it must read this file.** Reading means an actual file read, not relying on these examples or defaults. Never hardcode a provider string in another skill — resolve through this file.
Two parts:
- `providers` — map of role categories to provider strings. Pass straight to `create_agent`'s `provider` field.
- `preferences` — freeform string array. Read on startup; weave into agent prompts contextually.
Categories: `impl`, `ui`, `research`, `planning`, `audit`. Skills pick the category that matches the role they're launching.
```json
{
"providers": {
"impl": "codex/gpt-5.4",
"ui": "claude/opus",
"research": "codex/gpt-5.4",
"planning": "codex/gpt-5.4",
"audit": "codex/gpt-5.4"
},
"preferences": [
"Claude Opus is the right choice for anything artistic or human-skill-oriented: copywriting, naming, UX copy, visual design, styling. Codex is the workhorse for mechanical work."
]
}
```
If the file is missing, use sensible defaults and tell the user once.
All providers shown in this document are examples, you must resolve real providers either found in the preferences or by calling Paseo provider tooling to find a valid provider.
## Waiting
Agents take time — 10–30+ minutes is routine. Favor asynchronous workflows.
@@ -129,32 +112,4 @@ paseo heartbeat create --cron "*/15 * * * *" "check the build"
Discover with `paseo --help` and `paseo <cmd> --help`.
**If `paseo` isn't on PATH but the desktop app is installed**, the bundled CLI is at:
- macOS: `/Applications/Paseo.app/Contents/Resources/bin/paseo`
- Linux: `<install-dir>/resources/bin/paseo`
- Windows: `C:\Program Files\Paseo\resources\bin\paseo.cmd`
The desktop app's first-run hook (`installCli`) symlinks this to `~/.local/bin/paseo` (macOS/Linux) or drops a `.cmd` trampoline (Windows) and adds `~/.local/bin` to PATH via shell rc files. If that didn't take, offer to symlink it — don't do it silently.
## Ops and debugging
Daemon-client architecture: the daemon owns agent lifecycle, state, and the WebSocket API. Tools, CLI, mobile, and desktop apps are all clients.
| | Default |
| -------------- | --------------------------------------------------------------- |
| Listen address | `127.0.0.1:6767` (override `PASEO_LISTEN`) |
| Home | `~/.paseo` (override `PASEO_HOME`) |
| Daemon log | `$PASEO_HOME/daemon.log` |
| Agent state | `$PASEO_HOME/agents/<id>.json` |
| Worktrees | `$PASEO_HOME/worktrees/` (or `worktrees.root` in `config.json`) |
| PID file | `$PASEO_HOME/paseo.pid` |
| Health | `GET http://127.0.0.1:6767/api/health` |
Debug order:
1. `tail -n 200 ~/.paseo/daemon.log`.
2. `paseo daemon status` for liveness.
3. `curl -s localhost:6767/api/health` if the CLI itself is suspect.
**Never restart the daemon without explicit user approval** — it kills every running agent, including, often, the one asking.
For product questions, setup, logs, version problems, or troubleshooting, use the **paseo-help** skill.