From 42d1a2e4e3dd2d4a95c9a4d6c87a03d768db81ca Mon Sep 17 00:00:00 2001
From: Mohamed Boudra <boudra.moha@gmail.com>
Date: Sat, 4 Jul 2026 10:19:53 +0200
Subject: [PATCH] docs(browser): document browser automation

---
 CHANGELOG.md                 |  9 +++-
 public-docs/browser-tools.md | 78 ++++++++++++++++++++++++++++++++++
 public-docs/browser.md       | 82 ++++++++++++++++++++++++++++++++++++
 3 files changed, 167 insertions(+), 2 deletions(-)
 create mode 100644 public-docs/browser-tools.md
 create mode 100644 public-docs/browser.md

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6793f7466..dde425b44 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,21 +1,26 @@
 # Changelog
 
-## 0.1.104-beta.2 - 2026-07-03
+## 0.1.104-beta.3 - 2026-07-04
 
 ### Added
 
-- Agents can drive the in-app browser: open tabs, navigate, click, type, and take screenshots
+- Agents can drive the in-app browser with page snapshots, trusted input, dialogs, and tab controls ([#1881](https://github.com/getpaseo/paseo/pull/1881))
 - Inspect, annotate, and send page elements from a browser tab to the agent ([#1708](https://github.com/getpaseo/paseo/pull/1708) by [@huiliaoning](https://github.com/huiliaoning))
 - Schedules screen to create and manage recurring agents ([#1246](https://github.com/getpaseo/paseo/pull/1246))
 - Open a project from anywhere with Cmd+O ([#1849](https://github.com/getpaseo/paseo/pull/1849))
+- Agents can rename workspaces after they understand the task ([#1876](https://github.com/getpaseo/paseo/pull/1876))
+- Claude Ultra Code is available for supported Claude models ([#1872](https://github.com/getpaseo/paseo/pull/1872))
 - ByteDance TRAE CLI available as an agent provider ([#1831](https://github.com/getpaseo/paseo/pull/1831) by [@park0er](https://github.com/park0er))
 
 ### Improved
 
+- Workspaces created by agents now get readable generated names ([#1887](https://github.com/getpaseo/paseo/pull/1887))
+- Browser tabs opened by agents stay in the background until you switch to them ([#1875](https://github.com/getpaseo/paseo/pull/1875))
 - Clearer cards when an agent asks a question ([#1643](https://github.com/getpaseo/paseo/pull/1643) by [@cleiter](https://github.com/cleiter))
 
 ### Fixed
 
+- Docker images keep running during provider cleanup and diagnostics ([#1877](https://github.com/getpaseo/paseo/pull/1877))
 - New Workspace drafts survive archiving a workspace ([#1838](https://github.com/getpaseo/paseo/pull/1838))
 - Composer autocomplete stays open after switching screens ([#1851](https://github.com/getpaseo/paseo/pull/1851))
 - Claude usage appears when a quota window has no scheduled reset ([#1855](https://github.com/getpaseo/paseo/pull/1855))
diff --git a/public-docs/browser-tools.md b/public-docs/browser-tools.md
new file mode 100644
index 000000000..4d5b7a3f4
--- /dev/null
+++ b/public-docs/browser-tools.md
@@ -0,0 +1,78 @@
+---
+title: Browser tools
+description: The browser_* MCP tools agents use to drive Paseo browser tabs.
+nav: Tools reference
+order: 36
+category: Browser
+---
+
+# Browser tools
+
+The `browser_*` tools are injected into agents alongside the other [Paseo MCP tools](/docs/mcp) when [browser automation](/docs/browser) is enabled.
+
+Shared concepts:
+
+- **`browserId`** identifies a tab. It comes from `browser_new_tab` or `browser_list_tabs` and is required by every tab-scoped tool.
+- **`ref`** identifies an element, e.g. `@e3`. Refs come from the latest `browser_snapshot` of the same tab and expire when the page changes — stale refs return an error instead of acting on the wrong element.
+- Every result reports **dialogs** the page opened during the command (alerts accepted; confirm/prompt/beforeunload dismissed).
+
+Arguments marked `?` are optional.
+
+## Tabs
+
+| Tool                | Arguments                  | Purpose                                                                   |
+| ------------------- | -------------------------- | ------------------------------------------------------------------------- |
+| `browser_list_tabs` | —                          | List open tabs in the agent's workspace across connected hosts.           |
+| `browser_new_tab`   | `url?`                     | Open a tab in the background and return its `browserId`.                  |
+| `browser_close_tab` | `browserId`                | Close a tab and clean up its webview.                                     |
+| `browser_resize`    | `browserId, width, height` | Resize the tab's viewport — check a layout at phone or tablet dimensions. |
+
+## Reading the page
+
+| Tool                 | Arguments                              | Purpose                                                                               |
+| -------------------- | -------------------------------------- | ------------------------------------------------------------------------------------- |
+| `browser_snapshot`   | `browserId`                            | Return the page as an accessibility tree with element refs.                           |
+| `browser_screenshot` | `browserId, fullPage?`                 | Capture a PNG of the viewport, or the full page with `fullPage`.                      |
+| `browser_logs`       | `browserId, maxEntries?`               | Read recent console messages and network timing entries.                              |
+| `browser_wait`       | `browserId, text? \| url?, timeoutMs?` | Wait until the page contains text or reaches a URL fragment (exactly one of the two). |
+
+## Interacting
+
+| Tool               | Arguments                                           | Purpose                                                                              |
+| ------------------ | --------------------------------------------------- | ------------------------------------------------------------------------------------ |
+| `browser_click`    | `browserId, ref, button?, doubleClick?, modifiers?` | Click an element — left/right/middle, double-click, keyboard modifiers.              |
+| `browser_fill`     | `browserId, ref, value`                             | Set the value of an input-like element.                                              |
+| `browser_type`     | `browserId, text, ref?`                             | Type text into an element, or into the focused element when `ref` is omitted.        |
+| `browser_keypress` | `browserId, key, ref?`                              | Press a key (`Enter`, `Escape`, `Tab`, `Space`, …) on an element or the focused one. |
+| `browser_hover`    | `browserId, ref`                                    | Hover an element — triggers real CSS `:hover`.                                       |
+| `browser_select`   | `browserId, ref, value`                             | Choose an option in a `<select>`.                                                    |
+| `browser_drag`     | `browserId, sourceRef, targetRef`                   | Drag one element onto another.                                                       |
+| `browser_upload`   | `browserId, ref, filePaths`                         | Set files on a file input. Paths must be inside the agent's workspace.               |
+| `browser_scroll`   | `browserId, deltaX, deltaY, ref?`                   | Scroll the page, or center the wheel input over an element with `ref`.               |
+
+## Navigation
+
+| Tool               | Arguments        | Purpose                                             |
+| ------------------ | ---------------- | --------------------------------------------------- |
+| `browser_navigate` | `browserId, url` | Go to an `http(s)` URL.                             |
+| `browser_back`     | `browserId`      | Go back — errors when there is no history to go to. |
+| `browser_forward`  | `browserId`      | Go forward — errors when there is no forward entry. |
+| `browser_reload`   | `browserId`      | Reload the page.                                    |
+
+## Scripting
+
+| Tool               | Arguments                   | Purpose                                                                                                                                  |
+| ------------------ | --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
+| `browser_evaluate` | `browserId, function, ref?` | Run a JavaScript function in the page. With `ref`, the resolved element is passed as the first argument. Results return as bounded JSON. |
+
+## Errors
+
+Tools return structured errors rather than failing silently. The ones agents see most:
+
+| Code                | Meaning                                                                         |
+| ------------------- | ------------------------------------------------------------------------------- |
+| `browser_disabled`  | Browser tools are turned off on this host.                                      |
+| `browser_no_host`   | No browser host (desktop app) is connected. Retryable.                          |
+| `browser_stale_ref` | The ref no longer matches the page — take a new snapshot.                       |
+| `browser_timeout`   | The element never became actionable, or the wait condition never held.          |
+| `browser_denied`    | The action isn't allowed — e.g. a non-`http(s)` URL, or no history to navigate. |
diff --git a/public-docs/browser.md b/public-docs/browser.md
new file mode 100644
index 000000000..4117730d2
--- /dev/null
+++ b/public-docs/browser.md
@@ -0,0 +1,82 @@
+---
+title: Browser automation
+description: Give agents a real browser inside Paseo to open pages, click, type, and verify their work.
+nav: Browser automation
+order: 35
+category: Browser
+---
+
+# Browser automation
+
+Agents in Paseo can drive real browser tabs — the same tabs you see in the Paseo desktop app. An agent can open your dev server, read the page, click through a flow, fill a form, and take a screenshot, all without leaving your machine.
+
+This closes the loop on frontend work: instead of telling you "the change should work," an agent opens the page and checks.
+
+Typical uses:
+
+- **Verify its own changes.** After editing a component, the agent opens the dev server, snapshots the page, and confirms the new text or layout is actually there.
+- **Reproduce and diagnose bugs.** Click the exact sequence from a bug report, then read the console and network logs.
+- **Exercise full flows.** Forms, multi-step wizards, hover menus, drag and drop, file uploads.
+- **Work in logged-in sessions.** Tabs keep their session state. Log in once yourself, and the agent can work behind the login.
+
+Because you share the browser with the agent, you can watch it work — and step in at any point.
+
+## Enabling
+
+Browser tools are off by default. Turn them on per host:
+
+- **In the app:** open your host's settings and enable **Browser tools**.
+- **In `config.json`** (`~/.paseo/config.json`):
+
+```json
+{
+  "daemon": {
+    "browserTools": {
+      "enabled": true
+    }
+  }
+}
+```
+
+The tools are part of the [Paseo MCP toolset](/docs/mcp), so **Inject Paseo tools** (`daemon.mcp.injectIntoAgents`) must also be on for agents to receive them. Existing agents may need a reload to pick up new tools.
+
+> Browser tools let agents access and control Paseo browser tabs, including logged-in browser state. Only enable this for agents you trust.
+
+## Desktop only, for now
+
+Browser tabs are hosted by the Paseo desktop app. The daemon itself doesn't run a browser — it routes tool calls to a connected desktop app, and returns an error when none is connected. The wire contract is host-neutral, so other hosts can carry the same tools later.
+
+## How an agent sees a page
+
+The primary tool is `browser_snapshot`, which returns the page as an accessibility tree — headings, text, form state, and hierarchy — instead of raw HTML:
+
+```yaml
+- document "Settings"
+  - heading "Workspace" [level=1]
+  - text: "Connected as Maya"
+  - textbox "Display name" [ref=@e2]
+  - button "Save changes" [ref=@e3]
+```
+
+Interactive elements carry refs like `@e3`. The agent passes a ref to `browser_click`, `browser_fill`, and the other action tools. Refs come from the latest snapshot of that tab and expire when the page changes — a stale ref returns an error instead of acting on the wrong element.
+
+For anything the tree can't capture, agents fall back to `browser_screenshot`, and `browser_logs` exposes console messages and network timing.
+
+## Architecture
+
+```
+agent ──MCP──▶ daemon (broker) ──▶ browser host (desktop app) ──▶ webview
+```
+
+- **Workspace-scoped tabs.** An agent only sees and controls tabs in its own workspace. New tabs open in the background without stealing your focus.
+- **Tab-to-host routing.** The daemon remembers which host owns each tab and routes tab commands there. `browser_list_tabs` aggregates all connected hosts.
+- **Trusted input.** Clicks, keys, hovers, and drags are dispatched as real browser input events — CSS `:hover` triggers, and pages can't tell an agent's click from a user's. Every action first waits for its target to be visible, enabled, and stable.
+- **Dialogs never block.** `alert` is accepted; `confirm`, `prompt`, and `beforeunload` are dismissed. Every handled dialog is reported in the tool result so the agent knows the page flow changed.
+
+## Security
+
+- Navigation is restricted to `http(s)` URLs.
+- File uploads can only reference files inside the agent's workspace.
+- Tabs share the browser profile you use in Paseo, including cookies and logins — that's what makes logged-in testing work, and why the feature is opt-in per host.
+
+See the [tools reference](/docs/browser-tools) for the full tool list.