mirror of https://github.com/getpaseo/paseo.git synced 2026-07-29 12:01:31 +00:00

Files

Mohamed Boudra 20385bdb50 Add opt-in browser tools for desktop tabs

* Add opt-in browser tools for desktop tabs

Adds the daemon opt-in, desktop tab routing, MCP tools, and real browser automation surfaces for Paseo desktop browser tabs.

* Fix browser tools CI expectations

* Address browser tools review findings

* Restrict browser file automation paths

* Fix browser upload test on Windows

* Harden browser navigation inputs

* Make browser tools create usable tabs

* Update browser MCP empty-state test

* Fail browser tab creation when registration times out

* Fix browser screenshots for agents

* Hide disabled browser tools from agents

* Address browser tools architecture review

* Replace browser tools review tests

* Wrap browser tab registration errors

* Mock Expo Router in app unit tests

* Handle invalid browser automation requests

* Return browser failure on desktop disconnect

* Update browser disconnect websocket test

* Relax browser timeout polling test

* Handle invalid browser responses

* Return browser failure when send fails

* Remove local diagnostics and fixture paths

* Fix dev service home fallback

* Use worktree home for dev services

* Use managed daemon in desktop dev

* fix(browser): keep agent tabs addressable

Track agent-active browser targets separately from human-focused tabs and keep resident webviews alive for automation. Browser tool visibility now comes from registration while the broker reports disabled execution.

* refactor(browser): register tools through catalog

Move browser tool registration onto the shared Paseo tool catalog so the MCP server remains only the transport adapter.

* fix(settings): translate browser tools host error

2026-06-30 22:33:18 +02:00

scripts

Use separate OpenAI endpoints for speech-to-text and text-to-speech (#1823 )

2026-06-30 10:00:27 +02:00

src

Add opt-in browser tools for desktop tabs

2026-06-30 22:33:18 +02:00

.env.example

Use separate OpenAI endpoints for speech-to-text and text-to-speech (#1823 )

2026-06-30 10:00:27 +02:00

.gitignore

server(codex): normalize file paths, truncate large diffs, and drop turn/diff/updated telemetry

2026-02-06 11:51:59 +07:00

AGENTS.md

refactor(agent): add wait_for_agent tool and simplify prompt sending API

2025-11-03 11:47:15 +01:00

CLAUDE.md

Extract client SDK package (#1052 )

2026-05-28 01:58:18 +08:00

package.json

chore(release): cut 0.1.102

2026-06-30 12:00:31 +02:00

README.md

chore: reformat codebase with oxfmt

2026-04-23 21:44:48 +07:00

tsconfig.scripts.json

Align script TypeScript lib with server build

2026-05-05 18:54:29 +07:00

tsconfig.server.json

Move terminal sessions to worker process

2026-04-30 22:35:22 +07:00

tsconfig.server.typecheck.json

Fix repo typecheck failures

2025-11-15 20:17:47 +01:00

vitest.config.ts

Key workspace status and ownership by ID, not directory

2026-06-16 13:37:58 +07:00

README.md

Voice Assistant

A voice-controlled terminal assistant that runs as a single local service.

Quick Start

# Install dependencies
npm install

# Copy environment variables
cp .env.example .env
# Edit .env and add your API keys (OpenAI, Deepgram)

# Run development servers
npm run dev

# Open browser to http://localhost:5173

Architecture

Express Server (port 3000) - Serves API and built UI in production
Vite Dev Server (port 5173) - Hot-reload React UI in development
WebSocket (/ws) - Real-time bidirectional communication
Agent - STT → LLM → TTS pipeline with terminal control
Daemon - tmux-based terminal management (in-process)

Development

# Run both servers (recommended)
npm run dev

# Or run separately:
npm run dev:server  # Express on port 3000
npm run dev:ui      # Vite on port 5173

# Type checking
npm run typecheck

# Build for production
npm run build

# Start production server
npm start

Project Status

✅ Completed (Phases 1-2):

Package setup and configuration
Express server with WebSocket
React UI with Vite
WebSocket client with ping/pong testing

⏳ In Progress (Phase 3):

Terminal control (tmux integration)

📋 Planned (Phases 4-9):

LLM integration (OpenAI GPT-4)
Agent orchestrator
Speech-to-Text (Deepgram)
Text-to-Speech (OpenAI)
Audio streaming
UI polish

See IMPLEMENTATION_PLAN.md for complete details.

Environment Variables

OPENAI_API_KEY=your-openai-key-here      # GPT-4 and TTS
DEEPGRAM_API_KEY=your-deepgram-key-here  # Streaming STT
STT_MODEL=whisper-1        # Optional: override to gpt-4o-transcribe, etc.
STT_CONFIDENCE_THRESHOLD=-3.0  # Optional: reject low-confidence clips
STT_DEBUG_AUDIO_DIR=.stt-debug # Optional: persist raw dictation audio for debugging
PASEO_HOME=~/.paseo        # Runtime state directory (agents/, etc.)
PASEO_LISTEN=127.0.0.1:6767  # Listen address (host:port or /path/to/socket)

PASEO_HOME defaults to ~/.paseo and isolates runtime artifacts like agents/. PASEO_LISTEN controls the daemon listen address. For blue/green testing you can run a parallel server without touching production state:

PASEO_HOME=~/.paseo-blue PASEO_LISTEN=127.0.0.1:7777 npm run dev

Tech Stack

Server: Express, TypeScript, ws (WebSocket)
Client: React 18, Vite, TypeScript
Terminal: tmux (via child_process)
AI: OpenAI (LLM + TTS), Deepgram (STT)

Testing

Currently manual testing via:

Start servers: npm run dev
Open http://localhost:5173
Test WebSocket connection (green status indicator)
Click "Send Ping" button to test communication

More testing guidance as features are implemented.

License

MIT