mirror of
https://github.com/getpaseo/paseo.git
synced 2026-10-23 01:32:40 +00:00
8ddeb79c8d
* fix(claude): open autonomous turns only when Claude starts a turn Paseo guessed from frame shape whether the main session had started work on its own, and only a result could close the turn that guess opened. A background subagent's task frames, its own background jobs' task_notification and rate limit events all arrive with no parent tool use id, so they looked like main-session work and opened a turn no result would ever close. The agent then reported "running" while its main session sat idle, and the app's message queue never drained. Claude brackets every main-session turn itself: system/init opens it and exactly one result closes it, whether a prompt, a wake-up after background work, a slash command or an API error started it. Background subagents never emit either on the main stream. An autonomous turn now opens only on that init, and only when no foreground turn is open. The frame heuristic is removed. - Paseo launches the CLI with CLAUDE_CODE_EMIT_SESSION_STATE_EVENTS=1, added as an overlay in buildSdkEnv so the SDK and the usage probe share one env object. An "idle" session state closes an autonomous turn as a safety net, never a foreground one. "running" opens nothing: Claude Code 2.1.280 stays "running" for as long as any background subagent is alive. State frames never reach the timeline. - An init clears a leftover pendingInterruptAbort. A new turn has begun, so the interrupted one is over, and a stale flag would swallow the new turn's result. Tests that opened autonomous turns from bare frames now send the init Claude sends. New coverage: helper frames never open a turn while the main session is idle, a wake-up turn opens on init and closes on its result, a folded steer keeps one turn while a turn Claude starts after the result gets its own, an idle closes a turn no result closed, session state never opens a turn, and the env flag reaches the SDK launch. docs/agent-lifecycle.md records that only system/init and result bracket a main-session turn, since the code cannot tell a reader why frame shape fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(claude): interrupt Claude only while its main turn runs, and spare helpers Every Stop, and every send that replaces a running turn (paseo send, MCP send_agent_prompt, voice, the app's queue drain, Send now), ends in query.interrupt(). Without the SDK's perTaskStopAffordance declaration that call kills every running background subagent, even when the main session is idle and there is nothing of ours to stop. - Declare perTaskStopAffordance, so an interrupt aborts only the main turn, like Esc in Claude Code. Claude stops a helper itself with its TaskStop tool. - Track whether a main-session turn is in flight from Claude's own brackets: set by system/init and by a pushed message's command_lifecycle "started" (/compact works for seconds before its init), cleared by the result or an idle session state. - Call query.interrupt() and set pendingInterruptAbort only when that turn is in flight, or the cancelled turn's own message has already been started. Otherwise withdraw our not-yet-started messages with cancel_async_message instead: an idle interrupt reaches only helpers, and with no result to follow it the flag would linger. - Track the foreground prompt with the queued steers until Claude starts it, so Stop withdraws a message still waiting in the queue. A turn stopped while the Claude process is still starting never sends it. - Accept command_lifecycle "cancelled" as terminal, so a cancelled steer no longer waits to be withdrawn or keeps denying permission cards. Interrupt tests now wait for the pump to read the init before stopping, since that is when Paseo learns a turn is in flight. New coverage: a stop before Claude starts the message withdraws it without interrupting, a stop while only helpers run never reaches Claude, a stop mid-turn interrupts once and drops the aborted result, a new turn clears a flag no result will consume, the SDK option is passed, and a cancelled steer is not withdrawn again. docs/providers.md described the old unconditional interrupt; it now states when the Claude adapter interrupts and why it declares the affordance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(claude): keep a helper spawned in the background out of a turn's cancel When a turn is canceled, Paseo terminalizes the subagents that were running in its foreground and skips the backgrounded ones, which outlive it. It only learned a child was backgrounded from task_updated's patch.is_backgrounded, which Claude sends when a foreground task moves to the background later. A child spawned in the background carries the flag on task_started instead, and a resumed subagent always does, so the track marked those helpers "canceled" while they kept working. Read is_backgrounded from task_started as well. docs/agent-lifecycle.md said task_updated was the only signal; it now names both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(app): drain queued messages as steers The composer queue drains when an agent goes idle. It sent each message with no activeTurnBehavior, which the daemon treats as "interrupt": if the agent had just started a turn of its own (a Claude wake-up after background work finished), the drained message stopped that turn and replaced it. Send drained messages with activeTurnBehavior "steer". A drain that races such a turn folds into it; with no turn running the daemon starts a normal one, as before. "steer" is an existing protocol value, so older daemons need nothing new. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(claude): cover background helpers against real Claude Every other test in this file tells Claude "Do not use a background task", so a message sent or a Stop pressed while a backgrounded subagent runs was never exercised. Four real-Claude scenarios cover it: - "a background subagent keeps running after a message is sent while it works": one helper, the main turn ends, the parent is not reported running, a message in that window is answered, and the helper is never read as canceled and completes. - "a helper's own nested background job does not phantom-run the parent or kill the helper": a helper starts its own background Bash job, then keeps working in the foreground. The parent stays idle across the nested job's completion (sampled for 30 s around a 5 s job), a message sent in that window is answered, and the helper reports CHILD_DONE. - "the app's default steer survives a backgrounded helper while the main session works": the app's default send behavior (steer) does not kill a helper when it interrupts the main session's own turn. - "Stop cancels the main session's own turn but spares its background helper": the same for Stop. The wake-up turn Claude starts after a helper finishes lands a moment after the helper reads completed (0.1 to 5 s in recorded traces), and the daemon announces it as a pushed "running" agent_update, not as a turn_started stream event. The checks poll for it and read the final idle once the wake-up settles. waitForFinish can return a stale "idle" right after a send, and the push stream can miss a message's final text when it resolves, so the tests read fetchAgentTimeline like the rest of the file. Helper status comes from agent.provider_subagents.list, since agent_stream carries no provider_subagent events. The scenarios need the previous commits together. Run against the frame heuristic this branch removes, the nested-job test fails: the parent flips to running once the nested job finishes and stays there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(claude): interrupt a stopped prompt Claude has already taken Stop before Claude announces a prompt started withdraws it with cancel_async_message instead of interrupting. Claude can dequeue a message for its next turn before announcing it, and withdrawing it is then a no-op that reports cancelled: false, so the stopped prompt kept running. Interrupt whenever a message could not be withdrawn. Co-authored-by: Isak Persson <isak.a.persson@gmail.com> * fix(claude): withdraw a stopped prompt only once When the withdrawal of a stopped prompt failed, the interrupt path withdrew a second time, which could cancel a prompt sent while the first withdrawal was in flight. Withdraw once and interrupt on failure. The real-Claude steer scenario now waits for the background helper to exist before steering, so it proves the helper survives the steer. Co-authored-by: Isak Persson <isak.a.persson@gmail.com> --------- Co-authored-by: Isak Persson <isak.a.persson@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: paseo-bot[bot] <266920839+paseo-bot[bot]@users.noreply.github.com>
Voice Assistant
A voice-controlled terminal assistant that runs as a single local service.
Quick Start
# Install dependencies
npm install
# Copy environment variables
cp .env.example .env
# Edit .env and add your API keys (OpenAI, Deepgram)
# Run development servers
npm run dev
# Open browser to http://localhost:5173
Architecture
- Express Server (port 3000) - Serves API and built UI in production
- Vite Dev Server (port 5173) - Hot-reload React UI in development
- WebSocket (
/ws) - Real-time bidirectional communication - Agent - STT → LLM → TTS pipeline with terminal control
- Daemon - tmux-based terminal management (in-process)
Development
# Run both servers (recommended)
npm run dev
# Or run separately:
npm run dev:server # Express on port 3000
npm run dev:ui # Vite on port 5173
# Type checking
npm run typecheck
# Build for production
npm run build
# Start production server
npm start
Project Status
✅ Completed (Phases 1-2):
- Package setup and configuration
- Express server with WebSocket
- React UI with Vite
- WebSocket client with ping/pong testing
⏳ In Progress (Phase 3):
- Terminal control (tmux integration)
📋 Planned (Phases 4-9):
- LLM integration (OpenAI GPT-4)
- Agent orchestrator
- Speech-to-Text (Deepgram)
- Text-to-Speech (OpenAI)
- Audio streaming
- UI polish
See IMPLEMENTATION_PLAN.md for complete details.
Environment Variables
OPENAI_API_KEY=your-openai-key-here # GPT-4 and TTS
DEEPGRAM_API_KEY=your-deepgram-key-here # Streaming STT
STT_MODEL=whisper-1 # Optional: override to gpt-4o-transcribe, etc.
STT_CONFIDENCE_THRESHOLD=-3.0 # Optional: reject low-confidence clips
STT_DEBUG_AUDIO_DIR=.stt-debug # Optional: persist raw dictation audio for debugging
PASEO_HOME=~/.paseo # Runtime state directory (agents/, etc.)
PASEO_LISTEN=127.0.0.1:6767 # Listen address (host:port or /path/to/socket)
PASEO_HOME defaults to ~/.paseo and isolates runtime artifacts like agents/. PASEO_LISTEN controls the daemon listen address. For blue/green testing you can run a parallel server without touching production state:
PASEO_HOME=~/.paseo-blue PASEO_LISTEN=127.0.0.1:7777 npm run dev
Tech Stack
- Server: Express, TypeScript, ws (WebSocket)
- Client: React 18, Vite, TypeScript
- Terminal: tmux (via child_process)
- AI: OpenAI (LLM + TTS), Deepgram (STT)
Testing
Currently manual testing via:
- Start servers:
npm run dev - Open http://localhost:5173
- Test WebSocket connection (green status indicator)
- Click "Send Ping" button to test communication
More testing guidance as features are implemented.
License
MIT