mirror of
https://github.com/getpaseo/paseo.git
synced 2026-10-26 19:22:51 +00:00
Mohamed Boudra
74a7e5803b
* feat(voice): add a Voice settings page with provider choice and a local model catalog Each host gets a Voice page under host settings. Dictation and voice mode each pick a provider (Local or OpenAI) and a model. Provider settings sit in one group of collapsible rows: local speech-to-text and text-to-speech list every catalog model with its size, languages and description before download, and OpenAI takes a write-only API key and an optional base URL. The local catalog grows from Parakeet/Kokoro to Whisper, SenseVoice, Dolphin, GigaAM, Parakeet Japanese, multilingual Kokoro, Kitten and Piper voices. Each entry declares its sherpa engine family, so the recognizer and TTS build the matching model config and the downloader verifies the files that family needs. Kitten v0.8 is left out: it crashes the pinned sherpa-onnx 1.12.28. Voice mode is now opt-in. The default lived only in code (true), never in config.json, so hosts that never toggled it find it off after upgrading. OpenAI speech-to-text resolved one model for both features, with voice mode's choice taking precedence, so dictation silently used voice mode's model. The resolved config now carries a model per feature; STT_MODEL still overrides both. Rows revealed under a collapsible settings row are a nested density level set by the container: tighter padding, smaller text, and xs controls, with controls taking their size from the row instead of each caller. * test(app): pre-bundle the JSX dev runtime for browser tests Compiling test JSX with the automatic runtime imports react/jsx-dev-runtime. Vite discovered it mid-run on a cold cache and reloaded the page, which failed the first browser test file to import it. * refactor(voice): keep speech provider settings at each use site Store credentials and endpoint options beside the speech feature that consumes them so saved page values cannot be shadowed by a separate provider block. Normalize existing config on load and mirror the inline shape on config reads and patches. This is the config, resolver and wire phase; the provider sheet follows separately. * fix(voice): preserve saved keys when resaving provider settings Forms send the selected provider on each save but cannot read back its API key. Merge patches for the existing provider so these saves retain credentials; replace the site only when the provider changes. * feat(voice): configure each speech use site from a provider sheet The Voice page now lists Speech-to-text, Listening and Speaking as one-line summaries. Tapping one opens a sheet with the provider choice and only that provider's settings: the local catalog (size, languages and description before download, download state per model) or the OpenAI key, base URL, model and language or voice. The sheet's form model sends only the fields the user changed, so re-saving never clears a saved key. Fields an env var decides are read-only with a "Set by" hint; the API key and base URL stay editable because a saved value wins over the env fallback. Save failures show a toast and keep the sheet open with the input. E2E daemons pin speech providers through env vars to avoid model downloads, so the browser spec covers the locked-provider path, the key fallback and the failed save; provider switching is covered by the form model tests. * refactor(voice): press settings and catalog rows through PressHighlight Settings rows that lead somewhere and local model rows now use the standard row press primitive, so native shows the finger-down flash the rest of the app's rows have. Web hover and press styling are unchanged. * fix(voice): preserve STT language for local providers Language belongs to every STT use site. Preserve legacy local language settings through config loading and saving, resolve them without provider gating, and expose them in the provider form.
Voice Assistant
A voice-controlled terminal assistant that runs as a single local service.
Quick Start
# Install dependencies
npm install
# Copy environment variables
cp .env.example .env
# Edit .env and add your API keys (OpenAI, Deepgram)
# Run development servers
npm run dev
# Open browser to http://localhost:5173
Architecture
- Express Server (port 3000) - Serves API and built UI in production
- Vite Dev Server (port 5173) - Hot-reload React UI in development
- WebSocket (
/ws) - Real-time bidirectional communication - Agent - STT → LLM → TTS pipeline with terminal control
- Daemon - tmux-based terminal management (in-process)
Development
# Run both servers (recommended)
npm run dev
# Or run separately:
npm run dev:server # Express on port 3000
npm run dev:ui # Vite on port 5173
# Type checking
npm run typecheck
# Build for production
npm run build
# Start production server
npm start
Project Status
✅ Completed (Phases 1-2):
- Package setup and configuration
- Express server with WebSocket
- React UI with Vite
- WebSocket client with ping/pong testing
⏳ In Progress (Phase 3):
- Terminal control (tmux integration)
📋 Planned (Phases 4-9):
- LLM integration (OpenAI GPT-4)
- Agent orchestrator
- Speech-to-Text (Deepgram)
- Text-to-Speech (OpenAI)
- Audio streaming
- UI polish
See IMPLEMENTATION_PLAN.md for complete details.
Environment Variables
OPENAI_API_KEY=your-openai-key-here # GPT-4 and TTS
DEEPGRAM_API_KEY=your-deepgram-key-here # Streaming STT
STT_MODEL=whisper-1 # Optional: override to gpt-4o-transcribe, etc.
STT_CONFIDENCE_THRESHOLD=-3.0 # Optional: reject low-confidence clips
STT_DEBUG_AUDIO_DIR=.stt-debug # Optional: persist raw dictation audio for debugging
PASEO_HOME=~/.paseo # Runtime state directory (agents/, etc.)
PASEO_LISTEN=127.0.0.1:6767 # Listen address (host:port or /path/to/socket)
PASEO_HOME defaults to ~/.paseo and isolates runtime artifacts like agents/. PASEO_LISTEN controls the daemon listen address. For blue/green testing you can run a parallel server without touching production state:
PASEO_HOME=~/.paseo-blue PASEO_LISTEN=127.0.0.1:7777 npm run dev
Tech Stack
- Server: Express, TypeScript, ws (WebSocket)
- Client: React 18, Vite, TypeScript
- Terminal: tmux (via child_process)
- AI: OpenAI (LLM + TTS), Deepgram (STT)
Testing
Currently manual testing via:
- Start servers:
npm run dev - Open http://localhost:5173
- Test WebSocket connection (green status indicator)
- Click "Send Ping" button to test communication
More testing guidance as features are implemented.
License
MIT