diff --git a/README.md b/README.md index 69ab40bde..85e110481 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,15 @@ # Voice Dev -Voice-controlled development environment powered by OpenAI Realtime API. +Voice-controlled development environment powered by LiveKit and OpenAI Realtime API. ## Packages +- **`@voice-dev/agent`** - LiveKit agent with OpenAI Realtime API + - Located in `packages/agent/` + - Handles voice conversations via LiveKit + - Integrates OpenAI Realtime API + - Manages agent lifecycle + - **`@voice-dev/mcp-server`** - MCP server for terminal control - Located in `packages/mcp-server/` - Provides terminal session management via MCP protocol @@ -11,20 +17,37 @@ Voice-controlled development environment powered by OpenAI Realtime API. - **`@voice-dev/web`** - Next.js web interface - Located in `packages/web/` - - Voice interaction with OpenAI Realtime API + - Voice interaction via LiveKit Cloud - Password authentication - Agent activity transparency (debug panel) - Real-time status display ## Quick Start +For detailed setup instructions, see [QUICKSTART.md](./QUICKSTART.md). + +**You need TWO terminals:** + +```bash +# Terminal 1: Start the LiveKit agent +npm run dev:agent + +# Terminal 2: Start the web app +npm run dev +``` + +Then open http://localhost:3000 and click "Start Voice Chat". + +For more info: +- [QUICKSTART.md](./QUICKSTART.md) - Step-by-step setup guide +- [MIGRATION.md](./MIGRATION.md) - Details about LiveKit migration + +### Other Commands + ```bash # Install all dependencies npm install -# Run web app in dev mode -npm run dev - # Run MCP server in dev mode npm run dev:mcp @@ -37,6 +60,17 @@ npm run typecheck ## Development +### LiveKit Agent + +```bash +# Development +npm run dev:agent # Start agent in dev mode + +# Production +npm run build:agent # Build TypeScript +npm run start:agent # Run built agent +``` + ### Web App ```bash @@ -61,12 +95,20 @@ npm run start:mcp # Run built server ## Environment Variables -Create `.env.local` in `packages/web/`: +Create `.env.local` in both `packages/web/` and `packages/agent/`: ```bash +# OpenAI OPENAI_API_KEY=sk-your-api-key-here -AUTH_PASSWORD=your-secure-password-here -MCP_SERVER_URL=https://your-mcp-server-url + +# LiveKit Cloud +LIVEKIT_URL=wss://your-project.livekit.cloud +LIVEKIT_API_KEY=your-api-key +LIVEKIT_API_SECRET=your-api-secret + +# Optional +AUTH_PASSWORD=your-secure-password-here # For web auth +MCP_SERVER_URL=https://your-mcp-server-url # For MCP integration ``` See `packages/web/.env.local.example` for template. @@ -76,6 +118,13 @@ See `packages/web/.env.local.example` for template. ``` voice-dev/ ├── packages/ +│ ├── agent/ # LiveKit Agent Package +│ │ ├── src/ +│ │ │ └── agent.ts # Agent implementation +│ │ ├── dist/ # Compiled output +│ │ ├── package.json +│ │ └── tsconfig.json +│ │ │ ├── mcp-server/ # MCP Server Package │ │ ├── src/ │ │ │ ├── index.ts # CLI entry point @@ -87,9 +136,9 @@ voice-dev/ │ │ │ └── web/ # Web App Package │ ├── app/ -│ │ ├── api/ # API routes +│ │ ├── api/ # API routes (token generation) │ │ ├── components/ # React components -│ │ ├── hooks/ # Custom hooks +│ │ ├── hooks/ # Custom hooks (LiveKit) │ │ ├── types/ # TypeScript types │ │ └── voice-client.tsx # Main voice UI │ ├── package.json @@ -97,20 +146,31 @@ voice-dev/ │ ├── package.json # Root workspace config ├── tsconfig.base.json # Shared TypeScript config -└── README.md # This file +├── README.md # This file +├── QUICKSTART.md # Setup guide +└── MIGRATION.md # LiveKit migration details ``` ## Features +### LiveKit Agent +- OpenAI Realtime API integration +- Voice-to-voice conversation +- Low-latency audio processing +- Automatic room management +- System prompt customization +- Tool/function calling support + ### Web App -- Real-time voice interaction with OpenAI -- WebRTC-based low-latency audio streaming +- Real-time voice interaction via LiveKit +- LiveKit React SDK components - Live audio level visualization +- Device selection - Mute/unmute controls - Password authentication - Agent activity transparency (debug log panel) - Real-time agent status display -- Tool call visibility +- Automatic reconnection ### MCP Server - Full terminal session control @@ -155,12 +215,15 @@ npm run start:mcp -- --http --password your-password | Script | Description | |--------|-------------| | `npm run dev` | Start web app dev server | +| `npm run dev:agent` | Start LiveKit agent dev server | | `npm run dev:mcp` | Start MCP server dev server | | `npm run build` | Build all packages | +| `npm run build:agent` | Build agent only | | `npm run build:mcp` | Build MCP server only | | `npm run build:web` | Build web app only | | `npm run typecheck` | Type check all packages | | `npm run start` | Start web app production server | +| `npm run start:agent` | Start agent production | | `npm run start:mcp` | Start MCP server production | ### Package Level @@ -178,11 +241,27 @@ npm run