MCP Integration

Overview

This application now integrates with a Model Context Protocol (MCP) server to extend the AI's capabilities with custom tools.

How It Works

The integration uses Remote MCP Server mode, where OpenAI's servers directly communicate with your MCP server:

During session creation, the app sends MCP configuration to OpenAI
OpenAI connects to your MCP server and discovers available tools
During conversation, when the AI needs to use a tool, it calls your MCP server automatically
Your MCP server executes the tool and returns results to OpenAI
OpenAI incorporates the results into the conversation and responds to the user

Configuration

The MCP server is configured in app/api/session/route.ts:

tools: [
  {
    type: 'mcp',
    server_label: 'local-mcp',
    server_url: 'https://mohameds-macbook-pro.tail8fe838.ts.net/mcp?password=dev-password',
    require_approval: 'never',
  },
]

Configuration Options

type: Must be 'mcp' for MCP server integration
server_label: A friendly name for your MCP server (used in logs/debugging)
server_url: The HTTPS URL of your MCP server (including query parameters for auth)
require_approval: Set to 'never' for automatic tool execution, or 'always' for manual approval

Tool Discovery

The MCP protocol automatically discovers all available tools from your server. You don't need to:

Manually define each tool
Handle function call events in the client
Send results back to OpenAI

Everything is handled automatically by OpenAI's servers.

Benefits

Zero Client Complexity: No need to monitor WebRTC data channels or handle function calls
Automatic Tool Discovery: MCP servers expose their available tools automatically
Scalable: Easy to add more MCP servers or switch between them
Production Ready: This is OpenAI's recommended approach for tool integration

Testing

To test the MCP integration:

Start the development server: npm run dev
Open http://localhost:3000
Click "Start Voice Chat"
Ask the AI to perform actions that require your MCP tools
The AI will automatically use the appropriate tools and respond

Example prompts to test (depending on your MCP tools):

"What tools do you have access to?"
"Can you [action that requires your MCP tool]?"

Changing MCP Configuration

To use a different MCP server or modify settings:

Edit app/api/session/route.ts
Update the tools array in the session creation
Rebuild: npm run build
Restart the server

Security Notes

The MCP server URL includes authentication in the query string (?password=dev-password)
This is acceptable for development but consider using proper OAuth or API keys for production
The require_approval: 'never' setting means tools execute automatically without confirmation
For production, consider using require_approval: 'always' with a UI approval mechanism

Troubleshooting

Tools Not Working

Verify your MCP server is accessible at the configured URL
Check that the password/auth is correct
Ensure your MCP server is returning valid MCP protocol responses

Session Creation Fails

Check OpenAI API key is valid
Verify the MCP server URL is properly formatted
Check network connectivity to your MCP server

AI Doesn't Use Tools

The AI decides when to use tools based on the conversation context
Try more explicit prompts that clearly require tool usage
Verify tools are properly exposed by your MCP server

3.5 KiB Raw Blame History