Local streaming STT/TTS with Parakeet + Pocket TTS

- Add provider-agnostic streaming speech interfaces
- Local STT via Sherpa (Zipformer/Paraformer) + Parakeet v3 (offline)
- Local TTS via Pocket TTS ONNX (onnxruntime-node) with chunked streaming
- Auto-download models into PASEO_HOME and add Node download script
- Add E2E download test that round-trips TTS->STT
This commit is contained in:
Mohamed Boudra
2026-02-06 10:38:38 +07:00
parent b1b0be7b56
commit d791f79efe
40 changed files with 3421 additions and 144 deletions

View File

@@ -32,6 +32,37 @@ npm install -g @getpaseo/cli && paseo
Then open the app and connect to your daemon.
## Local speech (STT/TTS)
Paseo can run dictation + voice mode STT/TTS fully locally via `sherpa-onnx`.
When the daemon starts with a local speech provider selected, it will download any missing model files automatically (unless `PASEO_SHERPA_ONNX_AUTO_DOWNLOAD=0`).
```bash
npm run speech:download --workspace=@getpaseo/server
```
Optional configuration:
- `PASEO_SHERPA_ONNX_MODELS_DIR` (defaults to `~/.paseo/models/sherpa-onnx`)
- `PASEO_SHERPA_ONNX_AUTO_DOWNLOAD` (`1` by default; set `0` to disable automatic downloads on daemon start)
- `PASEO_SHERPA_STT_PRESET` (`zipformer`, `paraformer`, or `parakeet` for NVIDIA Parakeet TDT v3)
- `PASEO_SHERPA_TTS_PRESET` (`pocket-tts` (Kyutai Pocket TTS), `kitten`, or `kokoro`)
- `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER` (`sherpa` or `openai`)
To see all supported local model IDs:
```bash
npm run speech:models --workspace=@getpaseo/server
```
Optional: run an end-to-end test that downloads real models and exercises streaming STT + streaming TTS:
```bash
PASEO_SPEECH_E2E_DOWNLOAD=1 PASEO_SPEECH_E2E_MODEL_SET=parakeet-pocket \
npx vitest run --workspace=@getpaseo/server src/server/speech/sherpa/speech-download.e2e.test.ts
```
## Documentation
See [paseo.sh/docs](https://paseo.sh/docs) for full documentation.