puter/paseo
1
0
Fork 0
mirror of https://github.com/getpaseo/paseo.git synced 2026-10-27 19:52:49 +00:00
Code Issues Packages Projects Releases Wiki Activity

Choose voice providers and local speech models from a Voice settings page (#7295)

Browse Source 
* feat(voice): add a Voice settings page with provider choice and a local model catalog

Each host gets a Voice page under host settings. Dictation and voice mode
each pick a provider (Local or OpenAI) and a model. Provider settings sit in
one group of collapsible rows: local speech-to-text and text-to-speech list
every catalog model with its size, languages and description before download,
and OpenAI takes a write-only API key and an optional base URL.

The local catalog grows from Parakeet/Kokoro to Whisper, SenseVoice, Dolphin,
GigaAM, Parakeet Japanese, multilingual Kokoro, Kitten and Piper voices. Each
entry declares its sherpa engine family, so the recognizer and TTS build the
matching model config and the downloader verifies the files that family needs.
Kitten v0.8 is left out: it crashes the pinned sherpa-onnx 1.12.28.

Voice mode is now opt-in. The default lived only in code (true), never in
config.json, so hosts that never toggled it find it off after upgrading.

OpenAI speech-to-text resolved one model for both features, with voice mode's
choice taking precedence, so dictation silently used voice mode's model. The
resolved config now carries a model per feature; STT_MODEL still overrides both.

Rows revealed under a collapsible settings row are a nested density level set
by the container: tighter padding, smaller text, and xs controls, with
controls taking their size from the row instead of each caller.

* test(app): pre-bundle the JSX dev runtime for browser tests

Compiling test JSX with the automatic runtime imports react/jsx-dev-runtime. Vite discovered it mid-run on a cold cache and reloaded the page, which failed the first browser test file to import it.

* refactor(voice): keep speech provider settings at each use site

Store credentials and endpoint options beside the speech feature that consumes them so saved page values cannot be shadowed by a separate provider block. Normalize existing config on load and mirror the inline shape on config reads and patches.

This is the config, resolver and wire phase; the provider sheet follows separately.

* fix(voice): preserve saved keys when resaving provider settings

Forms send the selected provider on each save but cannot read back its API key. Merge patches for the existing provider so these saves retain credentials; replace the site only when the provider changes.

* feat(voice): configure each speech use site from a provider sheet

The Voice page now lists Speech-to-text, Listening and Speaking as one-line
summaries. Tapping one opens a sheet with the provider choice and only that
provider's settings: the local catalog (size, languages and description
before download, download state per model) or the OpenAI key, base URL,
model and language or voice.

The sheet's form model sends only the fields the user changed, so re-saving
never clears a saved key. Fields an env var decides are read-only with a
"Set by" hint; the API key and base URL stay editable because a saved value
wins over the env fallback. Save failures show a toast and keep the sheet
open with the input.

E2E daemons pin speech providers through env vars to avoid model downloads,
so the browser spec covers the locked-provider path, the key fallback and
the failed save; provider switching is covered by the form model tests.

* refactor(voice): press settings and catalog rows through PressHighlight

Settings rows that lead somewhere and local model rows now use the standard
row press primitive, so native shows the finger-down flash the rest of the
app's rows have. Web hover and press styling are unchanged.

* fix(voice): preserve STT language for local providers

Language belongs to every STT use site. Preserve legacy local language settings through config loading and saving, resolve them without provider gating, and expose them in the provider form.
This commit is contained in:
Mohamed Boudra
2026-10-26 14:19:02 +07:00
committed by GitHub
parent 88bd1cf05a
commit 74a7e5803b
83 changed files with 6060 additions and 1086 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/data-model.md
+20 -41
View File
@@ -221,12 +221,6 @@ snapshot so a mixed edit can apply its live subset and still name the paths that
}
},
providers: {
openai: {
apiKey?: string,
baseUrl?: string,
stt?: { apiKey?: string, baseUrl?: string },
tts?: { apiKey?: string, baseUrl?: string }
},
local: { modelsDir: string }
},
agents: {
@@ -244,8 +238,8 @@ snapshot so a mixed edit can apply its live subset and still name the paths that
pluginsEnabled: boolean,
plugins: Record<pluginId, { source: "directory", path: string, enabled?: boolean }>,
features: {
dictation: { enabled, stt: { provider, model, language, confidenceThreshold } },
voiceMode: { enabled, llm, stt: { provider, model, language }, turnDetection, tts: { provider, model, voice, speakerId, speed } }
dictation: { enabled, stt: SpeechSttConfig },
voiceMode: { enabled, llm, stt: SpeechSttConfig, turnDetection, tts: SpeechTtsConfig }
},
log: {
level, format,
@@ -353,42 +347,27 @@ Environment variables override `config.json`:
after changing `config.json`. Environment changes require a daemon restart; the launch environment
remains authoritative during reload.
Local speech model ids are intentionally narrow: STT uses `parakeet-tdt-0.6b-v2-int8`, TTS uses `kokoro-en-v0_19`, and turn detection uses the bundled Silero VAD model.
### Speech configuration
Set these to select OpenAI instead of local speech:
Each use site owns its provider settings: `features.dictation.stt`,
`features.voiceMode.stt`, and `features.voiceMode.tts`. `provider` selects a strict
`local` or `openai` variant. Both STT variants carry `language`; normalization preserves
it regardless of provider. `providers.local.modelsDir` remains shared storage.
See [Voice configuration](../public-docs/voice.md) for fields, examples and env precedence.
| Env var | Applies to |
| ------------------------------ | ------------------------------- |
| `PASEO_VOICE_STT_PROVIDER` | Voice mode STT provider |
| `PASEO_DICTATION_STT_PROVIDER` | Composer dictation STT provider |
| `PASEO_VOICE_TTS_PROVIDER` | Voice mode TTS provider |
On load, `persisted-config.ts` folds legacy `providers.openai` credentials and URLs
into OpenAI sites that lack their own values, using endpoint settings before shared
settings. A provider-less site becomes local; provider names are lower-cased, and
fields ignored by the selected speech provider are discarded. The next save writes only the
inline shape. A legacy key with no site to receive it is discarded with a warning
that names its path, without logging its value.
OpenAI speech can be configured under `providers.openai`. STT and TTS resolve independently, so they can point at different endpoints:
```json
{
"providers": {
"openai": {
"stt": {
"apiKey": "sk-...",
"baseUrl": "https://stt.example.com/v1"
},
"tts": {
"apiKey": "sk-...",
"baseUrl": "https://api.openai.com/v1"
}
}
}
}
```
`providers.openai.stt` is used for both composer dictation and voice mode speech-to-text; `providers.openai.tts` is used for voice mode text-to-speech. The equivalent env vars are `OPENAI_STT_API_KEY`/`OPENAI_STT_BASE_URL` and `OPENAI_TTS_API_KEY`/`OPENAI_TTS_BASE_URL`. Each feature falls back to `providers.openai.apiKey`/`providers.openai.baseUrl`, then `OPENAI_API_KEY`/`OPENAI_BASE_URL`, when its own fields are unset. These settings apply only to Paseo OpenAI speech features, not to Codex or other OpenAI-backed tools.
Paseo uses these paths under the configured OpenAI base URL:
- dictation STT: `/v1/audio/transcriptions`
- voice mode STT: `/v1/audio/transcriptions`
- voice mode TTS: `/v1/audio/speech`
`speech-settings.ts` owns mutable config reads and patches. The wire uses the same
`features` structure. Reads replace OpenAI keys with `hasApiKey` (saved in config)
and report effective env sources in `envOverrides`. A site patch that changes
`provider` replaces the entire object. The same or omitted `provider` merges fields
into the current variant, preserving the write-only key when a form re-saves it. `null` clears a value. Switching providers discards credentials and
options from the previous variant.
---
docs/design.md
+3 -1
View File
@@ -70,7 +70,7 @@ the sidebar. Apply it as `backgroundColor`; setting `opacity` on the control als
Sizes: `xs` for ultra-tight inline triggers. `sm` for any button sitting in a row. `md` is the page default. `lg` is reserved for large standalone CTAs.
Sizes are a shared contract across control kinds, defined once in `control-geometry.ts`: `xs` = 28px tall with `fontSize.sm` labels, `sm` = 32px with `fontSize.base`, `md`/`lg` = 44px with `fontSize.base`. `<SegmentedControl>` (`packages/app/src/components/ui/segmented-control.tsx`) takes the same `xs`/`sm`/`md` sizes — a segmented control next to a `<Button>` of the same size always matches in height, label size, and corner radius. Its segments run one padding step tighter than a button, because the gap between segments already reads as padding. The selected segment is a `surface3` fill with `foreground` text, not an inverted one — inverting it inside thin chrome puts a white slab in the toolbar. Thin chrome such as the file toolbar uses `xs`; settings rows use `sm`. Never shrink a control's font or padding locally to fit a context — if the context needs a smaller control, the size tier is missing or the wrong one is in use.
Sizes are a shared contract across control kinds, defined once in `control-geometry.ts`: `xs` = 28px tall with `fontSize.sm` labels, `sm` = 32px with `fontSize.base`, `md`/`lg` = 44px with `fontSize.base`. `<SegmentedControl>` (`packages/app/src/components/ui/segmented-control.tsx`) takes the same `xs`/`sm`/`md` sizes — a segmented control next to a `<Button>` of the same size always matches in height, label size, and corner radius. Its segments run one padding step tighter than a button, because the gap between segments already reads as padding. The selected segment is a `surface3` fill with `foreground` text, not an inverted one — inverting it inside thin chrome puts a white slab in the toolbar. Text fields take `xs`, `sm` and `md`. Thin chrome such as the file toolbar uses `xs`. A settings row sizes the controls in its trailing slot itself, `sm` in a card and `xs` when nested (`packages/app/src/components/ui/control-size.tsx`), so don't pass a size to a button, dropdown or field placed in a row. Never shrink a control's font or padding locally to fit a context — if the context needs a smaller control, the size tier is missing or the wrong one is in use.
A `<Pressable>` wrapping a `<Text>` is a sixth variant. It is wrong. `<Button>` accepts `style`, `textStyle`, `leftIcon`, `disabled`, `size`, and `variant`.
@@ -126,6 +126,8 @@ Cards inside a section sit closer than sections. Rows inside a card touch — on
Rows have generous vertical padding: roughly 16px of content plus 16px of vertical padding for settings rows, 8–12px for sidebar list items where many rows must fit. Compressing rows below the established density to fit more on the screen is wrong. Too many rows means more cards or more sections, not smaller rows.
Rows revealed under a `<SettingsCollapsibleRow>` are nested rows, one step smaller than card rows: 8px of vertical padding, `fontSize.sm` text, `xs` controls, and one fixed height so rows with and without a trailing button line up. The collapsible row sets this for everything under it (`packages/app/src/components/settings/row-density.tsx`). Never pass a density to a row, and never pad or resize a nested row by hand.
The whitespace is the design.
---