mirror of
https://github.com/getpaseo/paseo.git
synced 2026-10-27 03:32:55 +00:00
Choose voice providers and local speech models from a Voice settings page (#7295)
* feat(voice): add a Voice settings page with provider choice and a local model catalog Each host gets a Voice page under host settings. Dictation and voice mode each pick a provider (Local or OpenAI) and a model. Provider settings sit in one group of collapsible rows: local speech-to-text and text-to-speech list every catalog model with its size, languages and description before download, and OpenAI takes a write-only API key and an optional base URL. The local catalog grows from Parakeet/Kokoro to Whisper, SenseVoice, Dolphin, GigaAM, Parakeet Japanese, multilingual Kokoro, Kitten and Piper voices. Each entry declares its sherpa engine family, so the recognizer and TTS build the matching model config and the downloader verifies the files that family needs. Kitten v0.8 is left out: it crashes the pinned sherpa-onnx 1.12.28. Voice mode is now opt-in. The default lived only in code (true), never in config.json, so hosts that never toggled it find it off after upgrading. OpenAI speech-to-text resolved one model for both features, with voice mode's choice taking precedence, so dictation silently used voice mode's model. The resolved config now carries a model per feature; STT_MODEL still overrides both. Rows revealed under a collapsible settings row are a nested density level set by the container: tighter padding, smaller text, and xs controls, with controls taking their size from the row instead of each caller. * test(app): pre-bundle the JSX dev runtime for browser tests Compiling test JSX with the automatic runtime imports react/jsx-dev-runtime. Vite discovered it mid-run on a cold cache and reloaded the page, which failed the first browser test file to import it. * refactor(voice): keep speech provider settings at each use site Store credentials and endpoint options beside the speech feature that consumes them so saved page values cannot be shadowed by a separate provider block. Normalize existing config on load and mirror the inline shape on config reads and patches. This is the config, resolver and wire phase; the provider sheet follows separately. * fix(voice): preserve saved keys when resaving provider settings Forms send the selected provider on each save but cannot read back its API key. Merge patches for the existing provider so these saves retain credentials; replace the site only when the provider changes. * feat(voice): configure each speech use site from a provider sheet The Voice page now lists Speech-to-text, Listening and Speaking as one-line summaries. Tapping one opens a sheet with the provider choice and only that provider's settings: the local catalog (size, languages and description before download, download state per model) or the OpenAI key, base URL, model and language or voice. The sheet's form model sends only the fields the user changed, so re-saving never clears a saved key. Fields an env var decides are read-only with a "Set by" hint; the API key and base URL stay editable because a saved value wins over the env fallback. Save failures show a toast and keep the sheet open with the input. E2E daemons pin speech providers through env vars to avoid model downloads, so the browser spec covers the locked-provider path, the key fallback and the failed save; provider switching is covered by the form model tests. * refactor(voice): press settings and catalog rows through PressHighlight Settings rows that lead somewhere and local model rows now use the standard row press primitive, so native shows the finger-down flash the rest of the app's rows have. Web hover and press styling are unchanged. * fix(voice): preserve STT language for local providers Language belongs to every STT use site. Preserve legacy local language settings through config loading and saving, resolve them without provider gating, and expose them in the provider form.
This commit is contained in:
Mohamed Boudra
+20
-41
@@ -221,12 +221,6 @@ snapshot so a mixed edit can apply its live subset and still name the paths that
|
||||
}
|
||||
},
|
||||
providers: {
|
||||
openai: {
|
||||
apiKey?: string,
|
||||
baseUrl?: string,
|
||||
stt?: { apiKey?: string, baseUrl?: string },
|
||||
tts?: { apiKey?: string, baseUrl?: string }
|
||||
},
|
||||
local: { modelsDir: string }
|
||||
},
|
||||
agents: {
|
||||
@@ -244,8 +238,8 @@ snapshot so a mixed edit can apply its live subset and still name the paths that
|
||||
pluginsEnabled: boolean,
|
||||
plugins: Record<pluginId, { source: "directory", path: string, enabled?: boolean }>,
|
||||
features: {
|
||||
dictation: { enabled, stt: { provider, model, language, confidenceThreshold } },
|
||||
voiceMode: { enabled, llm, stt: { provider, model, language }, turnDetection, tts: { provider, model, voice, speakerId, speed } }
|
||||
dictation: { enabled, stt: SpeechSttConfig },
|
||||
voiceMode: { enabled, llm, stt: SpeechSttConfig, turnDetection, tts: SpeechTtsConfig }
|
||||
},
|
||||
log: {
|
||||
level, format,
|
||||
@@ -353,42 +347,27 @@ Environment variables override `config.json`:
|
||||
after changing `config.json`. Environment changes require a daemon restart; the launch environment
|
||||
remains authoritative during reload.
|
||||
|
||||
Local speech model ids are intentionally narrow: STT uses `parakeet-tdt-0.6b-v2-int8`, TTS uses `kokoro-en-v0_19`, and turn detection uses the bundled Silero VAD model.
|
||||
### Speech configuration
|
||||
|
||||
Set these to select OpenAI instead of local speech:
|
||||
Each use site owns its provider settings: `features.dictation.stt`,
|
||||
`features.voiceMode.stt`, and `features.voiceMode.tts`. `provider` selects a strict
|
||||
`local` or `openai` variant. Both STT variants carry `language`; normalization preserves
|
||||
it regardless of provider. `providers.local.modelsDir` remains shared storage.
|
||||
See [Voice configuration](../public-docs/voice.md) for fields, examples and env precedence.
|
||||
|
||||
| Env var | Applies to |
|
||||
| ------------------------------ | ------------------------------- |
|
||||
| `PASEO_VOICE_STT_PROVIDER` | Voice mode STT provider |
|
||||
| `PASEO_DICTATION_STT_PROVIDER` | Composer dictation STT provider |
|
||||
| `PASEO_VOICE_TTS_PROVIDER` | Voice mode TTS provider |
|
||||
On load, `persisted-config.ts` folds legacy `providers.openai` credentials and URLs
|
||||
into OpenAI sites that lack their own values, using endpoint settings before shared
|
||||
settings. A provider-less site becomes local; provider names are lower-cased, and
|
||||
fields ignored by the selected speech provider are discarded. The next save writes only the
|
||||
inline shape. A legacy key with no site to receive it is discarded with a warning
|
||||
that names its path, without logging its value.
|
||||
|
||||
OpenAI speech can be configured under `providers.openai`. STT and TTS resolve independently, so they can point at different endpoints:
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"openai": {
|
||||
"stt": {
|
||||
"apiKey": "sk-...",
|
||||
"baseUrl": "https://stt.example.com/v1"
|
||||
},
|
||||
"tts": {
|
||||
"apiKey": "sk-...",
|
||||
"baseUrl": "https://api.openai.com/v1"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`providers.openai.stt` is used for both composer dictation and voice mode speech-to-text; `providers.openai.tts` is used for voice mode text-to-speech. The equivalent env vars are `OPENAI_STT_API_KEY`/`OPENAI_STT_BASE_URL` and `OPENAI_TTS_API_KEY`/`OPENAI_TTS_BASE_URL`. Each feature falls back to `providers.openai.apiKey`/`providers.openai.baseUrl`, then `OPENAI_API_KEY`/`OPENAI_BASE_URL`, when its own fields are unset. These settings apply only to Paseo OpenAI speech features, not to Codex or other OpenAI-backed tools.
|
||||
|
||||
Paseo uses these paths under the configured OpenAI base URL:
|
||||
|
||||
- dictation STT: `/v1/audio/transcriptions`
|
||||
- voice mode STT: `/v1/audio/transcriptions`
|
||||
- voice mode TTS: `/v1/audio/speech`
|
||||
`speech-settings.ts` owns mutable config reads and patches. The wire uses the same
|
||||
`features` structure. Reads replace OpenAI keys with `hasApiKey` (saved in config)
|
||||
and report effective env sources in `envOverrides`. A site patch that changes
|
||||
`provider` replaces the entire object. The same or omitted `provider` merges fields
|
||||
into the current variant, preserving the write-only key when a form re-saves it. `null` clears a value. Switching providers discards credentials and
|
||||
options from the previous variant.
|
||||
|
||||
---
|
||||
|
||||
|
||||
+3
-1
@@ -70,7 +70,7 @@ the sidebar. Apply it as `backgroundColor`; setting `opacity` on the control als
|
||||
|
||||
Sizes: `xs` for ultra-tight inline triggers. `sm` for any button sitting in a row. `md` is the page default. `lg` is reserved for large standalone CTAs.
|
||||
|
||||
Sizes are a shared contract across control kinds, defined once in `control-geometry.ts`: `xs` = 28px tall with `fontSize.sm` labels, `sm` = 32px with `fontSize.base`, `md`/`lg` = 44px with `fontSize.base`. `<SegmentedControl>` (`packages/app/src/components/ui/segmented-control.tsx`) takes the same `xs`/`sm`/`md` sizes — a segmented control next to a `<Button>` of the same size always matches in height, label size, and corner radius. Its segments run one padding step tighter than a button, because the gap between segments already reads as padding. The selected segment is a `surface3` fill with `foreground` text, not an inverted one — inverting it inside thin chrome puts a white slab in the toolbar. Thin chrome such as the file toolbar uses `xs`; settings rows use `sm`. Never shrink a control's font or padding locally to fit a context — if the context needs a smaller control, the size tier is missing or the wrong one is in use.
|
||||
Sizes are a shared contract across control kinds, defined once in `control-geometry.ts`: `xs` = 28px tall with `fontSize.sm` labels, `sm` = 32px with `fontSize.base`, `md`/`lg` = 44px with `fontSize.base`. `<SegmentedControl>` (`packages/app/src/components/ui/segmented-control.tsx`) takes the same `xs`/`sm`/`md` sizes — a segmented control next to a `<Button>` of the same size always matches in height, label size, and corner radius. Its segments run one padding step tighter than a button, because the gap between segments already reads as padding. The selected segment is a `surface3` fill with `foreground` text, not an inverted one — inverting it inside thin chrome puts a white slab in the toolbar. Text fields take `xs`, `sm` and `md`. Thin chrome such as the file toolbar uses `xs`. A settings row sizes the controls in its trailing slot itself, `sm` in a card and `xs` when nested (`packages/app/src/components/ui/control-size.tsx`), so don't pass a size to a button, dropdown or field placed in a row. Never shrink a control's font or padding locally to fit a context — if the context needs a smaller control, the size tier is missing or the wrong one is in use.
|
||||
|
||||
A `<Pressable>` wrapping a `<Text>` is a sixth variant. It is wrong. `<Button>` accepts `style`, `textStyle`, `leftIcon`, `disabled`, `size`, and `variant`.
|
||||
|
||||
@@ -126,6 +126,8 @@ Cards inside a section sit closer than sections. Rows inside a card touch — on
|
||||
|
||||
Rows have generous vertical padding: roughly 16px of content plus 16px of vertical padding for settings rows, 8–12px for sidebar list items where many rows must fit. Compressing rows below the established density to fit more on the screen is wrong. Too many rows means more cards or more sections, not smaller rows.
|
||||
|
||||
Rows revealed under a `<SettingsCollapsibleRow>` are nested rows, one step smaller than card rows: 8px of vertical padding, `fontSize.sm` text, `xs` controls, and one fixed height so rows with and without a trailing button line up. The collapsible row sets this for everything under it (`packages/app/src/components/settings/row-density.tsx`). Never pass a density to a row, and never pad or resize a nested row by hand.
|
||||
|
||||
The whitespace is the design.
|
||||
|
||||
---
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
import { expect, test } from "../support/fixtures";
|
||||
import {
|
||||
apiKeyInput,
|
||||
baseUrlInput,
|
||||
expectSheetClosed,
|
||||
expectSpeechSiteSummary,
|
||||
openSpeechSite,
|
||||
openVoiceSettings,
|
||||
providerOption,
|
||||
readSpeechSite,
|
||||
resetSpeechSite,
|
||||
saveSpeechSite,
|
||||
withUnwritableDaemonConfig,
|
||||
} from "../support/helpers/voice-settings";
|
||||
|
||||
// E2E daemons pin every speech provider to OpenAI through env vars so no test starts a model
|
||||
// download. That makes the provider picker read-only here; the env key covers the key fallback.
|
||||
const ENV_API_KEY = "sk-e2e-env-fallback";
|
||||
const SAVED_API_KEY = "sk-e2e-saved-key";
|
||||
const BASE_URL = "http://127.0.0.1:9/v1";
|
||||
|
||||
test.use({ e2eDaemonEnvironment: { OPENAI_API_KEY: ENV_API_KEY } });
|
||||
|
||||
test("configures an env-pinned OpenAI site and keeps the saved key on later saves", async ({
|
||||
page,
|
||||
}) => {
|
||||
try {
|
||||
await openVoiceSettings(page);
|
||||
await expectSpeechSiteSummary(page, "Listening", "OpenAI · whisper-1");
|
||||
|
||||
const sheet = await test.step("the env var decides the provider", async () => {
|
||||
const opened = await openSpeechSite(page, "Listening");
|
||||
await expect(opened.getByText("Set by PASEO_VOICE_STT_PROVIDER")).toBeVisible();
|
||||
await expect(providerOption(opened, "Local")).toBeDisabled();
|
||||
await expect(providerOption(opened, "OpenAI")).toHaveAttribute("aria-selected", "true");
|
||||
await expect(opened.getByText("Set by OPENAI_API_KEY")).toBeVisible();
|
||||
return opened;
|
||||
});
|
||||
|
||||
await test.step("a saved key and base URL win over the env key", async () => {
|
||||
await apiKeyInput(sheet).fill(SAVED_API_KEY);
|
||||
await baseUrlInput(sheet).fill(BASE_URL);
|
||||
await saveSpeechSite(sheet);
|
||||
await expectSheetClosed(page);
|
||||
await expect
|
||||
.poll(() => readSpeechSite("voiceMode.stt"))
|
||||
.toMatchObject({ provider: "openai", hasApiKey: true, baseUrl: BASE_URL });
|
||||
});
|
||||
|
||||
await test.step("reopening shows the key as saved without revealing it", async () => {
|
||||
const reopened = await openSpeechSite(page, "Listening");
|
||||
await expect(apiKeyInput(reopened)).toHaveValue("");
|
||||
await expect(apiKeyInput(reopened)).toHaveAttribute("placeholder", "Saved");
|
||||
await expect(reopened.getByText("Set by OPENAI_API_KEY")).toHaveCount(0);
|
||||
await expect(baseUrlInput(reopened)).toHaveValue(BASE_URL);
|
||||
await expect(reopened.getByRole("button", { name: "Save", exact: true })).toBeDisabled();
|
||||
});
|
||||
|
||||
await test.step("clearing the base URL keeps the saved key", async () => {
|
||||
const sheetAgain = page.getByRole("dialog");
|
||||
await baseUrlInput(sheetAgain).fill("");
|
||||
await saveSpeechSite(sheetAgain);
|
||||
await expectSheetClosed(page);
|
||||
const site = await readSpeechSite("voiceMode.stt");
|
||||
expect(site).toMatchObject({ provider: "openai", hasApiKey: true });
|
||||
expect(site).not.toHaveProperty("baseUrl");
|
||||
});
|
||||
} finally {
|
||||
await resetSpeechSite("voiceMode.stt");
|
||||
}
|
||||
});
|
||||
|
||||
test("a failed save keeps the sheet open with the user's input", async ({ page }) => {
|
||||
try {
|
||||
await openVoiceSettings(page);
|
||||
const sheet = await openSpeechSite(page, "Speaking");
|
||||
await baseUrlInput(sheet).fill(BASE_URL);
|
||||
|
||||
await withUnwritableDaemonConfig(async () => {
|
||||
await saveSpeechSite(sheet);
|
||||
await expect(page.getByText(/^Unable to update voice settings: /)).toBeVisible();
|
||||
});
|
||||
|
||||
await expect(baseUrlInput(sheet)).toHaveValue(BASE_URL);
|
||||
await saveSpeechSite(sheet);
|
||||
await expectSheetClosed(page);
|
||||
await expect
|
||||
.poll(() => readSpeechSite("voiceMode.tts"))
|
||||
.toMatchObject({ provider: "openai", baseUrl: BASE_URL });
|
||||
} finally {
|
||||
await resetSpeechSite("voiceMode.tts");
|
||||
}
|
||||
});
|
||||
@@ -42,6 +42,7 @@ type HostSection =
|
||||
| "pair-device"
|
||||
| "agents"
|
||||
| "metadata"
|
||||
| "voice"
|
||||
| "workspaces"
|
||||
| "providers"
|
||||
| "usage"
|
||||
|
||||
@@ -0,0 +1,99 @@
|
||||
import { rename, mkdir, rmdir } from "node:fs/promises";
|
||||
import path from "node:path";
|
||||
import { expect, type Locator, type Page } from "@playwright/test";
|
||||
import { gotoAppShell, openSettings } from "./app";
|
||||
import { connectNewWorkspaceDaemonClient } from "./new-workspace";
|
||||
import { getServerId } from "./server-id";
|
||||
import { expectSettingsHeader, openHostSection, openSettingsHost } from "./settings";
|
||||
|
||||
export type SpeechSiteLabel = "Speech-to-text" | "Listening" | "Speaking";
|
||||
type SpeechSiteId = "dictation.stt" | "voiceMode.stt" | "voiceMode.tts";
|
||||
|
||||
export async function openVoiceSettings(page: Page): Promise<void> {
|
||||
const serverId = getServerId();
|
||||
await gotoAppShell(page);
|
||||
await openSettings(page);
|
||||
await openSettingsHost(page, serverId);
|
||||
await openHostSection(page, serverId, "voice");
|
||||
await expectSettingsHeader(page, "Voice");
|
||||
}
|
||||
|
||||
function speechSiteRow(page: Page, label: SpeechSiteLabel): Locator {
|
||||
return page.getByRole("button", { name: new RegExp(`^${label}, `) });
|
||||
}
|
||||
|
||||
export async function expectSpeechSiteSummary(
|
||||
page: Page,
|
||||
label: SpeechSiteLabel,
|
||||
summary: string,
|
||||
): Promise<void> {
|
||||
await expect(speechSiteRow(page, label)).toHaveAccessibleName(`${label}, ${summary}`);
|
||||
}
|
||||
|
||||
/** Opens the provider sheet for one use site and returns it. */
|
||||
export async function openSpeechSite(page: Page, label: SpeechSiteLabel): Promise<Locator> {
|
||||
await speechSiteRow(page, label).click();
|
||||
const sheet = page.getByRole("dialog");
|
||||
await expect(sheet.getByText(label, { exact: true })).toBeVisible();
|
||||
return sheet;
|
||||
}
|
||||
|
||||
export function providerOption(sheet: Locator, provider: "Local" | "OpenAI"): Locator {
|
||||
return sheet.getByRole("button", { name: provider, exact: true });
|
||||
}
|
||||
|
||||
export function apiKeyInput(sheet: Locator): Locator {
|
||||
return sheet.getByRole("textbox", { name: "API key" });
|
||||
}
|
||||
|
||||
export function baseUrlInput(sheet: Locator): Locator {
|
||||
return sheet.getByRole("textbox", { name: "Base URL" });
|
||||
}
|
||||
|
||||
export async function saveSpeechSite(sheet: Locator): Promise<void> {
|
||||
await sheet.getByRole("button", { name: "Save", exact: true }).click();
|
||||
}
|
||||
|
||||
export async function expectSheetClosed(page: Page): Promise<void> {
|
||||
await expect(page.getByRole("dialog")).toHaveCount(0);
|
||||
}
|
||||
|
||||
/** The speech site as the daemon reports it; the API key itself never leaves the daemon. */
|
||||
export async function readSpeechSite(siteId: SpeechSiteId): Promise<Record<string, unknown>> {
|
||||
const client = await connectNewWorkspaceDaemonClient({ ownProjects: false });
|
||||
try {
|
||||
const { config } = await client.getDaemonConfig();
|
||||
const features = config.features as Record<string, Record<string, unknown>>;
|
||||
const [feature, site] = siteId.split(".");
|
||||
return features[feature][site] as Record<string, unknown>;
|
||||
} finally {
|
||||
await client.close().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
/** Puts a speech site back to the default provider so later tests start from a known state. */
|
||||
export async function resetSpeechSite(siteId: SpeechSiteId): Promise<void> {
|
||||
const client = await connectNewWorkspaceDaemonClient({ ownProjects: false });
|
||||
try {
|
||||
const [feature, site] = siteId.split(".");
|
||||
await client.patchDaemonConfig({ features: { [feature]: { [site]: { provider: "local" } } } });
|
||||
} finally {
|
||||
await client.close().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
|
||||
/** Makes the daemon's config file unreadable for the duration of `run`. */
|
||||
export async function withUnwritableDaemonConfig(run: () => Promise<void>): Promise<void> {
|
||||
const paseoHome = process.env.E2E_PASEO_HOME;
|
||||
if (!paseoHome) throw new Error("E2E_PASEO_HOME is required to break the daemon config");
|
||||
const configPath = path.join(paseoHome, "config.json");
|
||||
const asidePath = `${configPath}.aside`;
|
||||
await rename(configPath, asidePath);
|
||||
await mkdir(configPath);
|
||||
try {
|
||||
await run();
|
||||
} finally {
|
||||
await rmdir(configPath);
|
||||
await rename(asidePath, configPath);
|
||||
}
|
||||
}
|
||||
@@ -1,20 +1,18 @@
|
||||
import { ChevronRight } from "lucide-react-native";
|
||||
import { Children, isValidElement, useCallback, useMemo, useState, type ReactNode } from "react";
|
||||
import { Pressable, Text, View, type PressableStateCallbackType } from "react-native";
|
||||
import { StyleSheet, withUnistyles } from "react-native-unistyles";
|
||||
import { mutedIconColorMapping } from "@/components/ui/icon-color";
|
||||
import { View } from "react-native";
|
||||
import { StyleSheet } from "react-native-unistyles";
|
||||
import { isWeb } from "@/constants/platform";
|
||||
import { settingsStyles } from "@/styles/settings";
|
||||
import { ICON_SIZE, type Theme } from "@/styles/theme";
|
||||
|
||||
const ThemedChevron = withUnistyles(ChevronRight);
|
||||
const hoveredIconColorMapping = (theme: Theme) => ({ color: theme.colors.foreground });
|
||||
import { NestedSettingsRows } from "./row-density";
|
||||
import { SettingsRowHeader } from "./row-header";
|
||||
|
||||
interface SettingsCollapsibleRowProps {
|
||||
label: string;
|
||||
/** A short status shown before the chevron, so the row says something while collapsed. */
|
||||
summary?: string;
|
||||
defaultExpanded?: boolean;
|
||||
testID?: string;
|
||||
/** The rows revealed underneath, separated like the rows of a `SettingsCard`. */
|
||||
/** The rows revealed underneath, separated like the rows of a `SettingsCard` and laid out as nested rows. */
|
||||
children: ReactNode;
|
||||
}
|
||||
|
||||
@@ -25,6 +23,7 @@ interface SettingsCollapsibleRowProps {
|
||||
*/
|
||||
export function SettingsCollapsibleRow({
|
||||
label,
|
||||
summary,
|
||||
defaultExpanded = false,
|
||||
testID,
|
||||
children,
|
||||
@@ -37,66 +36,39 @@ export function SettingsCollapsibleRow({
|
||||
() => (isWeb ? ({ "aria-expanded": expanded } as const) : null),
|
||||
[expanded],
|
||||
);
|
||||
const headerStyle = useCallback(
|
||||
({ hovered, pressed }: PressableStateCallbackType & { hovered?: boolean }) => [
|
||||
settingsStyles.row,
|
||||
hovered && styles.headerHovered,
|
||||
pressed && styles.headerPressed,
|
||||
],
|
||||
[],
|
||||
);
|
||||
const chevronStyle = useMemo(
|
||||
() => [styles.chevron, expanded && styles.chevronExpanded],
|
||||
[expanded],
|
||||
);
|
||||
|
||||
return (
|
||||
<View testID={testID}>
|
||||
<Pressable
|
||||
{...webExpandedProps}
|
||||
accessibilityRole="button"
|
||||
accessibilityLabel={label}
|
||||
accessibilityState={accessibilityState}
|
||||
<SettingsRowHeader
|
||||
label={label}
|
||||
summary={summary}
|
||||
onPress={toggle}
|
||||
style={headerStyle}
|
||||
expanded={expanded}
|
||||
accessibilityState={accessibilityState}
|
||||
webAccessibilityProps={webExpandedProps}
|
||||
testID={testID ? `${testID}-toggle` : undefined}
|
||||
>
|
||||
{({ hovered }: PressableStateCallbackType & { hovered?: boolean }) => (
|
||||
<>
|
||||
<View style={settingsStyles.rowContent}>
|
||||
<Text style={settingsStyles.rowTitle}>{label}</Text>
|
||||
</View>
|
||||
<View style={chevronStyle}>
|
||||
<ThemedChevron
|
||||
size={ICON_SIZE.sm}
|
||||
uniProps={hovered ? hoveredIconColorMapping : mutedIconColorMapping}
|
||||
/>
|
||||
</View>
|
||||
</>
|
||||
)}
|
||||
</Pressable>
|
||||
{expanded
|
||||
? Children.toArray(children).map((child, index) => (
|
||||
<View key={isValidElement(child) ? child.key : index} style={settingsStyles.rowBorder}>
|
||||
{child}
|
||||
</View>
|
||||
))
|
||||
: null}
|
||||
/>
|
||||
{expanded ? (
|
||||
<View style={styles.children}>
|
||||
<NestedSettingsRows>
|
||||
{Children.toArray(children).map((child, index) => (
|
||||
<View
|
||||
key={isValidElement(child) ? child.key : index}
|
||||
style={settingsStyles.rowBorder}
|
||||
>
|
||||
{child}
|
||||
</View>
|
||||
))}
|
||||
</NestedSettingsRows>
|
||||
</View>
|
||||
) : null}
|
||||
</View>
|
||||
);
|
||||
}
|
||||
|
||||
const styles = StyleSheet.create((theme) => ({
|
||||
headerHovered: {
|
||||
backgroundColor: theme.colors.surface2,
|
||||
},
|
||||
headerPressed: {
|
||||
backgroundColor: theme.colors.surface3,
|
||||
},
|
||||
// Optical: the chevron's ink sits about 4px inside its box whether it points right or down; this
|
||||
// puts the ink on the card's trailing rail, where the switch and the reports' refresh buttons end.
|
||||
chevron: { marginRight: -theme.spacing[1] },
|
||||
chevronExpanded: {
|
||||
transform: [{ rotate: "90deg" }],
|
||||
// The revealed rows keep the card's rails; the tint is what groups them under their header.
|
||||
children: {
|
||||
backgroundColor: theme.colors.surface0,
|
||||
},
|
||||
}));
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
import { SettingsRowHeader } from "./row-header";
|
||||
|
||||
interface SettingsDetailRowProps {
|
||||
label: string;
|
||||
/** What the detail is set to, so the row says it without being opened. */
|
||||
summary?: string;
|
||||
onPress: () => void;
|
||||
/** A row that cannot be opened keeps its summary and drops the chevron. */
|
||||
disabled?: boolean;
|
||||
testID?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* A settings row that opens a detail, such as a sheet, for one setting. The label, a one-line
|
||||
* summary of the current value and a chevron pointing right. Sits inside a `SettingsCard`.
|
||||
*/
|
||||
export function SettingsDetailRow({
|
||||
label,
|
||||
summary,
|
||||
onPress,
|
||||
disabled,
|
||||
testID,
|
||||
}: SettingsDetailRowProps) {
|
||||
return (
|
||||
<SettingsRowHeader
|
||||
label={label}
|
||||
summary={summary}
|
||||
onPress={onPress}
|
||||
disabled={disabled}
|
||||
testID={testID}
|
||||
/>
|
||||
);
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { ReactNode } from "react";
|
||||
import type { ComponentType, ReactNode } from "react";
|
||||
import { Pressable, Text } from "react-native";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import { Info } from "lucide-react-native";
|
||||
@@ -6,7 +6,13 @@ import { StyleSheet, withUnistyles } from "react-native-unistyles";
|
||||
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
|
||||
import { ICON_SIZE, type Theme } from "@/styles/theme";
|
||||
|
||||
const ThemedInfo = withUnistyles(Info);
|
||||
type TipIconComponent = ComponentType<{ size: number; color: string }>;
|
||||
|
||||
function TipIcon({ icon: Icon, color }: { icon: TipIconComponent; color: string }) {
|
||||
return <Icon size={ICON_SIZE.sm} color={color} />;
|
||||
}
|
||||
|
||||
const ThemedTipIcon = withUnistyles(TipIcon);
|
||||
|
||||
const mutedIconMapping = (theme: Theme) => ({ color: theme.colors.foregroundMuted });
|
||||
|
||||
@@ -14,27 +20,37 @@ interface SettingsInfoTipProps {
|
||||
/** Header the tip belongs to; used for the accessibility label. */
|
||||
title: string;
|
||||
info: ReactNode;
|
||||
/** Defaults to the info icon. */
|
||||
icon?: TipIconComponent;
|
||||
/** Defaults to "About {title}". */
|
||||
accessibilityLabel?: string;
|
||||
testID?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The one way a settings group or section explains itself: an info icon on the
|
||||
* header that opens a tooltip. Never a paragraph under the header — see
|
||||
* docs/design.md §7.
|
||||
* docs/design.md §7. Rows use it the same way to keep secondary details on one line.
|
||||
*/
|
||||
export function SettingsInfoTip({ title, info, testID }: SettingsInfoTipProps) {
|
||||
export function SettingsInfoTip({
|
||||
title,
|
||||
info,
|
||||
icon = Info,
|
||||
accessibilityLabel,
|
||||
testID,
|
||||
}: SettingsInfoTipProps) {
|
||||
const { t } = useTranslation();
|
||||
return (
|
||||
<Tooltip delayDuration={0} enabledOnDesktop enabledOnMobile>
|
||||
<TooltipTrigger asChild>
|
||||
<Pressable
|
||||
accessibilityRole="button"
|
||||
accessibilityLabel={t("settings.groupInfo", { title })}
|
||||
accessibilityLabel={accessibilityLabel ?? t("settings.groupInfo", { title })}
|
||||
testID={testID}
|
||||
hitSlop={8}
|
||||
style={styles.button}
|
||||
>
|
||||
<ThemedInfo size={ICON_SIZE.sm} uniProps={mutedIconMapping} />
|
||||
<ThemedTipIcon icon={icon} uniProps={mutedIconMapping} />
|
||||
</Pressable>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="top" align="start" offset={8}>
|
||||
|
||||
@@ -24,15 +24,23 @@ import { DropdownMenu, DropdownMenuContent, DropdownMenuItem } from "@/component
|
||||
import { DropdownTrigger } from "@/components/ui/dropdown-trigger";
|
||||
import { useIsCompactFormFactor } from "@/constants/layout";
|
||||
import { settingsStyles } from "@/styles/settings";
|
||||
import { ControlSizeProvider } from "@/components/ui/control-size";
|
||||
import { useSettingsRowLayout } from "./row-density";
|
||||
export { SettingsGroup } from "./headings/settings-group";
|
||||
export { SettingsSection } from "./headings/settings-section";
|
||||
export { SettingsCollapsibleRow } from "./collapsible-row";
|
||||
export { SettingsDetailRow } from "./detail-row";
|
||||
export { SettingsInfoTip } from "./headings/settings-info-tip";
|
||||
|
||||
interface AppSettingsRowProps extends Omit<SettingsRowProps, "hint"> {
|
||||
hint?: ReactNode;
|
||||
/** What app screens can add to a plugin-facing row: an accessory beside the label, such as an info tip. */
|
||||
interface AppRowExtras {
|
||||
labelAccessory?: ReactNode;
|
||||
}
|
||||
|
||||
interface AppSettingsRowProps extends Omit<SettingsRowProps, "hint">, AppRowExtras {
|
||||
hint?: ReactNode;
|
||||
}
|
||||
|
||||
export function SettingsCard({ children, testID }: { children: ReactNode; testID?: string }) {
|
||||
return (
|
||||
<View style={settingsStyles.card} testID={testID}>
|
||||
@@ -57,26 +65,32 @@ export function SettingsRow({
|
||||
testID,
|
||||
}: AppSettingsRowProps) {
|
||||
const compact = useIsCompactFormFactor();
|
||||
const rowStyle = useMemo(() => [settingsStyles.row, compact && styles.compactRow], [compact]);
|
||||
const layout = useSettingsRowLayout();
|
||||
const rowStyle = useMemo(() => [layout.row, compact && styles.compactRow], [compact, layout]);
|
||||
const accessoryTitleStyle = useMemo(() => [layout.title, styles.accessoryLabel], [layout]);
|
||||
return (
|
||||
<View style={rowStyle} testID={testID}>
|
||||
<View style={styles.label}>
|
||||
{labelAccessory ? (
|
||||
<View style={styles.labelRow}>
|
||||
<Text style={[settingsStyles.rowTitle, styles.accessoryLabel]}>{label}</Text>
|
||||
<Text style={accessoryTitleStyle}>{label}</Text>
|
||||
{labelAccessory}
|
||||
</View>
|
||||
) : (
|
||||
<Text style={settingsStyles.rowTitle}>{label}</Text>
|
||||
<Text style={layout.title}>{label}</Text>
|
||||
)}
|
||||
{typeof hint === "string" ? <Text style={settingsStyles.rowHint}>{hint}</Text> : hint}
|
||||
{typeof hint === "string" ? <Text style={layout.hint}>{hint}</Text> : hint}
|
||||
{error ? (
|
||||
<Text accessibilityRole="alert" style={settingsStyles.rowError}>
|
||||
{error}
|
||||
</Text>
|
||||
) : null}
|
||||
</View>
|
||||
{children ? <View style={styles.control}>{children}</View> : null}
|
||||
{children ? (
|
||||
<View style={styles.control}>
|
||||
<ControlSizeProvider size={layout.controlSize}>{children}</ControlSizeProvider>
|
||||
</View>
|
||||
) : null}
|
||||
</View>
|
||||
);
|
||||
}
|
||||
@@ -117,7 +131,7 @@ export function SettingsSelect<Value extends string>({
|
||||
onValueChange,
|
||||
disabled,
|
||||
...row
|
||||
}: SettingsSelectProps<Value>) {
|
||||
}: SettingsSelectProps<Value> & AppRowExtras) {
|
||||
const selectedLabel = options.find((option) => option.value === value)?.label ?? value;
|
||||
return (
|
||||
<SettingsRow {...row}>
|
||||
@@ -152,7 +166,7 @@ export function SettingsInput({
|
||||
secureTextEntry,
|
||||
ref,
|
||||
...row
|
||||
}: SettingsInputProps) {
|
||||
}: SettingsInputProps & AppRowExtras) {
|
||||
const compact = useIsCompactFormFactor();
|
||||
const input = useRef<EditingTextInputHandle>(null);
|
||||
useImperativeHandle(
|
||||
@@ -175,7 +189,7 @@ export function SettingsInput({
|
||||
editable={!disabled}
|
||||
secureTextEntry={secureTextEntry}
|
||||
accessibilityLabel={row.label}
|
||||
size={compact ? "md" : "sm"}
|
||||
size={compact ? "md" : undefined}
|
||||
style={styles.input}
|
||||
/>
|
||||
</SettingsRow>
|
||||
@@ -185,7 +199,7 @@ export function SettingsInput({
|
||||
export function SettingsAction({ actionLabel, onPress, disabled, ...row }: SettingsActionProps) {
|
||||
return (
|
||||
<SettingsRow {...row}>
|
||||
<Button variant="outline" size="sm" onPress={onPress} disabled={disabled}>
|
||||
<Button variant="outline" onPress={onPress} disabled={disabled}>
|
||||
{actionLabel}
|
||||
</Button>
|
||||
</SettingsRow>
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
import React, { act } from "react";
|
||||
import { createRoot, type Root } from "react-dom/client";
|
||||
import { afterEach, describe, expect, it } from "vitest";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { SettingsCard, SettingsCollapsibleRow, SettingsRow } from "@/components/settings";
|
||||
|
||||
let root: Root | null = null;
|
||||
let container: HTMLDivElement | null = null;
|
||||
|
||||
afterEach(() => {
|
||||
act(() => root?.unmount());
|
||||
container?.remove();
|
||||
root = null;
|
||||
container = null;
|
||||
});
|
||||
|
||||
function mountRows() {
|
||||
container = document.createElement("div");
|
||||
document.body.appendChild(container);
|
||||
root = createRoot(container);
|
||||
act(() =>
|
||||
root?.render(
|
||||
<SettingsCard>
|
||||
<SettingsRow label="Card row" testID="card-row">
|
||||
<Button variant="outline" testID="card-button">
|
||||
Download
|
||||
</Button>
|
||||
</SettingsRow>
|
||||
<SettingsCollapsibleRow label="Group" defaultExpanded testID="group">
|
||||
<SettingsRow label="Nested with a button" testID="nested-with-button">
|
||||
<Button variant="outline" testID="nested-button">
|
||||
Download
|
||||
</Button>
|
||||
</SettingsRow>
|
||||
<SettingsRow label="Nested without a button" testID="nested-without-button" />
|
||||
</SettingsCollapsibleRow>
|
||||
</SettingsCard>,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
function element(testID: string): HTMLElement {
|
||||
const found = document.querySelector(`[data-testid="${testID}"]`);
|
||||
if (!(found instanceof HTMLElement)) throw new Error(`Nothing rendered as ${testID}`);
|
||||
return found;
|
||||
}
|
||||
|
||||
function heightOf(testID: string): number {
|
||||
return element(testID).getBoundingClientRect().height;
|
||||
}
|
||||
|
||||
function titleFontSize(rowTestID: string, label: string): number {
|
||||
const title = [...element(rowTestID).querySelectorAll("div")].find(
|
||||
(node) => node.textContent === label && node.children.length === 0,
|
||||
);
|
||||
if (!title) throw new Error(`No title ${label}`);
|
||||
return Number.parseFloat(getComputedStyle(title).fontSize);
|
||||
}
|
||||
|
||||
describe("settings row density", () => {
|
||||
it("lines up nested rows with and without a button, tighter than card rows", () => {
|
||||
mountRows();
|
||||
|
||||
expect(heightOf("nested-with-button")).toBe(heightOf("nested-without-button"));
|
||||
expect(heightOf("nested-with-button")).toBeLessThan(heightOf("card-row"));
|
||||
expect(heightOf("group-toggle")).toBeGreaterThan(heightOf("nested-without-button"));
|
||||
});
|
||||
|
||||
it("sizes nested text and controls one step below card rows", () => {
|
||||
mountRows();
|
||||
|
||||
expect(heightOf("nested-button")).toBeLessThan(heightOf("card-button"));
|
||||
expect(titleFontSize("nested-with-button", "Nested with a button")).toBeLessThan(
|
||||
titleFontSize("card-row", "Card row"),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,57 @@
|
||||
import { createContext, useContext, useMemo, type ReactNode } from "react";
|
||||
import type { StyleProp, TextStyle, ViewStyle } from "react-native";
|
||||
import { StyleSheet } from "react-native-unistyles";
|
||||
import { CONTROL_HEIGHTS } from "@/components/ui/control-geometry";
|
||||
import type { ContainedControlSize } from "@/components/ui/control-size";
|
||||
import { settingsStyles } from "@/styles/settings";
|
||||
|
||||
// Rows revealed under a collapsible row belong to it, so they sit one step smaller than the rows
|
||||
// of the card: tighter padding, smaller text, smaller controls. The container decides; a row
|
||||
// never knows on its own that it is nested.
|
||||
const NestedRowsContext = createContext(false);
|
||||
|
||||
export function NestedSettingsRows({ children }: { children: ReactNode }) {
|
||||
return <NestedRowsContext.Provider value>{children}</NestedRowsContext.Provider>;
|
||||
}
|
||||
|
||||
export interface SettingsRowLayout {
|
||||
row: StyleProp<ViewStyle>;
|
||||
title: StyleProp<TextStyle>;
|
||||
hint: StyleProp<TextStyle>;
|
||||
/** The size of the controls in the row's trailing slot. */
|
||||
controlSize: ContainedControlSize;
|
||||
}
|
||||
|
||||
/** How a row lays out where it is rendered: in a card, or nested under a collapsible row. */
|
||||
export function useSettingsRowLayout(): SettingsRowLayout {
|
||||
const nested = useContext(NestedRowsContext);
|
||||
return useMemo(
|
||||
() =>
|
||||
nested
|
||||
? {
|
||||
row: [settingsStyles.row, styles.nestedRow],
|
||||
title: [settingsStyles.rowTitle, styles.nestedText],
|
||||
hint: [settingsStyles.rowHint, styles.nestedText],
|
||||
controlSize: "xs",
|
||||
}
|
||||
: {
|
||||
row: settingsStyles.row,
|
||||
title: settingsStyles.rowTitle,
|
||||
hint: settingsStyles.rowHint,
|
||||
controlSize: "sm",
|
||||
},
|
||||
[nested],
|
||||
);
|
||||
}
|
||||
|
||||
const styles = StyleSheet.create((theme) => ({
|
||||
// Every nested row is as tall as one holding an `xs` control, so rows with and without a
|
||||
// trailing button line up.
|
||||
nestedRow: {
|
||||
paddingVertical: theme.spacing[2],
|
||||
minHeight: CONTROL_HEIGHTS.tight + theme.spacing[2] * 2,
|
||||
},
|
||||
nestedText: {
|
||||
fontSize: theme.fontSize.sm,
|
||||
},
|
||||
}));
|
||||
@@ -0,0 +1,113 @@
|
||||
import { ChevronRight } from "lucide-react-native";
|
||||
import { useCallback, useMemo } from "react";
|
||||
import { Text, View, type AccessibilityState, type PressableStateCallbackType } from "react-native";
|
||||
import { StyleSheet, withUnistyles } from "react-native-unistyles";
|
||||
import { mutedIconColorMapping } from "@/components/ui/icon-color";
|
||||
import { PressHighlight } from "@/components/ui/press-highlight";
|
||||
import { settingsStyles } from "@/styles/settings";
|
||||
import { ICON_SIZE, type Theme } from "@/styles/theme";
|
||||
import { useSettingsRowLayout } from "./row-density";
|
||||
|
||||
const ThemedChevron = withUnistyles(ChevronRight);
|
||||
const hoveredIconColorMapping = (theme: Theme) => ({ color: theme.colors.foreground });
|
||||
|
||||
type HoverState = PressableStateCallbackType & { hovered?: boolean };
|
||||
|
||||
export interface SettingsRowHeaderProps {
|
||||
label: string;
|
||||
/** A short status shown before the chevron. */
|
||||
summary?: string;
|
||||
onPress: () => void;
|
||||
disabled?: boolean;
|
||||
/** Turns the chevron down, for a row whose children are revealed in place. */
|
||||
expanded?: boolean;
|
||||
accessibilityState?: AccessibilityState;
|
||||
/** Props React Native Web does not derive from `accessibilityState`, such as `aria-expanded`. */
|
||||
webAccessibilityProps?: Record<string, unknown> | null;
|
||||
testID?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The pressable line shared by settings rows that lead somewhere: a label, an optional summary and
|
||||
* a trailing chevron. A collapsible row turns the chevron down; a row that opens a sheet keeps it
|
||||
* pointing right.
|
||||
*/
|
||||
export function SettingsRowHeader({
|
||||
label,
|
||||
summary,
|
||||
onPress,
|
||||
disabled,
|
||||
expanded = false,
|
||||
accessibilityState,
|
||||
webAccessibilityProps,
|
||||
testID,
|
||||
}: SettingsRowHeaderProps) {
|
||||
const layout = useSettingsRowLayout();
|
||||
const rowStyle = useCallback(
|
||||
({ hovered, pressed }: HoverState) => [
|
||||
layout.row,
|
||||
!disabled && hovered && styles.hovered,
|
||||
!disabled && pressed && styles.pressed,
|
||||
],
|
||||
[disabled, layout],
|
||||
);
|
||||
const chevronStyle = useMemo(
|
||||
() => [styles.chevron, expanded && styles.chevronExpanded],
|
||||
[expanded],
|
||||
);
|
||||
return (
|
||||
<PressHighlight
|
||||
{...webAccessibilityProps}
|
||||
accessibilityRole="button"
|
||||
accessibilityLabel={summary ? `${label}, ${summary}` : label}
|
||||
accessibilityState={accessibilityState}
|
||||
disabled={disabled}
|
||||
onPress={onPress}
|
||||
style={rowStyle}
|
||||
highlightStyle={styles.pressed}
|
||||
testID={testID}
|
||||
>
|
||||
{({ hovered }: HoverState) => (
|
||||
<>
|
||||
<View style={settingsStyles.rowContent}>
|
||||
<Text style={layout.title}>{label}</Text>
|
||||
</View>
|
||||
{summary ? (
|
||||
<Text style={styles.summary} numberOfLines={1}>
|
||||
{summary}
|
||||
</Text>
|
||||
) : null}
|
||||
{disabled ? null : (
|
||||
<View style={chevronStyle}>
|
||||
<ThemedChevron
|
||||
size={ICON_SIZE.sm}
|
||||
uniProps={hovered ? hoveredIconColorMapping : mutedIconColorMapping}
|
||||
/>
|
||||
</View>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</PressHighlight>
|
||||
);
|
||||
}
|
||||
|
||||
const styles = StyleSheet.create((theme) => ({
|
||||
summary: {
|
||||
flexShrink: 1,
|
||||
color: theme.colors.foregroundMuted,
|
||||
fontSize: theme.fontSize.sm,
|
||||
marginRight: theme.spacing[2],
|
||||
},
|
||||
hovered: {
|
||||
backgroundColor: theme.colors.surface2,
|
||||
},
|
||||
pressed: {
|
||||
backgroundColor: theme.colors.surface3,
|
||||
},
|
||||
// Optical: the chevron's ink sits about 4px inside its box whether it points right or down; this
|
||||
// puts the ink on the card's trailing rail, where the switch and the reports' refresh buttons end.
|
||||
chevron: { marginRight: -theme.spacing[1] },
|
||||
chevronExpanded: {
|
||||
transform: [{ rotate: "90deg" }],
|
||||
},
|
||||
}));
|
||||
@@ -1,4 +1,5 @@
|
||||
import { LoadingSpinner } from "@/components/ui/loading-spinner";
|
||||
import { useContainedControlSize } from "@/components/ui/control-size";
|
||||
import {
|
||||
default as React,
|
||||
useCallback,
|
||||
@@ -162,7 +163,7 @@ const styles = StyleSheet.create((theme) => {
|
||||
export function Button({
|
||||
children,
|
||||
variant = "secondary",
|
||||
size = "md",
|
||||
size: sizeProp,
|
||||
leftIcon,
|
||||
trailing,
|
||||
style,
|
||||
@@ -185,6 +186,8 @@ export function Button({
|
||||
}
|
||||
>) {
|
||||
const [hovered, setHovered] = useState(false);
|
||||
const containedSize = useContainedControlSize();
|
||||
const size = sizeProp ?? containedSize ?? "md";
|
||||
const isDisabled = disabled || loading;
|
||||
|
||||
let variantStyle: ViewStyle;
|
||||
|
||||
@@ -81,6 +81,8 @@ describe("control geometry", () => {
|
||||
it("keeps field text sizing tied to control size", () => {
|
||||
const geometry = createControlGeometry(theme);
|
||||
|
||||
expect(geometry.fieldTextXs.fontSize).toBe(12);
|
||||
expect(geometry.fieldTextXs.lineHeight).toBe(17);
|
||||
expect(geometry.fieldTextSm.fontSize).toBe(14);
|
||||
expect(geometry.fieldTextSm.lineHeight).toBe(20);
|
||||
expect(geometry.fieldTextMd.fontSize).toBe(14);
|
||||
@@ -95,6 +97,13 @@ describe("control geometry", () => {
|
||||
const geometry = createControlGeometry(theme);
|
||||
const borderWidth = theme.borderWidth[1];
|
||||
|
||||
expect(geometry.fieldControlXs.minHeight).toBe(28);
|
||||
expect(
|
||||
geometry.fieldTextXs.lineHeight +
|
||||
geometry.fieldControlXs.paddingVertical * 2 +
|
||||
borderWidth * 2,
|
||||
).toBe(geometry.fieldControlXs.minHeight);
|
||||
|
||||
expect(geometry.fieldControlSm.minHeight).toBe(32);
|
||||
expect(geometry.fieldControlSm.paddingVertical).toBe(5);
|
||||
expect(
|
||||
|
||||
@@ -2,7 +2,7 @@ import type { StyleProp, ViewStyle } from "react-native";
|
||||
import { ICON_SIZE, type Theme } from "@/styles/theme";
|
||||
|
||||
export type ButtonControlSize = "xs" | "sm" | "md" | "lg";
|
||||
export type FieldControlSize = "sm" | "md";
|
||||
export type FieldControlSize = "xs" | "sm" | "md";
|
||||
export type SegmentedControlSize = "xs" | "sm" | "md";
|
||||
export type ControlInteractionPhase = "rest" | "hover" | "active";
|
||||
|
||||
@@ -188,8 +188,19 @@ export function resolveControlInteractionStyles(
|
||||
|
||||
export function createControlGeometry(theme: Theme) {
|
||||
const controlBorderWidth = theme.borderWidth[1];
|
||||
const fieldTextXsLineHeight = fieldLineHeight(theme.fontSize.sm);
|
||||
const fieldTextSmLineHeight = fieldLineHeight(theme.fontSize.base);
|
||||
const fieldTextMdLineHeight = fieldLineHeight(theme.fontSize.base);
|
||||
const fieldControlXs = {
|
||||
minHeight: CONTROL_HEIGHTS.tight,
|
||||
paddingHorizontal: theme.spacing[2],
|
||||
paddingVertical: fieldVerticalPadding(
|
||||
CONTROL_HEIGHTS.tight,
|
||||
fieldTextXsLineHeight,
|
||||
controlBorderWidth,
|
||||
),
|
||||
borderRadius: theme.borderRadius.md,
|
||||
};
|
||||
const fieldControlSm = {
|
||||
minHeight: CONTROL_HEIGHTS.compact,
|
||||
paddingHorizontal: theme.spacing[3],
|
||||
@@ -210,6 +221,10 @@ export function createControlGeometry(theme: Theme) {
|
||||
),
|
||||
borderRadius: theme.borderRadius.lg,
|
||||
};
|
||||
const fieldTextXs = {
|
||||
fontSize: theme.fontSize.sm,
|
||||
lineHeight: fieldTextXsLineHeight,
|
||||
};
|
||||
const fieldTextSm = {
|
||||
fontSize: theme.fontSize.base,
|
||||
lineHeight: fieldTextSmLineHeight,
|
||||
@@ -252,6 +267,10 @@ export function createControlGeometry(theme: Theme) {
|
||||
fontSize: theme.fontSize.sm,
|
||||
},
|
||||
alert,
|
||||
formTextInputXs: {
|
||||
...fieldControlXs,
|
||||
...fieldTextXs,
|
||||
},
|
||||
formTextInputSm: {
|
||||
...fieldControlSm,
|
||||
...fieldTextSm,
|
||||
@@ -264,8 +283,10 @@ export function createControlGeometry(theme: Theme) {
|
||||
...fieldControlMd,
|
||||
...fieldTextMd,
|
||||
},
|
||||
fieldControlXs,
|
||||
fieldControlSm,
|
||||
fieldControlMd,
|
||||
fieldTextXs,
|
||||
fieldTextSm,
|
||||
fieldTextMd,
|
||||
controlRest: {
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import { createContext, useContext, type ReactNode } from "react";
|
||||
|
||||
/** The control sizes a container can ask of the controls placed in it. */
|
||||
export type ContainedControlSize = "xs" | "sm";
|
||||
|
||||
// A row decides how big its controls are, so a button or field placed in it needs no size of
|
||||
// its own. A size passed to the control still wins.
|
||||
const ControlSizeContext = createContext<ContainedControlSize | null>(null);
|
||||
|
||||
export function ControlSizeProvider({
|
||||
size,
|
||||
children,
|
||||
}: {
|
||||
size: ContainedControlSize;
|
||||
children: ReactNode;
|
||||
}) {
|
||||
return <ControlSizeContext.Provider value={size}>{children}</ControlSizeContext.Provider>;
|
||||
}
|
||||
|
||||
/** The size the surrounding container asks for, or null outside one. */
|
||||
export function useContainedControlSize(): ContainedControlSize | null {
|
||||
return useContext(ControlSizeContext);
|
||||
}
|
||||
@@ -9,6 +9,7 @@ import {
|
||||
} from "@/components/ui/control-geometry";
|
||||
import type { Theme } from "@/styles/theme";
|
||||
import { DropdownMenuTrigger, type DropdownMenuTriggerProps } from "@/components/ui/dropdown-menu";
|
||||
import { useContainedControlSize } from "@/components/ui/control-size";
|
||||
|
||||
const ThemedChevronDown = withUnistyles(ChevronDown);
|
||||
|
||||
@@ -32,10 +33,12 @@ interface DropdownTriggerProps extends Omit<DropdownMenuTriggerProps, "children"
|
||||
export function DropdownTrigger({
|
||||
children,
|
||||
leading,
|
||||
size = "sm",
|
||||
size: sizeProp,
|
||||
disabled,
|
||||
...props
|
||||
}: DropdownTriggerProps): ReactElement {
|
||||
const containedSize = useContainedControlSize();
|
||||
const size = sizeProp ?? containedSize ?? "sm";
|
||||
let sizeStyle: ViewStyle;
|
||||
if (size === "xs") {
|
||||
sizeStyle = styles.xs;
|
||||
|
||||
@@ -23,6 +23,16 @@ import {
|
||||
resolveControlInteractionStyles,
|
||||
type FieldControlSize,
|
||||
} from "@/components/ui/control-geometry";
|
||||
import { useContainedControlSize } from "@/components/ui/control-size";
|
||||
|
||||
const FIELD_CHROME = { xs: "chromeXs", sm: "chromeSm", md: "chromeMd" } as const satisfies Record<
|
||||
FieldControlSize,
|
||||
string
|
||||
>;
|
||||
const FIELD_TEXT = { xs: "inputXs", sm: "inputSm", md: "inputMd" } as const satisfies Record<
|
||||
FieldControlSize,
|
||||
string
|
||||
>;
|
||||
|
||||
interface FieldProps {
|
||||
label: string;
|
||||
@@ -159,11 +169,13 @@ function assignTextInputRef(
|
||||
}
|
||||
|
||||
export const FormTextInput = forwardRef<EditingTextInputHandle, FormTextInputProps>(
|
||||
function FormTextInput({ size = "md", style, onFocus, onBlur, editable, ...props }, ref) {
|
||||
function FormTextInput({ size: sizeProp, style, onFocus, onBlur, editable, ...props }, ref) {
|
||||
const [focused, setFocused] = useState(false);
|
||||
const containedSize = useContainedControlSize();
|
||||
const size = sizeProp ?? containedSize ?? "md";
|
||||
const isDisabled = editable === false;
|
||||
const chromeSizeStyle = size === "sm" ? formInputStyles.chromeSm : formInputStyles.chromeMd;
|
||||
const inputSizeStyle = size === "sm" ? formInputStyles.inputSm : formInputStyles.inputMd;
|
||||
const chromeSizeStyle = formInputStyles[FIELD_CHROME[size]];
|
||||
const inputSizeStyle = formInputStyles[FIELD_TEXT[size]];
|
||||
const splitStyle = useMemo(() => splitFormTextInputStyle(style), [style]);
|
||||
const setInputRef = useCallback(
|
||||
(node: EditingTextInputHandle | null) => {
|
||||
@@ -262,6 +274,9 @@ const formInputStyles = StyleSheet.create((theme) => {
|
||||
chrome: {
|
||||
backgroundColor: theme.colors.surface2,
|
||||
},
|
||||
chromeXs: {
|
||||
...geometry.fieldControlXs,
|
||||
},
|
||||
chromeSm: {
|
||||
...geometry.fieldControlSm,
|
||||
},
|
||||
@@ -289,6 +304,9 @@ const formInputStyles = StyleSheet.create((theme) => {
|
||||
outlineColor: "transparent",
|
||||
outlineWidth: 0,
|
||||
},
|
||||
inputXs: {
|
||||
...geometry.fieldTextXs,
|
||||
},
|
||||
inputSm: {
|
||||
...geometry.fieldTextSm,
|
||||
},
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
import type { Theme } from "@/styles/theme";
|
||||
|
||||
export const foregroundIconColorMapping = (theme: Theme) => ({
|
||||
color: theme.colors.foreground,
|
||||
});
|
||||
|
||||
export const mutedIconColorMapping = (theme: Theme) => ({
|
||||
color: theme.colors.foregroundMuted,
|
||||
});
|
||||
|
||||
@@ -7,6 +7,7 @@ import {
|
||||
segmentedIconSize,
|
||||
type SegmentedControlSize,
|
||||
} from "@/components/ui/control-geometry";
|
||||
import { useContainedControlSize } from "@/components/ui/control-size";
|
||||
import type { Theme } from "@/styles/theme";
|
||||
|
||||
type SegmentedControlIconRenderer = (props: { color: string; size: number }) => ReactNode;
|
||||
@@ -48,11 +49,13 @@ export function SegmentedControl<T extends string>({
|
||||
options,
|
||||
value,
|
||||
onValueChange,
|
||||
size = "md",
|
||||
size: sizeProp,
|
||||
hideLabels = false,
|
||||
style,
|
||||
testID,
|
||||
}: SegmentedControlProps<T>) {
|
||||
const containedSize = useContainedControlSize();
|
||||
const size = sizeProp ?? containedSize ?? "md";
|
||||
const sizeStyles = {
|
||||
xs: { container: styles.containerXs, segment: styles.segmentXs, label: styles.labelXs },
|
||||
sm: { container: styles.containerSm, segment: styles.segmentSm, label: styles.labelSm },
|
||||
|
||||
@@ -2260,6 +2260,7 @@ export const ar: TranslationResources = {
|
||||
connections: "اتصالات",
|
||||
agents: "Agents",
|
||||
metadata: "البيانات الوصفية",
|
||||
voice: "الصوت",
|
||||
workspaces: "Workspaces",
|
||||
providers: "مقدمي الخدمات",
|
||||
usage: "الاستخدام",
|
||||
@@ -2282,6 +2283,69 @@ export const ar: TranslationResources = {
|
||||
docs: "الوثائق",
|
||||
saveError: "تعذر تحديث إنشاء البيانات الوصفية",
|
||||
},
|
||||
voice: {
|
||||
offline: "اتصل بهذا المضيف لضبط الصوت.",
|
||||
updateHost: "حدّث المضيف لضبط الصوت.",
|
||||
saveError: "تعذّر تحديث إعدادات الصوت",
|
||||
provider: "المزوّد",
|
||||
providers: {
|
||||
local: "محلي",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "الصوت",
|
||||
save: "حفظ",
|
||||
retry: "إعادة المحاولة",
|
||||
cancel: "إلغاء",
|
||||
model: "النموذج",
|
||||
language: "اللغة",
|
||||
languagePlaceholder: "اكتشاف تلقائي",
|
||||
setBy: "محدد بواسطة {{name}}",
|
||||
unknownProvider: "حدّث التطبيق لإعداد {{provider}}",
|
||||
sites: {
|
||||
dictationStt: "تحويل الكلام إلى نص",
|
||||
voiceModeStt: "الاستماع",
|
||||
voiceModeTts: "التحدث",
|
||||
},
|
||||
dictation: {
|
||||
title: "الإملاء",
|
||||
enabled: "الإملاء",
|
||||
enabledHint: "تحدّث في مربع الرسالة بدلًا من الكتابة",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "وضع الصوت",
|
||||
info: "يحوّل وضع الصوت كلامك إلى نص ويقرأ ردود الوكيل بصوت عالٍ.",
|
||||
enabled: "وضع الصوت",
|
||||
enabledHint: "تحدّث مع وكلائك دون استخدام يديك",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "مفتاح API",
|
||||
apiKeyHint: "يُحفظ على المضيف ولا يُعرض مرة أخرى.",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "محفوظ",
|
||||
missingKey: "مفتاح API مفقود",
|
||||
baseUrl: "عنوان URL الأساسي",
|
||||
baseUrlHint: "للخوادم المتوافقة مع OpenAI. اتركه فارغًا لاستخدام OpenAI.",
|
||||
},
|
||||
models: {
|
||||
loading: "جارٍ تحميل النماذج...",
|
||||
loadError: "تعذّر تحميل النماذج المحلية",
|
||||
none: "لا توجد نماذج متاحة",
|
||||
downloaded: "منزّل",
|
||||
inUse: "قيد الاستخدام",
|
||||
recommended: "موصى به",
|
||||
download: "تنزيل",
|
||||
downloadAction: "تنزيل {{label}}",
|
||||
downloading: "جارٍ التنزيل...",
|
||||
downloadingPercent: "جارٍ التنزيل {{percent}}%",
|
||||
extracting: "جارٍ الاستخراج...",
|
||||
delete: "حذف",
|
||||
deleteAction: "حذف {{label}}",
|
||||
deleteTitle: "حذف {{label}}؟",
|
||||
deleteMessage: "سيؤدي هذا إلى تحرير {{size}}. يمكنك تنزيله مرة أخرى لاحقًا.",
|
||||
languages: "لغات {{label}}",
|
||||
moreLanguages: "{{languages}} و{{count}} أخرى",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "عام",
|
||||
sending: "الإرسال",
|
||||
|
||||
@@ -2287,6 +2287,7 @@ export const en = {
|
||||
connections: "Connections",
|
||||
agents: "Agents",
|
||||
metadata: "Metadata",
|
||||
voice: "Voice",
|
||||
workspaces: "Workspaces",
|
||||
providers: "Providers",
|
||||
usage: "Usage",
|
||||
@@ -2371,6 +2372,69 @@ export const en = {
|
||||
docs: "Docs",
|
||||
saveError: "Unable to update metadata generation",
|
||||
},
|
||||
voice: {
|
||||
offline: "Connect to this host to configure voice.",
|
||||
updateHost: "Update the host to configure voice.",
|
||||
saveError: "Unable to update voice settings",
|
||||
provider: "Provider",
|
||||
providers: {
|
||||
local: "Local",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "Voice",
|
||||
save: "Save",
|
||||
retry: "Retry",
|
||||
cancel: "Cancel",
|
||||
model: "Model",
|
||||
language: "Language",
|
||||
languagePlaceholder: "Auto-detect",
|
||||
setBy: "Set by {{name}}",
|
||||
unknownProvider: "Update the app to configure {{provider}}",
|
||||
sites: {
|
||||
dictationStt: "Speech-to-text",
|
||||
voiceModeStt: "Listening",
|
||||
voiceModeTts: "Speaking",
|
||||
},
|
||||
dictation: {
|
||||
title: "Dictation",
|
||||
enabled: "Dictation",
|
||||
enabledHint: "Speak into the message box instead of typing",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "Voice mode",
|
||||
info: "Voice mode transcribes what you say and reads agent replies aloud.",
|
||||
enabled: "Voice mode",
|
||||
enabledHint: "Talk with your agents hands-free",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "API key",
|
||||
apiKeyHint: "Saved on the host and never shown again.",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "Saved",
|
||||
missingKey: "API key missing",
|
||||
baseUrl: "Base URL",
|
||||
baseUrlHint: "For OpenAI-compatible servers. Leave empty to use OpenAI.",
|
||||
},
|
||||
models: {
|
||||
loading: "Loading models...",
|
||||
loadError: "Couldn't load local models",
|
||||
none: "No models available",
|
||||
downloaded: "Downloaded",
|
||||
inUse: "In use",
|
||||
recommended: "Recommended",
|
||||
download: "Download",
|
||||
downloadAction: "Download {{label}}",
|
||||
downloading: "Downloading...",
|
||||
downloadingPercent: "Downloading {{percent}}%",
|
||||
extracting: "Extracting...",
|
||||
delete: "Delete",
|
||||
deleteAction: "Delete {{label}}",
|
||||
deleteTitle: "Delete {{label}}?",
|
||||
deleteMessage: "This frees {{size}}. You can download it again later.",
|
||||
languages: "{{label}} languages",
|
||||
moreLanguages: "{{languages}} and {{count}} more",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "General",
|
||||
sending: "Sending",
|
||||
|
||||
@@ -2272,6 +2272,7 @@ export const es: TranslationResources = {
|
||||
connections: "Conexiones",
|
||||
agents: "Agents",
|
||||
metadata: "Metadatos",
|
||||
voice: "Voz",
|
||||
workspaces: "Workspaces",
|
||||
providers: "Proveedores",
|
||||
usage: "Uso",
|
||||
@@ -2294,6 +2295,69 @@ export const es: TranslationResources = {
|
||||
docs: "Documentación",
|
||||
saveError: "No se pudo actualizar la generación de metadatos",
|
||||
},
|
||||
voice: {
|
||||
offline: "Conéctate a este host para configurar la voz.",
|
||||
updateHost: "Actualiza el host para configurar la voz.",
|
||||
saveError: "No se pudo actualizar la configuración de voz",
|
||||
provider: "Proveedor",
|
||||
providers: {
|
||||
local: "Local",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "Voz",
|
||||
save: "Guardar",
|
||||
retry: "Reintentar",
|
||||
cancel: "Cancelar",
|
||||
model: "Modelo",
|
||||
language: "Idioma",
|
||||
languagePlaceholder: "Detección automática",
|
||||
setBy: "Definido por {{name}}",
|
||||
unknownProvider: "Actualiza la app para configurar {{provider}}",
|
||||
sites: {
|
||||
dictationStt: "Voz a texto",
|
||||
voiceModeStt: "Escucha",
|
||||
voiceModeTts: "Habla",
|
||||
},
|
||||
dictation: {
|
||||
title: "Dictado",
|
||||
enabled: "Dictado",
|
||||
enabledHint: "Habla en el cuadro de mensaje en lugar de escribir",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "Modo de voz",
|
||||
info: "El modo de voz transcribe lo que dices y lee en voz alta las respuestas del agente.",
|
||||
enabled: "Modo de voz",
|
||||
enabledHint: "Habla con tus agentes sin usar las manos",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "Clave de API",
|
||||
apiKeyHint: "Se guarda en el host y no se vuelve a mostrar.",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "Guardada",
|
||||
missingKey: "Falta la clave de API",
|
||||
baseUrl: "URL base",
|
||||
baseUrlHint: "Para servidores compatibles con OpenAI. Déjala vacía para usar OpenAI.",
|
||||
},
|
||||
models: {
|
||||
loading: "Cargando modelos...",
|
||||
loadError: "No se pudieron cargar los modelos locales",
|
||||
none: "No hay modelos disponibles",
|
||||
downloaded: "Descargado",
|
||||
inUse: "En uso",
|
||||
recommended: "Recomendado",
|
||||
download: "Descargar",
|
||||
downloadAction: "Descargar {{label}}",
|
||||
downloading: "Descargando...",
|
||||
downloadingPercent: "Descargando {{percent}} %",
|
||||
extracting: "Extrayendo...",
|
||||
delete: "Eliminar",
|
||||
deleteAction: "Eliminar {{label}}",
|
||||
deleteTitle: "¿Eliminar {{label}}?",
|
||||
deleteMessage: "Esto libera {{size}}. Puedes volver a descargarlo más tarde.",
|
||||
languages: "Idiomas de {{label}}",
|
||||
moreLanguages: "{{languages}} y {{count}} más",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "General",
|
||||
sending: "Envío",
|
||||
|
||||
@@ -2272,6 +2272,7 @@ export const fr: TranslationResources = {
|
||||
connections: "Connexions",
|
||||
agents: "Agents",
|
||||
metadata: "Métadonnées",
|
||||
voice: "Voix",
|
||||
workspaces: "Espaces de travail",
|
||||
providers: "Fournisseurs",
|
||||
usage: "Utilisation",
|
||||
@@ -2294,6 +2295,69 @@ export const fr: TranslationResources = {
|
||||
docs: "Documentation",
|
||||
saveError: "Impossible de mettre à jour la génération de métadonnées",
|
||||
},
|
||||
voice: {
|
||||
offline: "Connectez-vous à cet hôte pour configurer la voix.",
|
||||
updateHost: "Mettez à jour l'hôte pour configurer la voix.",
|
||||
saveError: "Impossible de mettre à jour les réglages vocaux",
|
||||
provider: "Fournisseur",
|
||||
providers: {
|
||||
local: "Local",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "Voix",
|
||||
save: "Enregistrer",
|
||||
retry: "Réessayer",
|
||||
cancel: "Annuler",
|
||||
model: "Modèle",
|
||||
language: "Langue",
|
||||
languagePlaceholder: "Détection automatique",
|
||||
setBy: "Défini par {{name}}",
|
||||
unknownProvider: "Mettez à jour l'app pour configurer {{provider}}",
|
||||
sites: {
|
||||
dictationStt: "Transcription",
|
||||
voiceModeStt: "Écoute",
|
||||
voiceModeTts: "Parole",
|
||||
},
|
||||
dictation: {
|
||||
title: "Dictée",
|
||||
enabled: "Dictée",
|
||||
enabledHint: "Parlez dans la zone de message au lieu de taper",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "Mode vocal",
|
||||
info: "Le mode vocal transcrit ce que vous dites et lit à voix haute les réponses de l'agent.",
|
||||
enabled: "Mode vocal",
|
||||
enabledHint: "Parlez à vos agents sans les mains",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "Clé d'API",
|
||||
apiKeyHint: "Enregistrée sur l'hôte, jamais réaffichée.",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "Enregistrée",
|
||||
missingKey: "Clé d'API manquante",
|
||||
baseUrl: "URL de base",
|
||||
baseUrlHint: "Pour les serveurs compatibles OpenAI. Laissez vide pour utiliser OpenAI.",
|
||||
},
|
||||
models: {
|
||||
loading: "Chargement des modèles...",
|
||||
loadError: "Impossible de charger les modèles locaux",
|
||||
none: "Aucun modèle disponible",
|
||||
downloaded: "Téléchargé",
|
||||
inUse: "Utilisé",
|
||||
recommended: "Recommandé",
|
||||
download: "Télécharger",
|
||||
downloadAction: "Télécharger {{label}}",
|
||||
downloading: "Téléchargement...",
|
||||
downloadingPercent: "Téléchargement {{percent}} %",
|
||||
extracting: "Extraction...",
|
||||
delete: "Supprimer",
|
||||
deleteAction: "Supprimer {{label}}",
|
||||
deleteTitle: "Supprimer {{label}} ?",
|
||||
deleteMessage: "Cela libère {{size}}. Vous pourrez le retélécharger plus tard.",
|
||||
languages: "Langues de {{label}}",
|
||||
moreLanguages: "{{languages}} et {{count}} autres",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "Général",
|
||||
sending: "Envoi",
|
||||
|
||||
@@ -2245,6 +2245,7 @@ export const ja: TranslationResources = {
|
||||
connections: "接続",
|
||||
agents: "エージェント",
|
||||
metadata: "メタデータ",
|
||||
voice: "音声",
|
||||
workspaces: "ワークスペース",
|
||||
providers: "プロバイダー",
|
||||
usage: "使用状況",
|
||||
@@ -2267,6 +2268,69 @@ export const ja: TranslationResources = {
|
||||
docs: "ドキュメント",
|
||||
saveError: "メタデータ生成を更新できません",
|
||||
},
|
||||
voice: {
|
||||
offline: "音声を設定するには、このホストに接続してください。",
|
||||
updateHost: "音声を設定するには、ホストを更新してください。",
|
||||
saveError: "音声設定を更新できませんでした",
|
||||
provider: "プロバイダー",
|
||||
providers: {
|
||||
local: "ローカル",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "ボイス",
|
||||
save: "保存",
|
||||
retry: "再試行",
|
||||
cancel: "キャンセル",
|
||||
model: "モデル",
|
||||
language: "言語",
|
||||
languagePlaceholder: "自動検出",
|
||||
setBy: "{{name}} で設定",
|
||||
unknownProvider: "{{provider}} を設定するにはアプリを更新してください",
|
||||
sites: {
|
||||
dictationStt: "音声入力",
|
||||
voiceModeStt: "聞き取り",
|
||||
voiceModeTts: "読み上げ",
|
||||
},
|
||||
dictation: {
|
||||
title: "音声入力",
|
||||
enabled: "音声入力",
|
||||
enabledHint: "入力する代わりにメッセージ欄に話しかけます",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "ボイスモード",
|
||||
info: "ボイスモードは話した内容を文字起こしし、エージェントの返答を読み上げます。",
|
||||
enabled: "ボイスモード",
|
||||
enabledHint: "ハンズフリーでエージェントと会話します",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "API キー",
|
||||
apiKeyHint: "ホストに保存され、再表示されません。",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "保存済み",
|
||||
missingKey: "API キーがありません",
|
||||
baseUrl: "ベース URL",
|
||||
baseUrlHint: "OpenAI 互換サーバー用です。OpenAI を使う場合は空欄のままにします。",
|
||||
},
|
||||
models: {
|
||||
loading: "モデルを読み込み中...",
|
||||
loadError: "ローカルモデルを読み込めませんでした",
|
||||
none: "利用できるモデルがありません",
|
||||
downloaded: "ダウンロード済み",
|
||||
inUse: "使用中",
|
||||
recommended: "おすすめ",
|
||||
download: "ダウンロード",
|
||||
downloadAction: "{{label}} をダウンロード",
|
||||
downloading: "ダウンロード中...",
|
||||
downloadingPercent: "ダウンロード中 {{percent}}%",
|
||||
extracting: "展開中...",
|
||||
delete: "削除",
|
||||
deleteAction: "{{label}} を削除",
|
||||
deleteTitle: "{{label}} を削除しますか？",
|
||||
deleteMessage: "{{size}} が解放されます。後で再ダウンロードできます。",
|
||||
languages: "{{label}} の対応言語",
|
||||
moreLanguages: "{{languages}} ほか {{count}} 言語",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "一般",
|
||||
sending: "送信",
|
||||
|
||||
@@ -2234,6 +2234,7 @@ export const ko: TranslationResources = {
|
||||
connections: "연결",
|
||||
agents: "에이전트",
|
||||
metadata: "메타데이터",
|
||||
voice: "음성",
|
||||
workspaces: "워크스페이스",
|
||||
providers: "프로바이더",
|
||||
usage: "사용량",
|
||||
@@ -2256,6 +2257,69 @@ export const ko: TranslationResources = {
|
||||
docs: "문서",
|
||||
saveError: "메타데이터 생성을 업데이트할 수 없습니다",
|
||||
},
|
||||
voice: {
|
||||
offline: "음성을 설정하려면 이 호스트에 연결하세요.",
|
||||
updateHost: "음성을 설정하려면 호스트를 업데이트하세요.",
|
||||
saveError: "음성 설정을 업데이트할 수 없습니다",
|
||||
provider: "제공자",
|
||||
providers: {
|
||||
local: "로컬",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "목소리",
|
||||
save: "저장",
|
||||
retry: "다시 시도",
|
||||
cancel: "취소",
|
||||
model: "모델",
|
||||
language: "언어",
|
||||
languagePlaceholder: "자동 감지",
|
||||
setBy: "{{name}}에서 설정됨",
|
||||
unknownProvider: "{{provider}}을(를) 설정하려면 앱을 업데이트하세요",
|
||||
sites: {
|
||||
dictationStt: "음성 텍스트 변환",
|
||||
voiceModeStt: "듣기",
|
||||
voiceModeTts: "말하기",
|
||||
},
|
||||
dictation: {
|
||||
title: "받아쓰기",
|
||||
enabled: "받아쓰기",
|
||||
enabledHint: "입력하는 대신 메시지 상자에 말하세요",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "음성 모드",
|
||||
info: "음성 모드는 말한 내용을 텍스트로 옮기고 에이전트의 답변을 소리 내어 읽어 줍니다.",
|
||||
enabled: "음성 모드",
|
||||
enabledHint: "손을 쓰지 않고 에이전트와 대화하세요",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "API 키",
|
||||
apiKeyHint: "호스트에 저장되며 다시 표시되지 않습니다.",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "저장됨",
|
||||
missingKey: "API 키 없음",
|
||||
baseUrl: "기본 URL",
|
||||
baseUrlHint: "OpenAI 호환 서버용입니다. OpenAI를 사용하려면 비워 두세요.",
|
||||
},
|
||||
models: {
|
||||
loading: "모델을 불러오는 중...",
|
||||
loadError: "로컬 모델을 불러올 수 없습니다",
|
||||
none: "사용 가능한 모델 없음",
|
||||
downloaded: "다운로드됨",
|
||||
inUse: "사용 중",
|
||||
recommended: "추천",
|
||||
download: "다운로드",
|
||||
downloadAction: "{{label}} 다운로드",
|
||||
downloading: "다운로드 중...",
|
||||
downloadingPercent: "다운로드 중 {{percent}}%",
|
||||
extracting: "압축 해제 중...",
|
||||
delete: "삭제",
|
||||
deleteAction: "{{label}} 삭제",
|
||||
deleteTitle: "{{label}}을(를) 삭제할까요?",
|
||||
deleteMessage: "{{size}}가 확보됩니다. 나중에 다시 다운로드할 수 있습니다.",
|
||||
languages: "{{label}} 지원 언어",
|
||||
moreLanguages: "{{languages}} 외 {{count}}개",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "일반",
|
||||
sending: "전송",
|
||||
|
||||
@@ -2257,6 +2257,7 @@ export const ptBR: TranslationResources = {
|
||||
connections: "Conexões",
|
||||
agents: "Agentes",
|
||||
metadata: "Metadados",
|
||||
voice: "Voz",
|
||||
workspaces: "Workspaces",
|
||||
providers: "Provedores",
|
||||
usage: "Uso",
|
||||
@@ -2279,6 +2280,69 @@ export const ptBR: TranslationResources = {
|
||||
docs: "Documentação",
|
||||
saveError: "Não foi possível atualizar a geração de metadados",
|
||||
},
|
||||
voice: {
|
||||
offline: "Conecte-se a este host para configurar a voz.",
|
||||
updateHost: "Atualize o host para configurar a voz.",
|
||||
saveError: "Não foi possível atualizar as configurações de voz",
|
||||
provider: "Provedor",
|
||||
providers: {
|
||||
local: "Local",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "Voz",
|
||||
save: "Salvar",
|
||||
retry: "Tentar novamente",
|
||||
cancel: "Cancelar",
|
||||
model: "Modelo",
|
||||
language: "Idioma",
|
||||
languagePlaceholder: "Detecção automática",
|
||||
setBy: "Definido por {{name}}",
|
||||
unknownProvider: "Atualize o app para configurar {{provider}}",
|
||||
sites: {
|
||||
dictationStt: "Fala para texto",
|
||||
voiceModeStt: "Escuta",
|
||||
voiceModeTts: "Fala",
|
||||
},
|
||||
dictation: {
|
||||
title: "Ditado",
|
||||
enabled: "Ditado",
|
||||
enabledHint: "Fale na caixa de mensagem em vez de digitar",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "Modo de voz",
|
||||
info: "O modo de voz transcreve o que você diz e lê em voz alta as respostas do agente.",
|
||||
enabled: "Modo de voz",
|
||||
enabledHint: "Converse com seus agentes sem usar as mãos",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "Chave de API",
|
||||
apiKeyHint: "Salva no host e nunca mostrada novamente.",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "Salva",
|
||||
missingKey: "Chave de API ausente",
|
||||
baseUrl: "URL base",
|
||||
baseUrlHint: "Para servidores compatíveis com a OpenAI. Deixe vazio para usar a OpenAI.",
|
||||
},
|
||||
models: {
|
||||
loading: "Carregando modelos...",
|
||||
loadError: "Não foi possível carregar os modelos locais",
|
||||
none: "Nenhum modelo disponível",
|
||||
downloaded: "Baixado",
|
||||
inUse: "Em uso",
|
||||
recommended: "Recomendado",
|
||||
download: "Baixar",
|
||||
downloadAction: "Baixar {{label}}",
|
||||
downloading: "Baixando...",
|
||||
downloadingPercent: "Baixando {{percent}}%",
|
||||
extracting: "Extraindo...",
|
||||
delete: "Excluir",
|
||||
deleteAction: "Excluir {{label}}",
|
||||
deleteTitle: "Excluir {{label}}?",
|
||||
deleteMessage: "Isso libera {{size}}. Você pode baixá-lo de novo depois.",
|
||||
languages: "Idiomas do {{label}}",
|
||||
moreLanguages: "{{languages}} e mais {{count}}",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "Geral",
|
||||
sending: "Envio",
|
||||
|
||||
@@ -2257,6 +2257,7 @@ export const ru: TranslationResources = {
|
||||
connections: "Подключения",
|
||||
agents: "Агенты",
|
||||
metadata: "Метаданные",
|
||||
voice: "Голос",
|
||||
workspaces: "Рабочие пространства",
|
||||
providers: "Провайдеры",
|
||||
usage: "Использование",
|
||||
@@ -2280,6 +2281,70 @@ export const ru: TranslationResources = {
|
||||
docs: "Документация",
|
||||
saveError: "Не удалось обновить настройки генерации метаданных",
|
||||
},
|
||||
voice: {
|
||||
offline: "Подключитесь к этому хосту, чтобы настроить голос.",
|
||||
updateHost: "Обновите хост, чтобы настроить голос.",
|
||||
saveError: "Не удалось обновить голосовые настройки",
|
||||
provider: "Провайдер",
|
||||
providers: {
|
||||
local: "Локально",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "Голос",
|
||||
save: "Сохранить",
|
||||
retry: "Повторить",
|
||||
cancel: "Отмена",
|
||||
model: "Модель",
|
||||
language: "Язык",
|
||||
languagePlaceholder: "Автоопределение",
|
||||
setBy: "Задано через {{name}}",
|
||||
unknownProvider: "Обновите приложение, чтобы настроить {{provider}}",
|
||||
sites: {
|
||||
dictationStt: "Распознавание речи",
|
||||
voiceModeStt: "Прослушивание",
|
||||
voiceModeTts: "Озвучивание",
|
||||
},
|
||||
dictation: {
|
||||
title: "Диктовка",
|
||||
enabled: "Диктовка",
|
||||
enabledHint: "Говорите в поле сообщения вместо набора текста",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "Голосовой режим",
|
||||
info: "Голосовой режим расшифровывает вашу речь и зачитывает ответы агента вслух.",
|
||||
enabled: "Голосовой режим",
|
||||
enabledHint: "Общайтесь с агентами без рук",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "API-ключ",
|
||||
apiKeyHint: "Хранится на хосте и больше не показывается.",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "Сохранён",
|
||||
missingKey: "Нет API-ключа",
|
||||
baseUrl: "Базовый URL",
|
||||
baseUrlHint:
|
||||
"Для серверов, совместимых с OpenAI. Оставьте пустым, чтобы использовать OpenAI.",
|
||||
},
|
||||
models: {
|
||||
loading: "Загрузка моделей...",
|
||||
loadError: "Не удалось загрузить локальные модели",
|
||||
none: "Нет доступных моделей",
|
||||
downloaded: "Скачана",
|
||||
inUse: "Используется",
|
||||
recommended: "Рекомендуется",
|
||||
download: "Скачать",
|
||||
downloadAction: "Скачать {{label}}",
|
||||
downloading: "Скачивание...",
|
||||
downloadingPercent: "Скачивание {{percent}}%",
|
||||
extracting: "Распаковка...",
|
||||
delete: "Удалить",
|
||||
deleteAction: "Удалить {{label}}",
|
||||
deleteTitle: "Удалить {{label}}?",
|
||||
deleteMessage: "Освободится {{size}}. Позже её можно скачать снова.",
|
||||
languages: "Языки {{label}}",
|
||||
moreLanguages: "{{languages}} и ещё {{count}}",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "Основные",
|
||||
sending: "Отправка",
|
||||
|
||||
@@ -2202,6 +2202,7 @@ export const zhCN: TranslationResources = {
|
||||
connections: "连接",
|
||||
agents: "Agents",
|
||||
metadata: "元数据",
|
||||
voice: "语音",
|
||||
workspaces: "Workspaces",
|
||||
providers: "Providers",
|
||||
usage: "使用情况",
|
||||
@@ -2223,6 +2224,69 @@ export const zhCN: TranslationResources = {
|
||||
docs: "文档",
|
||||
saveError: "无法更新元数据生成设置",
|
||||
},
|
||||
voice: {
|
||||
offline: "连接到此主机以配置语音。",
|
||||
updateHost: "请更新主机以配置语音。",
|
||||
saveError: "无法更新语音设置",
|
||||
provider: "提供方",
|
||||
providers: {
|
||||
local: "本地",
|
||||
openai: "OpenAI",
|
||||
},
|
||||
voice: "音色",
|
||||
save: "保存",
|
||||
retry: "重试",
|
||||
cancel: "取消",
|
||||
model: "模型",
|
||||
language: "语言",
|
||||
languagePlaceholder: "自动检测",
|
||||
setBy: "由 {{name}} 设置",
|
||||
unknownProvider: "更新应用以配置 {{provider}}",
|
||||
sites: {
|
||||
dictationStt: "语音转文字",
|
||||
voiceModeStt: "聆听",
|
||||
voiceModeTts: "朗读",
|
||||
},
|
||||
dictation: {
|
||||
title: "听写",
|
||||
enabled: "听写",
|
||||
enabledHint: "对着消息框说话，无需打字",
|
||||
},
|
||||
voiceMode: {
|
||||
title: "语音模式",
|
||||
info: "语音模式会转写你说的话，并朗读智能体的回复。",
|
||||
enabled: "语音模式",
|
||||
enabledHint: "免手动与智能体对话",
|
||||
},
|
||||
openai: {
|
||||
apiKey: "API 密钥",
|
||||
apiKeyHint: "保存在主机上，不会再次显示。",
|
||||
keyPlaceholder: "sk-...",
|
||||
savedKeyPlaceholder: "已保存",
|
||||
missingKey: "缺少 API 密钥",
|
||||
baseUrl: "基础 URL",
|
||||
baseUrlHint: "用于兼容 OpenAI 的服务器。留空则使用 OpenAI。",
|
||||
},
|
||||
models: {
|
||||
loading: "正在加载模型...",
|
||||
loadError: "无法加载本地模型",
|
||||
none: "没有可用的模型",
|
||||
downloaded: "已下载",
|
||||
inUse: "使用中",
|
||||
recommended: "推荐",
|
||||
download: "下载",
|
||||
downloadAction: "下载 {{label}}",
|
||||
downloading: "正在下载...",
|
||||
downloadingPercent: "正在下载 {{percent}}%",
|
||||
extracting: "正在解压...",
|
||||
delete: "删除",
|
||||
deleteAction: "删除 {{label}}",
|
||||
deleteTitle: "删除 {{label}}？",
|
||||
deleteMessage: "这将释放 {{size}}。之后可以重新下载。",
|
||||
languages: "{{label}} 支持的语言",
|
||||
moreLanguages: "{{languages}} 等另外 {{count}} 种",
|
||||
},
|
||||
},
|
||||
general: {
|
||||
title: "通用",
|
||||
sending: "发送",
|
||||
|
||||
@@ -37,6 +37,7 @@ import {
|
||||
Files,
|
||||
Smartphone,
|
||||
Sparkles,
|
||||
Mic,
|
||||
Blocks,
|
||||
Globe,
|
||||
PanelLeft,
|
||||
@@ -118,6 +119,7 @@ import { PluginSettingsContent } from "@/plugins/settings";
|
||||
import { useInstalledPlugins } from "@/plugins/registry";
|
||||
import { HostPluginsPage } from "@/screens/settings/plugins-page";
|
||||
import { MetadataGenerationPage } from "@/screens/settings/metadata-generation-page";
|
||||
import { HostVoicePage } from "@/screens/settings/voice/voice-page";
|
||||
import ProjectsScreen from "@/screens/projects-screen";
|
||||
import ProjectSettingsScreen from "@/screens/project-settings-screen";
|
||||
import { SETTINGS_DESKTOP_SIDEBAR_WIDTH, useIsCompactFormFactor } from "@/constants/layout";
|
||||
@@ -240,6 +242,7 @@ const HOST_SECTION_ITEMS: HostSectionItem[] = [
|
||||
{ id: "pair-device", labelKey: "openProject.tiles.pairDevice.title", icon: Smartphone },
|
||||
{ id: "agents", labelKey: "settings.hostSections.agents", icon: Bot },
|
||||
{ id: "metadata", labelKey: "settings.hostSections.metadata", icon: Sparkles },
|
||||
{ id: "voice", labelKey: "settings.hostSections.voice", icon: Mic },
|
||||
{ id: "workspaces", labelKey: "settings.hostSections.workspaces", icon: FolderGit2 },
|
||||
{ id: "providers", labelKey: "settings.hostSections.providers", icon: Boxes },
|
||||
{ id: "usage", labelKey: "settings.hostSections.usage", icon: Gauge },
|
||||
@@ -262,6 +265,8 @@ function renderHostSettingsContent(
|
||||
return <HostAgentsPage serverId={view.serverId} />;
|
||||
case "metadata":
|
||||
return <MetadataGenerationPage serverId={view.serverId} />;
|
||||
case "voice":
|
||||
return <HostVoicePage serverId={view.serverId} />;
|
||||
case "workspaces":
|
||||
return <HostWorkspacesPage serverId={view.serverId} />;
|
||||
case "providers":
|
||||
|
||||
@@ -0,0 +1,174 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { SpeechSettingsPatch } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
import { openSpeechProviderForm, type SpeechProviderFormSnapshot } from "./speech-provider-form";
|
||||
|
||||
function open(overrides: Partial<SpeechProviderFormSnapshot> = {}) {
|
||||
return openSpeechProviderForm({
|
||||
siteId: "voiceMode.stt",
|
||||
site: { provider: "local" },
|
||||
envOverrides: {},
|
||||
...overrides,
|
||||
});
|
||||
}
|
||||
|
||||
async function savedPatch(
|
||||
model: ReturnType<typeof open>,
|
||||
): Promise<SpeechSettingsPatch | undefined> {
|
||||
let sent: SpeechSettingsPatch | undefined;
|
||||
const result = await model.save(async (patch) => {
|
||||
sent = patch;
|
||||
});
|
||||
expect(result).toEqual({ status: "saved" });
|
||||
return sent;
|
||||
}
|
||||
|
||||
describe("speech provider form", () => {
|
||||
it("edits and clears local STT language from the saved snapshot", async () => {
|
||||
const form = open({ site: { provider: "local", language: "fr" } });
|
||||
expect(form.getState()).toMatchObject({ language: "fr", canSave: false });
|
||||
form.setLanguage(" de ");
|
||||
expect(await savedPatch(form)).toEqual({
|
||||
voiceMode: { stt: { provider: "local", language: "de" } },
|
||||
});
|
||||
const clear = open({ site: { provider: "local", language: "fr" } });
|
||||
clear.setLanguage("");
|
||||
expect(await savedPatch(clear)).toEqual({
|
||||
voiceMode: { stt: { provider: "local", language: null } },
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps STT language when switching either provider", async () => {
|
||||
for (const provider of ["local", "openai"] as const) {
|
||||
const form = open({
|
||||
site:
|
||||
provider === "local"
|
||||
? { provider, language: "fr" }
|
||||
: { provider, hasApiKey: false, language: "fr" },
|
||||
});
|
||||
form.setProvider(provider === "local" ? "openai" : "local");
|
||||
expect(await savedPatch(form)).toEqual({
|
||||
voiceMode: { stt: { provider: provider === "local" ? "openai" : "local", language: "fr" } },
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it("keeps env-controlled language read-only after switching providers", async () => {
|
||||
const form = open({
|
||||
site: { provider: "local", language: "fr" },
|
||||
envOverrides: { "features.voiceMode.stt.language": "PASEO_VOICE_LANGUAGE" },
|
||||
});
|
||||
form.setProvider("openai");
|
||||
expect(form.getState().fields.language).toEqual({
|
||||
envVar: "PASEO_VOICE_LANGUAGE",
|
||||
readOnly: true,
|
||||
});
|
||||
expect(await savedPatch(form)).toEqual({ voiceMode: { stt: { provider: "openai" } } });
|
||||
});
|
||||
|
||||
it("switches a site to OpenAI with a key and base URL", async () => {
|
||||
const form = open();
|
||||
expect(form.getState().canSave).toBe(false);
|
||||
|
||||
form.setProvider("openai");
|
||||
form.setApiKey(" sk-test ");
|
||||
form.setBaseUrl("http://127.0.0.1:9999/v1");
|
||||
|
||||
expect(await savedPatch(form)).toEqual({
|
||||
voiceMode: {
|
||||
stt: { provider: "openai", apiKey: "sk-test", baseUrl: "http://127.0.0.1:9999/v1" },
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it("re-saves OpenAI settings without touching the saved key", async () => {
|
||||
const form = open({
|
||||
siteId: "dictation.stt",
|
||||
site: { provider: "openai", hasApiKey: true, baseUrl: "https://a.example/v1" },
|
||||
});
|
||||
expect(form.getState()).toMatchObject({ apiKey: "", hasSavedApiKey: true, canSave: false });
|
||||
|
||||
form.setOpenAiModel("gpt-4o-transcribe");
|
||||
form.setBaseUrl("");
|
||||
|
||||
expect(await savedPatch(form)).toEqual({
|
||||
dictation: { stt: { provider: "openai", model: "gpt-4o-transcribe", baseUrl: null } },
|
||||
});
|
||||
});
|
||||
|
||||
it("selects a local model", async () => {
|
||||
const form = open({
|
||||
siteId: "dictation.stt",
|
||||
site: { provider: "local", model: "whisper-tiny" },
|
||||
});
|
||||
form.setLocalModel("whisper-tiny");
|
||||
expect(form.getState().canSave).toBe(false);
|
||||
|
||||
form.setLocalModel("whisper-base");
|
||||
|
||||
expect(await savedPatch(form)).toEqual({
|
||||
dictation: { stt: { provider: "local", model: "whisper-base" } },
|
||||
});
|
||||
});
|
||||
|
||||
it("locks fields an env var decides and keeps fallback fields editable", async () => {
|
||||
const form = open({
|
||||
site: { provider: "openai", hasApiKey: false },
|
||||
envOverrides: {
|
||||
"features.voiceMode.stt.provider": "PASEO_VOICE_STT_PROVIDER",
|
||||
"features.voiceMode.stt.model": "STT_MODEL",
|
||||
"features.voiceMode.stt.apiKey": "OPENAI_API_KEY",
|
||||
},
|
||||
});
|
||||
expect(form.getState().fields).toMatchObject({
|
||||
provider: { envVar: "PASEO_VOICE_STT_PROVIDER", readOnly: true },
|
||||
model: { envVar: "STT_MODEL", readOnly: true },
|
||||
apiKey: { envVar: "OPENAI_API_KEY", readOnly: false },
|
||||
baseUrl: { envVar: null, readOnly: false },
|
||||
});
|
||||
|
||||
form.setProvider("local");
|
||||
form.setOpenAiModel("gpt-4o-transcribe");
|
||||
form.setApiKey("sk-saved");
|
||||
|
||||
expect(form.getState().provider).toBe("openai");
|
||||
expect(await savedPatch(form)).toEqual({
|
||||
voiceMode: { stt: { provider: "openai", apiKey: "sk-saved" } },
|
||||
});
|
||||
});
|
||||
|
||||
it("does not apply the saved provider's env vars to another provider", () => {
|
||||
const form = open({
|
||||
envOverrides: { "features.voiceMode.stt.model": "PASEO_VOICE_LOCAL_STT_MODEL" },
|
||||
});
|
||||
expect(form.getState().fields.model.readOnly).toBe(true);
|
||||
|
||||
form.setProvider("openai");
|
||||
|
||||
expect(form.getState().fields.model).toEqual({ envVar: null, readOnly: false });
|
||||
});
|
||||
|
||||
it("keeps the input and allows a retry after a failed save", async () => {
|
||||
const form = open({ siteId: "voiceMode.tts" });
|
||||
form.setProvider("openai");
|
||||
form.setApiKey("sk-test");
|
||||
form.setVoice("nova");
|
||||
const failure = new Error("disk full");
|
||||
|
||||
const result = await form.save(async () => {
|
||||
expect(form.getState()).toMatchObject({ saving: true, canSave: false });
|
||||
throw failure;
|
||||
});
|
||||
|
||||
expect(result).toEqual({ status: "failed", error: failure });
|
||||
expect(form.getState()).toMatchObject({
|
||||
provider: "openai",
|
||||
apiKey: "sk-test",
|
||||
voice: "nova",
|
||||
saving: false,
|
||||
canSave: true,
|
||||
});
|
||||
expect(await savedPatch(form)).toEqual({
|
||||
voiceMode: { tts: { provider: "openai", apiKey: "sk-test", voice: "nova" } },
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,240 @@
|
||||
import type { SpeechSettingsPatch } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
import {
|
||||
SPEECH_SITE_KIND,
|
||||
buildSpeechSitePatch,
|
||||
resolveOpenAiModel,
|
||||
resolveOpenAiVoice,
|
||||
speechSiteFieldPath,
|
||||
type SpeechModelKind,
|
||||
type SpeechProvider,
|
||||
type SpeechSite,
|
||||
type SpeechSiteId,
|
||||
type SpeechSitePatch,
|
||||
} from "./voice-settings";
|
||||
|
||||
// The provider sheet's form: one speech site, its provider and that provider's fields.
|
||||
// Plain TypeScript so the save rules are testable without rendering (docs/forms.md).
|
||||
|
||||
export interface SpeechProviderFormSnapshot {
|
||||
siteId: SpeechSiteId;
|
||||
site: Extract<SpeechSite, { provider: SpeechProvider }>;
|
||||
envOverrides: Record<string, string>;
|
||||
}
|
||||
|
||||
export type SpeechProviderField =
|
||||
| "provider"
|
||||
| "model"
|
||||
| "apiKey"
|
||||
| "baseUrl"
|
||||
| "language"
|
||||
| "voice";
|
||||
|
||||
// An env var set for one of these is only a fallback: a value saved in the sheet wins,
|
||||
// so the field stays editable. Every other field is decided by its env var.
|
||||
const FALLBACK_FIELDS: ReadonlySet<SpeechProviderField> = new Set(["apiKey", "baseUrl"]);
|
||||
|
||||
export interface SpeechProviderFieldSource {
|
||||
/** The env var that decides this field's effective value, when one does. */
|
||||
envVar: string | null;
|
||||
readOnly: boolean;
|
||||
}
|
||||
|
||||
export interface SpeechProviderFormState {
|
||||
kind: SpeechModelKind;
|
||||
provider: SpeechProvider;
|
||||
/** The local model the user picked in this sheet; null keeps the saved or default model. */
|
||||
localModel: string | null;
|
||||
savedLocalModel: string | undefined;
|
||||
apiKey: string;
|
||||
hasSavedApiKey: boolean;
|
||||
baseUrl: string;
|
||||
openAiModel: string;
|
||||
language: string;
|
||||
voice: string;
|
||||
fields: Record<SpeechProviderField, SpeechProviderFieldSource>;
|
||||
saving: boolean;
|
||||
canSave: boolean;
|
||||
}
|
||||
|
||||
export type SpeechProviderSaveResult =
|
||||
| { status: "saved" }
|
||||
| { status: "skipped" }
|
||||
| { status: "failed"; error: unknown };
|
||||
|
||||
export interface SpeechProviderFormModel {
|
||||
getState: () => SpeechProviderFormState;
|
||||
subscribe: (listener: () => void) => () => void;
|
||||
close: () => void;
|
||||
setProvider: (provider: SpeechProvider) => void;
|
||||
setLocalModel: (modelId: string) => void;
|
||||
setApiKey: (value: string) => void;
|
||||
setBaseUrl: (value: string) => void;
|
||||
setOpenAiModel: (value: string) => void;
|
||||
setLanguage: (value: string) => void;
|
||||
setVoice: (value: string) => void;
|
||||
/** Sends the changed fields through `save`; the inputs survive a failed save. */
|
||||
save: (
|
||||
save: (patch: SpeechSettingsPatch) => Promise<unknown>,
|
||||
) => Promise<SpeechProviderSaveResult>;
|
||||
}
|
||||
|
||||
interface OpenAiValues {
|
||||
baseUrl: string;
|
||||
openAiModel: string;
|
||||
voice: string;
|
||||
}
|
||||
|
||||
export function openSpeechProviderForm(
|
||||
snapshot: SpeechProviderFormSnapshot,
|
||||
): SpeechProviderFormModel {
|
||||
const { siteId, site, envOverrides } = snapshot;
|
||||
const kind = SPEECH_SITE_KIND[siteId];
|
||||
const savedOpenAi = site.provider === "openai" ? site : undefined;
|
||||
const initialOpenAi: OpenAiValues = {
|
||||
baseUrl: savedOpenAi?.baseUrl ?? "",
|
||||
openAiModel: resolveOpenAiModel(kind, savedOpenAi?.model),
|
||||
voice: resolveOpenAiVoice(savedOpenAi?.voice),
|
||||
};
|
||||
const initialLanguage = site.language ?? "";
|
||||
// The daemon reports env vars for the provider that is in effect, so they only describe the
|
||||
// provider-specific fields while that provider is selected. Language applies to either STT provider.
|
||||
const savedFields = buildFieldSources(siteId, envOverrides);
|
||||
const switchedFields = {
|
||||
...buildFieldSources(siteId, {}),
|
||||
provider: savedFields.provider,
|
||||
language: savedFields.language,
|
||||
};
|
||||
const listeners = new Set<() => void>();
|
||||
let closed = false;
|
||||
|
||||
let state: SpeechProviderFormState = derive({
|
||||
kind,
|
||||
provider: site.provider,
|
||||
localModel: null,
|
||||
savedLocalModel: site.provider === "local" ? site.model : undefined,
|
||||
apiKey: "",
|
||||
hasSavedApiKey: savedOpenAi?.hasApiKey ?? false,
|
||||
...initialOpenAi,
|
||||
language: initialLanguage,
|
||||
fields: savedFields,
|
||||
saving: false,
|
||||
canSave: false,
|
||||
});
|
||||
|
||||
function derive(next: SpeechProviderFormState): SpeechProviderFormState {
|
||||
return {
|
||||
...next,
|
||||
fields: next.provider === site.provider ? savedFields : switchedFields,
|
||||
canSave: !next.saving && isChanged(next),
|
||||
};
|
||||
}
|
||||
|
||||
function isChanged(next: SpeechProviderFormState): boolean {
|
||||
if (next.provider !== site.provider) return true;
|
||||
if (kind === "stt" && next.language.trim() !== initialLanguage) return true;
|
||||
if (next.provider === "local") {
|
||||
return next.localModel !== null && next.localModel !== next.savedLocalModel;
|
||||
}
|
||||
return next.apiKey.trim().length > 0 || changedOpenAiFields(next).length > 0;
|
||||
}
|
||||
|
||||
function changedOpenAiFields(next: SpeechProviderFormState): (keyof OpenAiValues)[] {
|
||||
const keys: (keyof OpenAiValues)[] =
|
||||
kind === "stt" ? ["baseUrl", "openAiModel"] : ["baseUrl", "openAiModel", "voice"];
|
||||
return keys.filter((key) => next[key].trim() !== initialOpenAi[key]);
|
||||
}
|
||||
|
||||
function buildSitePatch(next: SpeechProviderFormState): SpeechSitePatch {
|
||||
const language = next.language.trim();
|
||||
const languagePatch: Record<string, string | null> =
|
||||
kind === "stt" &&
|
||||
!next.fields.language.readOnly &&
|
||||
(language !== initialLanguage || (next.provider !== site.provider && language.length > 0))
|
||||
? { language: language || null }
|
||||
: {};
|
||||
if (next.provider === "local") {
|
||||
return {
|
||||
provider: "local",
|
||||
...languagePatch,
|
||||
...(next.localModel !== null && !next.fields.model.readOnly
|
||||
? { model: next.localModel }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
const patch: Record<string, string | null> = { provider: "openai", ...languagePatch };
|
||||
const apiKey = next.apiKey.trim();
|
||||
if (apiKey) patch.apiKey = apiKey;
|
||||
for (const key of changedOpenAiFields(next)) {
|
||||
const field = key === "openAiModel" ? "model" : key;
|
||||
if (next.fields[field].readOnly) continue;
|
||||
patch[field] = next[key].trim() || null;
|
||||
}
|
||||
return patch as SpeechSitePatch;
|
||||
}
|
||||
|
||||
function publish(next: SpeechProviderFormState): void {
|
||||
if (closed) return;
|
||||
state = derive(next);
|
||||
for (const listener of listeners) listener();
|
||||
}
|
||||
|
||||
function edit(change: Partial<SpeechProviderFormState>): void {
|
||||
if (state.saving) return;
|
||||
publish({ ...state, ...change });
|
||||
}
|
||||
|
||||
return {
|
||||
getState: () => state,
|
||||
subscribe(listener) {
|
||||
listeners.add(listener);
|
||||
return () => {
|
||||
listeners.delete(listener);
|
||||
};
|
||||
},
|
||||
close() {
|
||||
closed = true;
|
||||
listeners.clear();
|
||||
},
|
||||
setProvider: (provider) => {
|
||||
if (!state.fields.provider.readOnly) edit({ provider });
|
||||
},
|
||||
setLocalModel: (localModel) => {
|
||||
if (!state.fields.model.readOnly) edit({ localModel });
|
||||
},
|
||||
setApiKey: (apiKey) => edit({ apiKey }),
|
||||
setBaseUrl: (baseUrl) => edit({ baseUrl }),
|
||||
setOpenAiModel: (openAiModel) => edit({ openAiModel }),
|
||||
setLanguage: (language) => edit({ language }),
|
||||
setVoice: (voice) => edit({ voice }),
|
||||
async save(save) {
|
||||
if (!state.canSave) return { status: "skipped" };
|
||||
const patch = buildSpeechSitePatch(siteId, buildSitePatch(state));
|
||||
publish({ ...state, saving: true });
|
||||
try {
|
||||
await save(patch);
|
||||
return { status: "saved" };
|
||||
} catch (error) {
|
||||
publish({ ...state, saving: false });
|
||||
return { status: "failed", error };
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function buildFieldSources(
|
||||
siteId: SpeechSiteId,
|
||||
envOverrides: Record<string, string>,
|
||||
): Record<SpeechProviderField, SpeechProviderFieldSource> {
|
||||
const source = (field: SpeechProviderField): SpeechProviderFieldSource => {
|
||||
const envVar = envOverrides[speechSiteFieldPath(siteId, field)] ?? null;
|
||||
return { envVar, readOnly: envVar !== null && !FALLBACK_FIELDS.has(field) };
|
||||
};
|
||||
return {
|
||||
provider: source("provider"),
|
||||
model: source("model"),
|
||||
apiKey: source("apiKey"),
|
||||
baseUrl: source("baseUrl"),
|
||||
language: source("language"),
|
||||
voice: source("voice"),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,585 @@
|
||||
import {
|
||||
useCallback,
|
||||
useEffect,
|
||||
useMemo,
|
||||
useState,
|
||||
useSyncExternalStore,
|
||||
type ReactNode,
|
||||
} from "react";
|
||||
import { Text, View, type PressableStateCallbackType } from "react-native";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import type { TFunction } from "i18next";
|
||||
import { StyleSheet, withUnistyles } from "react-native-unistyles";
|
||||
import { Check, Download, Languages, Trash2 } from "lucide-react-native";
|
||||
import type { LocalSpeechModel, SpeechSettingsPatch } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
import { AdaptiveModalSheet, type SheetHeader } from "@/components/adaptive-modal-sheet";
|
||||
import {
|
||||
SettingsCard,
|
||||
SettingsInfoTip,
|
||||
SettingsInput,
|
||||
SettingsRow,
|
||||
SettingsSelect,
|
||||
} from "@/components/settings";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { buttonControlHeight } from "@/components/ui/control-geometry";
|
||||
import { useContainedControlSize } from "@/components/ui/control-size";
|
||||
import { foregroundIconColorMapping } from "@/components/ui/icon-color";
|
||||
import { PressHighlight } from "@/components/ui/press-highlight";
|
||||
import { LoadingSpinner } from "@/components/ui/loading-spinner";
|
||||
import { SegmentedControl } from "@/components/ui/segmented-control";
|
||||
import { StatusBadge } from "@/components/ui/status-badge";
|
||||
import { useIsCompactFormFactor } from "@/constants/layout";
|
||||
import { useToast } from "@/contexts/toast-api-context";
|
||||
import { ICON_SIZE } from "@/styles/theme";
|
||||
import { confirmDialog } from "@/utils/confirm-dialog";
|
||||
import type { LocalSpeechModels } from "./use-local-speech-models";
|
||||
import {
|
||||
openSpeechProviderForm,
|
||||
type SpeechProviderFieldSource,
|
||||
type SpeechProviderFormModel,
|
||||
type SpeechProviderFormSnapshot,
|
||||
type SpeechProviderFormState,
|
||||
} from "./speech-provider-form";
|
||||
import {
|
||||
SPEECH_PROVIDERS,
|
||||
buildOpenAiModelOptions,
|
||||
buildOpenAiVoiceOptions,
|
||||
formatLanguages,
|
||||
formatModelSize,
|
||||
getLocalModelState,
|
||||
resolveLocalModel,
|
||||
type LocalModelState,
|
||||
type SpeechProvider,
|
||||
} from "./voice-settings";
|
||||
|
||||
const ThemedCheck = withUnistyles(Check);
|
||||
|
||||
export interface SpeechProviderSheetProps extends SpeechProviderFormSnapshot {
|
||||
title: string;
|
||||
localModels: LocalSpeechModels;
|
||||
visible: boolean;
|
||||
onClose: () => void;
|
||||
onDismiss: () => void;
|
||||
save: (patch: SpeechSettingsPatch) => Promise<unknown>;
|
||||
}
|
||||
|
||||
function errorMessage(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
|
||||
// The form opens from the props of the first render; later config changes do not reset input.
|
||||
function useSpeechProviderForm(snapshot: SpeechProviderFormSnapshot): SpeechProviderFormModel {
|
||||
const [model] = useState(() => openSpeechProviderForm(snapshot));
|
||||
useEffect(() => () => model.close(), [model]);
|
||||
return model;
|
||||
}
|
||||
|
||||
/** One speech site's provider and that provider's settings. Mount it fresh for each open. */
|
||||
export function SpeechProviderSheet({
|
||||
title,
|
||||
siteId,
|
||||
site,
|
||||
envOverrides,
|
||||
localModels,
|
||||
visible,
|
||||
onClose,
|
||||
onDismiss,
|
||||
save,
|
||||
}: SpeechProviderSheetProps) {
|
||||
const { t } = useTranslation();
|
||||
const toast = useToast();
|
||||
const model = useSpeechProviderForm({ siteId, site, envOverrides });
|
||||
const state = useSyncExternalStore(model.subscribe, model.getState, model.getState);
|
||||
|
||||
const submit = useCallback(() => {
|
||||
void model.save(save).then((result) => {
|
||||
if (result.status === "saved") onClose();
|
||||
if (result.status === "failed") {
|
||||
toast.error(`${t("settings.voice.saveError")}: ${errorMessage(result.error)}`);
|
||||
}
|
||||
return undefined;
|
||||
});
|
||||
}, [model, onClose, save, t, toast]);
|
||||
|
||||
const header = useMemo<SheetHeader>(() => ({ title }), [title]);
|
||||
const footer = useMemo(
|
||||
() => (
|
||||
<View style={styles.footer}>
|
||||
<Button
|
||||
style={styles.footerButton}
|
||||
variant="secondary"
|
||||
onPress={onClose}
|
||||
disabled={state.saving}
|
||||
>
|
||||
{t("settings.voice.cancel")}
|
||||
</Button>
|
||||
<Button
|
||||
style={styles.footerButton}
|
||||
variant="default"
|
||||
onPress={submit}
|
||||
disabled={!state.canSave}
|
||||
loading={state.saving}
|
||||
testID="voice-provider-save"
|
||||
>
|
||||
{t("settings.voice.save")}
|
||||
</Button>
|
||||
</View>
|
||||
),
|
||||
[onClose, state.canSave, state.saving, submit, t],
|
||||
);
|
||||
|
||||
return (
|
||||
<AdaptiveModalSheet
|
||||
header={header}
|
||||
visible={visible}
|
||||
onClose={onClose}
|
||||
onDismiss={onDismiss}
|
||||
footer={footer}
|
||||
testID="voice-provider-sheet"
|
||||
>
|
||||
<SettingsCard>
|
||||
<ProviderRow state={state} model={model} />
|
||||
</SettingsCard>
|
||||
{state.provider === "local" ? (
|
||||
<>
|
||||
{state.kind === "stt" ? (
|
||||
<SettingsCard>
|
||||
<LanguageField state={state} model={model} />
|
||||
</SettingsCard>
|
||||
) : null}
|
||||
<LocalModelList state={state} model={model} localModels={localModels} />
|
||||
</>
|
||||
) : (
|
||||
<OpenAiFields state={state} model={model} />
|
||||
)}
|
||||
</AdaptiveModalSheet>
|
||||
);
|
||||
}
|
||||
|
||||
interface FormProps {
|
||||
state: SpeechProviderFormState;
|
||||
model: SpeechProviderFormModel;
|
||||
}
|
||||
|
||||
function setByHint(t: TFunction, source: SpeechProviderFieldSource): string | undefined {
|
||||
return source.envVar ? t("settings.voice.setBy", { name: source.envVar }) : undefined;
|
||||
}
|
||||
|
||||
function ProviderRow({ state, model }: FormProps) {
|
||||
const { t } = useTranslation();
|
||||
const source = state.fields.provider;
|
||||
const options = useMemo(
|
||||
() =>
|
||||
SPEECH_PROVIDERS.map((provider) => ({
|
||||
value: provider,
|
||||
label: t(`settings.voice.providers.${provider}`),
|
||||
disabled: source.readOnly,
|
||||
})),
|
||||
[source.readOnly, t],
|
||||
);
|
||||
return (
|
||||
<SettingsRow label={t("settings.voice.provider")} hint={setByHint(t, source)}>
|
||||
<SegmentedControl<SpeechProvider>
|
||||
options={options}
|
||||
value={state.provider}
|
||||
onValueChange={model.setProvider}
|
||||
testID="voice-provider-picker"
|
||||
/>
|
||||
</SettingsRow>
|
||||
);
|
||||
}
|
||||
|
||||
function LanguageField({ state, model }: FormProps) {
|
||||
const { t } = useTranslation();
|
||||
return (
|
||||
<SettingsInput
|
||||
label={t("settings.voice.language")}
|
||||
hint={setByHint(t, state.fields.language)}
|
||||
initialValue={state.language}
|
||||
onChangeText={model.setLanguage}
|
||||
placeholder={t("settings.voice.languagePlaceholder")}
|
||||
disabled={state.fields.language.readOnly}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// OpenAI
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function OpenAiFields({ state, model }: FormProps) {
|
||||
const { t } = useTranslation();
|
||||
const modelOptions = useMemo(
|
||||
() => buildOpenAiModelOptions(state.kind, state.openAiModel),
|
||||
[state.kind, state.openAiModel],
|
||||
);
|
||||
const voiceOptions = useMemo(() => buildOpenAiVoiceOptions(state.voice), [state.voice]);
|
||||
const apiKeyInfo = useMemo(
|
||||
() => (
|
||||
<SettingsInfoTip
|
||||
title={t("settings.voice.openai.apiKey")}
|
||||
info={t("settings.voice.openai.apiKeyHint")}
|
||||
/>
|
||||
),
|
||||
[t],
|
||||
);
|
||||
const baseUrlInfo = useMemo(
|
||||
() => (
|
||||
<SettingsInfoTip
|
||||
title={t("settings.voice.openai.baseUrl")}
|
||||
info={t("settings.voice.openai.baseUrlHint")}
|
||||
/>
|
||||
),
|
||||
[t],
|
||||
);
|
||||
|
||||
return (
|
||||
<SettingsCard>
|
||||
<SettingsInput
|
||||
label={t("settings.voice.openai.apiKey")}
|
||||
labelAccessory={apiKeyInfo}
|
||||
hint={setByHint(t, state.fields.apiKey)}
|
||||
initialValue={state.apiKey}
|
||||
onChangeText={model.setApiKey}
|
||||
secureTextEntry
|
||||
placeholder={
|
||||
state.hasSavedApiKey
|
||||
? t("settings.voice.openai.savedKeyPlaceholder")
|
||||
: t("settings.voice.openai.keyPlaceholder")
|
||||
}
|
||||
/>
|
||||
<SettingsInput
|
||||
label={t("settings.voice.openai.baseUrl")}
|
||||
labelAccessory={baseUrlInfo}
|
||||
hint={setByHint(t, state.fields.baseUrl)}
|
||||
initialValue={state.baseUrl}
|
||||
onChangeText={model.setBaseUrl}
|
||||
placeholder="https://api.openai.com/v1"
|
||||
/>
|
||||
<SettingsSelect
|
||||
label={t("settings.voice.model")}
|
||||
hint={setByHint(t, state.fields.model)}
|
||||
value={state.openAiModel}
|
||||
options={modelOptions}
|
||||
onValueChange={model.setOpenAiModel}
|
||||
disabled={state.fields.model.readOnly}
|
||||
/>
|
||||
{state.kind === "stt" ? (
|
||||
<LanguageField state={state} model={model} />
|
||||
) : (
|
||||
<SettingsSelect
|
||||
label={t("settings.voice.voice")}
|
||||
hint={setByHint(t, state.fields.voice)}
|
||||
value={state.voice}
|
||||
options={voiceOptions}
|
||||
onValueChange={model.setVoice}
|
||||
disabled={state.fields.voice.readOnly}
|
||||
/>
|
||||
)}
|
||||
</SettingsCard>
|
||||
);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Local: the catalog, with the selected model checked
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function LocalModelList({
|
||||
state,
|
||||
model,
|
||||
localModels,
|
||||
}: FormProps & { localModels: LocalSpeechModels }) {
|
||||
const { t } = useTranslation();
|
||||
const { models, isLoading, error, refresh, download, remove } = localModels;
|
||||
const kindModels = useMemo(
|
||||
() => models.filter((entry) => entry.kind === state.kind),
|
||||
[models, state.kind],
|
||||
);
|
||||
const selectedId =
|
||||
state.localModel ??
|
||||
resolveLocalModel({ kind: state.kind, model: state.savedLocalModel, localModels: models })?.id;
|
||||
const source = state.fields.model;
|
||||
|
||||
if (kindModels.length === 0) {
|
||||
if (error) {
|
||||
return (
|
||||
<SettingsCard>
|
||||
<SettingsRow label={t("settings.voice.models.loadError")} hint={error.message}>
|
||||
<Button variant="outline" onPress={refresh}>
|
||||
{t("settings.voice.retry")}
|
||||
</Button>
|
||||
</SettingsRow>
|
||||
</SettingsCard>
|
||||
);
|
||||
}
|
||||
return (
|
||||
<SettingsCard>
|
||||
<SettingsRow
|
||||
label={isLoading ? t("settings.voice.models.loading") : t("settings.voice.models.none")}
|
||||
/>
|
||||
</SettingsCard>
|
||||
);
|
||||
}
|
||||
return (
|
||||
<SettingsCard testID="voice-local-models">
|
||||
{kindModels.map((entry) => (
|
||||
<LocalModelRow
|
||||
key={entry.id}
|
||||
model={entry}
|
||||
selected={entry.id === selectedId}
|
||||
selectionLocked={source.readOnly}
|
||||
hint={entry.id === selectedId ? setByHint(t, source) : undefined}
|
||||
onSelect={model.setLocalModel}
|
||||
onDownload={download}
|
||||
onDelete={remove}
|
||||
/>
|
||||
))}
|
||||
</SettingsCard>
|
||||
);
|
||||
}
|
||||
|
||||
function progressLabel(
|
||||
t: TFunction,
|
||||
state: Extract<LocalModelState, { kind: "downloading" | "extracting" }>,
|
||||
): string {
|
||||
if (state.kind === "extracting") return t("settings.voice.models.extracting");
|
||||
if (state.percent === null) return t("settings.voice.models.downloading");
|
||||
return t("settings.voice.models.downloadingPercent", { percent: state.percent });
|
||||
}
|
||||
|
||||
type HoverState = PressableStateCallbackType & { hovered?: boolean };
|
||||
|
||||
function LocalModelRow({
|
||||
model,
|
||||
selected,
|
||||
selectionLocked,
|
||||
hint,
|
||||
onSelect,
|
||||
onDownload,
|
||||
onDelete,
|
||||
}: {
|
||||
model: LocalSpeechModel;
|
||||
selected: boolean;
|
||||
selectionLocked: boolean;
|
||||
/** Why the selection is fixed, shown on the selected row. */
|
||||
hint: string | undefined;
|
||||
onSelect: (modelId: string) => void;
|
||||
onDownload: (modelId: string) => Promise<void>;
|
||||
onDelete: (modelId: string) => Promise<void>;
|
||||
}) {
|
||||
const { t, i18n } = useTranslation();
|
||||
const state = getLocalModelState(model);
|
||||
const [actionError, setActionError] = useState<string | null>(null);
|
||||
|
||||
const select = useCallback(() => onSelect(model.id), [model.id, onSelect]);
|
||||
const download = useCallback(() => {
|
||||
setActionError(null);
|
||||
onDownload(model.id).catch((error: unknown) => setActionError(errorMessage(error)));
|
||||
}, [model.id, onDownload]);
|
||||
const remove = useCallback(() => {
|
||||
void confirmDialog({
|
||||
title: t("settings.voice.models.deleteTitle", { label: model.label }),
|
||||
message: t("settings.voice.models.deleteMessage", {
|
||||
size: formatModelSize(model.downloadSizeBytes),
|
||||
}),
|
||||
confirmLabel: t("settings.voice.models.delete"),
|
||||
destructive: true,
|
||||
})
|
||||
.then((confirmed) => {
|
||||
if (!confirmed) return undefined;
|
||||
setActionError(null);
|
||||
return onDelete(model.id);
|
||||
})
|
||||
.catch((error: unknown) => setActionError(errorMessage(error)));
|
||||
}, [model.downloadSizeBytes, model.id, model.label, onDelete, t]);
|
||||
|
||||
const languages = formatLanguages({
|
||||
languages: model.languages,
|
||||
locale: i18n.language,
|
||||
formatMore: (listed, count) =>
|
||||
t("settings.voice.models.moreLanguages", { languages: listed, count }),
|
||||
});
|
||||
const details = useMemo(
|
||||
() => (
|
||||
<View style={styles.details}>
|
||||
<SettingsInfoTip
|
||||
title={model.label}
|
||||
info={model.description}
|
||||
testID={`voice-model-${model.id}-description`}
|
||||
/>
|
||||
<SettingsInfoTip
|
||||
title={model.label}
|
||||
info={languages}
|
||||
icon={Languages}
|
||||
accessibilityLabel={t("settings.voice.models.languages", { label: model.label })}
|
||||
testID={`voice-model-${model.id}-languages`}
|
||||
/>
|
||||
<Text style={styles.metaText}>{formatModelSize(model.downloadSizeBytes)}</Text>
|
||||
{state.kind === "installed" ? (
|
||||
<StatusBadge
|
||||
size="xs"
|
||||
variant="success"
|
||||
label={
|
||||
model.inUse ? t("settings.voice.models.inUse") : t("settings.voice.models.downloaded")
|
||||
}
|
||||
/>
|
||||
) : null}
|
||||
{hint ? <Text style={styles.metaText}>{hint}</Text> : null}
|
||||
</View>
|
||||
),
|
||||
[
|
||||
hint,
|
||||
languages,
|
||||
model.description,
|
||||
model.downloadSizeBytes,
|
||||
model.id,
|
||||
model.inUse,
|
||||
model.label,
|
||||
state.kind,
|
||||
t,
|
||||
],
|
||||
);
|
||||
|
||||
const recommended = useMemo(
|
||||
() =>
|
||||
model.isDefault ? (
|
||||
<StatusBadge size="xs" label={t("settings.voice.models.recommended")} />
|
||||
) : undefined,
|
||||
[model.isDefault, t],
|
||||
);
|
||||
|
||||
// Phone rows have no room beside the label for a labelled button, so actions show their icon.
|
||||
const compact = useIsCompactFormFactor();
|
||||
const downloadLabel =
|
||||
state.kind === "failed" ? t("settings.voice.retry") : t("settings.voice.models.download");
|
||||
let control = null;
|
||||
if (state.kind === "downloading" || state.kind === "extracting") {
|
||||
control = (
|
||||
<View style={styles.progress}>
|
||||
<LoadingSpinner size="small" color={styles.spinnerColor.color} />
|
||||
<Text style={styles.metaText}>{progressLabel(t, state)}</Text>
|
||||
</View>
|
||||
);
|
||||
} else if (state.kind === "installed") {
|
||||
control = model.inUse ? null : (
|
||||
<Button
|
||||
variant="ghost"
|
||||
leftIcon={Trash2}
|
||||
onPress={remove}
|
||||
accessibilityLabel={t("settings.voice.models.deleteAction", { label: model.label })}
|
||||
testID={`voice-model-delete-${model.id}`}
|
||||
>
|
||||
{compact ? null : t("settings.voice.models.delete")}
|
||||
</Button>
|
||||
);
|
||||
} else {
|
||||
control = (
|
||||
<Button
|
||||
variant="outline"
|
||||
leftIcon={Download}
|
||||
onPress={download}
|
||||
accessibilityLabel={t("settings.voice.models.downloadAction", { label: model.label })}
|
||||
testID={`voice-model-download-${model.id}`}
|
||||
>
|
||||
{compact ? null : downloadLabel}
|
||||
</Button>
|
||||
);
|
||||
}
|
||||
|
||||
const rowStyle = useCallback(
|
||||
({ hovered, pressed }: HoverState) => [
|
||||
!selectionLocked && hovered && styles.rowHovered,
|
||||
!selectionLocked && pressed && styles.rowPressed,
|
||||
],
|
||||
[selectionLocked],
|
||||
);
|
||||
const accessibilityState = useMemo(
|
||||
() => ({ checked: selected, disabled: selectionLocked }),
|
||||
[selected, selectionLocked],
|
||||
);
|
||||
|
||||
return (
|
||||
<PressHighlight
|
||||
accessibilityRole="radio"
|
||||
accessibilityLabel={model.label}
|
||||
accessibilityState={accessibilityState}
|
||||
aria-checked={selected}
|
||||
disabled={selectionLocked}
|
||||
onPress={select}
|
||||
style={rowStyle}
|
||||
highlightStyle={styles.rowPressed}
|
||||
testID={`voice-model-${model.id}`}
|
||||
>
|
||||
<SettingsRow
|
||||
label={model.label}
|
||||
labelAccessory={recommended}
|
||||
hint={details}
|
||||
error={actionError ?? (state.kind === "failed" ? state.error : null)}
|
||||
>
|
||||
<TrailingControls>
|
||||
{control}
|
||||
<View style={styles.checkSlot}>
|
||||
{selected ? (
|
||||
<ThemedCheck size={ICON_SIZE.sm} uniProps={foregroundIconColorMapping} />
|
||||
) : null}
|
||||
</View>
|
||||
</TrailingControls>
|
||||
</SettingsRow>
|
||||
</PressHighlight>
|
||||
);
|
||||
}
|
||||
|
||||
// Every catalog row is as tall as one holding a button, so a row does not shrink when its
|
||||
// download finishes and the button goes away.
|
||||
function TrailingControls({ children }: { children: ReactNode }) {
|
||||
const size = useContainedControlSize() ?? "sm";
|
||||
const style = useMemo(() => [styles.trailing, { minHeight: buttonControlHeight[size] }], [size]);
|
||||
return <View style={style}>{children}</View>;
|
||||
}
|
||||
|
||||
const styles = StyleSheet.create((theme) => ({
|
||||
footer: {
|
||||
flex: 1,
|
||||
flexDirection: "row",
|
||||
gap: theme.spacing[3],
|
||||
},
|
||||
footerButton: {
|
||||
flex: 1,
|
||||
},
|
||||
rowHovered: {
|
||||
backgroundColor: theme.colors.surface2,
|
||||
},
|
||||
rowPressed: {
|
||||
backgroundColor: theme.colors.surface3,
|
||||
},
|
||||
// The model's details sit on the hint line, so every catalog row has the same two lines.
|
||||
details: {
|
||||
flexDirection: "row",
|
||||
flexWrap: "wrap",
|
||||
alignItems: "center",
|
||||
gap: theme.spacing[2],
|
||||
marginTop: theme.spacing[1],
|
||||
},
|
||||
metaText: {
|
||||
color: theme.colors.foregroundMuted,
|
||||
fontSize: theme.fontSize.sm,
|
||||
fontVariant: ["tabular-nums"],
|
||||
},
|
||||
trailing: {
|
||||
flexDirection: "row",
|
||||
alignItems: "center",
|
||||
gap: theme.spacing[2],
|
||||
},
|
||||
checkSlot: {
|
||||
width: ICON_SIZE.sm,
|
||||
alignItems: "center",
|
||||
},
|
||||
progress: {
|
||||
flexDirection: "row",
|
||||
alignItems: "center",
|
||||
gap: theme.spacing[2],
|
||||
},
|
||||
spinnerColor: {
|
||||
color: theme.colors.foregroundMuted,
|
||||
},
|
||||
}));
|
||||
@@ -0,0 +1,71 @@
|
||||
import { useCallback } from "react";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import type { LocalSpeechModel } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
import { useFetchQuery } from "@/data/query";
|
||||
import { useHostRuntimeClient, useHostRuntimeIsConnected } from "@/runtime/host-runtime";
|
||||
import { isLocalModelBusy } from "./voice-settings";
|
||||
|
||||
const DOWNLOAD_POLL_INTERVAL_MS = 1000;
|
||||
|
||||
const localSpeechModelsQueryKey = (serverId: string) => ["speech-local-models", serverId] as const;
|
||||
|
||||
export interface LocalSpeechModels {
|
||||
models: LocalSpeechModel[];
|
||||
isLoading: boolean;
|
||||
error: Error | null;
|
||||
refresh: () => void;
|
||||
download: (modelId: string) => Promise<void>;
|
||||
remove: (modelId: string) => Promise<void>;
|
||||
}
|
||||
|
||||
// The daemon reports download progress through the list, so the list polls while any
|
||||
// model is downloading or extracting.
|
||||
export function useLocalSpeechModels(serverId: string): LocalSpeechModels {
|
||||
const { t } = useTranslation();
|
||||
const client = useHostRuntimeClient(serverId);
|
||||
const isConnected = useHostRuntimeIsConnected(serverId);
|
||||
const query = useFetchQuery({
|
||||
queryKey: localSpeechModelsQueryKey(serverId),
|
||||
enabled: Boolean(client && isConnected),
|
||||
dataShape: "list",
|
||||
staleTimeMs: 0,
|
||||
queryFn: () => {
|
||||
if (!client) throw new Error(t("workspace.terminal.hostDisconnected"));
|
||||
return client.listLocalSpeechModels();
|
||||
},
|
||||
refetchInterval: (current) =>
|
||||
current.state.data?.some(isLocalModelBusy) ? DOWNLOAD_POLL_INTERVAL_MS : false,
|
||||
});
|
||||
const { refetch } = query;
|
||||
|
||||
const refresh = useCallback(() => {
|
||||
void refetch();
|
||||
}, [refetch]);
|
||||
|
||||
const download = useCallback(
|
||||
async (modelId: string) => {
|
||||
if (!client) return;
|
||||
await client.downloadLocalSpeechModel(modelId);
|
||||
await refetch();
|
||||
},
|
||||
[client, refetch],
|
||||
);
|
||||
|
||||
const remove = useCallback(
|
||||
async (modelId: string) => {
|
||||
if (!client) return;
|
||||
await client.deleteLocalSpeechModel(modelId);
|
||||
await refetch();
|
||||
},
|
||||
[client, refetch],
|
||||
);
|
||||
|
||||
return {
|
||||
models: query.data ?? [],
|
||||
isLoading: query.isLoading,
|
||||
error: query.error,
|
||||
refresh,
|
||||
download,
|
||||
remove,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,252 @@
|
||||
import { useCallback, useState } from "react";
|
||||
import { View } from "react-native";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import type { TFunction } from "i18next";
|
||||
import { StyleSheet } from "react-native-unistyles";
|
||||
import type { SpeechSettingsPatch } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
import {
|
||||
SettingsCard,
|
||||
SettingsDetailRow,
|
||||
SettingsRow,
|
||||
SettingsSection,
|
||||
} from "@/components/settings";
|
||||
import { Alert } from "@/components/ui/alert";
|
||||
import { LoadingSpinner } from "@/components/ui/loading-spinner";
|
||||
import { Switch } from "@/components/ui/switch";
|
||||
import { useToast } from "@/contexts/toast-api-context";
|
||||
import { useDaemonConfig } from "@/hooks/use-daemon-config";
|
||||
import { useHostFeature } from "@/runtime/host-features";
|
||||
import { useHostRuntimeIsConnected } from "@/runtime/host-runtime";
|
||||
import { SpeechProviderSheet } from "./speech-provider-sheet";
|
||||
import { useLocalSpeechModels } from "./use-local-speech-models";
|
||||
import {
|
||||
getSpeechSite,
|
||||
summarizeSpeechSite,
|
||||
type SpeechSiteId,
|
||||
type SpeechSiteSummary,
|
||||
} from "./voice-settings";
|
||||
|
||||
type SavePatch = (patch: SpeechSettingsPatch) => Promise<unknown>;
|
||||
const EMPTY_ENV_OVERRIDES: Record<string, string> = {};
|
||||
|
||||
// The sheet mounts fresh for every open (docs/forms.md); `openCount` keys each mount.
|
||||
interface OpenSheet {
|
||||
siteId: SpeechSiteId;
|
||||
visible: boolean;
|
||||
openCount: number;
|
||||
}
|
||||
|
||||
function errorMessage(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
|
||||
export function HostVoicePage({ serverId }: { serverId: string }) {
|
||||
const { t } = useTranslation();
|
||||
const toast = useToast();
|
||||
const isConnected = useHostRuntimeIsConnected(serverId);
|
||||
const supported = useHostFeature(serverId, "speechSettings");
|
||||
const { config, patchConfig } = useDaemonConfig(serverId);
|
||||
const localModels = useLocalSpeechModels(serverId);
|
||||
const { refresh: refreshModels } = localModels;
|
||||
const [sheet, setSheet] = useState<OpenSheet | null>(null);
|
||||
|
||||
// Selecting a model changes which models are in use, so the catalog is re-read after a save.
|
||||
const save = useCallback<SavePatch>(
|
||||
async (patch) => {
|
||||
await patchConfig({ features: patch });
|
||||
refreshModels();
|
||||
},
|
||||
[patchConfig, refreshModels],
|
||||
);
|
||||
const saveSwitch = useCallback(
|
||||
(patch: SpeechSettingsPatch) => {
|
||||
save(patch).catch((error: unknown) => {
|
||||
toast.error(`${t("settings.voice.saveError")}: ${errorMessage(error)}`);
|
||||
});
|
||||
},
|
||||
[save, t, toast],
|
||||
);
|
||||
const setDictationEnabled = useCallback(
|
||||
(enabled: boolean) => saveSwitch({ dictation: { enabled } }),
|
||||
[saveSwitch],
|
||||
);
|
||||
const setVoiceModeEnabled = useCallback(
|
||||
(enabled: boolean) => saveSwitch({ voiceMode: { enabled } }),
|
||||
[saveSwitch],
|
||||
);
|
||||
const openSheet = useCallback((siteId: SpeechSiteId) => {
|
||||
setSheet((current) => ({
|
||||
siteId,
|
||||
visible: true,
|
||||
openCount: (current?.openCount ?? 0) + 1,
|
||||
}));
|
||||
}, []);
|
||||
const closeSheet = useCallback(() => {
|
||||
setSheet((current) => (current ? { ...current, visible: false } : null));
|
||||
}, []);
|
||||
const dismissSheet = useCallback(() => setSheet(null), []);
|
||||
|
||||
if (!isConnected) {
|
||||
return <Alert size="md" variant="warning" title={t("settings.voice.offline")} />;
|
||||
}
|
||||
if (!supported) {
|
||||
return <Alert size="md" variant="warning" title={t("settings.voice.updateHost")} />;
|
||||
}
|
||||
const features = config?.features;
|
||||
if (!features) {
|
||||
return (
|
||||
<View style={styles.loading}>
|
||||
<LoadingSpinner size="large" color={styles.spinnerColor.color} />
|
||||
</View>
|
||||
);
|
||||
}
|
||||
const envOverrides = config?.envOverrides ?? EMPTY_ENV_OVERRIDES;
|
||||
const siteRow = (siteId: SpeechSiteId, label: string) => (
|
||||
<SpeechSiteRow
|
||||
siteId={siteId}
|
||||
label={label}
|
||||
summary={summarizeSpeechSite({
|
||||
siteId,
|
||||
site: getSpeechSite(features, siteId),
|
||||
envOverrides,
|
||||
localModels: localModels.models,
|
||||
})}
|
||||
onOpen={openSheet}
|
||||
/>
|
||||
);
|
||||
const sheetSite = sheet ? getSpeechSite(features, sheet.siteId) : null;
|
||||
|
||||
return (
|
||||
<View testID="voice-settings">
|
||||
<SettingsSection title={t("settings.voice.dictation.title")} testID="voice-dictation">
|
||||
<SettingsCard>
|
||||
<FeatureSwitchRow
|
||||
label={t("settings.voice.dictation.enabled")}
|
||||
hint={t("settings.voice.dictation.enabledHint")}
|
||||
envVar={envOverrides["features.dictation.enabled"]}
|
||||
value={features.dictation.enabled}
|
||||
onChange={setDictationEnabled}
|
||||
/>
|
||||
{siteRow("dictation.stt", t("settings.voice.sites.dictationStt"))}
|
||||
</SettingsCard>
|
||||
</SettingsSection>
|
||||
<SettingsSection
|
||||
title={t("settings.voice.voiceMode.title")}
|
||||
info={t("settings.voice.voiceMode.info")}
|
||||
testID="voice-voice-mode"
|
||||
>
|
||||
<SettingsCard>
|
||||
<FeatureSwitchRow
|
||||
label={t("settings.voice.voiceMode.enabled")}
|
||||
hint={t("settings.voice.voiceMode.enabledHint")}
|
||||
envVar={envOverrides["features.voiceMode.enabled"]}
|
||||
value={features.voiceMode.enabled}
|
||||
onChange={setVoiceModeEnabled}
|
||||
/>
|
||||
{siteRow("voiceMode.stt", t("settings.voice.sites.voiceModeStt"))}
|
||||
{siteRow("voiceMode.tts", t("settings.voice.sites.voiceModeTts"))}
|
||||
</SettingsCard>
|
||||
</SettingsSection>
|
||||
{sheet && sheetSite && sheetSite.provider !== "unknown" ? (
|
||||
<SpeechProviderSheet
|
||||
key={sheet.openCount}
|
||||
title={siteLabel(t, sheet.siteId)}
|
||||
siteId={sheet.siteId}
|
||||
site={sheetSite}
|
||||
envOverrides={envOverrides}
|
||||
localModels={localModels}
|
||||
visible={sheet.visible}
|
||||
onClose={closeSheet}
|
||||
onDismiss={dismissSheet}
|
||||
save={save}
|
||||
/>
|
||||
) : null}
|
||||
</View>
|
||||
);
|
||||
}
|
||||
|
||||
function siteLabel(t: TFunction, siteId: SpeechSiteId): string {
|
||||
switch (siteId) {
|
||||
case "dictation.stt":
|
||||
return t("settings.voice.sites.dictationStt");
|
||||
case "voiceMode.stt":
|
||||
return t("settings.voice.sites.voiceModeStt");
|
||||
case "voiceMode.tts":
|
||||
return t("settings.voice.sites.voiceModeTts");
|
||||
}
|
||||
}
|
||||
|
||||
function FeatureSwitchRow({
|
||||
label,
|
||||
hint,
|
||||
envVar,
|
||||
value,
|
||||
onChange,
|
||||
}: {
|
||||
label: string;
|
||||
hint: string;
|
||||
envVar: string | undefined;
|
||||
value: boolean;
|
||||
onChange: (value: boolean) => void;
|
||||
}) {
|
||||
const { t } = useTranslation();
|
||||
return (
|
||||
<SettingsRow label={label} hint={envVar ? t("settings.voice.setBy", { name: envVar }) : hint}>
|
||||
<Switch
|
||||
value={value}
|
||||
onValueChange={onChange}
|
||||
disabled={Boolean(envVar)}
|
||||
accessibilityLabel={label}
|
||||
/>
|
||||
</SettingsRow>
|
||||
);
|
||||
}
|
||||
|
||||
function SpeechSiteRow({
|
||||
siteId,
|
||||
label,
|
||||
summary,
|
||||
onOpen,
|
||||
}: {
|
||||
siteId: SpeechSiteId;
|
||||
label: string;
|
||||
summary: SpeechSiteSummary;
|
||||
onOpen: (siteId: SpeechSiteId) => void;
|
||||
}) {
|
||||
const { t } = useTranslation();
|
||||
const open = useCallback(() => onOpen(siteId), [onOpen, siteId]);
|
||||
return (
|
||||
<SettingsDetailRow
|
||||
label={label}
|
||||
summary={summaryLabel(t, summary)}
|
||||
onPress={open}
|
||||
disabled={summary.kind === "unknown"}
|
||||
testID={`voice-site-${siteId}`}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function summaryLabel(t: TFunction, summary: SpeechSiteSummary): string {
|
||||
switch (summary.kind) {
|
||||
case "local":
|
||||
return summary.model
|
||||
? `${t("settings.voice.providers.local")} · ${summary.model}`
|
||||
: t("settings.voice.providers.local");
|
||||
case "openai":
|
||||
return `${t("settings.voice.providers.openai")} · ${
|
||||
summary.missingKey ? t("settings.voice.openai.missingKey") : summary.model
|
||||
}`;
|
||||
case "unknown":
|
||||
return t("settings.voice.unknownProvider", { provider: summary.provider });
|
||||
}
|
||||
}
|
||||
|
||||
const styles = StyleSheet.create((theme) => ({
|
||||
loading: {
|
||||
paddingVertical: theme.spacing[8],
|
||||
alignItems: "center",
|
||||
},
|
||||
spinnerColor: {
|
||||
color: theme.colors.foregroundMuted,
|
||||
},
|
||||
}));
|
||||
@@ -0,0 +1,122 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import type { LocalSpeechModel } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
import {
|
||||
buildOpenAiModelOptions,
|
||||
getSpeechSite,
|
||||
formatLanguages,
|
||||
formatModelSize,
|
||||
getLocalModelState,
|
||||
summarizeSpeechSite,
|
||||
} from "./voice-settings";
|
||||
|
||||
function model(overrides: Partial<LocalSpeechModel>): LocalSpeechModel {
|
||||
return {
|
||||
id: "parakeet-tdt-0.6b-v2-int8",
|
||||
kind: "stt",
|
||||
label: "Parakeet v2",
|
||||
description: "Accurate English transcription.",
|
||||
languages: ["en"],
|
||||
downloadSizeBytes: 482_468_385,
|
||||
isDefault: true,
|
||||
inUse: true,
|
||||
status: "installed",
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
const catalog = [
|
||||
model({}),
|
||||
model({
|
||||
id: "parakeet-tdt-110m-en-int8",
|
||||
label: "Parakeet Small",
|
||||
isDefault: false,
|
||||
status: "not_installed",
|
||||
inUse: false,
|
||||
}),
|
||||
model({ id: "kokoro-en-v0_19", kind: "tts", label: "Kokoro", inUse: false }),
|
||||
];
|
||||
|
||||
describe("voice settings", () => {
|
||||
it("keeps local STT language from the wire when opening a provider sheet", () => {
|
||||
const features = {
|
||||
dictation: { enabled: true, stt: { provider: "local", language: "fr" } },
|
||||
voiceMode: {
|
||||
enabled: false,
|
||||
stt: { provider: "local", language: "de" },
|
||||
tts: { provider: "local" },
|
||||
},
|
||||
};
|
||||
expect(getSpeechSite(features, "dictation.stt")).toMatchObject({
|
||||
provider: "local",
|
||||
language: "fr",
|
||||
});
|
||||
expect(getSpeechSite(features, "voiceMode.stt")).toMatchObject({
|
||||
provider: "local",
|
||||
language: "de",
|
||||
});
|
||||
});
|
||||
|
||||
it("summarizes a local site by its model, falling back to the catalog default", () => {
|
||||
const summary = (modelId?: string) =>
|
||||
summarizeSpeechSite({
|
||||
siteId: "dictation.stt",
|
||||
site: { provider: "local", model: modelId },
|
||||
envOverrides: {},
|
||||
localModels: catalog,
|
||||
});
|
||||
expect(summary()).toEqual({ kind: "local", model: "Parakeet v2" });
|
||||
expect(summary("parakeet-tdt-110m-en-int8")).toEqual({
|
||||
kind: "local",
|
||||
model: "Parakeet Small",
|
||||
});
|
||||
});
|
||||
|
||||
it("flags an OpenAI site with no saved key and no env key", () => {
|
||||
const summary = (envOverrides: Record<string, string>) =>
|
||||
summarizeSpeechSite({
|
||||
siteId: "voiceMode.tts",
|
||||
site: { provider: "openai", hasApiKey: false },
|
||||
envOverrides,
|
||||
localModels: catalog,
|
||||
});
|
||||
expect(summary({})).toEqual({ kind: "openai", model: "tts-1", missingKey: true });
|
||||
expect(summary({ "features.voiceMode.tts.apiKey": "OPENAI_API_KEY" })).toEqual({
|
||||
kind: "openai",
|
||||
model: "tts-1",
|
||||
missingKey: false,
|
||||
});
|
||||
});
|
||||
|
||||
it("keeps a custom OpenAI-compatible model id selectable", () => {
|
||||
expect(buildOpenAiModelOptions("stt", "my-whisper").at(-1)).toEqual({
|
||||
value: "my-whisper",
|
||||
label: "my-whisper",
|
||||
});
|
||||
});
|
||||
|
||||
it("reports download progress as a percentage", () => {
|
||||
expect(
|
||||
getLocalModelState(model({ status: "downloading", receivedBytes: 241, totalBytes: 482 })),
|
||||
).toEqual({ kind: "downloading", percent: 50 });
|
||||
expect(getLocalModelState(model({ status: "downloading" }))).toEqual({
|
||||
kind: "downloading",
|
||||
percent: null,
|
||||
});
|
||||
});
|
||||
|
||||
it("formats download sizes", () => {
|
||||
expect(formatModelSize(482_468_385)).toBe("482 MB");
|
||||
expect(formatModelSize(1_250_000_000)).toBe("1.3 GB");
|
||||
});
|
||||
|
||||
it("names a short language list and counts the rest of a long one", () => {
|
||||
const formatMore = (languages: string, count: number) => `${languages} and ${count} more`;
|
||||
expect(formatLanguages({ languages: ["en", "de"], locale: "en", formatMore })).toBe(
|
||||
"English, German",
|
||||
);
|
||||
const whisper = ["en", "zh", "de", "es", "ru", "ko", "fr", "ja", "pt", "tr", "pl", "ca"];
|
||||
expect(formatLanguages({ languages: whisper, locale: "en", formatMore })).toBe(
|
||||
"English, Chinese, German, Spanish, Russian, Korean, French, Japanese, Portuguese, Turkish and 2 more",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,240 @@
|
||||
import type {
|
||||
LocalSpeechModel,
|
||||
SpeechSettings,
|
||||
SpeechSettingsPatch,
|
||||
} from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
|
||||
// What the Voice page needs to know about speech providers and models, kept free of React
|
||||
// so the option lists and model states are testable on their own.
|
||||
|
||||
export type SpeechProvider = "local" | "openai";
|
||||
export type SpeechModelKind = "stt" | "tts";
|
||||
|
||||
export const SPEECH_PROVIDERS: readonly SpeechProvider[] = ["local", "openai"];
|
||||
|
||||
// The three places that use speech. Each carries its own provider settings in config.json
|
||||
// under `features.<site>`.
|
||||
export type SpeechSiteId = "dictation.stt" | "voiceMode.stt" | "voiceMode.tts";
|
||||
|
||||
export const SPEECH_SITE_KIND: Record<SpeechSiteId, SpeechModelKind> = {
|
||||
"dictation.stt": "stt",
|
||||
"voiceMode.stt": "stt",
|
||||
"voiceMode.tts": "tts",
|
||||
};
|
||||
|
||||
/** The config path the daemon names in `envOverrides` for a field of a site. */
|
||||
export function speechSiteFieldPath(siteId: SpeechSiteId, field: string): string {
|
||||
return `features.${siteId}.${field}`;
|
||||
}
|
||||
|
||||
/** One site's settings as the daemon reports them, narrowed to what the app can edit. */
|
||||
export type SpeechSite =
|
||||
| { provider: "local"; model?: string; language?: string }
|
||||
| {
|
||||
provider: "openai";
|
||||
hasApiKey: boolean;
|
||||
baseUrl?: string;
|
||||
model?: string;
|
||||
language?: string;
|
||||
voice?: string;
|
||||
}
|
||||
| { provider: "unknown"; id: string };
|
||||
|
||||
type SpeechSiteSettings = SpeechSettings["dictation"]["stt"] | SpeechSettings["voiceMode"]["tts"];
|
||||
|
||||
export function getSpeechSite(features: SpeechSettings, siteId: SpeechSiteId): SpeechSite {
|
||||
switch (siteId) {
|
||||
case "dictation.stt":
|
||||
return toSpeechSite(features.dictation.stt);
|
||||
case "voiceMode.stt":
|
||||
return toSpeechSite(features.voiceMode.stt);
|
||||
case "voiceMode.tts":
|
||||
return toSpeechSite(features.voiceMode.tts);
|
||||
}
|
||||
}
|
||||
|
||||
function toSpeechSite(settings: SpeechSiteSettings): SpeechSite {
|
||||
const text = (value: unknown) => (typeof value === "string" && value ? value : undefined);
|
||||
if (settings.provider === "local")
|
||||
return { provider: "local", model: text(settings.model), language: text(settings.language) };
|
||||
if (settings.provider === "openai") {
|
||||
return {
|
||||
provider: "openai",
|
||||
hasApiKey: settings.hasApiKey === true,
|
||||
baseUrl: text(settings.baseUrl),
|
||||
model: text(settings.model),
|
||||
language: text(settings.language),
|
||||
voice: text(settings.voice),
|
||||
};
|
||||
}
|
||||
return { provider: "unknown", id: settings.provider };
|
||||
}
|
||||
|
||||
export type SpeechSitePatch = NonNullable<
|
||||
| NonNullable<SpeechSettingsPatch["voiceMode"]>["stt"]
|
||||
| NonNullable<SpeechSettingsPatch["voiceMode"]>["tts"]
|
||||
>;
|
||||
|
||||
export function buildSpeechSitePatch(
|
||||
siteId: SpeechSiteId,
|
||||
patch: SpeechSitePatch,
|
||||
): SpeechSettingsPatch {
|
||||
switch (siteId) {
|
||||
case "dictation.stt":
|
||||
return { dictation: { stt: patch } };
|
||||
case "voiceMode.stt":
|
||||
return { voiceMode: { stt: patch } };
|
||||
case "voiceMode.tts":
|
||||
return { voiceMode: { tts: patch } };
|
||||
}
|
||||
}
|
||||
|
||||
// OpenAI accepts free-form ids so self-hosted servers can use their own; these are the
|
||||
// ones the page offers. A saved id outside the list is still shown as an option.
|
||||
const OPENAI_MODELS: Record<SpeechModelKind, { ids: readonly string[]; defaultId: string }> = {
|
||||
stt: {
|
||||
ids: ["whisper-1", "gpt-4o-transcribe", "gpt-4o-mini-transcribe"],
|
||||
defaultId: "whisper-1",
|
||||
},
|
||||
tts: { ids: ["tts-1", "tts-1-hd", "gpt-4o-mini-tts"], defaultId: "tts-1" },
|
||||
};
|
||||
|
||||
const OPENAI_VOICES = ["alloy", "ash", "coral", "echo", "fable", "nova", "onyx", "sage", "shimmer"];
|
||||
const DEFAULT_OPENAI_VOICE = "alloy";
|
||||
|
||||
export interface SpeechModelOption {
|
||||
value: string;
|
||||
label: string;
|
||||
}
|
||||
|
||||
export function resolveOpenAiModel(kind: SpeechModelKind, model: string | undefined): string {
|
||||
return model ?? OPENAI_MODELS[kind].defaultId;
|
||||
}
|
||||
|
||||
export function resolveLocalModel(input: {
|
||||
kind: SpeechModelKind;
|
||||
model: string | undefined;
|
||||
localModels: readonly LocalSpeechModel[];
|
||||
}): LocalSpeechModel | undefined {
|
||||
const kindModels = input.localModels.filter((model) => model.kind === input.kind);
|
||||
if (input.model) return kindModels.find((model) => model.id === input.model);
|
||||
return kindModels.find((model) => model.isDefault);
|
||||
}
|
||||
|
||||
export function buildOpenAiModelOptions(
|
||||
kind: SpeechModelKind,
|
||||
selected: string,
|
||||
): SpeechModelOption[] {
|
||||
return withSelected(
|
||||
OPENAI_MODELS[kind].ids.map((id) => ({ value: id, label: id })),
|
||||
selected,
|
||||
);
|
||||
}
|
||||
|
||||
export function resolveOpenAiVoice(voice: string | undefined): string {
|
||||
return voice ?? DEFAULT_OPENAI_VOICE;
|
||||
}
|
||||
|
||||
export function buildOpenAiVoiceOptions(selected: string): SpeechModelOption[] {
|
||||
return withSelected(
|
||||
OPENAI_VOICES.map((voice) => ({ value: voice, label: voice })),
|
||||
selected,
|
||||
);
|
||||
}
|
||||
|
||||
function withSelected(options: SpeechModelOption[], selected: string): SpeechModelOption[] {
|
||||
if (!selected || options.some((option) => option.value === selected)) return options;
|
||||
return [...options, { value: selected, label: selected }];
|
||||
}
|
||||
|
||||
// What a page row says about a site before it is opened.
|
||||
export type SpeechSiteSummary =
|
||||
| { kind: "local"; model: string | null }
|
||||
| { kind: "openai"; model: string; missingKey: boolean }
|
||||
| { kind: "unknown"; provider: string };
|
||||
|
||||
export function summarizeSpeechSite(input: {
|
||||
siteId: SpeechSiteId;
|
||||
site: SpeechSite;
|
||||
envOverrides: Record<string, string>;
|
||||
localModels: readonly LocalSpeechModel[];
|
||||
}): SpeechSiteSummary {
|
||||
const { site, siteId } = input;
|
||||
const kind = SPEECH_SITE_KIND[siteId];
|
||||
switch (site.provider) {
|
||||
case "local": {
|
||||
const model = resolveLocalModel({ kind, model: site.model, localModels: input.localModels });
|
||||
return { kind: "local", model: model?.label ?? site.model ?? null };
|
||||
}
|
||||
case "openai":
|
||||
return {
|
||||
kind: "openai",
|
||||
model: resolveOpenAiModel(kind, site.model),
|
||||
missingKey: !site.hasApiKey && !input.envOverrides[speechSiteFieldPath(siteId, "apiKey")],
|
||||
};
|
||||
case "unknown":
|
||||
return { kind: "unknown", provider: site.id };
|
||||
}
|
||||
}
|
||||
|
||||
export type LocalModelState =
|
||||
| { kind: "installed" }
|
||||
| { kind: "not_installed" }
|
||||
| { kind: "downloading"; percent: number | null }
|
||||
| { kind: "extracting" }
|
||||
| { kind: "failed"; error: string };
|
||||
|
||||
export function getLocalModelState(model: LocalSpeechModel): LocalModelState {
|
||||
switch (model.status) {
|
||||
case "installed":
|
||||
return { kind: "installed" };
|
||||
case "downloading": {
|
||||
const total = model.totalBytes ?? 0;
|
||||
const received = model.receivedBytes ?? 0;
|
||||
return {
|
||||
kind: "downloading",
|
||||
percent: total > 0 ? Math.min(100, Math.floor((received / total) * 100)) : null,
|
||||
};
|
||||
}
|
||||
case "extracting":
|
||||
return { kind: "extracting" };
|
||||
case "failed":
|
||||
return { kind: "failed", error: model.error ?? "" };
|
||||
default:
|
||||
return { kind: "not_installed" };
|
||||
}
|
||||
}
|
||||
|
||||
export function isLocalModelBusy(model: LocalSpeechModel): boolean {
|
||||
return model.status === "downloading" || model.status === "extracting";
|
||||
}
|
||||
|
||||
export function formatModelSize(bytes: number): string {
|
||||
const megabytes = bytes / 1_000_000;
|
||||
if (megabytes >= 1000) return `${(megabytes / 1000).toFixed(1)} GB`;
|
||||
return `${Math.round(megabytes)} MB`;
|
||||
}
|
||||
|
||||
// Multilingual models list close to a hundred languages; past this many the tooltip names
|
||||
// the first ones and counts the rest.
|
||||
const MAX_LISTED_LANGUAGES = 10;
|
||||
|
||||
export function formatLanguages(input: {
|
||||
languages: readonly string[];
|
||||
locale: string;
|
||||
formatMore: (languages: string, count: number) => string;
|
||||
}): string {
|
||||
const names = languageNames(input.locale);
|
||||
const listed = input.languages
|
||||
.slice(0, MAX_LISTED_LANGUAGES)
|
||||
.map((code) => names?.of(code) ?? code.toUpperCase())
|
||||
.join(", ");
|
||||
const hidden = input.languages.length - MAX_LISTED_LANGUAGES;
|
||||
return hidden > 0 ? input.formatMore(listed, hidden) : listed;
|
||||
}
|
||||
|
||||
// Hermes ships without Intl.DisplayNames; the language code is the fallback label there.
|
||||
function languageNames(locale: string): Intl.DisplayNames | null {
|
||||
if (typeof Intl.DisplayNames !== "function") return null;
|
||||
return new Intl.DisplayNames([locale], { type: "language" });
|
||||
}
|
||||
@@ -518,6 +518,7 @@ export const HOST_SECTION_SLUGS = [
|
||||
"pair-device",
|
||||
"agents",
|
||||
"metadata",
|
||||
"voice",
|
||||
"workspaces",
|
||||
"providers",
|
||||
"usage",
|
||||
|
||||
@@ -161,6 +161,7 @@ export default defineConfig({
|
||||
"i18next",
|
||||
"@marijn/find-cluster-break",
|
||||
"tiny-invariant",
|
||||
"react/jsx-dev-runtime",
|
||||
"react-native-gesture-handler > hoist-non-react-statics",
|
||||
"react-native-gesture-handler > invariant",
|
||||
],
|
||||
@@ -171,6 +172,8 @@ export default defineConfig({
|
||||
"react-native-keyboard-controller",
|
||||
],
|
||||
},
|
||||
// babel-preset-expo compiles JSX with the automatic runtime, so app modules don't import React.
|
||||
esbuild: { jsx: "automatic" },
|
||||
// The globals a React Native bundler defines, which esbuild is no longer there to supply for
|
||||
// the package excluded above.
|
||||
define: {
|
||||
|
||||
@@ -19,6 +19,7 @@ import { CreationClient } from "./creation/index.js";
|
||||
import type { CreationSnapshot } from "@getpaseo/protocol/messages";
|
||||
import type { z } from "zod";
|
||||
import type { SessionEventSubscription } from "@getpaseo/protocol/messages";
|
||||
import type { LocalSpeechModel } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
import { CLIENT_CAPS, type ClientCapability } from "@getpaseo/protocol/client-capabilities";
|
||||
import type { AgentAttentionNotificationPayload } from "@getpaseo/protocol/agent-attention-notification";
|
||||
import { MAX_WEBSOCKET_MESSAGE_BYTES } from "@getpaseo/protocol/websocket-message-limit";
|
||||
@@ -5547,6 +5548,34 @@ export class DaemonClient {
|
||||
});
|
||||
}
|
||||
|
||||
async listLocalSpeechModels(requestId?: string): Promise<LocalSpeechModel[]> {
|
||||
const payload = await this.sendCorrelatedSessionRequest({
|
||||
requestId,
|
||||
message: { type: "speech.local_models.list.request" },
|
||||
responseType: "speech.local_models.list.response",
|
||||
});
|
||||
if (payload.error) throw new Error(payload.error);
|
||||
return payload.models;
|
||||
}
|
||||
|
||||
async downloadLocalSpeechModel(modelId: string, requestId?: string): Promise<void> {
|
||||
const payload = await this.sendCorrelatedSessionRequest({
|
||||
requestId,
|
||||
message: { type: "speech.local_models.download.request", modelId },
|
||||
responseType: "speech.local_models.download.response",
|
||||
});
|
||||
if (payload.error) throw new Error(payload.error);
|
||||
}
|
||||
|
||||
async deleteLocalSpeechModel(modelId: string, requestId?: string): Promise<void> {
|
||||
const payload = await this.sendCorrelatedSessionRequest({
|
||||
requestId,
|
||||
message: { type: "speech.local_models.delete.request", modelId },
|
||||
responseType: "speech.local_models.delete.response",
|
||||
});
|
||||
if (payload.error) throw new Error(payload.error);
|
||||
}
|
||||
|
||||
sendBrowserAutomationExecuteResponse(response: BrowserAutomationExecuteResponse): void {
|
||||
this.sendSessionMessageStrict(response);
|
||||
}
|
||||
|
||||
@@ -82,6 +82,16 @@ import {
|
||||
ScheduleRunOnceResponseSchema,
|
||||
ScheduleUpdateResponseSchema,
|
||||
} from "./schedule/rpc-schemas.js";
|
||||
import {
|
||||
SpeechLocalModelsDeleteRequestSchema,
|
||||
SpeechLocalModelsDeleteResponseSchema,
|
||||
SpeechLocalModelsDownloadRequestSchema,
|
||||
SpeechLocalModelsDownloadResponseSchema,
|
||||
SpeechLocalModelsListRequestSchema,
|
||||
SpeechLocalModelsListResponseSchema,
|
||||
SpeechSettingsPatchSchema,
|
||||
SpeechSettingsSchema,
|
||||
} from "./speech/rpc-schemas.js";
|
||||
import {
|
||||
LoopRunRequestSchema,
|
||||
LoopListRequestSchema,
|
||||
@@ -233,6 +243,9 @@ export const MutableDaemonConfigSchema = z
|
||||
skills: z.object({ selection: AgentSkillSelectionSchema.optional() }).strict().optional(),
|
||||
pluginsEnabled: z.boolean().optional(),
|
||||
plugins: z.record(PluginIdSchema, PluginSourceSchema).optional(),
|
||||
// COMPAT(speechSettings): added in v0.12.0, optional because older daemons omit it.
|
||||
features: SpeechSettingsSchema.optional(),
|
||||
envOverrides: z.record(z.string(), z.string()).optional(),
|
||||
})
|
||||
.passthrough();
|
||||
|
||||
@@ -254,6 +267,7 @@ export const MutableDaemonConfigPatchSchema = z
|
||||
agentProfiles: z.array(AgentProfileSchema).optional(),
|
||||
pluginsEnabled: z.boolean().optional(),
|
||||
plugins: z.record(PluginIdSchema, PluginSourceSchema).optional(),
|
||||
features: SpeechSettingsPatchSchema.optional(),
|
||||
})
|
||||
.partial()
|
||||
.passthrough();
|
||||
@@ -3440,6 +3454,9 @@ export const SessionInboundMessageSchema = z.discriminatedUnion("type", [
|
||||
ChatWaitRequestSchema,
|
||||
ScheduleCreateRequestSchema,
|
||||
ScheduleListRequestSchema,
|
||||
SpeechLocalModelsListRequestSchema,
|
||||
SpeechLocalModelsDownloadRequestSchema,
|
||||
SpeechLocalModelsDeleteRequestSchema,
|
||||
ScheduleInspectRequestSchema,
|
||||
ScheduleLogsRequestSchema,
|
||||
SchedulePauseRequestSchema,
|
||||
@@ -3674,6 +3691,8 @@ export const ServerInfoStatusPayloadSchema = z
|
||||
daemonStatusRpc: z.boolean().optional(),
|
||||
// COMPAT(daemonConfigReload): added in v0.4.0, remove gate after 2027-02-14.
|
||||
daemonConfigReload: z.boolean().optional(),
|
||||
// COMPAT(speechSettings): added in v0.12.0, remove gate after 2027-04-18.
|
||||
speechSettings: z.boolean().optional(),
|
||||
// COMPAT(relayConfig): added in v0.2.6, remove gate after 2027-01-31.
|
||||
relayConfig: z.boolean().optional(),
|
||||
// COMPAT(pushTokenRevocation): added in v0.3.2, remove gate after 2027-02-10.
|
||||
@@ -7152,6 +7171,9 @@ export const SessionOutboundMessageSchema = z.discriminatedUnion("type", [
|
||||
ChatWaitResponseSchema,
|
||||
ScheduleCreateResponseSchema,
|
||||
ScheduleListResponseSchema,
|
||||
SpeechLocalModelsListResponseSchema,
|
||||
SpeechLocalModelsDownloadResponseSchema,
|
||||
SpeechLocalModelsDeleteResponseSchema,
|
||||
ScheduleInspectResponseSchema,
|
||||
ScheduleLogsResponseSchema,
|
||||
SchedulePauseResponseSchema,
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
import { describe, expect, test } from "vitest";
|
||||
import { z } from "zod";
|
||||
import {
|
||||
MutableDaemonConfigSchema,
|
||||
MutableDaemonConfigPatchSchema,
|
||||
UsageReportEntrySchema,
|
||||
AgentSnapshotPayloadSchema,
|
||||
AgentTimelineItemPayloadSchema,
|
||||
@@ -525,3 +527,50 @@ test("released setup clients parse empty combined logs and retain command output
|
||||
}).detail.log,
|
||||
).toBe("old daemon combined output");
|
||||
});
|
||||
|
||||
test("local STT language survives wire reads and patches", () => {
|
||||
const patch = {
|
||||
features: {
|
||||
dictation: { stt: { provider: "local", language: "fr" } },
|
||||
voiceMode: { stt: { provider: "local", language: null } },
|
||||
},
|
||||
};
|
||||
expect(MutableDaemonConfigPatchSchema.parse(patch)).toEqual(patch);
|
||||
});
|
||||
|
||||
test("speech config reads remain compatible with clients before features and future providers", () => {
|
||||
const oldSchema = MutableDaemonConfigSchema.omit({ features: true, envOverrides: true });
|
||||
const oldRead = oldSchema.parse({
|
||||
providers: {},
|
||||
relay: { enabled: false },
|
||||
mcp: { enabled: true, injectIntoAgents: true },
|
||||
});
|
||||
const read = {
|
||||
...oldRead,
|
||||
features: {
|
||||
dictation: { enabled: true, stt: { provider: "local", language: "fr" } },
|
||||
voiceMode: {
|
||||
enabled: false,
|
||||
stt: { provider: "future", model: "future-model", region: "eu" },
|
||||
tts: { provider: "openai", hasApiKey: false },
|
||||
},
|
||||
},
|
||||
envOverrides: { "features.voiceMode.stt.provider": "PASEO_VOICE_STT_PROVIDER" },
|
||||
};
|
||||
expect(oldSchema.parse(read)).toEqual(read);
|
||||
expect(MutableDaemonConfigSchema.parse(oldRead)).toEqual(oldRead);
|
||||
expect(MutableDaemonConfigSchema.parse(read).features?.dictation.stt).toEqual(
|
||||
read.features.dictation.stt,
|
||||
);
|
||||
expect(MutableDaemonConfigSchema.parse(read).features?.voiceMode.stt).toEqual(
|
||||
read.features.voiceMode.stt,
|
||||
);
|
||||
const oldPatchSchema = MutableDaemonConfigPatchSchema.omit({ features: true });
|
||||
const oldPatch = oldPatchSchema.parse({ relay: { enabled: true } });
|
||||
expect(MutableDaemonConfigPatchSchema.parse(oldPatch)).toEqual(oldPatch);
|
||||
expect(
|
||||
MutableDaemonConfigPatchSchema.parse({
|
||||
features: { voiceMode: { stt: { provider: "future", region: "eu" } } },
|
||||
}).features?.voiceMode?.stt,
|
||||
).toEqual({ provider: "future", region: "eu" });
|
||||
});
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
import { z } from "zod";
|
||||
|
||||
// Per-site settings mirror config.json. Unknown providers remain parseable by older
|
||||
// clients; the feature UI decides whether it knows how to edit them.
|
||||
const ModelFields = { model: z.string().optional() };
|
||||
const OpenAiReadFields = {
|
||||
...ModelFields,
|
||||
hasApiKey: z.boolean(),
|
||||
baseUrl: z.string().optional(),
|
||||
};
|
||||
const SttReadFields = {
|
||||
language: z.string().optional(),
|
||||
confidenceThreshold: z.number().optional(),
|
||||
};
|
||||
const LocalTtsFields = { speakerId: z.number().int().optional(), speed: z.number().optional() };
|
||||
|
||||
export const SpeechSttSettingsSchema = z.union([
|
||||
z
|
||||
.object({ provider: z.literal("local"), ...ModelFields, language: z.string().optional() })
|
||||
.passthrough(),
|
||||
z.object({ provider: z.literal("openai"), ...OpenAiReadFields, ...SttReadFields }).passthrough(),
|
||||
z.object({ provider: z.string(), ...ModelFields }).passthrough(),
|
||||
]);
|
||||
export const SpeechTtsSettingsSchema = z.union([
|
||||
z.object({ provider: z.literal("local"), ...ModelFields, ...LocalTtsFields }).passthrough(),
|
||||
z
|
||||
.object({ provider: z.literal("openai"), ...OpenAiReadFields, voice: z.string().optional() })
|
||||
.passthrough(),
|
||||
z.object({ provider: z.string(), ...ModelFields }).passthrough(),
|
||||
]);
|
||||
|
||||
export const SpeechSettingsSchema = z
|
||||
.object({
|
||||
dictation: z.object({ enabled: z.boolean(), stt: SpeechSttSettingsSchema }).passthrough(),
|
||||
voiceMode: z
|
||||
.object({
|
||||
enabled: z.boolean(),
|
||||
stt: SpeechSttSettingsSchema,
|
||||
tts: SpeechTtsSettingsSchema,
|
||||
llm: z
|
||||
.object({ provider: z.string().optional(), model: z.string().optional() })
|
||||
.passthrough()
|
||||
.optional(),
|
||||
turnDetection: z.object({ provider: z.string().optional() }).passthrough().optional(),
|
||||
})
|
||||
.passthrough(),
|
||||
})
|
||||
.passthrough();
|
||||
|
||||
const ModelPatchFields = { model: z.string().nullable().optional() };
|
||||
const OpenAiPatchFields = {
|
||||
...ModelPatchFields,
|
||||
apiKey: z.string().nullable().optional(),
|
||||
baseUrl: z.string().nullable().optional(),
|
||||
};
|
||||
const SttPatchFields = {
|
||||
language: z.string().nullable().optional(),
|
||||
confidenceThreshold: z.number().nullable().optional(),
|
||||
};
|
||||
const LocalTtsPatchFields = {
|
||||
speakerId: z.number().int().nullable().optional(),
|
||||
speed: z.number().nullable().optional(),
|
||||
};
|
||||
const SpeechSttSettingsPatchSchema = z.union([
|
||||
z
|
||||
.object({
|
||||
provider: z.literal("local"),
|
||||
...ModelPatchFields,
|
||||
language: z.string().nullable().optional(),
|
||||
})
|
||||
.passthrough(),
|
||||
z
|
||||
.object({ provider: z.literal("openai"), ...OpenAiPatchFields, ...SttPatchFields })
|
||||
.passthrough(),
|
||||
z.object({ provider: z.string().nullable() }).passthrough(),
|
||||
z
|
||||
.object({ provider: z.undefined().optional(), ...OpenAiPatchFields, ...SttPatchFields })
|
||||
.passthrough(),
|
||||
]);
|
||||
const SpeechTtsSettingsPatchSchema = z.union([
|
||||
z
|
||||
.object({ provider: z.literal("local"), ...ModelPatchFields, ...LocalTtsPatchFields })
|
||||
.passthrough(),
|
||||
z
|
||||
.object({
|
||||
provider: z.literal("openai"),
|
||||
...OpenAiPatchFields,
|
||||
voice: z.string().nullable().optional(),
|
||||
})
|
||||
.passthrough(),
|
||||
z.object({ provider: z.string().nullable() }).passthrough(),
|
||||
z
|
||||
.object({
|
||||
provider: z.undefined().optional(),
|
||||
...OpenAiPatchFields,
|
||||
...LocalTtsPatchFields,
|
||||
voice: z.string().nullable().optional(),
|
||||
})
|
||||
.passthrough(),
|
||||
]);
|
||||
export const SpeechSettingsPatchSchema = z
|
||||
.object({
|
||||
dictation: z
|
||||
.object({
|
||||
enabled: z.boolean().nullable().optional(),
|
||||
stt: SpeechSttSettingsPatchSchema.optional(),
|
||||
})
|
||||
.passthrough()
|
||||
.optional(),
|
||||
voiceMode: z
|
||||
.object({
|
||||
enabled: z.boolean().nullable().optional(),
|
||||
stt: SpeechSttSettingsPatchSchema.optional(),
|
||||
tts: SpeechTtsSettingsPatchSchema.optional(),
|
||||
llm: z
|
||||
.object({
|
||||
provider: z.string().nullable().optional(),
|
||||
model: z.string().nullable().optional(),
|
||||
})
|
||||
.passthrough()
|
||||
.optional(),
|
||||
turnDetection: z
|
||||
.object({ provider: z.string().nullable().optional() })
|
||||
.passthrough()
|
||||
.optional(),
|
||||
})
|
||||
.passthrough()
|
||||
.optional(),
|
||||
})
|
||||
.passthrough();
|
||||
|
||||
export type SpeechSettings = z.infer<typeof SpeechSettingsSchema>;
|
||||
export type SpeechSettingsPatch = z.infer<typeof SpeechSettingsPatchSchema>;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Local speech models: catalog, install state, download, delete.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const LocalSpeechModelSchema = z
|
||||
.object({
|
||||
id: z.string(),
|
||||
// "stt" or "tts".
|
||||
kind: z.string(),
|
||||
label: z.string(),
|
||||
description: z.string(),
|
||||
languages: z.array(z.string()),
|
||||
downloadSizeBytes: z.number(),
|
||||
isDefault: z.boolean(),
|
||||
// Selected by an enabled feature; the daemon keeps it installed.
|
||||
inUse: z.boolean(),
|
||||
// "installed" | "not_installed" | "downloading" | "extracting" | "failed".
|
||||
status: z.string(),
|
||||
receivedBytes: z.number().optional(),
|
||||
totalBytes: z.number().optional(),
|
||||
error: z.string().optional(),
|
||||
})
|
||||
.passthrough();
|
||||
|
||||
export type LocalSpeechModel = z.infer<typeof LocalSpeechModelSchema>;
|
||||
|
||||
export const SpeechLocalModelsListRequestSchema = z.object({
|
||||
type: z.literal("speech.local_models.list.request"),
|
||||
requestId: z.string(),
|
||||
});
|
||||
|
||||
export const SpeechLocalModelsDownloadRequestSchema = z.object({
|
||||
type: z.literal("speech.local_models.download.request"),
|
||||
requestId: z.string(),
|
||||
modelId: z.string(),
|
||||
});
|
||||
|
||||
export const SpeechLocalModelsDeleteRequestSchema = z.object({
|
||||
type: z.literal("speech.local_models.delete.request"),
|
||||
requestId: z.string(),
|
||||
modelId: z.string(),
|
||||
});
|
||||
|
||||
export const SpeechLocalModelsListResponseSchema = z.object({
|
||||
type: z.literal("speech.local_models.list.response"),
|
||||
payload: z.object({
|
||||
requestId: z.string(),
|
||||
models: z.array(LocalSpeechModelSchema),
|
||||
error: z.string().nullable(),
|
||||
}),
|
||||
});
|
||||
|
||||
// Resolves once the download starts; poll the list for progress.
|
||||
export const SpeechLocalModelsDownloadResponseSchema = z.object({
|
||||
type: z.literal("speech.local_models.download.response"),
|
||||
payload: z.object({
|
||||
requestId: z.string(),
|
||||
error: z.string().nullable(),
|
||||
}),
|
||||
});
|
||||
|
||||
export const SpeechLocalModelsDeleteResponseSchema = z.object({
|
||||
type: z.literal("speech.local_models.delete.response"),
|
||||
payload: z.object({
|
||||
requestId: z.string(),
|
||||
error: z.string().nullable(),
|
||||
}),
|
||||
});
|
||||
@@ -1,9 +1,12 @@
|
||||
import { mkdtempSync, writeFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import pino from "pino";
|
||||
import { EventEmitter } from "node:events";
|
||||
|
||||
import { STTManager } from "./stt-manager.js";
|
||||
import { PersistedConfigSchema } from "../persisted-config.js";
|
||||
import { loadPersistedConfig } from "../persisted-config.js";
|
||||
import { resolveSpeechConfig } from "../speech/speech-config-resolver.js";
|
||||
import type {
|
||||
SpeechToTextProvider,
|
||||
@@ -97,12 +100,18 @@ class SequencedFakeStt implements SpeechToTextProvider {
|
||||
|
||||
describe("STTManager", () => {
|
||||
function resolveVoiceLanguage(params: { env?: NodeJS.ProcessEnv; persisted?: unknown }): string {
|
||||
const result = resolveSpeechConfig({
|
||||
paseoHome: "/tmp/paseo-home",
|
||||
env: params.env ?? ({} as NodeJS.ProcessEnv),
|
||||
persisted: PersistedConfigSchema.parse(params.persisted ?? {}),
|
||||
});
|
||||
return result.speech.sttLanguages.voice;
|
||||
const home = mkdtempSync(path.join(tmpdir(), "stt-language-"));
|
||||
try {
|
||||
writeFileSync(path.join(home, "config.json"), JSON.stringify(params.persisted ?? {}));
|
||||
const result = resolveSpeechConfig({
|
||||
paseoHome: home,
|
||||
env: params.env ?? {},
|
||||
persisted: loadPersistedConfig(home),
|
||||
});
|
||||
return result.speech.sttLanguages.voice;
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function transcribeWithResolvedVoiceLanguage(params: {
|
||||
|
||||
@@ -182,6 +182,9 @@ const INBOUND_PERMISSION = {
|
||||
set_agent_model_request: "workspace.write",
|
||||
set_agent_thinking_request: "workspace.write",
|
||||
set_daemon_config_request: "daemon.manage",
|
||||
"speech.local_models.list.request": "daemon.read",
|
||||
"speech.local_models.download.request": "daemon.manage",
|
||||
"speech.local_models.delete.request": "daemon.manage",
|
||||
set_voice_mode: "workspace.write",
|
||||
shutdown_server_request: "daemon.manage",
|
||||
start_workspace_script_request: "workspace.write",
|
||||
@@ -417,6 +420,9 @@ const OUTBOUND_PERMISSION = {
|
||||
set_agent_model_response: "workspace.write",
|
||||
set_agent_thinking_response: "workspace.write",
|
||||
set_daemon_config_response: "daemon.manage",
|
||||
"speech.local_models.list.response": "daemon.read",
|
||||
"speech.local_models.download.response": "daemon.manage",
|
||||
"speech.local_models.delete.response": "daemon.manage",
|
||||
set_voice_mode_response: "workspace.write",
|
||||
start_workspace_script_response: "workspace.write",
|
||||
stash_list_response: "workspace.read",
|
||||
|
||||
@@ -138,6 +138,7 @@ import type { OpenAiSpeechProviderConfig } from "./speech/providers/openai/confi
|
||||
import type { LocalSpeechProviderConfig } from "./speech/providers/local/config.js";
|
||||
import type { RequestedSpeechProviders } from "./speech/speech-types.js";
|
||||
import { createSpeechService } from "./speech/speech-runtime.js";
|
||||
import { createSpeechSettingsOwner } from "./speech/speech-settings.js";
|
||||
import { AgentManager } from "./agent/agent-manager.js";
|
||||
import { AgentStorage } from "./agent/agent-storage.js";
|
||||
import { attachAgentStoragePersistence } from "./persistence-hooks.js";
|
||||
@@ -379,6 +380,8 @@ export interface PaseoSpeechSttLanguages {
|
||||
|
||||
export interface PaseoSpeechConfig {
|
||||
providers: RequestedSpeechProviders;
|
||||
// Where local models live, whether or not a feature currently uses the local provider.
|
||||
localModelsDir: string;
|
||||
sttLanguages?: PaseoSpeechSttLanguages;
|
||||
local?: PaseoLocalSpeechConfig;
|
||||
}
|
||||
@@ -555,6 +558,10 @@ function createInitialRelayConfig(config: PaseoDaemonConfig): MutableDaemonConfi
|
||||
};
|
||||
}
|
||||
|
||||
function resolveReloadEnv(config: PaseoDaemonConfig): NodeJS.ProcessEnv {
|
||||
return config.configReload?.env ?? process.env;
|
||||
}
|
||||
|
||||
function createInitialMutableDaemonConfig(config: PaseoDaemonConfig): MutableDaemonConfig {
|
||||
const providers = config.providerOverrides ?? {};
|
||||
|
||||
@@ -615,13 +622,24 @@ export async function createPaseoDaemon(
|
||||
const elapsed = () => `${(performance.now() - bootstrapStart).toFixed(0)}ms`;
|
||||
const daemonVersion = config.daemonVersion ?? resolveDaemonVersion(import.meta.url);
|
||||
const initialMutableConfig = createInitialMutableDaemonConfig(config);
|
||||
const speechService = createSpeechService({
|
||||
logger,
|
||||
openaiConfig: config.openai,
|
||||
speechConfig: config.speech,
|
||||
});
|
||||
logger.info({ elapsed: elapsed() }, "Speech service created");
|
||||
const daemonConfigStore = new DaemonConfigStore(config.paseoHome, initialMutableConfig, logger, {
|
||||
relayEnabledMutable: config.relayEnabledMutable ?? true,
|
||||
startupPersisted: config.configReload?.startupPersisted,
|
||||
speech: createSpeechSettingsOwner({
|
||||
paseoHome: config.paseoHome,
|
||||
env: resolveReloadEnv(config),
|
||||
speechService,
|
||||
}),
|
||||
reloadSource: {
|
||||
resolve: (persisted) => {
|
||||
const reloaded = resolveConfigFromPersisted(config.paseoHome, persisted, {
|
||||
env: config.configReload?.env ?? process.env,
|
||||
env: resolveReloadEnv(config),
|
||||
cli: config.configReload?.cli,
|
||||
relayEnabledFallback: config.configReload?.relayEnabledFallback,
|
||||
});
|
||||
@@ -1677,13 +1695,6 @@ export async function createPaseoDaemon(
|
||||
logger.info({ route: agentMcpRoute, enabled: mcpEnabled }, "Agent MCP route mounted");
|
||||
}
|
||||
|
||||
const speechService = createSpeechService({
|
||||
logger,
|
||||
openaiConfig: config.openai,
|
||||
speechConfig: config.speech,
|
||||
});
|
||||
logger.info({ elapsed: elapsed() }, "Speech service created");
|
||||
|
||||
logger.info({ elapsed: elapsed() }, "Bootstrap complete, ready to start listening");
|
||||
|
||||
const start = async () => {
|
||||
|
||||
@@ -115,9 +115,13 @@ describe("server config", () => {
|
||||
name: "OpenAI speech providers",
|
||||
providers: { dictation: "openai", voiceStt: "openai", voiceTts: "openai" },
|
||||
expected: [
|
||||
"features.dictation.stt.apiKey",
|
||||
"features.dictation.stt.confidenceThreshold",
|
||||
"features.dictation.stt.model",
|
||||
"features.voiceMode.stt.apiKey",
|
||||
"features.voiceMode.stt.confidenceThreshold",
|
||||
"features.voiceMode.stt.model",
|
||||
"features.voiceMode.tts.apiKey",
|
||||
"features.voiceMode.tts.model",
|
||||
"features.voiceMode.tts.voice",
|
||||
],
|
||||
@@ -126,8 +130,9 @@ describe("server config", () => {
|
||||
name: "mixed local and OpenAI speech providers",
|
||||
providers: { dictation: "local", voiceStt: "openai", voiceTts: "local" },
|
||||
expected: [
|
||||
"features.dictation.stt.confidenceThreshold",
|
||||
"features.dictation.stt.model",
|
||||
"features.voiceMode.stt.apiKey",
|
||||
"features.voiceMode.stt.confidenceThreshold",
|
||||
"features.voiceMode.stt.model",
|
||||
"features.voiceMode.tts.model",
|
||||
],
|
||||
|
||||
@@ -22,7 +22,7 @@ import { ProviderOverrideSchema } from "./agent/provider-launch-config.js";
|
||||
import { AgentProviderSchema } from "@getpaseo/protocol/provider-manifest";
|
||||
import { hashDaemonPassword } from "./auth.js";
|
||||
import { resolveSpeechConfig } from "./speech/speech-config-resolver.js";
|
||||
import type { RequestedSpeechProviders } from "./speech/speech-types.js";
|
||||
import { readSpeechSettings } from "./speech/speech-settings.js";
|
||||
import { mergeHostnames, parseHostnamesEnv, type HostnamesConfig } from "./hostnames.js";
|
||||
import { resolveGitProcessPolicy } from "../utils/git-process-scheduler.js";
|
||||
|
||||
@@ -614,7 +614,7 @@ export function resolveConfigFromPersisted(
|
||||
persisted.agents?.providers as Record<string, unknown> | undefined,
|
||||
);
|
||||
|
||||
const overrideControlledPaths = resolveOverrideControlledPaths(env, cli, speech.providers);
|
||||
const overrideControlledPaths = resolveOverrideControlledPaths(env, cli, persisted);
|
||||
|
||||
return {
|
||||
listen,
|
||||
@@ -692,13 +692,13 @@ function parsePositiveGitOverride(value: string | undefined): boolean {
|
||||
function resolveOverrideControlledPaths(
|
||||
env: NodeJS.ProcessEnv,
|
||||
cli: CliConfigOverrides | undefined,
|
||||
speechProviders: RequestedSpeechProviders,
|
||||
persisted: PersistedConfig,
|
||||
): string[] {
|
||||
return Array.from(
|
||||
new Set([
|
||||
...resolveDaemonOverrideControlledPaths(env, cli),
|
||||
...resolveLogOverrideControlledPaths(env),
|
||||
...resolveSpeechOverrideControlledPaths(env, speechProviders),
|
||||
...resolveSpeechOverrideControlledPaths(env, persisted),
|
||||
]),
|
||||
).sort();
|
||||
}
|
||||
@@ -807,67 +807,12 @@ function resolveLogOverrideControlledPaths(env: NodeJS.ProcessEnv): string[] {
|
||||
return paths;
|
||||
}
|
||||
|
||||
function isEnabledSpeechProvider(
|
||||
provider: RequestedSpeechProviders[keyof RequestedSpeechProviders],
|
||||
expected: "local" | "openai",
|
||||
): boolean {
|
||||
return provider.enabled !== false && provider.provider === expected;
|
||||
}
|
||||
|
||||
function resolveSpeechOverrideControlledPaths(
|
||||
env: NodeJS.ProcessEnv,
|
||||
providers: RequestedSpeechProviders,
|
||||
persisted: PersistedConfig,
|
||||
): string[] {
|
||||
const paths: string[] = [];
|
||||
const add = (envName: string, ...configPaths: string[]) => {
|
||||
if (env[envName] !== undefined) paths.push(...configPaths);
|
||||
};
|
||||
|
||||
add("PASEO_DICTATION_ENABLED", "features.dictation.enabled");
|
||||
add("PASEO_DICTATION_STT_PROVIDER", "features.dictation.stt.provider");
|
||||
if (
|
||||
env.PASEO_DICTATION_LOCAL_STT_MODEL !== undefined &&
|
||||
isEnabledSpeechProvider(providers.dictationStt, "local")
|
||||
) {
|
||||
paths.push("features.dictation.stt.model");
|
||||
}
|
||||
add("PASEO_DICTATION_LANGUAGE", "features.dictation.stt.language");
|
||||
add("PASEO_VOICE_MODE_ENABLED", "features.voiceMode.enabled");
|
||||
add("PASEO_VOICE_LLM_PROVIDER", "features.voiceMode.llm.provider");
|
||||
add("PASEO_VOICE_STT_PROVIDER", "features.voiceMode.stt.provider");
|
||||
if (
|
||||
env.PASEO_VOICE_LOCAL_STT_MODEL !== undefined &&
|
||||
isEnabledSpeechProvider(providers.voiceStt, "local")
|
||||
) {
|
||||
paths.push("features.voiceMode.stt.model");
|
||||
}
|
||||
add("PASEO_VOICE_LANGUAGE", "features.voiceMode.stt.language");
|
||||
add("PASEO_VOICE_TURN_DETECTION_PROVIDER", "features.voiceMode.turnDetection.provider");
|
||||
add("PASEO_VOICE_TTS_PROVIDER", "features.voiceMode.tts.provider");
|
||||
if (
|
||||
env.PASEO_VOICE_LOCAL_TTS_MODEL !== undefined &&
|
||||
isEnabledSpeechProvider(providers.voiceTts, "local")
|
||||
) {
|
||||
paths.push("features.voiceMode.tts.model");
|
||||
}
|
||||
add("PASEO_VOICE_LOCAL_TTS_SPEAKER_ID", "features.voiceMode.tts.speakerId");
|
||||
add("PASEO_VOICE_LOCAL_TTS_SPEED", "features.voiceMode.tts.speed");
|
||||
add("PASEO_LOCAL_MODELS_DIR", "providers.local.modelsDir");
|
||||
const openAiDictationStt = isEnabledSpeechProvider(providers.dictationStt, "openai");
|
||||
const openAiVoiceStt = isEnabledSpeechProvider(providers.voiceStt, "openai");
|
||||
if (env.STT_CONFIDENCE_THRESHOLD !== undefined && (openAiDictationStt || openAiVoiceStt)) {
|
||||
paths.push("features.dictation.stt.confidenceThreshold");
|
||||
}
|
||||
if (env.STT_MODEL !== undefined) {
|
||||
if (openAiDictationStt) paths.push("features.dictation.stt.model");
|
||||
if (openAiVoiceStt) paths.push("features.voiceMode.stt.model");
|
||||
}
|
||||
if (isEnabledSpeechProvider(providers.voiceTts, "openai")) {
|
||||
add("TTS_MODEL", "features.voiceMode.tts.model");
|
||||
add("TTS_VOICE", "features.voiceMode.tts.voice");
|
||||
}
|
||||
if (env.PASEO_DICTATION_LANGUAGE !== undefined && env.PASEO_VOICE_LANGUAGE === undefined) {
|
||||
paths.push("features.voiceMode.stt.language");
|
||||
}
|
||||
const paths = Object.keys(readSpeechSettings(persisted, env).envOverrides);
|
||||
if (env.PASEO_VOICE_LLM_PROVIDER !== undefined) paths.push("features.voiceMode.llm.provider");
|
||||
if (env.PASEO_LOCAL_MODELS_DIR !== undefined) paths.push("providers.local.modelsDir");
|
||||
return paths;
|
||||
}
|
||||
|
||||
@@ -1318,9 +1318,9 @@ test("emits disabled voice capability reasons on fresh daemon startup", async ()
|
||||
expect(voice).toBeTruthy();
|
||||
|
||||
expect(voice?.dictation.enabled).toBe(false);
|
||||
expect(voice?.dictation.reason).toBe("Dictation is disabled in daemon config.");
|
||||
expect(voice?.dictation.reason).toBe("Dictation is off. Turn it on in Settings > Voice.");
|
||||
expect(voice?.voice.enabled).toBe(false);
|
||||
expect(voice?.voice.reason).toBe("Realtime voice is disabled in daemon config.");
|
||||
expect(voice?.voice.reason).toBe("Voice mode is off. Turn it on in Settings > Voice.");
|
||||
} finally {
|
||||
await client.close().catch(() => undefined);
|
||||
await isolatedCtx.cleanup();
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { afterEach, describe, expect, test } from "vitest";
|
||||
@@ -6,6 +6,7 @@ import { afterEach, describe, expect, test } from "vitest";
|
||||
import { DaemonConfigStore, applyMutableProviderConfigToOverrides } from "./daemon-config-store.js";
|
||||
import { loadConfig } from "./config.js";
|
||||
import { loadPersistedConfig } from "./persisted-config.js";
|
||||
import { readSpeechSettings } from "./speech/speech-settings.js";
|
||||
import type { PersistedConfig } from "./persisted-config.js";
|
||||
import type { MutableDaemonConfig } from "@getpaseo/protocol/messages";
|
||||
|
||||
@@ -1284,3 +1285,239 @@ describe("DaemonConfigStore reload", () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("DaemonConfigStore speech settings", () => {
|
||||
const tempDirs: string[] = [];
|
||||
|
||||
afterEach(() => {
|
||||
for (const dir of tempDirs) rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function createSpeechStore(initialPersisted?: unknown, env: NodeJS.ProcessEnv = {}) {
|
||||
const paseoHome = mkdtempSync(path.join(tmpdir(), "paseo-daemon-config-speech-"));
|
||||
tempDirs.push(paseoHome);
|
||||
if (initialPersisted) {
|
||||
writeFileSync(
|
||||
path.join(paseoHome, "config.json"),
|
||||
`${JSON.stringify(initialPersisted, null, 2)}\n`,
|
||||
);
|
||||
}
|
||||
const persisted = loadPersistedConfig(paseoHome);
|
||||
const applied: PersistedConfig[] = [];
|
||||
const speech = {
|
||||
read: (next: PersistedConfig) => readSpeechSettings(next, env),
|
||||
apply: (next: PersistedConfig) => applied.push(next),
|
||||
};
|
||||
const store = new DaemonConfigStore(paseoHome, reloadableConfig(persisted), undefined, {
|
||||
speech,
|
||||
reloadSource: {
|
||||
resolve: (next) => ({ mutable: reloadableConfig(next), overrideControlledPaths: [] }),
|
||||
},
|
||||
});
|
||||
return { paseoHome, store, applied, persisted };
|
||||
}
|
||||
|
||||
test("reads, patches, and clears local STT language without losing it on other saves", () => {
|
||||
const { paseoHome, store } = createSpeechStore({
|
||||
features: { dictation: { stt: { language: "fr" } } },
|
||||
});
|
||||
expect(store.get().features?.dictation.stt).toEqual({ provider: "local", language: "fr" });
|
||||
store.patch({ features: { voiceMode: { enabled: true } } });
|
||||
expect(loadPersistedConfig(paseoHome).features?.dictation?.stt).toEqual({
|
||||
provider: "local",
|
||||
language: "fr",
|
||||
});
|
||||
const patched = store.patch({
|
||||
features: { dictation: { stt: { provider: "local", language: "de" } } },
|
||||
});
|
||||
expect(patched.features?.dictation.stt).toEqual({ provider: "local", language: "de" });
|
||||
store.patch({ features: { dictation: { stt: { language: null } } } });
|
||||
expect(loadPersistedConfig(paseoHome).features?.dictation?.stt).toEqual({ provider: "local" });
|
||||
});
|
||||
|
||||
test("reports language env overrides for both local STT sites", () => {
|
||||
const { store } = createSpeechStore(
|
||||
{},
|
||||
{ PASEO_DICTATION_LANGUAGE: "fr", PASEO_VOICE_LANGUAGE: "de" },
|
||||
);
|
||||
expect(store.get().envOverrides).toMatchObject({
|
||||
"features.dictation.stt.language": "PASEO_DICTATION_LANGUAGE",
|
||||
"features.voiceMode.stt.language": "PASEO_VOICE_LANGUAGE",
|
||||
});
|
||||
});
|
||||
|
||||
test("keeps voice mode off until it is turned on", () => {
|
||||
const { paseoHome, store } = createSpeechStore();
|
||||
|
||||
expect(store.get().features?.dictation.enabled).toBe(true);
|
||||
expect(store.get().features?.voiceMode.enabled).toBe(false);
|
||||
|
||||
const result = store.patch({ features: { voiceMode: { enabled: true } } });
|
||||
|
||||
expect(result.features?.voiceMode.enabled).toBe(true);
|
||||
expect(loadPersistedConfig(paseoHome).features?.voiceMode?.enabled).toBe(true);
|
||||
});
|
||||
|
||||
test("stores the OpenAI key without ever returning it", () => {
|
||||
const { paseoHome, store, applied } = createSpeechStore();
|
||||
|
||||
const result = store.patch({
|
||||
features: { dictation: { stt: { provider: "openai", apiKey: " sk-test " } } },
|
||||
});
|
||||
|
||||
expect(result.features?.dictation.stt).toEqual({ provider: "openai", hasApiKey: true });
|
||||
expect(JSON.stringify(result)).not.toContain("sk-test");
|
||||
expect(loadPersistedConfig(paseoHome).features?.dictation?.stt).toMatchObject({
|
||||
apiKey: "sk-test",
|
||||
});
|
||||
expect(applied).toHaveLength(1);
|
||||
});
|
||||
|
||||
test("applies a replaced key even though the visible settings do not change", () => {
|
||||
const { store, applied } = createSpeechStore({
|
||||
version: 1,
|
||||
features: { dictation: { stt: { provider: "openai", apiKey: "sk-old" } } },
|
||||
});
|
||||
|
||||
store.patch({ features: { dictation: { stt: { apiKey: "sk-new" } } } });
|
||||
|
||||
expect(applied.map((next) => next.features?.dictation?.stt)).toEqual([
|
||||
{ provider: "openai", apiKey: "sk-new" },
|
||||
]);
|
||||
});
|
||||
|
||||
test("switching provider clears the previous provider's model", () => {
|
||||
const { paseoHome, store } = createSpeechStore({
|
||||
version: 1,
|
||||
features: { dictation: { stt: { provider: "local", model: "parakeet-tdt-0.6b-v3-int8" } } },
|
||||
});
|
||||
|
||||
const result = store.patch({ features: { dictation: { stt: { provider: "openai" } } } });
|
||||
|
||||
expect(result.features?.dictation.stt).toEqual({ provider: "openai", hasApiKey: false });
|
||||
expect(loadPersistedConfig(paseoHome).features?.dictation?.stt).toEqual({
|
||||
provider: "openai",
|
||||
});
|
||||
});
|
||||
|
||||
test("rejects a local model id outside the catalog without persisting it", () => {
|
||||
const { paseoHome, store, applied } = createSpeechStore();
|
||||
|
||||
expect(() =>
|
||||
store.patch({ features: { voiceMode: { tts: { provider: "local", model: "nope" } } } }),
|
||||
).toThrow("Unknown local voice model: nope");
|
||||
expect(loadPersistedConfig(paseoHome).features?.voiceMode).toBeUndefined();
|
||||
expect(applied).toEqual([]);
|
||||
});
|
||||
|
||||
test("reload applies hand-edited speech settings without a restart", () => {
|
||||
const { paseoHome, store, applied, persisted } = createSpeechStore();
|
||||
writeFileSync(
|
||||
path.join(paseoHome, "config.json"),
|
||||
`${JSON.stringify({ ...persisted, features: { voiceMode: { enabled: false } } }, null, 2)}\n`,
|
||||
);
|
||||
|
||||
const result = store.reload();
|
||||
|
||||
expect(result.appliedPaths).toContain("features.voiceMode");
|
||||
expect(result.restartRequiredPaths).toEqual([]);
|
||||
expect(store.get().features?.voiceMode.enabled).toBe(false);
|
||||
expect(applied).toHaveLength(1);
|
||||
});
|
||||
test("merges same-provider fields and replaces the whole variant when provider changes", () => {
|
||||
const { paseoHome, store } = createSpeechStore();
|
||||
store.patch({
|
||||
features: {
|
||||
voiceMode: {
|
||||
stt: { provider: "openai", apiKey: "secret", baseUrl: "https://site/v1", model: "first" },
|
||||
},
|
||||
},
|
||||
});
|
||||
store.patch({ features: { voiceMode: { stt: { model: "second", baseUrl: null } } } });
|
||||
expect(loadPersistedConfig(paseoHome).features?.voiceMode?.stt).toEqual({
|
||||
provider: "openai",
|
||||
apiKey: "secret",
|
||||
model: "second",
|
||||
});
|
||||
store.patch({ features: { voiceMode: { stt: { provider: "openai", model: "third" } } } });
|
||||
expect(store.get().features?.voiceMode.stt).toEqual({
|
||||
provider: "openai",
|
||||
model: "third",
|
||||
hasApiKey: true,
|
||||
});
|
||||
store.patch({ features: { voiceMode: { stt: { apiKey: "new-secret" } } } });
|
||||
store.patch({ features: { voiceMode: { stt: { provider: "local" } } } });
|
||||
store.patch({ features: { voiceMode: { stt: { provider: "openai" } } } });
|
||||
expect(store.get().features?.voiceMode.stt).toEqual({ provider: "openai", hasApiKey: false });
|
||||
});
|
||||
|
||||
test("keeps a saved OpenAI key when a full variant patch omits the write-only key", () => {
|
||||
const { paseoHome, store } = createSpeechStore({
|
||||
features: { voiceMode: { stt: { provider: "openai", apiKey: "saved-secret" } } },
|
||||
});
|
||||
const result = store.patch({
|
||||
features: { voiceMode: { stt: { provider: "openai", model: "whisper-1" } } },
|
||||
});
|
||||
expect(result.features?.voiceMode.stt).toEqual({
|
||||
provider: "openai",
|
||||
model: "whisper-1",
|
||||
hasApiKey: true,
|
||||
});
|
||||
expect(loadPersistedConfig(paseoHome).features?.voiceMode?.stt).toEqual({
|
||||
provider: "openai",
|
||||
apiKey: "saved-secret",
|
||||
model: "whisper-1",
|
||||
});
|
||||
expect(JSON.stringify(result)).not.toContain("saved-secret");
|
||||
});
|
||||
|
||||
test("loads a legacy host, redacts its key and writes the normalized config after any save", () => {
|
||||
const { paseoHome, store } = createSpeechStore({
|
||||
providers: { openai: { apiKey: "legacy-secret", stt: { baseUrl: "https://old/v1" } } },
|
||||
features: { voiceMode: { enabled: true, stt: { provider: "openai" } } },
|
||||
});
|
||||
expect(store.get().features?.voiceMode.stt).toEqual({
|
||||
provider: "openai",
|
||||
baseUrl: "https://old/v1",
|
||||
hasApiKey: true,
|
||||
});
|
||||
expect(JSON.stringify(store.get())).not.toContain("legacy-secret");
|
||||
store.patch({ autoArchiveAfterMerge: true });
|
||||
const disk = JSON.parse(readFileSync(path.join(paseoHome, "config.json"), "utf8"));
|
||||
expect(disk.providers).not.toHaveProperty("openai");
|
||||
expect(disk.features.voiceMode.stt.apiKey).toBe("legacy-secret");
|
||||
});
|
||||
|
||||
test("reports env overrides and removes fallback provenance after saving a key", () => {
|
||||
const { store } = createSpeechStore(
|
||||
{ features: { voiceMode: { stt: { provider: "openai", model: "configured" } } } },
|
||||
{
|
||||
PASEO_VOICE_STT_PROVIDER: "local",
|
||||
OPENAI_API_KEY: "env-secret",
|
||||
},
|
||||
);
|
||||
expect(store.get().features?.voiceMode.stt.provider).toBe("local");
|
||||
expect(store.get().envOverrides).toEqual({
|
||||
"features.voiceMode.stt.provider": "PASEO_VOICE_STT_PROVIDER",
|
||||
});
|
||||
const other = createSpeechStore(
|
||||
{ features: { voiceMode: { stt: { provider: "openai" } } } },
|
||||
{
|
||||
OPENAI_API_KEY: "env-secret",
|
||||
OPENAI_STT_BASE_URL: "https://env/v1",
|
||||
STT_MODEL: "override",
|
||||
},
|
||||
).store;
|
||||
expect(other.get().envOverrides).toEqual({
|
||||
"features.voiceMode.stt.apiKey": "OPENAI_API_KEY",
|
||||
"features.voiceMode.stt.baseUrl": "OPENAI_STT_BASE_URL",
|
||||
"features.voiceMode.stt.model": "STT_MODEL",
|
||||
});
|
||||
expect(other.get().features?.voiceMode.stt.hasApiKey).toBe(false);
|
||||
other.patch({
|
||||
features: { voiceMode: { stt: { apiKey: "saved-secret", baseUrl: "https://saved/v1" } } },
|
||||
});
|
||||
expect(other.get().envOverrides).toEqual({ "features.voiceMode.stt.model": "STT_MODEL" });
|
||||
expect(JSON.stringify(other.get())).not.toContain("secret");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -9,6 +9,8 @@ import {
|
||||
MutableDaemonConfigPatchSchema,
|
||||
} from "@getpaseo/protocol/messages";
|
||||
import type { AgentSkillSelection } from "@getpaseo/protocol/messages";
|
||||
import type { SpeechSettingsPatch } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
import { mergeSpeechSettingsPatch, type SpeechSettingsOwner } from "./speech/speech-settings.js";
|
||||
|
||||
export type { MutableDaemonConfig, MutableDaemonConfigPatch } from "@getpaseo/protocol/messages";
|
||||
|
||||
@@ -32,8 +34,11 @@ interface SupportedMutableConfigPatch {
|
||||
skills?: MutableDaemonConfig["skills"];
|
||||
pluginsEnabled?: boolean;
|
||||
plugins?: MutableDaemonConfig["plugins"];
|
||||
features?: SpeechSettingsPatch;
|
||||
}
|
||||
|
||||
const SPEECH_PERSISTED_PATHS = ["features.dictation", "features.voiceMode"];
|
||||
|
||||
interface LoggerLike {
|
||||
child(bindings: Record<string, unknown>): LoggerLike;
|
||||
info(...args: unknown[]): void;
|
||||
@@ -296,6 +301,7 @@ function pickSupportedPatchFields(patch: MutableDaemonConfigPatch): SupportedMut
|
||||
...(patch.agentProfiles !== undefined ? { agentProfiles: patch.agentProfiles } : {}),
|
||||
...(patch.pluginsEnabled !== undefined ? { pluginsEnabled: patch.pluginsEnabled } : {}),
|
||||
...(patch.plugins !== undefined ? { plugins: patch.plugins } : {}),
|
||||
...(patch.features !== undefined ? { features: patch.features } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -338,6 +344,7 @@ export class DaemonConfigStore {
|
||||
private readonly relayEnabledMutable: boolean;
|
||||
private readonly reloadSource: DaemonConfigReloadSource | undefined;
|
||||
private readonly startupPersisted: PersistedConfig;
|
||||
private readonly speechOwner: SpeechSettingsOwner | undefined;
|
||||
private lastKnownPersisted: PersistedConfig;
|
||||
|
||||
constructor(
|
||||
@@ -348,18 +355,21 @@ export class DaemonConfigStore {
|
||||
relayEnabledMutable?: boolean;
|
||||
reloadSource?: DaemonConfigReloadSource;
|
||||
startupPersisted?: PersistedConfig;
|
||||
speech?: SpeechSettingsOwner;
|
||||
} = {},
|
||||
) {
|
||||
this.paseoHome = paseoHome;
|
||||
this.logger = getLogger(logger);
|
||||
this.relayEnabledMutable = options.relayEnabledMutable ?? true;
|
||||
this.reloadSource = options.reloadSource;
|
||||
this.speechOwner = options.speech;
|
||||
this.startupPersisted = options.startupPersisted ?? loadPersistedConfig(paseoHome, this.logger);
|
||||
this.lastKnownPersisted = this.startupPersisted;
|
||||
this.current = MutableDaemonConfigSchema.parse({
|
||||
...initial,
|
||||
relay: initial.relay ?? { enabled: true },
|
||||
...this.readSpeech(this.startupPersisted),
|
||||
});
|
||||
this.relayEnabledMutable = options.relayEnabledMutable ?? true;
|
||||
this.reloadSource = options.reloadSource;
|
||||
this.startupPersisted = options.startupPersisted ?? loadPersistedConfig(paseoHome, this.logger);
|
||||
this.lastKnownPersisted = this.startupPersisted;
|
||||
}
|
||||
|
||||
public get(): MutableDaemonConfig {
|
||||
@@ -381,9 +391,11 @@ export class DaemonConfigStore {
|
||||
"Relay is controlled by a daemon launch override. Remove PASEO_RELAY_ENABLED or the relay CLI flag before changing it here.",
|
||||
);
|
||||
}
|
||||
const { removeProviders = [], ...configPatch } = parsedPatch;
|
||||
const { removeProviders = [], ...patch } = parsedPatch;
|
||||
const { features: speechPatch, ...configPatch } = patch;
|
||||
const removedProviders = Array.from(new Set(removeProviders));
|
||||
const merged = deepMerge(this.current, configPatch);
|
||||
if (speechPatch !== undefined) Object.assign(merged, this.readPatchedSpeech(speechPatch));
|
||||
if (parsedPatch.skills?.selection !== undefined) {
|
||||
merged.skills = { selection: parsedPatch.skills.selection };
|
||||
}
|
||||
@@ -397,30 +409,51 @@ export class DaemonConfigStore {
|
||||
|
||||
const configChanged = !isEqualValue(this.current, next);
|
||||
|
||||
if (!configChanged && removedProviders.length === 0) {
|
||||
// A replaced API key leaves the speech view unchanged, so a speech patch always persists.
|
||||
if (!configChanged && removedProviders.length === 0 && speechPatch === undefined) {
|
||||
return this.current;
|
||||
}
|
||||
|
||||
const { previous: persistedBeforePatch, knownNext } = this.persistConfig(
|
||||
configPatch,
|
||||
patch,
|
||||
removedProviders,
|
||||
);
|
||||
if (!configChanged) {
|
||||
this.lastKnownPersisted = knownNext;
|
||||
return this.current;
|
||||
}
|
||||
|
||||
try {
|
||||
this.applyReplacement(next, { removedProviders });
|
||||
this.lastKnownPersisted = knownNext;
|
||||
} catch (error) {
|
||||
savePersistedConfig(this.paseoHome, persistedBeforePatch, this.logger);
|
||||
throw error;
|
||||
const speechChanged = this.speechPersistedChanged(this.lastKnownPersisted, knownNext);
|
||||
if (configChanged) {
|
||||
try {
|
||||
this.applyReplacement(next, { removedProviders });
|
||||
} catch (error) {
|
||||
savePersistedConfig(this.paseoHome, persistedBeforePatch, this.logger);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
this.lastKnownPersisted = knownNext;
|
||||
if (speechChanged) this.speechOwner?.apply(knownNext);
|
||||
|
||||
return this.current;
|
||||
}
|
||||
|
||||
private readSpeech(
|
||||
persisted: PersistedConfig,
|
||||
): Pick<MutableDaemonConfig, "features" | "envOverrides"> {
|
||||
return this.speechOwner ? this.speechOwner.read(persisted) : {};
|
||||
}
|
||||
|
||||
private readPatchedSpeech(
|
||||
speechPatch: SpeechSettingsPatch,
|
||||
): Pick<MutableDaemonConfig, "features" | "envOverrides"> {
|
||||
if (!this.speechOwner) {
|
||||
throw new Error("Speech settings are unavailable for this daemon instance");
|
||||
}
|
||||
return this.speechOwner.read(mergeSpeechSettingsPatch(this.lastKnownPersisted, speechPatch));
|
||||
}
|
||||
|
||||
private speechPersistedChanged(previous: PersistedConfig, next: PersistedConfig): boolean {
|
||||
return diffPaths(previous, next).some((path) =>
|
||||
SPEECH_PERSISTED_PATHS.some((owner) => pathBelongsTo(path, owner)),
|
||||
);
|
||||
}
|
||||
|
||||
public reload(): DaemonConfigReloadResult {
|
||||
if (!this.reloadSource) {
|
||||
throw new Error("Daemon config reload is unavailable for this daemon instance");
|
||||
@@ -433,6 +466,7 @@ export class DaemonConfigStore {
|
||||
const desired = MutableDaemonConfigSchema.parse({
|
||||
...resolved.mutable,
|
||||
plugins: this.current.plugins,
|
||||
...this.readSpeech(persisted),
|
||||
});
|
||||
const changedSinceLastApply = diffPaths(this.lastKnownPersisted, persisted);
|
||||
const overrideControlledPaths = compactOwnedPaths(
|
||||
@@ -441,7 +475,7 @@ export class DaemonConfigStore {
|
||||
),
|
||||
resolved.overrideControlledPaths,
|
||||
);
|
||||
const appliedPaths = RELOADABLE_PATHS.filter((persistedPath) => {
|
||||
const appliedPaths: string[] = RELOADABLE_PATHS.filter((persistedPath) => {
|
||||
if (resolved.overrideControlledPaths.some((owner) => pathBelongsTo(persistedPath, owner))) {
|
||||
return false;
|
||||
}
|
||||
@@ -458,6 +492,8 @@ export class DaemonConfigStore {
|
||||
diffPaths(this.startupPersisted, persisted).filter((path) => {
|
||||
if (path === "$schema" || path === "version") return false;
|
||||
if (RELOADABLE_PATHS.some((owner) => pathBelongsTo(path, owner))) return false;
|
||||
if (this.speechOwner && SPEECH_PERSISTED_PATHS.some((owner) => pathBelongsTo(path, owner)))
|
||||
return false;
|
||||
return !resolved.overrideControlledPaths.some((owner) => pathBelongsTo(path, owner));
|
||||
}),
|
||||
[],
|
||||
@@ -466,8 +502,17 @@ export class DaemonConfigStore {
|
||||
const removedProviders = Object.keys(this.current.providers).filter(
|
||||
(provider) => !(provider in desired.providers),
|
||||
);
|
||||
const speechChanged = this.speechPersistedChanged(this.lastKnownPersisted, persisted);
|
||||
this.applyReplacement(desired, { removedProviders });
|
||||
this.lastKnownPersisted = persisted;
|
||||
if (speechChanged && this.speechOwner) {
|
||||
this.speechOwner.apply(persisted);
|
||||
appliedPaths.push(
|
||||
...SPEECH_PERSISTED_PATHS.filter((owner) =>
|
||||
changedSinceLastApply.some((path) => pathBelongsTo(path, owner)),
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
appliedPaths: [...appliedPaths].sort(),
|
||||
@@ -604,8 +649,11 @@ function mergeMutablePatchIntoPersistedConfig(params: {
|
||||
const { persisted, patch, removeProviders, persistRelayEnabled } = params;
|
||||
const daemon = mergeMutableDaemonPatch(persisted.daemon, patch, persistRelayEnabled);
|
||||
const agents = mergeMutableAgentPatch(persisted.agents, patch, removeProviders);
|
||||
const withSpeech = patch.features
|
||||
? mergeSpeechSettingsPatch(persisted, patch.features)
|
||||
: persisted;
|
||||
return {
|
||||
...persisted,
|
||||
...withSpeech,
|
||||
...(patch.pluginsEnabled !== undefined ? { pluginsEnabled: patch.pluginsEnabled } : {}),
|
||||
...(patch.plugins !== undefined ? { plugins: patch.plugins } : {}),
|
||||
...(daemon ? { daemon } : { daemon: undefined }),
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
import { mkdtempSync, writeFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import { describe, expect, it, vi, beforeEach, afterEach } from "vitest";
|
||||
import { EventEmitter } from "node:events";
|
||||
import pino from "pino";
|
||||
|
||||
import { DictationStreamManager } from "./dictation-stream-manager.js";
|
||||
import { PersistedConfigSchema } from "../persisted-config.js";
|
||||
import { loadPersistedConfig } from "../persisted-config.js";
|
||||
import { resolveSpeechConfig } from "../speech/speech-config-resolver.js";
|
||||
import type {
|
||||
SpeechToTextProvider,
|
||||
@@ -131,12 +134,18 @@ describe("DictationStreamManager (provider-agnostic provider)", () => {
|
||||
env?: NodeJS.ProcessEnv;
|
||||
persisted?: unknown;
|
||||
}): string {
|
||||
const result = resolveSpeechConfig({
|
||||
paseoHome: "/tmp/paseo-home",
|
||||
env: params.env ?? ({} as NodeJS.ProcessEnv),
|
||||
persisted: PersistedConfigSchema.parse(params.persisted ?? {}),
|
||||
});
|
||||
return result.speech.sttLanguages.dictation;
|
||||
const home = mkdtempSync(path.join(tmpdir(), "stt-language-"));
|
||||
try {
|
||||
writeFileSync(path.join(home, "config.json"), JSON.stringify(params.persisted ?? {}));
|
||||
const result = resolveSpeechConfig({
|
||||
paseoHome: home,
|
||||
env: params.env ?? {},
|
||||
persisted: loadPersistedConfig(home),
|
||||
});
|
||||
return result.speech.sttLanguages.dictation;
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function startWithResolvedDictationLanguage(params: {
|
||||
|
||||
@@ -43,3 +43,19 @@ test("configuration edits validate before writing and preserve unrelated setting
|
||||
await rm(root, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("configuration paths edit fields inside speech provider variants", async () => {
|
||||
const home = await mkdtemp(path.join(tmpdir(), "paseo-speech-edit-"));
|
||||
try {
|
||||
editPersistedConfig(home, "features.voiceMode.stt", { value: { provider: "openai" } });
|
||||
editPersistedConfig(home, "features.voiceMode.stt.apiKey", { value: "site-secret" });
|
||||
const config = readPersistedConfig(home);
|
||||
expect(getPersistedConfigValue(config, "features.voiceMode.stt.apiKey")).toBe("site-secret");
|
||||
editPersistedConfig(home, "features.voiceMode.stt.apiKey", { unset: true });
|
||||
expect(
|
||||
getPersistedConfigValue(readPersistedConfig(home), "features.voiceMode.stt.apiKey"),
|
||||
).toBeUndefined();
|
||||
} finally {
|
||||
await rm(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { resolveSpeechConfig } from "./speech/speech-config-resolver.js";
|
||||
import {
|
||||
chmodSync,
|
||||
lstatSync,
|
||||
@@ -149,26 +150,33 @@ describe("PersistedConfigSchema worktrees config", () => {
|
||||
});
|
||||
|
||||
describe("PersistedConfigSchema provider credentials", () => {
|
||||
test("accepts separate OpenAI STT and TTS credentials", () => {
|
||||
test("accepts separate inline STT and TTS credentials", () => {
|
||||
const parsed = PersistedConfigSchema.parse({
|
||||
providers: {
|
||||
openai: {
|
||||
features: {
|
||||
voiceMode: {
|
||||
stt: {
|
||||
provider: "openai",
|
||||
apiKey: " stt-secret ",
|
||||
baseUrl: " https://stt.example.com/v1 ",
|
||||
},
|
||||
tts: {
|
||||
provider: "openai",
|
||||
apiKey: " tts-secret ",
|
||||
baseUrl: " https://tts.example.com/v1 ",
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
expect(parsed.providers?.openai?.stt?.apiKey).toBe("stt-secret");
|
||||
expect(parsed.providers?.openai?.stt?.baseUrl).toBe("https://stt.example.com/v1");
|
||||
expect(parsed.providers?.openai?.tts?.apiKey).toBe("tts-secret");
|
||||
expect(parsed.providers?.openai?.tts?.baseUrl).toBe("https://tts.example.com/v1");
|
||||
expect(parsed.features?.voiceMode?.stt).toEqual({
|
||||
provider: "openai",
|
||||
apiKey: "stt-secret",
|
||||
baseUrl: "https://stt.example.com/v1",
|
||||
});
|
||||
expect(parsed.features?.voiceMode?.tts).toEqual({
|
||||
provider: "openai",
|
||||
apiKey: "tts-secret",
|
||||
baseUrl: "https://tts.example.com/v1",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -660,19 +668,21 @@ describe("PersistedConfigSchema voice mode config", () => {
|
||||
features: {
|
||||
dictation: {
|
||||
stt: {
|
||||
provider: "openai",
|
||||
language: " fr ",
|
||||
},
|
||||
},
|
||||
voiceMode: {
|
||||
stt: {
|
||||
provider: "openai",
|
||||
language: " de ",
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
expect(parsed.features?.dictation?.stt?.language).toBe("fr");
|
||||
expect(parsed.features?.voiceMode?.stt?.language).toBe("de");
|
||||
expect(parsed.features?.dictation?.stt).toMatchObject({ language: "fr" });
|
||||
expect(parsed.features?.voiceMode?.stt).toMatchObject({ language: "de" });
|
||||
});
|
||||
|
||||
test("accepts a voice and model from a self-hosted OpenAI-compatible server", () => {
|
||||
@@ -685,7 +695,7 @@ describe("PersistedConfigSchema voice mode config", () => {
|
||||
});
|
||||
|
||||
expect(parsed.features?.voiceMode?.tts?.model).toBe("kokoro-v1:Q8_0");
|
||||
expect(parsed.features?.voiceMode?.tts?.voice).toBe("aiden");
|
||||
expect(parsed.features?.voiceMode?.tts).toMatchObject({ voice: "aiden" });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -754,10 +764,7 @@ describe("loadPersistedConfig", () => {
|
||||
|
||||
const config = loadPersistedConfig(home);
|
||||
|
||||
expect(config.providers?.openai?.apiKey).toBe("global-key");
|
||||
expect((config.providers?.openai as Record<string, unknown>)?.voice).toBeUndefined();
|
||||
expect(config.providers?.openai?.stt).toBeUndefined();
|
||||
expect(config.providers?.openai?.tts).toBeUndefined();
|
||||
expect(config.providers).not.toHaveProperty("openai");
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
@@ -852,11 +859,7 @@ describe.skipIf(process.platform === "win32")("persisted config file permissions
|
||||
const home = path.join(parent, "home");
|
||||
try {
|
||||
savePersistedConfig(home, {
|
||||
providers: {
|
||||
openai: {
|
||||
apiKey: "secret",
|
||||
},
|
||||
},
|
||||
features: { dictation: { stt: { provider: "openai", apiKey: "secret" } } },
|
||||
});
|
||||
|
||||
expect(modeOf(home)).toBe(PRIVATE_DIRECTORY_MODE);
|
||||
@@ -898,3 +901,173 @@ describe.skipIf(process.platform === "win32")(
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
describe("speech provider config normalization", () => {
|
||||
test.each([undefined, "local"])(
|
||||
"preserves local STT language on load and save (provider: %s)",
|
||||
(provider) => {
|
||||
const home = createTempHome();
|
||||
try {
|
||||
writeFileSync(
|
||||
path.join(home, "config.json"),
|
||||
JSON.stringify({
|
||||
features: { dictation: { stt: { provider, language: "fr" } } },
|
||||
}),
|
||||
);
|
||||
const loaded = loadPersistedConfig(home);
|
||||
expect(loaded.features?.dictation?.stt).toEqual({ provider: "local", language: "fr" });
|
||||
expect(
|
||||
resolveSpeechConfig({ paseoHome: home, env: {}, persisted: loaded }).speech.sttLanguages
|
||||
.dictation,
|
||||
).toBe("fr");
|
||||
savePersistedConfig(home, loaded);
|
||||
expect(loadPersistedConfig(home).features?.dictation?.stt).toEqual({
|
||||
provider: "local",
|
||||
language: "fr",
|
||||
});
|
||||
expect(
|
||||
JSON.parse(readFileSync(path.join(home, "config.json"), "utf8")).features.dictation.stt,
|
||||
).toEqual({ provider: "local", language: "fr" });
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
},
|
||||
);
|
||||
|
||||
test("loads and resolves legacy fields ignored by each site's provider", () => {
|
||||
const home = mkdtempSync(path.join(tmpdir(), "speech-unused-options-"));
|
||||
try {
|
||||
const legacy = {
|
||||
features: {
|
||||
dictation: { stt: { provider: "local", language: "fr", confidenceThreshold: -1 } },
|
||||
voiceMode: {
|
||||
enabled: true,
|
||||
tts: { provider: "local", voice: "alloy", speakerId: 3, speed: 1.2 },
|
||||
},
|
||||
},
|
||||
};
|
||||
writeFileSync(path.join(home, "config.json"), JSON.stringify(legacy));
|
||||
const local = loadPersistedConfig(home);
|
||||
expect(local.features?.dictation?.stt).toEqual({ provider: "local", language: "fr" });
|
||||
expect(local.features?.voiceMode?.tts).toEqual({
|
||||
provider: "local",
|
||||
speakerId: 3,
|
||||
speed: 1.2,
|
||||
});
|
||||
expect(
|
||||
resolveSpeechConfig({ paseoHome: home, env: {}, persisted: local }).speech.local?.models,
|
||||
).toMatchObject({ voiceTtsSpeakerId: 3, voiceTtsSpeed: 1.2 });
|
||||
writeFileSync(
|
||||
path.join(home, "config.json"),
|
||||
JSON.stringify({
|
||||
features: {
|
||||
voiceMode: {
|
||||
enabled: true,
|
||||
tts: { provider: "openai", apiKey: "key", voice: "alloy", speakerId: 3, speed: 1.2 },
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
const openai = loadPersistedConfig(home);
|
||||
expect(openai.features?.voiceMode?.tts).toEqual({
|
||||
provider: "openai",
|
||||
apiKey: "key",
|
||||
voice: "alloy",
|
||||
});
|
||||
expect(
|
||||
resolveSpeechConfig({ paseoHome: home, env: {}, persisted: openai }).openai?.tts,
|
||||
).toMatchObject({ apiKey: "key", voice: "alloy" });
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("warns with the path, never the discarded legacy key", () => {
|
||||
const home = mkdtempSync(path.join(tmpdir(), "speech-warning-"));
|
||||
try {
|
||||
writeFileSync(
|
||||
path.join(home, "config.json"),
|
||||
JSON.stringify({
|
||||
providers: { openai: { apiKey: "discarded-secret" } },
|
||||
features: { voiceMode: { enabled: true, tts: { provider: "local" } } },
|
||||
}),
|
||||
);
|
||||
const warnings: unknown[][] = [];
|
||||
const logger = {
|
||||
child: () => logger,
|
||||
info: () => {},
|
||||
warn: (...args: unknown[]) => {
|
||||
warnings.push(args);
|
||||
},
|
||||
};
|
||||
const config = loadPersistedConfig(home, logger);
|
||||
expect(config.providers).not.toHaveProperty("openai");
|
||||
expect(warnings).toHaveLength(1);
|
||||
expect(JSON.stringify(warnings)).toContain("providers.openai.apiKey");
|
||||
expect(JSON.stringify(warnings)).toContain("no use site in config.json selects OpenAI");
|
||||
const resolved = resolveSpeechConfig({
|
||||
paseoHome: home,
|
||||
persisted: config,
|
||||
env: { PASEO_VOICE_TTS_PROVIDER: "openai" },
|
||||
});
|
||||
expect(resolved.speech.providers.voiceTts.provider).toBe("openai");
|
||||
expect(resolved.openai).toBeUndefined();
|
||||
expect(JSON.stringify(warnings)).not.toContain("discarded-secret");
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test("folds legacy endpoint credentials into OpenAI sites on load and saves inline", () => {
|
||||
const home = mkdtempSync(path.join(tmpdir(), "speech-normalize-"));
|
||||
try {
|
||||
writeFileSync(
|
||||
path.join(home, "config.json"),
|
||||
JSON.stringify({
|
||||
providers: {
|
||||
openai: {
|
||||
apiKey: "global-key",
|
||||
baseUrl: "https://global/v1",
|
||||
stt: { apiKey: "stt-key" },
|
||||
},
|
||||
},
|
||||
features: {
|
||||
dictation: { stt: { model: "whisper-base", language: "en" } },
|
||||
voiceMode: {
|
||||
enabled: true,
|
||||
stt: { provider: "OPENAI", baseUrl: "https://site/v1" },
|
||||
tts: { provider: "openai" },
|
||||
},
|
||||
},
|
||||
}),
|
||||
);
|
||||
const config = loadPersistedConfig(home);
|
||||
expect(config.features?.dictation?.stt).toEqual({
|
||||
provider: "local",
|
||||
model: "whisper-base",
|
||||
language: "en",
|
||||
});
|
||||
expect(config.features?.voiceMode?.stt).toEqual({
|
||||
provider: "openai",
|
||||
apiKey: "stt-key",
|
||||
baseUrl: "https://site/v1",
|
||||
});
|
||||
expect(config.features?.voiceMode?.tts).toEqual({
|
||||
provider: "openai",
|
||||
apiKey: "global-key",
|
||||
baseUrl: "https://global/v1",
|
||||
});
|
||||
expect(config.providers).not.toHaveProperty("openai");
|
||||
const resolved = resolveSpeechConfig({ paseoHome: home, env: {}, persisted: config });
|
||||
expect(resolved.openai?.stt?.apiKey).toBe("stt-key");
|
||||
expect(resolved.speech.providers.voiceStt).toMatchObject({
|
||||
provider: "openai",
|
||||
enabled: true,
|
||||
});
|
||||
savePersistedConfig(home, config);
|
||||
expect(JSON.parse(readFileSync(path.join(home, "config.json"), "utf8"))).toEqual(config);
|
||||
} finally {
|
||||
rmSync(home, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -49,22 +49,6 @@ const LogConfigSchema = z
|
||||
})
|
||||
.strict();
|
||||
|
||||
const OpenAiSpeechEndpointSchema = z
|
||||
.object({
|
||||
apiKey: z.string().trim().min(1).optional(),
|
||||
baseUrl: z.string().trim().min(1).optional(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
const OpenAiProviderSchema = z
|
||||
.object({
|
||||
apiKey: z.string().min(1).optional(),
|
||||
baseUrl: z.string().trim().min(1).optional(),
|
||||
stt: OpenAiSpeechEndpointSchema.optional(),
|
||||
tts: OpenAiSpeechEndpointSchema.optional(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
const LocalSpeechProviderSchema = z
|
||||
.object({
|
||||
modelsDir: z.string().min(1).optional(),
|
||||
@@ -73,7 +57,6 @@ const LocalSpeechProviderSchema = z
|
||||
|
||||
const ProvidersSchema = z
|
||||
.object({
|
||||
openai: OpenAiProviderSchema.optional(),
|
||||
local: LocalSpeechProviderSchema.optional(),
|
||||
})
|
||||
.strict();
|
||||
@@ -101,18 +84,54 @@ const SpeechProviderIdSchema = z
|
||||
.toLowerCase()
|
||||
.pipe(z.enum(["openai", "local"]));
|
||||
|
||||
const SpeechModelSchema = z.string().min(1).optional();
|
||||
const OpenAiConnectionFields = {
|
||||
apiKey: z.string().trim().min(1).optional(),
|
||||
baseUrl: z.string().trim().min(1).optional(),
|
||||
};
|
||||
|
||||
export const SpeechSttConfigSchema = z.discriminatedUnion("provider", [
|
||||
z
|
||||
.object({
|
||||
provider: z.literal("local"),
|
||||
model: SpeechModelSchema,
|
||||
language: z.string().trim().min(1).optional(),
|
||||
})
|
||||
.strict(),
|
||||
z
|
||||
.object({
|
||||
provider: z.literal("openai"),
|
||||
...OpenAiConnectionFields,
|
||||
model: SpeechModelSchema,
|
||||
language: z.string().trim().min(1).optional(),
|
||||
confidenceThreshold: z.number().optional(),
|
||||
})
|
||||
.strict(),
|
||||
]);
|
||||
|
||||
export const SpeechTtsConfigSchema = z.discriminatedUnion("provider", [
|
||||
z
|
||||
.object({
|
||||
provider: z.literal("local"),
|
||||
model: SpeechModelSchema,
|
||||
speakerId: z.number().int().optional(),
|
||||
speed: z.number().optional(),
|
||||
})
|
||||
.strict(),
|
||||
z
|
||||
.object({
|
||||
provider: z.literal("openai"),
|
||||
...OpenAiConnectionFields,
|
||||
model: SpeechModelSchema,
|
||||
voice: z.string().trim().min(1).optional(),
|
||||
})
|
||||
.strict(),
|
||||
]);
|
||||
|
||||
const FeatureDictationSchema = z
|
||||
.object({
|
||||
enabled: z.boolean().optional(),
|
||||
stt: z
|
||||
.object({
|
||||
provider: SpeechProviderIdSchema.optional(),
|
||||
model: z.string().min(1).optional(),
|
||||
language: z.string().trim().min(1).optional(),
|
||||
confidenceThreshold: z.number().optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
stt: SpeechSttConfigSchema.optional(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
@@ -126,31 +145,14 @@ const FeatureVoiceModeSchema = z
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
stt: z
|
||||
.object({
|
||||
provider: SpeechProviderIdSchema.optional(),
|
||||
model: z.string().min(1).optional(),
|
||||
language: z.string().trim().min(1).optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
stt: SpeechSttConfigSchema.optional(),
|
||||
turnDetection: z
|
||||
.object({
|
||||
provider: SpeechProviderIdSchema.optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
tts: z
|
||||
.object({
|
||||
provider: SpeechProviderIdSchema.optional(),
|
||||
model: z.string().min(1).optional(),
|
||||
// Free-form: self-hosted OpenAI-compatible servers expose their own voice names.
|
||||
voice: z.string().trim().min(1).optional(),
|
||||
speakerId: z.number().int().optional(),
|
||||
speed: z.number().optional(),
|
||||
})
|
||||
.strict()
|
||||
.optional(),
|
||||
tts: SpeechTtsConfigSchema.optional(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
@@ -387,6 +389,7 @@ const DEFAULT_PERSISTED_CONFIG = PersistedConfigSchema.parse({
|
||||
interface LoggerLike {
|
||||
child(bindings: Record<string, unknown>): LoggerLike;
|
||||
info(...args: unknown[]): void;
|
||||
warn?(...args: unknown[]): void;
|
||||
}
|
||||
|
||||
function getConfigPath(paseoHome: string): string {
|
||||
@@ -397,45 +400,96 @@ function getLogger(logger: LoggerLike | undefined): LoggerLike | undefined {
|
||||
return logger?.child({ module: "config" });
|
||||
}
|
||||
|
||||
// Removed config fields are stripped before parsing so the strict schema does not
|
||||
// reject a config written by an older release. The stripped values are discarded,
|
||||
// not migrated — there is no back-compat for the removed `providers.openai.voice`
|
||||
// block (use `providers.openai.stt` / `providers.openai.tts`).
|
||||
function stripRemovedConfigFields(parsed: unknown): unknown {
|
||||
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
|
||||
return parsed;
|
||||
function isConfigRecord(value: unknown): value is Record<string, unknown> {
|
||||
return value !== null && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
function configRecord(value: unknown): Record<string, unknown> | undefined {
|
||||
return isConfigRecord(value) ? value : undefined;
|
||||
}
|
||||
|
||||
// COMPAT(speechProviderConfig): added in v0.12.0, remove after 2027-04-26.
|
||||
// Fold shared credentials once at the file boundary; consumers see only the strict
|
||||
// per-site shape. Never include credential values in diagnostics.
|
||||
function normalizeSpeechProviderConfig(parsed: unknown, logger?: LoggerLike): unknown {
|
||||
if (!isConfigRecord(parsed)) return parsed;
|
||||
const root = structuredClone(parsed);
|
||||
const providers = configRecord(root.providers);
|
||||
const legacy = configRecord(providers?.openai);
|
||||
const features = configRecord(root.features);
|
||||
const foldedKeys = new Set<string>();
|
||||
for (const [feature, kind] of [
|
||||
["dictation", "stt"],
|
||||
["voiceMode", "stt"],
|
||||
["voiceMode", "tts"],
|
||||
] as const) {
|
||||
const site = configRecord(configRecord(features?.[feature])?.[kind]);
|
||||
if (site) normalizeSpeechSite(site, kind, legacy, foldedKeys);
|
||||
}
|
||||
|
||||
const root = { ...(parsed as Record<string, unknown>) };
|
||||
const providers = root.providers;
|
||||
if (!providers || typeof providers !== "object" || Array.isArray(providers)) {
|
||||
return root;
|
||||
if (legacy) warnDiscardedSpeechKeys(legacy, foldedKeys, logger);
|
||||
if (providers) {
|
||||
delete providers.openai;
|
||||
const local = configRecord(providers.local);
|
||||
if (local) delete local.autoDownload;
|
||||
}
|
||||
|
||||
const providersRecord = { ...(providers as Record<string, unknown>) };
|
||||
|
||||
const local = providersRecord.local;
|
||||
if (local && typeof local === "object" && !Array.isArray(local)) {
|
||||
const localRecord = { ...(local as Record<string, unknown>) };
|
||||
delete localRecord.autoDownload;
|
||||
providersRecord.local = localRecord;
|
||||
}
|
||||
|
||||
const openai = providersRecord.openai;
|
||||
if (openai && typeof openai === "object" && !Array.isArray(openai)) {
|
||||
const openaiRecord = { ...(openai as Record<string, unknown>) };
|
||||
// COMPAT(openaiVoiceConfig): added 2026-06-30, remove after 2026-12-30.
|
||||
// Drop a `providers.openai.voice` block left by an older release so the strict
|
||||
// schema doesn't reject it. The value is discarded, not migrated — there is no
|
||||
// back-compat; configure `providers.openai.stt` / `providers.openai.tts` instead.
|
||||
delete openaiRecord.voice;
|
||||
providersRecord.openai = openaiRecord;
|
||||
}
|
||||
|
||||
root.providers = providersRecord;
|
||||
return root;
|
||||
}
|
||||
|
||||
function normalizeSpeechSite(
|
||||
site: Record<string, unknown>,
|
||||
kind: "stt" | "tts",
|
||||
legacy: Record<string, unknown> | undefined,
|
||||
foldedKeys: Set<string>,
|
||||
) {
|
||||
site.provider =
|
||||
typeof site.provider === "string"
|
||||
? site.provider.trim().toLowerCase()
|
||||
: (site.provider ?? "local");
|
||||
if (kind === "stt" && site.provider === "local") {
|
||||
delete site.confidenceThreshold;
|
||||
}
|
||||
if (kind === "tts") {
|
||||
if (site.provider === "local") delete site.voice;
|
||||
if (site.provider === "openai") {
|
||||
delete site.speakerId;
|
||||
delete site.speed;
|
||||
}
|
||||
}
|
||||
if (site.provider !== "openai" || !legacy) return;
|
||||
const endpoint = configRecord(legacy[kind]);
|
||||
for (const field of ["apiKey", "baseUrl"] as const) {
|
||||
if (site[field] !== undefined) continue;
|
||||
if (endpoint?.[field] !== undefined) {
|
||||
site[field] = endpoint[field];
|
||||
foldedKeys.add(`providers.openai.${kind}.${field}`);
|
||||
} else if (legacy[field] !== undefined) {
|
||||
site[field] = legacy[field];
|
||||
foldedKeys.add(`providers.openai.${field}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function warnDiscardedSpeechKeys(
|
||||
legacy: Record<string, unknown>,
|
||||
foldedKeys: Set<string>,
|
||||
logger?: LoggerLike,
|
||||
) {
|
||||
const keys = [
|
||||
["providers.openai.apiKey", legacy.apiKey],
|
||||
...["stt", "tts"].map((kind) => [
|
||||
`providers.openai.${kind}.apiKey`,
|
||||
configRecord(legacy[kind])?.apiKey,
|
||||
]),
|
||||
] as const;
|
||||
for (const [keyPath, key] of keys) {
|
||||
if (key && typeof keyPath === "string" && !foldedKeys.has(keyPath)) {
|
||||
const message = `Dropped ${keyPath}: no use site in config.json selects OpenAI; re-enter it when selecting OpenAI.`;
|
||||
if (logger?.warn) logger.warn(message);
|
||||
else console.warn(message);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export function loadPersistedConfig(paseoHome: string, logger?: LoggerLike): PersistedConfig {
|
||||
const log = getLogger(logger);
|
||||
const configPath = getConfigPath(paseoHome);
|
||||
@@ -464,7 +518,7 @@ export function loadPersistedConfig(paseoHome: string, logger?: LoggerLike): Per
|
||||
});
|
||||
}
|
||||
|
||||
const config = parseConfigFile(configPath, raw);
|
||||
const config = parseConfigFile(configPath, raw, log);
|
||||
log?.info(`Loaded from ${configPath}`);
|
||||
return config;
|
||||
}
|
||||
@@ -486,7 +540,7 @@ export function readPersistedConfig(
|
||||
return parseConfigFile(configPath, raw);
|
||||
}
|
||||
|
||||
function parseConfigFile(configPath: string, raw: string): PersistedConfig {
|
||||
function parseConfigFile(configPath: string, raw: string, logger?: LoggerLike): PersistedConfig {
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = parseConfigText(raw);
|
||||
@@ -497,7 +551,7 @@ function parseConfigFile(configPath: string, raw: string): PersistedConfig {
|
||||
});
|
||||
}
|
||||
|
||||
const result = PersistedConfigSchema.safeParse(stripRemovedConfigFields(parsed));
|
||||
const result = PersistedConfigSchema.safeParse(normalizeSpeechProviderConfig(parsed, logger));
|
||||
if (!result.success) {
|
||||
const issues = result.error.issues
|
||||
.map((i) => ` - ${i.path.join(".")}: ${i.message}`)
|
||||
@@ -525,6 +579,12 @@ function configPathParts(field: string): string[] {
|
||||
schema instanceof z.ZodPipe
|
||||
)
|
||||
schema = (schema instanceof z.ZodPipe ? schema.in : schema.unwrap()) as z.ZodType;
|
||||
if (schema instanceof z.ZodUnion) {
|
||||
const variant = schema.options.find(
|
||||
(option) => option instanceof z.ZodObject && Object.hasOwn(option.shape, part),
|
||||
);
|
||||
if (variant instanceof z.ZodObject) schema = variant;
|
||||
}
|
||||
if (!(schema instanceof z.ZodObject) || !Object.hasOwn(schema.shape, part)) {
|
||||
throw new Error(
|
||||
`Unknown configuration path: ${field}. Replace its containing object with JSON for dynamic keys.`,
|
||||
|
||||
@@ -182,6 +182,10 @@ import {
|
||||
import { ScheduleSession } from "./session/schedule/schedule-session.js";
|
||||
import { ProviderCatalogSession } from "./session/provider/provider-catalog-session.js";
|
||||
import { UsageSession } from "./session/usage/usage-session.js";
|
||||
import {
|
||||
SpeechModelsSession,
|
||||
type SpeechModelsSessionOptions,
|
||||
} from "./session/speech/speech-models-session.js";
|
||||
import { WorkspaceFilesSession } from "./session/files/workspace-files-session.js";
|
||||
import { AgentConfigSession } from "./session/agent-config/agent-config-session.js";
|
||||
import { ProjectConfigSession } from "./session/project-config/project-config-session.js";
|
||||
@@ -528,6 +532,7 @@ export interface SessionOptions {
|
||||
stt: Resolvable<SpeechToTextProvider | null>;
|
||||
sttLanguage?: string;
|
||||
tts: Resolvable<TextToSpeechProvider | null>;
|
||||
speechModels?: SpeechModelsSessionOptions["runtime"];
|
||||
terminalManager: TerminalManager | null;
|
||||
providerSnapshotManager: ProviderSnapshotManager;
|
||||
hubExecutionAgents?: HubExecutionAgents;
|
||||
@@ -811,6 +816,7 @@ export class Session {
|
||||
private readonly scheduleSession: ScheduleSession;
|
||||
private readonly providerCatalogSession: ProviderCatalogSession;
|
||||
private readonly usageSession: UsageSession;
|
||||
private readonly speechModelsSession: SpeechModelsSession;
|
||||
private readonly workspaceFilesSession: WorkspaceFilesSession;
|
||||
private readonly agentConfigSession: AgentConfigSession;
|
||||
private readonly projectConfigSession: ProjectConfigSession;
|
||||
@@ -861,6 +867,7 @@ export class Session {
|
||||
stt,
|
||||
sttLanguage,
|
||||
tts,
|
||||
speechModels,
|
||||
terminalManager,
|
||||
providerSnapshotManager,
|
||||
serviceProxy,
|
||||
@@ -1037,6 +1044,10 @@ export class Session {
|
||||
runtime: pluginRuntime,
|
||||
logger: this.sessionLogger,
|
||||
});
|
||||
this.speechModelsSession = new SpeechModelsSession({
|
||||
emit: (msg) => this.emit(msg),
|
||||
runtime: speechModels,
|
||||
});
|
||||
this.agentConfigSession = new AgentConfigSession({
|
||||
host: {
|
||||
emit: (msg) => this.emit(msg),
|
||||
@@ -2421,8 +2432,7 @@ export class Session {
|
||||
this.dispatchOrchestrationSkillsMessage(msg) ??
|
||||
this.dispatchPluginDirectoryMessage(msg) ??
|
||||
this.dispatchPluginMessage(msg) ??
|
||||
this.dispatchTerminalMessage(msg) ??
|
||||
this.dispatchScheduleMessage(msg) ??
|
||||
this.dispatchHostServiceMessage(msg) ??
|
||||
this.dispatchMiscMessage(msg);
|
||||
if (promise) await promise;
|
||||
}
|
||||
@@ -2438,6 +2448,14 @@ export class Session {
|
||||
);
|
||||
}
|
||||
|
||||
private dispatchHostServiceMessage(msg: SessionInboundMessage): Promise<void> | undefined {
|
||||
return (
|
||||
this.dispatchTerminalMessage(msg) ??
|
||||
this.dispatchScheduleMessage(msg) ??
|
||||
this.speechModelsSession.dispatch(msg)
|
||||
);
|
||||
}
|
||||
|
||||
private dispatchWorkspaceLifecycleMessage(msg: SessionInboundMessage): Promise<void> | undefined {
|
||||
return (
|
||||
this.dispatchWorkspaceStateMessage(msg) ??
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
import { describe, expect, test } from "vitest";
|
||||
|
||||
import type { SessionOutboundMessage } from "../../messages.js";
|
||||
import {
|
||||
listLocalSpeechModels,
|
||||
type LocalSpeechModelId,
|
||||
} from "../../speech/providers/local/models.js";
|
||||
import {
|
||||
LocalSpeechModelInUseError,
|
||||
type LocalSpeechModelStatus,
|
||||
} from "../../speech/speech-runtime.js";
|
||||
import { SpeechModelsSession } from "./speech-models-session.js";
|
||||
|
||||
const [firstModel] = listLocalSpeechModels();
|
||||
|
||||
function createSession(models: LocalSpeechModelStatus[], inUse: LocalSpeechModelId[] = []) {
|
||||
const emitted: SessionOutboundMessage[] = [];
|
||||
const downloads: LocalSpeechModelId[] = [];
|
||||
const session = new SpeechModelsSession({
|
||||
emit: (message) => emitted.push(message),
|
||||
runtime: {
|
||||
listLocalModels: async () => models,
|
||||
downloadLocalModel: (modelId) => {
|
||||
downloads.push(modelId);
|
||||
},
|
||||
deleteLocalModel: async (modelId) => {
|
||||
if (inUse.includes(modelId)) throw new LocalSpeechModelInUseError(modelId);
|
||||
},
|
||||
},
|
||||
});
|
||||
return { session, emitted, downloads };
|
||||
}
|
||||
|
||||
describe("SpeechModelsSession", () => {
|
||||
test("lists catalog details and download progress before anything is installed", async () => {
|
||||
const { session, emitted } = createSession([
|
||||
{
|
||||
spec: firstModel,
|
||||
state: { status: "downloading", receivedBytes: 10, totalBytes: 40 },
|
||||
inUse: true,
|
||||
},
|
||||
]);
|
||||
|
||||
await session.dispatch({ type: "speech.local_models.list.request", requestId: "r1" });
|
||||
|
||||
expect(emitted).toEqual([
|
||||
{
|
||||
type: "speech.local_models.list.response",
|
||||
payload: {
|
||||
requestId: "r1",
|
||||
error: null,
|
||||
models: [
|
||||
{
|
||||
id: firstModel.id,
|
||||
kind: "stt",
|
||||
label: firstModel.label,
|
||||
description: firstModel.description,
|
||||
languages: [...firstModel.languages],
|
||||
downloadSizeBytes: firstModel.downloadSizeBytes,
|
||||
isDefault: true,
|
||||
inUse: true,
|
||||
status: "downloading",
|
||||
receivedBytes: 10,
|
||||
totalBytes: 40,
|
||||
},
|
||||
],
|
||||
},
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test("rejects a model id outside the catalog without starting a download", () => {
|
||||
const { session, emitted, downloads } = createSession([]);
|
||||
|
||||
void session.dispatch({
|
||||
type: "speech.local_models.download.request",
|
||||
requestId: "r2",
|
||||
modelId: "../../etc",
|
||||
});
|
||||
|
||||
expect(downloads).toEqual([]);
|
||||
expect(emitted).toEqual([
|
||||
{
|
||||
type: "speech.local_models.download.response",
|
||||
payload: { requestId: "r2", error: "Unknown local speech model: ../../etc" },
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
test("reports why an in-use model cannot be deleted", async () => {
|
||||
const { session, emitted } = createSession([], [firstModel.id]);
|
||||
|
||||
await session.dispatch({
|
||||
type: "speech.local_models.delete.request",
|
||||
requestId: "r3",
|
||||
modelId: firstModel.id,
|
||||
});
|
||||
|
||||
expect(emitted).toEqual([
|
||||
{
|
||||
type: "speech.local_models.delete.response",
|
||||
payload: { requestId: "r3", error: new LocalSpeechModelInUseError(firstModel.id).message },
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,124 @@
|
||||
import type { LocalSpeechModel } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
|
||||
import type { SessionInboundMessage, SessionOutboundMessage } from "../../messages.js";
|
||||
import { LocalSpeechModelIdSchema } from "../../speech/providers/local/models.js";
|
||||
import type { LocalSpeechModelStatus, SpeechService } from "../../speech/speech-runtime.js";
|
||||
|
||||
type SpeechModelsRuntime = Pick<
|
||||
SpeechService,
|
||||
"listLocalModels" | "downloadLocalModel" | "deleteLocalModel"
|
||||
>;
|
||||
|
||||
export interface SpeechModelsSessionOptions {
|
||||
emit(message: SessionOutboundMessage): void;
|
||||
// Absent when the daemon runs without speech.
|
||||
runtime?: SpeechModelsRuntime | null;
|
||||
}
|
||||
|
||||
export class SpeechModelsSession {
|
||||
constructor(private readonly options: SpeechModelsSessionOptions) {}
|
||||
|
||||
// Handles the speech.local_models.* requests; returns undefined for any other message.
|
||||
dispatch(msg: SessionInboundMessage): Promise<void> | undefined {
|
||||
switch (msg.type) {
|
||||
case "speech.local_models.list.request":
|
||||
return this.handleList(msg);
|
||||
case "speech.local_models.download.request":
|
||||
this.handleDownload(msg);
|
||||
return Promise.resolve();
|
||||
case "speech.local_models.delete.request":
|
||||
return this.handleDelete(msg);
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
private async handleList(
|
||||
msg: Extract<SessionInboundMessage, { type: "speech.local_models.list.request" }>,
|
||||
): Promise<void> {
|
||||
try {
|
||||
const models = await this.requireRuntime().listLocalModels();
|
||||
this.options.emit({
|
||||
type: "speech.local_models.list.response",
|
||||
payload: { requestId: msg.requestId, models: models.map(toWireModel), error: null },
|
||||
});
|
||||
} catch (error) {
|
||||
this.options.emit({
|
||||
type: "speech.local_models.list.response",
|
||||
payload: { requestId: msg.requestId, models: [], error: describeError(error) },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private handleDownload(
|
||||
msg: Extract<SessionInboundMessage, { type: "speech.local_models.download.request" }>,
|
||||
): void {
|
||||
try {
|
||||
this.requireRuntime().downloadLocalModel(parseModelId(msg.modelId));
|
||||
this.options.emit({
|
||||
type: "speech.local_models.download.response",
|
||||
payload: { requestId: msg.requestId, error: null },
|
||||
});
|
||||
} catch (error) {
|
||||
this.options.emit({
|
||||
type: "speech.local_models.download.response",
|
||||
payload: { requestId: msg.requestId, error: describeError(error) },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private async handleDelete(
|
||||
msg: Extract<SessionInboundMessage, { type: "speech.local_models.delete.request" }>,
|
||||
): Promise<void> {
|
||||
try {
|
||||
await this.requireRuntime().deleteLocalModel(parseModelId(msg.modelId));
|
||||
this.options.emit({
|
||||
type: "speech.local_models.delete.response",
|
||||
payload: { requestId: msg.requestId, error: null },
|
||||
});
|
||||
} catch (error) {
|
||||
this.options.emit({
|
||||
type: "speech.local_models.delete.response",
|
||||
payload: { requestId: msg.requestId, error: describeError(error) },
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
private requireRuntime(): SpeechModelsRuntime {
|
||||
if (!this.options.runtime) {
|
||||
throw new Error("Speech is unavailable on this daemon");
|
||||
}
|
||||
return this.options.runtime;
|
||||
}
|
||||
}
|
||||
|
||||
function parseModelId(modelId: string) {
|
||||
const parsed = LocalSpeechModelIdSchema.safeParse(modelId);
|
||||
if (!parsed.success) {
|
||||
throw new Error(`Unknown local speech model: ${modelId}`);
|
||||
}
|
||||
return parsed.data;
|
||||
}
|
||||
|
||||
function toWireModel(model: LocalSpeechModelStatus): LocalSpeechModel {
|
||||
const { spec, state, inUse } = model;
|
||||
return {
|
||||
id: spec.id,
|
||||
kind: spec.kind === "tts" ? "tts" : "stt",
|
||||
label: spec.label,
|
||||
description: spec.description,
|
||||
languages: [...spec.languages],
|
||||
downloadSizeBytes: spec.downloadSizeBytes,
|
||||
isDefault: spec.defaultFor !== undefined,
|
||||
inUse,
|
||||
status: state.status,
|
||||
...(state.status === "downloading"
|
||||
? { receivedBytes: state.receivedBytes, totalBytes: state.totalBytes }
|
||||
: {}),
|
||||
...(state.status === "failed" ? { error: state.error } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function describeError(error: unknown): string {
|
||||
return error instanceof Error ? error.message : String(error);
|
||||
}
|
||||
@@ -29,6 +29,7 @@ export interface LocalSpeechProviderConfig {
|
||||
|
||||
export interface ResolvedLocalSpeechConfig {
|
||||
local: LocalSpeechProviderConfig | undefined;
|
||||
modelsDir: string;
|
||||
sttLanguages: LocalSpeechSttLanguageConfig;
|
||||
}
|
||||
|
||||
@@ -68,12 +69,12 @@ const LocalSpeechResolutionSchema = z.object({
|
||||
function persistedLocalFeatureModel(
|
||||
provider: RequestedSpeechProviders[keyof RequestedSpeechProviders]["provider"],
|
||||
enabled: boolean | undefined,
|
||||
model: string | undefined,
|
||||
site: { provider: string; model?: string } | undefined,
|
||||
): string | undefined {
|
||||
if (provider !== "local" || enabled === false) {
|
||||
return undefined;
|
||||
}
|
||||
return model;
|
||||
return site?.provider === "local" ? site.model : undefined;
|
||||
}
|
||||
|
||||
function shouldIncludeLocalProviderConfig(params: {
|
||||
@@ -143,6 +144,8 @@ function buildLocalSpeechResolutionInput(params: {
|
||||
includeProviderConfig: boolean;
|
||||
}): Record<string, unknown> {
|
||||
const { paseoHome, env, persisted, providers, includeProviderConfig } = params;
|
||||
const tts = persisted.features?.voiceMode?.tts;
|
||||
const localTts = tts?.provider === "local" ? tts : undefined;
|
||||
return {
|
||||
includeProviderConfig,
|
||||
modelsDir: firstDefinedValue<string>([
|
||||
@@ -155,7 +158,7 @@ function buildLocalSpeechResolutionInput(params: {
|
||||
persistedLocalFeatureModel(
|
||||
providers.dictationStt.provider,
|
||||
providers.dictationStt.enabled,
|
||||
persisted.features?.dictation?.stt?.model,
|
||||
persisted.features?.dictation?.stt,
|
||||
),
|
||||
DEFAULT_LOCAL_STT_MODEL,
|
||||
]),
|
||||
@@ -164,7 +167,7 @@ function buildLocalSpeechResolutionInput(params: {
|
||||
persistedLocalFeatureModel(
|
||||
providers.voiceStt.provider,
|
||||
providers.voiceStt.enabled,
|
||||
persisted.features?.voiceMode?.stt?.model,
|
||||
persisted.features?.voiceMode?.stt,
|
||||
),
|
||||
DEFAULT_LOCAL_STT_MODEL,
|
||||
]),
|
||||
@@ -173,18 +176,18 @@ function buildLocalSpeechResolutionInput(params: {
|
||||
persistedLocalFeatureModel(
|
||||
providers.voiceTts.provider,
|
||||
providers.voiceTts.enabled,
|
||||
persisted.features?.voiceMode?.tts?.model,
|
||||
persisted.features?.voiceMode?.tts,
|
||||
),
|
||||
DEFAULT_LOCAL_TTS_MODEL,
|
||||
]),
|
||||
...buildLocalSpeechLanguageResolutionInput({ env, persisted }),
|
||||
voiceLocalTtsSpeakerId: firstDefinedValue<string | number>([
|
||||
env.PASEO_VOICE_LOCAL_TTS_SPEAKER_ID,
|
||||
persisted.features?.voiceMode?.tts?.speakerId,
|
||||
localTts?.speakerId,
|
||||
]),
|
||||
voiceLocalTtsSpeed: firstDefinedValue<string | number>([
|
||||
env.PASEO_VOICE_LOCAL_TTS_SPEED,
|
||||
persisted.features?.voiceMode?.tts?.speed,
|
||||
localTts?.speed,
|
||||
]),
|
||||
};
|
||||
}
|
||||
@@ -200,11 +203,10 @@ export function resolveLocalSpeechConfig(params: {
|
||||
buildLocalSpeechResolutionInput({ ...params, includeProviderConfig }),
|
||||
);
|
||||
|
||||
const resolvedVoiceTtsSpeakerId =
|
||||
parsed.voiceLocalTtsSpeakerId ??
|
||||
(parsed.voiceLocalTtsModel === "kokoro-en-v0_19" ? 0 : undefined);
|
||||
const resolvedVoiceTtsSpeakerId = parsed.voiceLocalTtsSpeakerId ?? 0;
|
||||
|
||||
return {
|
||||
modelsDir: parsed.modelsDir,
|
||||
sttLanguages: {
|
||||
dictation: parsed.dictationLanguage,
|
||||
voice: parsed.voiceLanguage,
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
import type { Logger } from "pino";
|
||||
|
||||
import {
|
||||
deleteSherpaOnnxModel,
|
||||
ensureSherpaOnnxModel,
|
||||
isSherpaOnnxModelInstalled,
|
||||
} from "./sherpa/model-downloader.js";
|
||||
import {
|
||||
listLocalSpeechModels,
|
||||
type LocalSpeechModelId,
|
||||
type LocalSpeechModelSpec,
|
||||
} from "./models.js";
|
||||
|
||||
export type LocalSpeechModelInstallState =
|
||||
| { status: "installed" }
|
||||
| { status: "not_installed" }
|
||||
| { status: "downloading"; receivedBytes: number; totalBytes: number }
|
||||
| { status: "extracting" }
|
||||
| { status: "failed"; error: string };
|
||||
|
||||
export interface LocalSpeechModelEntry {
|
||||
spec: LocalSpeechModelSpec;
|
||||
state: LocalSpeechModelInstallState;
|
||||
}
|
||||
|
||||
type InFlightState = Extract<
|
||||
LocalSpeechModelInstallState,
|
||||
{ status: "downloading" | "extracting" }
|
||||
>;
|
||||
|
||||
interface InFlightDownload {
|
||||
promise: Promise<string>;
|
||||
state: InFlightState;
|
||||
}
|
||||
|
||||
export class LocalSpeechModelBusyError extends Error {
|
||||
constructor(public readonly modelId: LocalSpeechModelId) {
|
||||
super(`${modelId} is downloading. Wait for the download to finish before deleting it.`);
|
||||
this.name = "LocalSpeechModelBusyError";
|
||||
}
|
||||
}
|
||||
|
||||
// Owns the models directory: which catalog models are installed, which are downloading,
|
||||
// and the last failure per model. Concurrent downloads of one model share a single transfer.
|
||||
export class LocalSpeechModelLibrary {
|
||||
private readonly inFlight = new Map<LocalSpeechModelId, InFlightDownload>();
|
||||
private readonly failures = new Map<LocalSpeechModelId, string>();
|
||||
private readonly abortController = new AbortController();
|
||||
private readonly logger: Logger;
|
||||
|
||||
constructor(
|
||||
private readonly modelsDir: string,
|
||||
logger: Logger,
|
||||
) {
|
||||
this.logger = logger.child({ module: "speech", component: "local-model-library" });
|
||||
}
|
||||
|
||||
getModelsDir(): string {
|
||||
return this.modelsDir;
|
||||
}
|
||||
|
||||
async list(): Promise<LocalSpeechModelEntry[]> {
|
||||
return Promise.all(
|
||||
listLocalSpeechModels().map(async (spec) => ({ spec, state: await this.getState(spec.id) })),
|
||||
);
|
||||
}
|
||||
|
||||
async listMissing(modelIds: LocalSpeechModelId[]): Promise<LocalSpeechModelId[]> {
|
||||
const installed = await Promise.all(
|
||||
modelIds.map((modelId) => isSherpaOnnxModelInstalled(this.modelsDir, modelId)),
|
||||
);
|
||||
return modelIds.filter((_modelId, index) => !installed[index]);
|
||||
}
|
||||
|
||||
download(modelId: LocalSpeechModelId): Promise<string> {
|
||||
const existing = this.inFlight.get(modelId);
|
||||
if (existing) {
|
||||
return existing.promise;
|
||||
}
|
||||
this.failures.delete(modelId);
|
||||
const entry: InFlightDownload = {
|
||||
state: { status: "downloading", receivedBytes: 0, totalBytes: 0 },
|
||||
promise: Promise.resolve(""),
|
||||
};
|
||||
entry.promise = ensureSherpaOnnxModel({
|
||||
modelsDir: this.modelsDir,
|
||||
modelId,
|
||||
logger: this.logger,
|
||||
signal: this.abortController.signal,
|
||||
onProgress: (progress) => {
|
||||
entry.state =
|
||||
progress.phase === "downloading"
|
||||
? {
|
||||
status: "downloading",
|
||||
receivedBytes: progress.receivedBytes,
|
||||
totalBytes: progress.totalBytes,
|
||||
}
|
||||
: { status: "extracting" };
|
||||
},
|
||||
})
|
||||
.catch((error: unknown) => {
|
||||
this.failures.set(modelId, error instanceof Error ? error.message : String(error));
|
||||
throw error;
|
||||
})
|
||||
.finally(() => {
|
||||
this.inFlight.delete(modelId);
|
||||
});
|
||||
this.inFlight.set(modelId, entry);
|
||||
return entry.promise;
|
||||
}
|
||||
|
||||
async delete(modelId: LocalSpeechModelId): Promise<void> {
|
||||
if (this.inFlight.has(modelId)) {
|
||||
throw new LocalSpeechModelBusyError(modelId);
|
||||
}
|
||||
this.failures.delete(modelId);
|
||||
await deleteSherpaOnnxModel(this.modelsDir, modelId);
|
||||
this.logger.info({ modelId }, "Deleted local speech model");
|
||||
}
|
||||
|
||||
async stop(): Promise<void> {
|
||||
this.abortController.abort();
|
||||
await Promise.allSettled(Array.from(this.inFlight.values(), (entry) => entry.promise));
|
||||
}
|
||||
|
||||
private async getState(modelId: LocalSpeechModelId): Promise<LocalSpeechModelInstallState> {
|
||||
const inFlight = this.inFlight.get(modelId);
|
||||
if (inFlight) {
|
||||
return inFlight.state;
|
||||
}
|
||||
if (await isSherpaOnnxModelInstalled(this.modelsDir, modelId)) {
|
||||
return { status: "installed" };
|
||||
}
|
||||
const failure = this.failures.get(modelId);
|
||||
return failure ? { status: "failed", error: failure } : { status: "not_installed" };
|
||||
}
|
||||
}
|
||||
@@ -13,12 +13,8 @@ import path from "node:path";
|
||||
import pino from "pino";
|
||||
import { describe, expect, onTestFinished, test } from "vitest";
|
||||
|
||||
import {
|
||||
ensureLocalSpeechModels,
|
||||
getLocalSpeechModelDir,
|
||||
listLocalSpeechModels,
|
||||
listMissingLocalSpeechModels,
|
||||
} from "./models.js";
|
||||
import { LocalSpeechModelLibrary } from "./model-library.js";
|
||||
import { getLocalSpeechModelDir, listLocalSpeechModels } from "./models.js";
|
||||
|
||||
const logger = pino({ level: "silent" });
|
||||
const modelId = "parakeet-tdt-0.6b-v2-int8";
|
||||
@@ -55,16 +51,31 @@ describe("local speech models", () => {
|
||||
const encoder = path.join(modelDir, spec.requiredFiles[0]);
|
||||
truncateSync(encoder, 1_000);
|
||||
|
||||
expect(await listMissingLocalSpeechModels({ modelsDir, modelIds: [modelId] })).toEqual([
|
||||
modelId,
|
||||
]);
|
||||
const library = new LocalSpeechModelLibrary(modelsDir, logger);
|
||||
expect(await library.listMissing([modelId])).toEqual([modelId]);
|
||||
|
||||
await ensureLocalSpeechModels({ modelsDir, modelIds: [modelId], logger });
|
||||
await library.download(modelId);
|
||||
|
||||
expect(readFileSync(encoder, "utf8")).toBe(
|
||||
`complete ${spec.requiredFiles[0]}`.padEnd(64_000, "."),
|
||||
);
|
||||
expect(existsSync(archive)).toBe(false);
|
||||
expect(await listMissingLocalSpeechModels({ modelsDir, modelIds: [modelId] })).toEqual([]);
|
||||
expect(await library.listMissing([modelId])).toEqual([]);
|
||||
});
|
||||
|
||||
test("lists install state and deletes an installed model", async () => {
|
||||
const { modelsDir } = setupModelsDirWithArchive();
|
||||
const library = new LocalSpeechModelLibrary(modelsDir, logger);
|
||||
const stateOf = async () =>
|
||||
(await library.list()).find((entry) => entry.spec.id === modelId)?.state;
|
||||
|
||||
expect(await stateOf()).toEqual({ status: "not_installed" });
|
||||
await library.download(modelId);
|
||||
expect(await stateOf()).toEqual({ status: "installed" });
|
||||
|
||||
await library.delete(modelId);
|
||||
|
||||
expect(await stateOf()).toEqual({ status: "not_installed" });
|
||||
expect(existsSync(getLocalSpeechModelDir(modelsDir, modelId))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,11 +1,8 @@
|
||||
import {
|
||||
ensureSherpaOnnxModels,
|
||||
getSherpaOnnxModelDir,
|
||||
isSherpaOnnxModelInstalled,
|
||||
} from "./sherpa/model-downloader.js";
|
||||
import { ensureSherpaOnnxModels, getSherpaOnnxModelDir } from "./sherpa/model-downloader.js";
|
||||
import {
|
||||
DEFAULT_LOCAL_STT_MODEL,
|
||||
DEFAULT_LOCAL_TTS_MODEL,
|
||||
LocalSpeechModelIdSchema,
|
||||
LocalSttModelIdSchema,
|
||||
LocalTtsModelIdSchema,
|
||||
listSherpaOnnxModels,
|
||||
@@ -17,6 +14,7 @@ import {
|
||||
export {
|
||||
DEFAULT_LOCAL_STT_MODEL,
|
||||
DEFAULT_LOCAL_TTS_MODEL,
|
||||
LocalSpeechModelIdSchema,
|
||||
LocalSttModelIdSchema,
|
||||
LocalTtsModelIdSchema,
|
||||
type LocalSpeechModelId,
|
||||
@@ -34,16 +32,6 @@ export function getLocalSpeechModelDir(modelsDir: string, modelId: LocalSpeechMo
|
||||
return getSherpaOnnxModelDir(modelsDir, modelId);
|
||||
}
|
||||
|
||||
export async function listMissingLocalSpeechModels(options: {
|
||||
modelsDir: string;
|
||||
modelIds: LocalSpeechModelId[];
|
||||
}): Promise<LocalSpeechModelId[]> {
|
||||
const installed = await Promise.all(
|
||||
options.modelIds.map((modelId) => isSherpaOnnxModelInstalled(options.modelsDir, modelId)),
|
||||
);
|
||||
return options.modelIds.filter((_modelId, index) => !installed[index]);
|
||||
}
|
||||
|
||||
export async function ensureLocalSpeechModels(options: {
|
||||
modelsDir: string;
|
||||
modelIds: LocalSpeechModelId[];
|
||||
|
||||
@@ -4,43 +4,467 @@ export type SherpaOnnxModelKind = "stt-offline" | "tts";
|
||||
|
||||
type DefaultModelRole = "stt" | "tts";
|
||||
|
||||
interface SherpaOnnxCatalogEntry {
|
||||
kind: SherpaOnnxModelKind;
|
||||
// How sherpa-onnx loads a model, with file names relative to the extracted directory.
|
||||
export type SttEngine =
|
||||
| { family: "transducer"; encoder: string; decoder: string; joiner: string; tokens: string }
|
||||
| { family: "whisper"; encoder: string; decoder: string; tokens: string }
|
||||
| { family: "sense_voice"; model: string; tokens: string }
|
||||
| { family: "dolphin"; model: string; tokens: string }
|
||||
| { family: "nemo_ctc"; model: string; tokens: string };
|
||||
|
||||
export type TtsEngine =
|
||||
| {
|
||||
family: "kokoro";
|
||||
model: string;
|
||||
voices: string;
|
||||
tokens: string;
|
||||
dataDir: string;
|
||||
// Multi-language Kokoro models read pronunciations from these instead of espeak alone.
|
||||
lexicons?: readonly string[];
|
||||
dictDir?: string;
|
||||
ruleFsts?: readonly string[];
|
||||
}
|
||||
| { family: "kitten"; model: string; voices: string; tokens: string; dataDir: string }
|
||||
| { family: "vits"; model: string; tokens: string; dataDir: string };
|
||||
|
||||
interface SherpaOnnxCatalogEntryBase {
|
||||
label: string;
|
||||
// One user-facing sentence shown before download.
|
||||
description: string;
|
||||
// BCP 47 language tags the model transcribes or speaks.
|
||||
languages: readonly string[];
|
||||
// Exact size of the release archive. The downloader rejects any other size.
|
||||
downloadSizeBytes: number;
|
||||
archiveUrl: string;
|
||||
extractedDir: string;
|
||||
requiredFiles: string[];
|
||||
description: string;
|
||||
defaultFor?: DefaultModelRole;
|
||||
}
|
||||
|
||||
type SherpaOnnxCatalogEntry =
|
||||
| (SherpaOnnxCatalogEntryBase & { kind: "stt-offline"; engine: SttEngine })
|
||||
| (SherpaOnnxCatalogEntryBase & { kind: "tts"; engine: TtsEngine });
|
||||
|
||||
// Every sherpa-onnx release archive extracts to a directory named after the archive.
|
||||
function releaseArchive(tag: "asr-models" | "tts-models", name: string) {
|
||||
return {
|
||||
archiveUrl: `https://github.com/k2-fsa/sherpa-onnx/releases/download/${tag}/${name}.tar.bz2`,
|
||||
extractedDir: name,
|
||||
};
|
||||
}
|
||||
|
||||
const PARAKEET_INT8_ENGINE = {
|
||||
family: "transducer",
|
||||
encoder: "encoder.int8.onnx",
|
||||
decoder: "decoder.int8.onnx",
|
||||
joiner: "joiner.int8.onnx",
|
||||
tokens: "tokens.txt",
|
||||
} as const satisfies SttEngine;
|
||||
|
||||
function whisperEngine(size: string): SttEngine {
|
||||
return {
|
||||
family: "whisper",
|
||||
encoder: `${size}-encoder.int8.onnx`,
|
||||
decoder: `${size}-decoder.int8.onnx`,
|
||||
tokens: `${size}-tokens.txt`,
|
||||
};
|
||||
}
|
||||
|
||||
function piperEngine(voice: string): TtsEngine {
|
||||
return {
|
||||
family: "vits",
|
||||
model: `${voice}.onnx`,
|
||||
tokens: "tokens.txt",
|
||||
dataDir: "espeak-ng-data",
|
||||
};
|
||||
}
|
||||
|
||||
// Parakeet TDT v3 transcribes these 25 European languages and detects which one is spoken.
|
||||
const PARAKEET_V3_LANGUAGES = [
|
||||
"bg",
|
||||
"hr",
|
||||
"cs",
|
||||
"da",
|
||||
"nl",
|
||||
"en",
|
||||
"et",
|
||||
"fi",
|
||||
"fr",
|
||||
"de",
|
||||
"el",
|
||||
"hu",
|
||||
"it",
|
||||
"lv",
|
||||
"lt",
|
||||
"mt",
|
||||
"pl",
|
||||
"pt",
|
||||
"ro",
|
||||
"sk",
|
||||
"sl",
|
||||
"es",
|
||||
"sv",
|
||||
"ru",
|
||||
"uk",
|
||||
] as const;
|
||||
|
||||
// The 99 languages multilingual Whisper models transcribe, detecting which one is spoken.
|
||||
const WHISPER_LANGUAGES = [
|
||||
"en",
|
||||
"zh",
|
||||
"de",
|
||||
"es",
|
||||
"ru",
|
||||
"ko",
|
||||
"fr",
|
||||
"ja",
|
||||
"pt",
|
||||
"tr",
|
||||
"pl",
|
||||
"ca",
|
||||
"nl",
|
||||
"ar",
|
||||
"sv",
|
||||
"it",
|
||||
"id",
|
||||
"hi",
|
||||
"fi",
|
||||
"vi",
|
||||
"he",
|
||||
"uk",
|
||||
"el",
|
||||
"ms",
|
||||
"cs",
|
||||
"ro",
|
||||
"da",
|
||||
"hu",
|
||||
"ta",
|
||||
"no",
|
||||
"th",
|
||||
"ur",
|
||||
"hr",
|
||||
"bg",
|
||||
"lt",
|
||||
"la",
|
||||
"mi",
|
||||
"ml",
|
||||
"cy",
|
||||
"sk",
|
||||
"te",
|
||||
"fa",
|
||||
"lv",
|
||||
"bn",
|
||||
"sr",
|
||||
"az",
|
||||
"sl",
|
||||
"kn",
|
||||
"et",
|
||||
"mk",
|
||||
"br",
|
||||
"eu",
|
||||
"is",
|
||||
"hy",
|
||||
"ne",
|
||||
"mn",
|
||||
"bs",
|
||||
"kk",
|
||||
"sq",
|
||||
"sw",
|
||||
"gl",
|
||||
"mr",
|
||||
"pa",
|
||||
"si",
|
||||
"km",
|
||||
"sn",
|
||||
"yo",
|
||||
"so",
|
||||
"af",
|
||||
"oc",
|
||||
"ka",
|
||||
"be",
|
||||
"tg",
|
||||
"sd",
|
||||
"gu",
|
||||
"am",
|
||||
"yi",
|
||||
"lo",
|
||||
"uz",
|
||||
"fo",
|
||||
"ht",
|
||||
"ps",
|
||||
"tk",
|
||||
"nn",
|
||||
"mt",
|
||||
"sa",
|
||||
"lb",
|
||||
"my",
|
||||
"bo",
|
||||
"tl",
|
||||
"mg",
|
||||
"as",
|
||||
"tt",
|
||||
"haw",
|
||||
"ln",
|
||||
"ha",
|
||||
"ba",
|
||||
"jv",
|
||||
"su",
|
||||
] as const;
|
||||
|
||||
// Dolphin's 40 Eastern languages. It also covers 22 Chinese dialects under "zh".
|
||||
const DOLPHIN_LANGUAGES = [
|
||||
"zh",
|
||||
"ja",
|
||||
"th",
|
||||
"ru",
|
||||
"ko",
|
||||
"id",
|
||||
"vi",
|
||||
"yue",
|
||||
"hi",
|
||||
"ur",
|
||||
"ms",
|
||||
"uz",
|
||||
"ar",
|
||||
"fa",
|
||||
"bn",
|
||||
"ta",
|
||||
"te",
|
||||
"ug",
|
||||
"gu",
|
||||
"my",
|
||||
"tl",
|
||||
"kk",
|
||||
"or",
|
||||
"ne",
|
||||
"mn",
|
||||
"km",
|
||||
"jv",
|
||||
"lo",
|
||||
"si",
|
||||
"fil",
|
||||
"ps",
|
||||
"pa",
|
||||
"kab",
|
||||
"ba",
|
||||
"ks",
|
||||
"tg",
|
||||
"su",
|
||||
"mr",
|
||||
"ky",
|
||||
"az",
|
||||
] as const;
|
||||
|
||||
export const SHERPA_ONNX_MODEL_CATALOG = {
|
||||
"parakeet-tdt-0.6b-v2-int8": {
|
||||
kind: "stt-offline",
|
||||
archiveUrl:
|
||||
"https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8.tar.bz2",
|
||||
extractedDir: "sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8",
|
||||
requiredFiles: ["encoder.int8.onnx", "decoder.int8.onnx", "joiner.int8.onnx", "tokens.txt"],
|
||||
description: "NVIDIA Parakeet TDT v2 (offline NeMo transducer, English).",
|
||||
label: "Parakeet v2",
|
||||
description: "Accurate English transcription that runs well on a laptop CPU.",
|
||||
languages: ["en"],
|
||||
downloadSizeBytes: 482_468_385,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8"),
|
||||
engine: PARAKEET_INT8_ENGINE,
|
||||
defaultFor: "stt",
|
||||
},
|
||||
"parakeet-tdt-110m-en-int8": {
|
||||
kind: "stt-offline",
|
||||
label: "Parakeet Small",
|
||||
description: "Small, fast English model with punctuation. A quick download for any laptop.",
|
||||
languages: ["en"],
|
||||
downloadSizeBytes: 108_035_095,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-nemo-parakeet_tdt_transducer_110m-en-36000-int8"),
|
||||
engine: PARAKEET_INT8_ENGINE,
|
||||
},
|
||||
"parakeet-tdt-0.6b-v3-int8": {
|
||||
kind: "stt-offline",
|
||||
archiveUrl:
|
||||
"https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8.tar.bz2",
|
||||
extractedDir: "sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8",
|
||||
requiredFiles: ["encoder.int8.onnx", "decoder.int8.onnx", "joiner.int8.onnx", "tokens.txt"],
|
||||
description:
|
||||
"NVIDIA Parakeet TDT v3 (offline NeMo transducer, 25 European languages, auto-detected).",
|
||||
label: "Parakeet v3",
|
||||
description: "Parakeet for 25 European languages, detecting which one you speak.",
|
||||
languages: PARAKEET_V3_LANGUAGES,
|
||||
downloadSizeBytes: 487_170_055,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8"),
|
||||
engine: PARAKEET_INT8_ENGINE,
|
||||
},
|
||||
"whisper-tiny-en": {
|
||||
kind: "stt-offline",
|
||||
label: "Whisper Tiny (English)",
|
||||
description: "Small English Whisper model. Quick to download, less accurate than Parakeet.",
|
||||
languages: ["en"],
|
||||
downloadSizeBytes: 118_071_777,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-whisper-tiny.en"),
|
||||
engine: whisperEngine("tiny.en"),
|
||||
},
|
||||
"whisper-base": {
|
||||
kind: "stt-offline",
|
||||
label: "Whisper Base",
|
||||
description: "Compact Whisper model for 99 languages, detecting which one you speak.",
|
||||
languages: WHISPER_LANGUAGES,
|
||||
downloadSizeBytes: 207_557_382,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-whisper-base"),
|
||||
engine: whisperEngine("base"),
|
||||
},
|
||||
"whisper-turbo": {
|
||||
kind: "stt-offline",
|
||||
label: "Whisper Turbo",
|
||||
description: "Whisper's most accurate model for 99 languages and Cantonese. Slower on a CPU.",
|
||||
languages: [...WHISPER_LANGUAGES, "yue"],
|
||||
downloadSizeBytes: 563_790_207,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-whisper-turbo"),
|
||||
engine: whisperEngine("turbo"),
|
||||
},
|
||||
"sense-voice-int8": {
|
||||
kind: "stt-offline",
|
||||
label: "SenseVoice",
|
||||
description: "Chinese, Cantonese, English, Japanese and Korean, detecting which one you speak.",
|
||||
languages: ["zh", "yue", "en", "ja", "ko"],
|
||||
downloadSizeBytes: 165_783_878,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2025-09-09"),
|
||||
engine: { family: "sense_voice", model: "model.int8.onnx", tokens: "tokens.txt" },
|
||||
},
|
||||
"dolphin-base-int8": {
|
||||
kind: "stt-offline",
|
||||
label: "Dolphin",
|
||||
description: "Small model for 40 Asian and Middle Eastern languages and Chinese dialects.",
|
||||
languages: DOLPHIN_LANGUAGES,
|
||||
downloadSizeBytes: 80_671_385,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-dolphin-base-ctc-multi-lang-int8-2025-04-02"),
|
||||
engine: { family: "dolphin", model: "model.int8.onnx", tokens: "tokens.txt" },
|
||||
},
|
||||
"giga-am-v3-ru": {
|
||||
kind: "stt-offline",
|
||||
label: "GigaAM v3 (Russian)",
|
||||
description: "Russian transcription with punctuation.",
|
||||
languages: ["ru"],
|
||||
downloadSizeBytes: 170_197_019,
|
||||
...releaseArchive(
|
||||
"asr-models",
|
||||
"sherpa-onnx-nemo-transducer-punct-giga-am-v3-russian-2025-12-16",
|
||||
),
|
||||
engine: {
|
||||
family: "transducer",
|
||||
encoder: "encoder.int8.onnx",
|
||||
decoder: "decoder.onnx",
|
||||
joiner: "joiner.onnx",
|
||||
tokens: "tokens.txt",
|
||||
},
|
||||
},
|
||||
"parakeet-tdt-ctc-0.6b-ja-int8": {
|
||||
kind: "stt-offline",
|
||||
label: "Parakeet (Japanese)",
|
||||
description: "Japanese transcription with punctuation.",
|
||||
languages: ["ja"],
|
||||
downloadSizeBytes: 489_389_564,
|
||||
...releaseArchive("asr-models", "sherpa-onnx-nemo-parakeet-tdt_ctc-0.6b-ja-35000-int8"),
|
||||
engine: { family: "nemo_ctc", model: "model.int8.onnx", tokens: "tokens.txt" },
|
||||
},
|
||||
"kokoro-en-v0_19": {
|
||||
kind: "tts",
|
||||
archiveUrl:
|
||||
"https://github.com/k2-fsa/sherpa-onnx/releases/download/tts-models/kokoro-en-v0_19.tar.bz2",
|
||||
extractedDir: "kokoro-en-v0_19",
|
||||
requiredFiles: ["model.onnx", "voices.bin", "tokens.txt", "espeak-ng-data"],
|
||||
description: "Kokoro TTS (higher quality; larger).",
|
||||
label: "Kokoro (English)",
|
||||
description: "Natural-sounding English voice for voice mode replies.",
|
||||
languages: ["en"],
|
||||
downloadSizeBytes: 319_625_534,
|
||||
...releaseArchive("tts-models", "kokoro-en-v0_19"),
|
||||
engine: {
|
||||
family: "kokoro",
|
||||
model: "model.onnx",
|
||||
voices: "voices.bin",
|
||||
tokens: "tokens.txt",
|
||||
dataDir: "espeak-ng-data",
|
||||
},
|
||||
defaultFor: "tts",
|
||||
},
|
||||
"kokoro-int8-en-v0_19": {
|
||||
kind: "tts",
|
||||
label: "Kokoro Lite (English)",
|
||||
description: "The same Kokoro English voices, quantized to a third of the download.",
|
||||
languages: ["en"],
|
||||
downloadSizeBytes: 103_248_205,
|
||||
...releaseArchive("tts-models", "kokoro-int8-en-v0_19"),
|
||||
engine: {
|
||||
family: "kokoro",
|
||||
model: "model.int8.onnx",
|
||||
voices: "voices.bin",
|
||||
tokens: "tokens.txt",
|
||||
dataDir: "espeak-ng-data",
|
||||
},
|
||||
},
|
||||
"kokoro-int8-multi-lang-v1_1": {
|
||||
kind: "tts",
|
||||
label: "Kokoro (Chinese and English)",
|
||||
description: "Kokoro voices that speak both Mandarin Chinese and English.",
|
||||
languages: ["zh", "en"],
|
||||
downloadSizeBytes: 147_031_220,
|
||||
...releaseArchive("tts-models", "kokoro-int8-multi-lang-v1_1"),
|
||||
engine: {
|
||||
family: "kokoro",
|
||||
model: "model.int8.onnx",
|
||||
voices: "voices.bin",
|
||||
tokens: "tokens.txt",
|
||||
dataDir: "espeak-ng-data",
|
||||
lexicons: ["lexicon-us-en.txt", "lexicon-zh.txt"],
|
||||
dictDir: "dict",
|
||||
ruleFsts: ["date-zh.fst", "phone-zh.fst", "number-zh.fst"],
|
||||
},
|
||||
},
|
||||
// Kitten v0.8 needs a newer sherpa-onnx than we pin; v0.2 loads on 1.12.28.
|
||||
"kitten-nano-en-v0_2-fp16": {
|
||||
kind: "tts",
|
||||
label: "Kitten Nano (English)",
|
||||
description: "Tiny, very fast English voice.",
|
||||
languages: ["en"],
|
||||
downloadSizeBytes: 26_586_708,
|
||||
...releaseArchive("tts-models", "kitten-nano-en-v0_2-fp16"),
|
||||
engine: {
|
||||
family: "kitten",
|
||||
model: "model.fp16.onnx",
|
||||
voices: "voices.bin",
|
||||
tokens: "tokens.txt",
|
||||
dataDir: "espeak-ng-data",
|
||||
},
|
||||
},
|
||||
"piper-en-us-libritts-r": {
|
||||
kind: "tts",
|
||||
label: "Piper LibriTTS (US English)",
|
||||
description: "Lightweight American English voice.",
|
||||
languages: ["en-US"],
|
||||
downloadSizeBytes: 23_398_348,
|
||||
...releaseArchive("tts-models", "vits-piper-en_US-libritts_r-medium-int8"),
|
||||
engine: piperEngine("en_US-libritts_r-medium"),
|
||||
},
|
||||
"piper-en-gb-alba": {
|
||||
kind: "tts",
|
||||
label: "Piper Alba (British English)",
|
||||
description: "Lightweight British English voice.",
|
||||
languages: ["en-GB"],
|
||||
downloadSizeBytes: 21_104_326,
|
||||
...releaseArchive("tts-models", "vits-piper-en_GB-alba-medium-int8"),
|
||||
engine: piperEngine("en_GB-alba-medium"),
|
||||
},
|
||||
"piper-de-thorsten": {
|
||||
kind: "tts",
|
||||
label: "Piper Thorsten (German)",
|
||||
description: "Lightweight German voice.",
|
||||
languages: ["de"],
|
||||
downloadSizeBytes: 20_949_833,
|
||||
...releaseArchive("tts-models", "vits-piper-de_DE-thorsten-medium-int8"),
|
||||
engine: piperEngine("de_DE-thorsten-medium"),
|
||||
},
|
||||
"piper-fr-siwis": {
|
||||
kind: "tts",
|
||||
label: "Piper Siwis (French)",
|
||||
description: "Lightweight French voice.",
|
||||
languages: ["fr"],
|
||||
downloadSizeBytes: 20_914_888,
|
||||
...releaseArchive("tts-models", "vits-piper-fr_FR-siwis-medium-int8"),
|
||||
engine: piperEngine("fr_FR-siwis-medium"),
|
||||
},
|
||||
"piper-es-davefx": {
|
||||
kind: "tts",
|
||||
label: "Piper Davefx (Spanish)",
|
||||
description: "Lightweight Spanish voice.",
|
||||
languages: ["es"],
|
||||
downloadSizeBytes: 21_171_632,
|
||||
...releaseArchive("tts-models", "vits-piper-es_ES-davefx-medium-int8"),
|
||||
engine: piperEngine("es_ES-davefx-medium"),
|
||||
},
|
||||
} as const satisfies Record<string, SherpaOnnxCatalogEntry>;
|
||||
|
||||
export type SherpaOnnxModelId = keyof typeof SHERPA_ONNX_MODEL_CATALOG;
|
||||
@@ -97,24 +521,44 @@ function createModelIdSchema<T extends string>(modelIds: readonly T[]): z.ZodTyp
|
||||
.transform((value) => value as T);
|
||||
}
|
||||
|
||||
export const LocalSpeechModelIdSchema = createModelIdSchema(ALL_MODEL_IDS);
|
||||
export const LocalSttModelIdSchema = createModelIdSchema(LOCAL_STT_MODEL_IDS);
|
||||
export const LocalTtsModelIdSchema = createModelIdSchema(LOCAL_TTS_MODEL_IDS);
|
||||
|
||||
export type SherpaOnnxModelSpec = SherpaOnnxCatalogEntry & {
|
||||
id: SherpaOnnxModelId;
|
||||
// Every file and directory that must exist for the model to count as installed.
|
||||
requiredFiles: string[];
|
||||
};
|
||||
|
||||
export function listSherpaOnnxModels(): SherpaOnnxModelSpec[] {
|
||||
return ALL_MODEL_IDS.map((id) => Object.assign({ id }, SHERPA_ONNX_MODEL_CATALOG[id]));
|
||||
return ALL_MODEL_IDS.map(getSherpaOnnxModelSpec);
|
||||
}
|
||||
|
||||
export function getSherpaOnnxModelSpec(id: SherpaOnnxModelId): SherpaOnnxModelSpec {
|
||||
const spec = SHERPA_ONNX_MODEL_CATALOG[id];
|
||||
const spec: SherpaOnnxCatalogEntry | undefined = SHERPA_ONNX_MODEL_CATALOG[id];
|
||||
if (!spec) {
|
||||
throw new Error(`Unknown local speech model id: ${id}`);
|
||||
}
|
||||
return {
|
||||
id,
|
||||
...spec,
|
||||
requiredFiles: engineFiles(spec.engine),
|
||||
};
|
||||
}
|
||||
|
||||
// Every path an engine names, so the downloader can verify an install.
|
||||
export function engineFiles(engine: SttEngine | TtsEngine): string[] {
|
||||
return Object.entries(engine).flatMap(([key, value]) => {
|
||||
if (key === "family") return [];
|
||||
return typeof value === "string" ? [value] : (value as readonly string[]);
|
||||
});
|
||||
}
|
||||
|
||||
export function getLocalSttEngine(id: LocalSttModelId): SttEngine {
|
||||
return SHERPA_ONNX_MODEL_CATALOG[id].engine;
|
||||
}
|
||||
|
||||
export function getLocalTtsEngine(id: LocalTtsModelId): TtsEngine {
|
||||
return SHERPA_ONNX_MODEL_CATALOG[id].engine;
|
||||
}
|
||||
|
||||
@@ -8,11 +8,16 @@ import type pino from "pino";
|
||||
import { getSherpaOnnxModelSpec, type SherpaOnnxModelId } from "./model-catalog.js";
|
||||
import { spawnProcess } from "../../../../../utils/spawn.js";
|
||||
|
||||
export type SherpaOnnxModelInstallProgress =
|
||||
| { phase: "downloading"; receivedBytes: number; totalBytes: number }
|
||||
| { phase: "extracting" };
|
||||
|
||||
export interface EnsureSherpaOnnxModelOptions {
|
||||
modelsDir: string;
|
||||
modelId: SherpaOnnxModelId;
|
||||
logger: pino.Logger;
|
||||
signal?: AbortSignal;
|
||||
onProgress?: (progress: SherpaOnnxModelInstallProgress) => void;
|
||||
}
|
||||
|
||||
export function getSherpaOnnxModelDir(modelsDir: string, modelId: SherpaOnnxModelId): string {
|
||||
@@ -41,7 +46,9 @@ async function hasRequiredFiles(modelDir: string, requiredFiles: string[]): Prom
|
||||
interface DownloadToFileOptions {
|
||||
url: string;
|
||||
outputPath: string;
|
||||
expectedBytes: number;
|
||||
signal?: AbortSignal;
|
||||
onProgress?: (receivedBytes: number, totalBytes: number) => void;
|
||||
}
|
||||
|
||||
async function downloadToFile(options: DownloadToFileOptions): Promise<void> {
|
||||
@@ -60,6 +67,13 @@ async function downloadToFile(options: DownloadToFileOptions): Promise<void> {
|
||||
// The fetch ReadableStream type is slightly different from what Readable.fromWeb expects
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
const nodeStream = Readable.fromWeb(res.body as any);
|
||||
const contentLength = Number(res.headers.get("content-length"));
|
||||
const totalBytes = contentLength > 0 ? contentLength : options.expectedBytes;
|
||||
let receivedBytes = 0;
|
||||
nodeStream.on("data", (chunk: Buffer) => {
|
||||
receivedBytes += chunk.length;
|
||||
options.onProgress?.(receivedBytes, totalBytes);
|
||||
});
|
||||
|
||||
try {
|
||||
await pipeline(nodeStream, createWriteStream(tmpPath), { signal: options.signal });
|
||||
@@ -157,9 +171,13 @@ export async function ensureSherpaOnnxModel(
|
||||
await downloadToFile({
|
||||
url: spec.archiveUrl,
|
||||
outputPath: archivePath,
|
||||
expectedBytes: spec.downloadSizeBytes,
|
||||
signal: options.signal,
|
||||
onProgress: (receivedBytes, totalBytes) =>
|
||||
options.onProgress?.({ phase: "downloading", receivedBytes, totalBytes }),
|
||||
});
|
||||
}
|
||||
options.onProgress?.({ phase: "extracting" });
|
||||
|
||||
logger.info(
|
||||
{
|
||||
@@ -213,6 +231,16 @@ export async function ensureSherpaOnnxModel(
|
||||
}
|
||||
}
|
||||
|
||||
export async function deleteSherpaOnnxModel(
|
||||
modelsDir: string,
|
||||
modelId: SherpaOnnxModelId,
|
||||
): Promise<void> {
|
||||
const { modelDir, archivePath, extractionDir } = getSherpaOnnxModelPaths(modelsDir, modelId);
|
||||
await rm(modelDir, { recursive: true, force: true });
|
||||
await rm(archivePath, { force: true });
|
||||
await rm(extractionDir, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
export async function ensureSherpaOnnxModels(options: {
|
||||
modelsDir: string;
|
||||
modelIds: SherpaOnnxModelId[];
|
||||
|
||||
+41
-18
@@ -1,6 +1,8 @@
|
||||
import { existsSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import type pino from "pino";
|
||||
|
||||
import { engineFiles, type SttEngine } from "./model-catalog.js";
|
||||
import { loadSherpaOnnxNode } from "./sherpa-onnx-node-loader.js";
|
||||
|
||||
function assertFileExists(filePath: string, label: string): void {
|
||||
@@ -9,16 +11,43 @@ function assertFileExists(filePath: string, label: string): void {
|
||||
}
|
||||
}
|
||||
|
||||
export interface SherpaOfflineRecognizerModel {
|
||||
kind: "nemo_transducer";
|
||||
encoder: string;
|
||||
decoder: string;
|
||||
joiner: string;
|
||||
tokens: string;
|
||||
// Each engine family takes its own block under sherpa's modelConfig.
|
||||
function buildModelConfig(modelDir: string, engine: SttEngine): Record<string, unknown> {
|
||||
const file = (name: string) => path.join(modelDir, name);
|
||||
switch (engine.family) {
|
||||
case "transducer":
|
||||
return {
|
||||
transducer: {
|
||||
encoder: file(engine.encoder),
|
||||
decoder: file(engine.decoder),
|
||||
joiner: file(engine.joiner),
|
||||
},
|
||||
modelType: "nemo_transducer",
|
||||
};
|
||||
case "whisper":
|
||||
// An empty language makes Whisper detect the spoken language.
|
||||
return {
|
||||
whisper: {
|
||||
encoder: file(engine.encoder),
|
||||
decoder: file(engine.decoder),
|
||||
language: "",
|
||||
task: "transcribe",
|
||||
},
|
||||
};
|
||||
case "sense_voice":
|
||||
return {
|
||||
senseVoice: { model: file(engine.model), language: "", useInverseTextNormalization: 1 },
|
||||
};
|
||||
case "dolphin":
|
||||
return { dolphin: { model: file(engine.model) } };
|
||||
case "nemo_ctc":
|
||||
return { nemoCtc: { model: file(engine.model) } };
|
||||
}
|
||||
}
|
||||
|
||||
export interface SherpaOfflineRecognizerConfig {
|
||||
model: SherpaOfflineRecognizerModel;
|
||||
modelDir: string;
|
||||
engine: SttEngine;
|
||||
numThreads?: number;
|
||||
provider?: "cpu";
|
||||
debug?: 0 | 1;
|
||||
@@ -54,10 +83,9 @@ export class SherpaOfflineRecognizerEngine {
|
||||
component: "offline-recognizer",
|
||||
});
|
||||
|
||||
assertFileExists(config.model.encoder, "offline encoder");
|
||||
assertFileExists(config.model.decoder, "offline decoder");
|
||||
assertFileExists(config.model.joiner, "offline joiner");
|
||||
assertFileExists(config.model.tokens, "tokens");
|
||||
for (const name of engineFiles(config.engine)) {
|
||||
assertFileExists(path.join(config.modelDir, name), `${config.engine.family} model file`);
|
||||
}
|
||||
|
||||
const sherpa = loadSherpaOnnxNode();
|
||||
|
||||
@@ -67,13 +95,8 @@ export class SherpaOfflineRecognizerEngine {
|
||||
featureDim: config.featureDim ?? 80,
|
||||
},
|
||||
modelConfig: {
|
||||
transducer: {
|
||||
encoder: config.model.encoder,
|
||||
decoder: config.model.decoder,
|
||||
joiner: config.model.joiner,
|
||||
},
|
||||
tokens: config.model.tokens,
|
||||
modelType: "nemo_transducer",
|
||||
...buildModelConfig(config.modelDir, config.engine),
|
||||
tokens: path.join(config.modelDir, config.engine.tokens),
|
||||
numThreads: config.numThreads ?? 1,
|
||||
provider: config.provider ?? "cpu",
|
||||
debug: config.debug ?? 0,
|
||||
|
||||
@@ -3,6 +3,7 @@ import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import pino from "pino";
|
||||
import { engineFiles, getLocalTtsEngine, type LocalTtsModelId } from "./model-catalog.js";
|
||||
import { SherpaOnnxTTS } from "./sherpa-tts.js";
|
||||
|
||||
const directories: string[] = [];
|
||||
@@ -11,17 +12,28 @@ afterEach(() => {
|
||||
rmSync(directory, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function createTts(numThreads?: number, beforeGenerate?: () => Promise<void>) {
|
||||
function createTts(
|
||||
numThreads?: number,
|
||||
beforeGenerate?: () => Promise<void>,
|
||||
modelId: LocalTtsModelId = "kokoro-en-v0_19",
|
||||
) {
|
||||
const modelDir = mkdtempSync(join(tmpdir(), "paseo-tts-"));
|
||||
directories.push(modelDir);
|
||||
for (const name of ["model.onnx", "voices.bin", "tokens.txt"])
|
||||
writeFileSync(join(modelDir, name), "");
|
||||
mkdirSync(join(modelDir, "espeak-ng-data"));
|
||||
// Data directories have no extension; everything else is a model file.
|
||||
for (const name of engineFiles(getLocalTtsEngine(modelId))) {
|
||||
if (name.includes(".")) writeFileSync(join(modelDir, name), "");
|
||||
else mkdirSync(join(modelDir, name));
|
||||
}
|
||||
let nativeConfig: unknown;
|
||||
const requests: unknown[] = [];
|
||||
const released: boolean[] = [];
|
||||
const tts = new SherpaOnnxTTS(
|
||||
{ preset: "kokoro-en-v0_19", modelDir, numThreads },
|
||||
{
|
||||
modelId,
|
||||
modelDir,
|
||||
engine: getLocalTtsEngine(modelId),
|
||||
numThreads,
|
||||
},
|
||||
pino({ level: "silent" }),
|
||||
() => ({
|
||||
OfflineTts: class {
|
||||
@@ -138,4 +150,31 @@ describe("SherpaOnnxTTS", () => {
|
||||
expect(nativeConfig).not.toHaveProperty("provider");
|
||||
},
|
||||
);
|
||||
|
||||
it("loads Piper voices through the vits engine", () => {
|
||||
const { nativeConfig } = createTts(undefined, undefined, "piper-de-thorsten");
|
||||
expect(nativeConfig).toMatchObject({
|
||||
model: {
|
||||
vits: {
|
||||
model: expect.stringMatching(/de_DE-thorsten-medium\.onnx$/),
|
||||
tokens: expect.stringMatching(/tokens\.txt$/),
|
||||
dataDir: expect.stringMatching(/espeak-ng-data$/),
|
||||
},
|
||||
},
|
||||
});
|
||||
expect(nativeConfig).not.toHaveProperty("model.kokoro");
|
||||
});
|
||||
|
||||
it("gives multi-language Kokoro its lexicons, dictionary and number rules", () => {
|
||||
const { nativeConfig } = createTts(undefined, undefined, "kokoro-int8-multi-lang-v1_1");
|
||||
expect(nativeConfig).toMatchObject({
|
||||
model: {
|
||||
kokoro: {
|
||||
lexicon: expect.stringMatching(/lexicon-us-en\.txt,.*lexicon-zh\.txt$/),
|
||||
dictDir: expect.stringMatching(/dict$/),
|
||||
},
|
||||
},
|
||||
ruleFsts: expect.stringMatching(/date-zh\.fst,.*phone-zh\.fst,.*number-zh\.fst$/),
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,16 +1,17 @@
|
||||
import type pino from "pino";
|
||||
import { Readable } from "node:stream";
|
||||
import { existsSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
|
||||
import type { SpeechStreamResult, TextToSpeechProvider } from "../../../speech-provider.js";
|
||||
import { chunkBuffer, float32ToPcm16le } from "../../../audio.js";
|
||||
import { loadSherpaOnnxNode } from "./sherpa-onnx-node-loader.js";
|
||||
|
||||
export type SherpaTtsPreset = "kokoro-en-v0_19";
|
||||
import { engineFiles, type LocalTtsModelId, type TtsEngine } from "./model-catalog.js";
|
||||
|
||||
export interface SherpaTtsConfig {
|
||||
preset: SherpaTtsPreset;
|
||||
modelId: LocalTtsModelId;
|
||||
modelDir: string;
|
||||
engine: TtsEngine;
|
||||
speakerId?: number;
|
||||
speed?: number;
|
||||
lengthScale?: number;
|
||||
@@ -23,6 +24,53 @@ function assertFileExists(filePath: string, label: string): void {
|
||||
}
|
||||
}
|
||||
|
||||
// Each engine family takes its own block under sherpa's TTS model config.
|
||||
function buildTtsModelConfig(
|
||||
modelDir: string,
|
||||
engine: TtsEngine,
|
||||
lengthScale: number,
|
||||
): Record<string, unknown> {
|
||||
const file = (name: string) => path.join(modelDir, name);
|
||||
switch (engine.family) {
|
||||
case "kokoro":
|
||||
return {
|
||||
kokoro: {
|
||||
model: file(engine.model),
|
||||
voices: file(engine.voices),
|
||||
tokens: file(engine.tokens),
|
||||
dataDir: file(engine.dataDir),
|
||||
lexicon: (engine.lexicons ?? []).map(file).join(","),
|
||||
dictDir: engine.dictDir ? file(engine.dictDir) : "",
|
||||
lengthScale,
|
||||
},
|
||||
};
|
||||
case "kitten":
|
||||
return {
|
||||
kitten: {
|
||||
model: file(engine.model),
|
||||
voices: file(engine.voices),
|
||||
tokens: file(engine.tokens),
|
||||
dataDir: file(engine.dataDir),
|
||||
lengthScale,
|
||||
},
|
||||
};
|
||||
case "vits":
|
||||
return {
|
||||
vits: {
|
||||
model: file(engine.model),
|
||||
tokens: file(engine.tokens),
|
||||
dataDir: file(engine.dataDir),
|
||||
lengthScale,
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
function ruleFsts(modelDir: string, engine: TtsEngine): string {
|
||||
if (engine.family !== "kokoro" || !engine.ruleFsts) return "";
|
||||
return engine.ruleFsts.map((name) => path.join(modelDir, name)).join(",");
|
||||
}
|
||||
|
||||
interface SherpaOfflineTtsNative {
|
||||
sampleRate?: number;
|
||||
generateAsync: (args: {
|
||||
@@ -59,39 +107,28 @@ export class SherpaOnnxTTS implements TextToSpeechProvider {
|
||||
throw new Error("sherpa-onnx-node OfflineTts is unavailable");
|
||||
}
|
||||
|
||||
const modelPath = `${config.modelDir}/model.onnx`;
|
||||
const voicesPath = `${config.modelDir}/voices.bin`;
|
||||
const tokensPath = `${config.modelDir}/tokens.txt`;
|
||||
const dataDir = `${config.modelDir}/espeak-ng-data`;
|
||||
|
||||
assertFileExists(modelPath, "TTS model");
|
||||
assertFileExists(voicesPath, "TTS voices");
|
||||
assertFileExists(tokensPath, "TTS tokens");
|
||||
assertFileExists(dataDir, "TTS espeak-ng dataDir");
|
||||
for (const name of engineFiles(config.engine)) {
|
||||
assertFileExists(path.join(config.modelDir, name), `${config.engine.family} TTS file`);
|
||||
}
|
||||
|
||||
const modelConfig = {
|
||||
// The native parser reads these under model, despite the upstream JS typedef.
|
||||
numThreads: config.numThreads ?? 2,
|
||||
provider: "cpu",
|
||||
kokoro: {
|
||||
model: modelPath,
|
||||
voices: voicesPath,
|
||||
tokens: tokensPath,
|
||||
dataDir,
|
||||
lengthScale: config.lengthScale ?? 1.0,
|
||||
},
|
||||
...buildTtsModelConfig(config.modelDir, config.engine, config.lengthScale ?? 1.0),
|
||||
};
|
||||
|
||||
const offlineTtsConfig = {
|
||||
model: modelConfig,
|
||||
maxNumSentences: 1,
|
||||
ruleFsts: ruleFsts(config.modelDir, config.engine),
|
||||
};
|
||||
|
||||
this.tts = new (
|
||||
sherpa as unknown as { OfflineTts: new (config: unknown) => SherpaOfflineTtsNative }
|
||||
).OfflineTts(offlineTtsConfig);
|
||||
this.logger.info(
|
||||
{ preset: config.preset, modelDir: config.modelDir },
|
||||
{ modelId: config.modelId, modelDir: config.modelDir },
|
||||
"Sherpa offline TTS initialized",
|
||||
);
|
||||
}
|
||||
|
||||
+9
-8
@@ -7,6 +7,7 @@ import pino from "pino";
|
||||
import { ensureSherpaOnnxModels, getSherpaOnnxModelDir } from "./model-downloader.js";
|
||||
import { createDaemonTestContext } from "../../../../test-utils/index.js";
|
||||
import { parsePcm16MonoWav, wordSimilarity } from "../../../../test-utils/dictation-e2e.js";
|
||||
import { getLocalSttEngine, getLocalTtsEngine } from "./model-catalog.js";
|
||||
import { SherpaOnnxTTS } from "./sherpa-tts.js";
|
||||
import { SherpaOfflineRecognizerEngine } from "./sherpa-offline-recognizer.js";
|
||||
import { SherpaOnnxParakeetSTT } from "./sherpa-parakeet-stt.js";
|
||||
@@ -192,7 +193,12 @@ downloadTest(
|
||||
const ttsText = "This is a voice note.";
|
||||
const ttsModelDir = getSherpaOnnxModelDir(modelsDir, "kokoro-en-v0_19");
|
||||
const tts = new SherpaOnnxTTS(
|
||||
{ preset: "kokoro-en-v0_19", modelDir: ttsModelDir, speakerId: 0 },
|
||||
{
|
||||
modelId: "kokoro-en-v0_19",
|
||||
modelDir: ttsModelDir,
|
||||
engine: getLocalTtsEngine("kokoro-en-v0_19"),
|
||||
speakerId: 0,
|
||||
},
|
||||
logger,
|
||||
);
|
||||
const { stream, format: ttsFormat } = await tts.synthesizeSpeech(ttsText);
|
||||
@@ -206,13 +212,8 @@ downloadTest(
|
||||
const sttModelDir = getSherpaOnnxModelDir(modelsDir, "parakeet-tdt-0.6b-v2-int8");
|
||||
const engine = new SherpaOfflineRecognizerEngine(
|
||||
{
|
||||
model: {
|
||||
kind: "nemo_transducer",
|
||||
encoder: `${sttModelDir}/encoder.int8.onnx`,
|
||||
decoder: `${sttModelDir}/decoder.int8.onnx`,
|
||||
joiner: `${sttModelDir}/joiner.int8.onnx`,
|
||||
tokens: `${sttModelDir}/tokens.txt`,
|
||||
},
|
||||
modelDir: sttModelDir,
|
||||
engine: getLocalSttEngine("parakeet-tdt-0.6b-v2-int8"),
|
||||
numThreads: 2,
|
||||
debug: 0,
|
||||
},
|
||||
|
||||
@@ -3,6 +3,7 @@ import pino from "pino";
|
||||
import type { StreamingTranscriptionSession } from "../../speech-provider.js";
|
||||
import type { TurnDetectionSession } from "../../turn-detection-provider.js";
|
||||
import { getLocalSpeechModelDir, type LocalSttModelId, type LocalTtsModelId } from "./models.js";
|
||||
import { getLocalSttEngine, getLocalTtsEngine } from "./sherpa/model-catalog.js";
|
||||
import { SherpaOfflineRecognizerEngine } from "./sherpa/sherpa-offline-recognizer.js";
|
||||
import { SherpaOnnxParakeetSTT } from "./sherpa/sherpa-parakeet-stt.js";
|
||||
import { SherpaParakeetRealtimeTranscriptionSession } from "./sherpa/sherpa-parakeet-realtime-session.js";
|
||||
@@ -85,13 +86,8 @@ function getSttEngine(
|
||||
const modelDir = getLocalSpeechModelDir(config.modelsDir, modelId);
|
||||
const created = new SherpaOfflineRecognizerEngine(
|
||||
{
|
||||
model: {
|
||||
kind: "nemo_transducer",
|
||||
encoder: `${modelDir}/encoder.int8.onnx`,
|
||||
decoder: `${modelDir}/decoder.int8.onnx`,
|
||||
joiner: `${modelDir}/joiner.int8.onnx`,
|
||||
tokens: `${modelDir}/tokens.txt`,
|
||||
},
|
||||
modelDir,
|
||||
engine: getLocalSttEngine(modelId),
|
||||
numThreads: 2,
|
||||
debug: 0,
|
||||
},
|
||||
@@ -125,8 +121,9 @@ function getTtsProvider(config: LocalSpeechWorkerConfig): SherpaOnnxTTS {
|
||||
const modelDir = getLocalSpeechModelDir(config.modelsDir, ttsModelId(config));
|
||||
const created = new SherpaOnnxTTS(
|
||||
{
|
||||
preset: ttsModelId(config),
|
||||
modelId: ttsModelId(config),
|
||||
modelDir,
|
||||
engine: getLocalTtsEngine(ttsModelId(config)),
|
||||
speakerId: config.voiceTtsSpeakerId,
|
||||
speed: config.voiceTtsSpeed,
|
||||
},
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import { describe, expect, test } from "vitest";
|
||||
|
||||
import { PersistedConfigSchema } from "../../../persisted-config.js";
|
||||
import { resolveOpenAiSpeechConfig } from "./config.js";
|
||||
import type { RequestedSpeechProviders } from "../../speech-types.js";
|
||||
@@ -10,233 +9,139 @@ const ALL_OPENAI: RequestedSpeechProviders = {
|
||||
voiceStt: { provider: "openai", explicit: true },
|
||||
voiceTts: { provider: "openai", explicit: true },
|
||||
};
|
||||
function resolve(input: unknown, env: NodeJS.ProcessEnv = {}) {
|
||||
return resolveOpenAiSpeechConfig({
|
||||
persisted: PersistedConfigSchema.parse(input),
|
||||
env,
|
||||
providers: ALL_OPENAI,
|
||||
});
|
||||
}
|
||||
|
||||
describe("resolveOpenAiSpeechConfig", () => {
|
||||
test("treats empty OPENAI_API_KEY as unset", () => {
|
||||
const persisted = PersistedConfigSchema.parse({});
|
||||
const env = {
|
||||
OPENAI_API_KEY: "",
|
||||
} as NodeJS.ProcessEnv;
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({
|
||||
env,
|
||||
persisted,
|
||||
providers: {
|
||||
...ALL_OPENAI,
|
||||
dictationStt: { provider: "local", explicit: false },
|
||||
voiceStt: { provider: "local", explicit: false },
|
||||
voiceTts: { provider: "local", explicit: false },
|
||||
test("treats empty credentials as unset", () => {
|
||||
expect(resolve({}, { OPENAI_API_KEY: "", OPENAI_STT_API_KEY: " " })).toBeUndefined();
|
||||
});
|
||||
test("uses endpoint env before global env and ignores empty fallbacks", () => {
|
||||
const result = resolve(
|
||||
{},
|
||||
{
|
||||
OPENAI_API_KEY: " global ",
|
||||
OPENAI_BASE_URL: " https://global/v1 ",
|
||||
OPENAI_STT_API_KEY: " stt ",
|
||||
OPENAI_STT_BASE_URL: " https://stt/v1 ",
|
||||
OPENAI_TTS_API_KEY: " ",
|
||||
OPENAI_TTS_BASE_URL: "",
|
||||
},
|
||||
);
|
||||
expect(result?.dictationStt).toEqual({ apiKey: "stt", baseUrl: "https://stt/v1" });
|
||||
expect(result?.stt).toEqual({ apiKey: "stt", baseUrl: "https://stt/v1" });
|
||||
expect(result?.tts).toEqual({
|
||||
apiKey: "global",
|
||||
baseUrl: "https://global/v1",
|
||||
model: "tts-1",
|
||||
voice: "alloy",
|
||||
responseFormat: "pcm",
|
||||
});
|
||||
|
||||
expect(resolved).toBeUndefined();
|
||||
});
|
||||
|
||||
test("applies trimmed OPENAI_API_KEY to both STT and TTS", () => {
|
||||
const persisted = PersistedConfigSchema.parse({});
|
||||
const env = {
|
||||
OPENAI_API_KEY: " sk-test ",
|
||||
} as NodeJS.ProcessEnv;
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("sk-test");
|
||||
expect(resolved?.tts?.apiKey).toBe("sk-test");
|
||||
});
|
||||
|
||||
test("resolves distinct endpoints for STT and TTS", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
providers: {
|
||||
openai: {
|
||||
stt: {
|
||||
apiKey: "stt-key",
|
||||
baseUrl: " https://stt.example.com/v1 ",
|
||||
test("each site wins over endpoint and global env credentials and URL", () => {
|
||||
const result = resolve(
|
||||
{
|
||||
features: {
|
||||
dictation: {
|
||||
stt: {
|
||||
provider: "openai",
|
||||
apiKey: "dictation",
|
||||
baseUrl: "https://dictation/v1",
|
||||
model: "dictation-model",
|
||||
confidenceThreshold: -1,
|
||||
},
|
||||
},
|
||||
tts: {
|
||||
apiKey: "tts-key",
|
||||
baseUrl: " https://tts.example.com/v1 ",
|
||||
voiceMode: {
|
||||
stt: {
|
||||
provider: "openai",
|
||||
apiKey: "voice",
|
||||
baseUrl: "https://voice/v1",
|
||||
model: "voice-model",
|
||||
confidenceThreshold: -2,
|
||||
},
|
||||
tts: {
|
||||
provider: "openai",
|
||||
apiKey: "tts",
|
||||
baseUrl: "https://tts/v1",
|
||||
model: "kokoro-v1:Q8_0",
|
||||
voice: "Aiden",
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
OPENAI_API_KEY: "global",
|
||||
OPENAI_BASE_URL: "https://global/v1",
|
||||
OPENAI_STT_API_KEY: "endpoint",
|
||||
OPENAI_STT_BASE_URL: "https://endpoint/v1",
|
||||
OPENAI_TTS_API_KEY: "endpoint",
|
||||
OPENAI_TTS_BASE_URL: "https://endpoint/v1",
|
||||
},
|
||||
);
|
||||
expect(result?.dictationStt).toEqual({
|
||||
apiKey: "dictation",
|
||||
baseUrl: "https://dictation/v1",
|
||||
model: "dictation-model",
|
||||
confidenceThreshold: -1,
|
||||
});
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({
|
||||
env: {} as NodeJS.ProcessEnv,
|
||||
persisted,
|
||||
providers: ALL_OPENAI,
|
||||
expect(result?.stt).toEqual({
|
||||
apiKey: "voice",
|
||||
baseUrl: "https://voice/v1",
|
||||
model: "voice-model",
|
||||
confidenceThreshold: -2,
|
||||
});
|
||||
expect(result?.tts).toEqual({
|
||||
apiKey: "tts",
|
||||
baseUrl: "https://tts/v1",
|
||||
model: "kokoro-v1:Q8_0",
|
||||
voice: "Aiden",
|
||||
responseFormat: "pcm",
|
||||
});
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("stt-key");
|
||||
expect(resolved?.stt?.baseUrl).toBe("https://stt.example.com/v1");
|
||||
expect(resolved?.tts?.apiKey).toBe("tts-key");
|
||||
expect(resolved?.tts?.baseUrl).toBe("https://tts.example.com/v1");
|
||||
});
|
||||
|
||||
test("prefers nested STT/TTS config over env and global fallbacks", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
providers: {
|
||||
openai: {
|
||||
apiKey: "fallback-config-key",
|
||||
baseUrl: "https://global-config.example.com/v1",
|
||||
stt: { apiKey: "stt-config-key", baseUrl: " https://stt.example.com/v1 " },
|
||||
tts: { apiKey: "tts-config-key", baseUrl: " https://tts.example.com/v1 " },
|
||||
test("env overrides model, voice and threshold at every OpenAI site", () => {
|
||||
const result = resolve(
|
||||
{
|
||||
features: {
|
||||
dictation: { stt: { provider: "openai", model: "dictation", confidenceThreshold: -1 } },
|
||||
voiceMode: {
|
||||
stt: { provider: "openai", model: "voice", confidenceThreshold: -2 },
|
||||
tts: { provider: "openai", model: "configured", voice: "Aiden" },
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
const env = {
|
||||
OPENAI_API_KEY: "env-key",
|
||||
OPENAI_STT_API_KEY: "stt-env-key",
|
||||
OPENAI_STT_BASE_URL: "https://stt-env.example.com/v1",
|
||||
OPENAI_TTS_API_KEY: "tts-env-key",
|
||||
OPENAI_TTS_BASE_URL: "https://tts-env.example.com/v1",
|
||||
OPENAI_BASE_URL: "https://env.example.com/v1",
|
||||
} as NodeJS.ProcessEnv;
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("stt-config-key");
|
||||
expect(resolved?.stt?.baseUrl).toBe("https://stt.example.com/v1");
|
||||
expect(resolved?.tts?.apiKey).toBe("tts-config-key");
|
||||
expect(resolved?.tts?.baseUrl).toBe("https://tts.example.com/v1");
|
||||
});
|
||||
|
||||
test("uses STT/TTS env config when nested config is unset", () => {
|
||||
const persisted = PersistedConfigSchema.parse({});
|
||||
const env = {
|
||||
OPENAI_API_KEY: "sk-test",
|
||||
OPENAI_STT_API_KEY: "stt-env-key",
|
||||
OPENAI_STT_BASE_URL: " https://stt-env.example.com/v1 ",
|
||||
OPENAI_TTS_API_KEY: "tts-env-key",
|
||||
OPENAI_TTS_BASE_URL: " https://tts-env.example.com/v1 ",
|
||||
OPENAI_BASE_URL: "https://env.example.com/v1",
|
||||
} as NodeJS.ProcessEnv;
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("stt-env-key");
|
||||
expect(resolved?.stt?.baseUrl).toBe("https://stt-env.example.com/v1");
|
||||
expect(resolved?.tts?.apiKey).toBe("tts-env-key");
|
||||
expect(resolved?.tts?.baseUrl).toBe("https://tts-env.example.com/v1");
|
||||
});
|
||||
|
||||
test("falls back to global OpenAI config for both STT and TTS", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
providers: {
|
||||
openai: {
|
||||
apiKey: "fallback-config-key",
|
||||
baseUrl: " https://global-config.example.com/v1 ",
|
||||
},
|
||||
{
|
||||
OPENAI_API_KEY: "key",
|
||||
STT_MODEL: "override",
|
||||
TTS_MODEL: "tts-override",
|
||||
TTS_VOICE: "Nova",
|
||||
STT_CONFIDENCE_THRESHOLD: "-3",
|
||||
},
|
||||
);
|
||||
expect(result?.dictationStt).toEqual({
|
||||
apiKey: "key",
|
||||
model: "override",
|
||||
confidenceThreshold: -3,
|
||||
});
|
||||
const env = {} as NodeJS.ProcessEnv;
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("fallback-config-key");
|
||||
expect(resolved?.stt?.baseUrl).toBe("https://global-config.example.com/v1");
|
||||
expect(resolved?.tts?.apiKey).toBe("fallback-config-key");
|
||||
expect(resolved?.tts?.baseUrl).toBe("https://global-config.example.com/v1");
|
||||
expect(result?.stt).toEqual({ apiKey: "key", model: "override", confidenceThreshold: -3 });
|
||||
expect(result?.tts).toMatchObject({ model: "tts-override", voice: "Nova" });
|
||||
});
|
||||
|
||||
test("falls back to global OpenAI env config when feature inputs are unset", () => {
|
||||
const persisted = PersistedConfigSchema.parse({});
|
||||
const env = {
|
||||
OPENAI_API_KEY: "env-key",
|
||||
OPENAI_BASE_URL: " https://env.example.com/v1 ",
|
||||
} as NodeJS.ProcessEnv;
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("env-key");
|
||||
expect(resolved?.stt?.baseUrl).toBe("https://env.example.com/v1");
|
||||
expect(resolved?.tts?.apiKey).toBe("env-key");
|
||||
expect(resolved?.tts?.baseUrl).toBe("https://env.example.com/v1");
|
||||
test("only configures endpoints with credentials and ignores unused TTS options", () => {
|
||||
const result = resolve(
|
||||
{ features: { dictation: { stt: { provider: "openai", apiKey: "dictation" } } } },
|
||||
{ TTS_VOICE: "arbitrary", TTS_MODEL: "arbitrary" },
|
||||
);
|
||||
expect(result).toEqual({ dictationStt: { apiKey: "dictation" } });
|
||||
});
|
||||
|
||||
test("ignores empty endpoint env vars and falls back to OPENAI_API_KEY", () => {
|
||||
const persisted = PersistedConfigSchema.parse({});
|
||||
const env = {
|
||||
OPENAI_API_KEY: "global-key",
|
||||
OPENAI_STT_API_KEY: "",
|
||||
OPENAI_STT_BASE_URL: " ",
|
||||
OPENAI_TTS_API_KEY: "",
|
||||
OPENAI_TTS_BASE_URL: "",
|
||||
} as NodeJS.ProcessEnv;
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("global-key");
|
||||
expect(resolved?.tts?.apiKey).toBe("global-key");
|
||||
});
|
||||
|
||||
test("omits TTS when only an STT key is configured", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
providers: {
|
||||
openai: {
|
||||
stt: { apiKey: "stt-only-key" },
|
||||
},
|
||||
},
|
||||
test("does not borrow another site's key", () => {
|
||||
const result = resolve({
|
||||
features: { voiceMode: { stt: { provider: "openai", apiKey: "voice-only" } } },
|
||||
});
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({
|
||||
env: {} as NodeJS.ProcessEnv,
|
||||
persisted,
|
||||
providers: ALL_OPENAI,
|
||||
});
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("stt-only-key");
|
||||
expect(resolved?.tts).toBeUndefined();
|
||||
});
|
||||
|
||||
test("resolves STT even when an unused TTS env var is invalid", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
providers: {
|
||||
openai: {
|
||||
stt: { apiKey: "stt-only-key" },
|
||||
},
|
||||
},
|
||||
});
|
||||
const env = { TTS_VOICE: "not-a-real-voice", TTS_MODEL: "bogus-model" } as NodeJS.ProcessEnv;
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
|
||||
|
||||
expect(resolved?.stt?.apiKey).toBe("stt-only-key");
|
||||
expect(resolved?.tts).toBeUndefined();
|
||||
});
|
||||
|
||||
test("sends the model and voice an OpenAI-compatible server serves, as written", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
providers: {
|
||||
openai: { tts: { apiKey: "tts-key", baseUrl: "https://tts.example.com/v1" } },
|
||||
},
|
||||
features: {
|
||||
voiceMode: { tts: { provider: "openai", model: "kokoro-v1:Q8_0", voice: "Aiden" } },
|
||||
},
|
||||
});
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({
|
||||
env: {} as NodeJS.ProcessEnv,
|
||||
persisted,
|
||||
providers: ALL_OPENAI,
|
||||
});
|
||||
|
||||
expect(resolved?.tts?.model).toBe("kokoro-v1:Q8_0");
|
||||
expect(resolved?.tts?.voice).toBe("Aiden");
|
||||
});
|
||||
|
||||
test("accepts OpenAI's own gpt-4o-mini-tts model", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
providers: { openai: { tts: { apiKey: "tts-key" } } },
|
||||
features: { voiceMode: { tts: { provider: "openai", model: "gpt-4o-mini-tts" } } },
|
||||
});
|
||||
|
||||
const resolved = resolveOpenAiSpeechConfig({
|
||||
env: {} as NodeJS.ProcessEnv,
|
||||
persisted,
|
||||
providers: ALL_OPENAI,
|
||||
});
|
||||
|
||||
expect(resolved?.tts?.model).toBe("gpt-4o-mini-tts");
|
||||
expect(resolved?.tts?.voice).toBe("alloy");
|
||||
expect(result?.dictationStt).toBeUndefined();
|
||||
expect(result?.stt).toEqual({ apiKey: "voice-only" });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,147 +6,61 @@ import type { STTConfig } from "./stt.js";
|
||||
import type { TTSConfig } from "./tts.js";
|
||||
|
||||
export const DEFAULT_OPENAI_TTS_MODEL = "tts-1";
|
||||
export type OpenAiSttFeature = "dictation" | "voiceMode";
|
||||
|
||||
export interface OpenAiSpeechProviderConfig {
|
||||
stt?: Partial<STTConfig> & { apiKey?: string };
|
||||
tts?: Partial<TTSConfig> & { apiKey?: string };
|
||||
dictationStt?: Partial<STTConfig>;
|
||||
stt?: Partial<STTConfig>;
|
||||
tts?: Partial<TTSConfig>;
|
||||
}
|
||||
|
||||
const NumberLikeSchema = z.union([z.number(), z.string().trim().min(1)]);
|
||||
const OptionalFiniteNumberSchema = z
|
||||
.union([z.number(), z.string().trim().min(1)])
|
||||
.pipe(z.coerce.number<string | number>().finite())
|
||||
.optional();
|
||||
|
||||
const OptionalFiniteNumberSchema = NumberLikeSchema.pipe(
|
||||
z.coerce.number<string | number>().finite(),
|
||||
).optional();
|
||||
|
||||
const OptionalTrimmedStringSchema = z
|
||||
.string()
|
||||
.trim()
|
||||
.optional()
|
||||
.transform((value) => (value && value.length > 0 ? value : undefined));
|
||||
|
||||
// Endpoint credentials only — plain trimmed strings, so this never throws on a
|
||||
// malformed value. The STT/TTS option groups parse separately and only for the
|
||||
// endpoint that is actually configured, so a stale env var for an unused endpoint
|
||||
// (e.g. a leftover TTS_VOICE in an STT-only setup) can't break the other one.
|
||||
const OpenAiEndpointKeysSchema = z.object({
|
||||
sttApiKey: OptionalTrimmedStringSchema,
|
||||
sttBaseUrl: OptionalTrimmedStringSchema,
|
||||
ttsApiKey: OptionalTrimmedStringSchema,
|
||||
ttsBaseUrl: OptionalTrimmedStringSchema,
|
||||
});
|
||||
|
||||
const OpenAiSttOptionsSchema = z.object({
|
||||
sttConfidenceThreshold: OptionalFiniteNumberSchema,
|
||||
sttModel: OptionalTrimmedStringSchema,
|
||||
});
|
||||
|
||||
// Voice and model are free-form so a self-hosted server can use its own ids; the
|
||||
// OpenAI catalog is only the default. Both are sent as written, because the
|
||||
// server owns those ids and their case (rosters use ids like kokoro-v1:Q8_0).
|
||||
const OpenAiTtsOptionsSchema = z.object({
|
||||
ttsVoice: z.string().trim().default("alloy"),
|
||||
ttsModel: z.string().trim().default(DEFAULT_OPENAI_TTS_MODEL),
|
||||
});
|
||||
|
||||
function isOpenAiProviderActive(provider: { enabled?: boolean; provider: string }): boolean {
|
||||
return provider.enabled !== false && provider.provider === "openai";
|
||||
function firstString(...values: (string | undefined)[]): string | undefined {
|
||||
return values.map((value) => value?.trim()).find((value) => Boolean(value));
|
||||
}
|
||||
|
||||
function pickIfOpenAi<T>(
|
||||
provider: { enabled?: boolean; provider: string },
|
||||
value: T | undefined,
|
||||
): T | undefined {
|
||||
return isOpenAiProviderActive(provider) ? value : undefined;
|
||||
}
|
||||
type Features = NonNullable<PersistedConfig["features"]>;
|
||||
type SttSite = NonNullable<NonNullable<Features["dictation"]>["stt"]>;
|
||||
type TtsSite = NonNullable<NonNullable<Features["voiceMode"]>["tts"]>;
|
||||
|
||||
function firstDefined<T>(values: Array<T | null | undefined>): T | undefined {
|
||||
for (const value of values) {
|
||||
if (value === undefined || value === null) {
|
||||
continue;
|
||||
}
|
||||
// Empty/whitespace env vars (e.g. a copied .env.example with OPENAI_STT_API_KEY=)
|
||||
// must not shadow a later fallback such as OPENAI_API_KEY.
|
||||
if (typeof value === "string" && value.trim().length === 0) {
|
||||
continue;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function buildOpenAiSttInput(params: {
|
||||
env: NodeJS.ProcessEnv;
|
||||
persisted: PersistedConfig;
|
||||
providers: RequestedSpeechProviders;
|
||||
}): Record<string, unknown> {
|
||||
const { env, persisted, providers } = params;
|
||||
function resolveStt(
|
||||
site: SttSite | undefined,
|
||||
env: NodeJS.ProcessEnv,
|
||||
): Partial<STTConfig> | undefined {
|
||||
const config = site?.provider === "openai" ? site : undefined;
|
||||
const apiKey = firstString(config?.apiKey, env.OPENAI_STT_API_KEY, env.OPENAI_API_KEY);
|
||||
if (!apiKey) return undefined;
|
||||
const baseUrl = firstString(config?.baseUrl, env.OPENAI_STT_BASE_URL, env.OPENAI_BASE_URL);
|
||||
const model = firstString(env.STT_MODEL, config?.model);
|
||||
const confidenceThreshold = OptionalFiniteNumberSchema.parse(
|
||||
firstString(env.STT_CONFIDENCE_THRESHOLD) ?? config?.confidenceThreshold,
|
||||
);
|
||||
return {
|
||||
sttConfidenceThreshold: firstDefined<string | number>([
|
||||
env.STT_CONFIDENCE_THRESHOLD,
|
||||
persisted.features?.dictation?.stt?.confidenceThreshold,
|
||||
]),
|
||||
sttModel: firstDefined<string>([
|
||||
env.STT_MODEL,
|
||||
pickIfOpenAi(providers.voiceStt, persisted.features?.voiceMode?.stt?.model),
|
||||
pickIfOpenAi(providers.dictationStt, persisted.features?.dictation?.stt?.model),
|
||||
]),
|
||||
apiKey,
|
||||
...(baseUrl ? { baseUrl } : {}),
|
||||
...(model ? { model } : {}),
|
||||
...(confidenceThreshold !== undefined ? { confidenceThreshold } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function buildOpenAiTtsInput(params: {
|
||||
env: NodeJS.ProcessEnv;
|
||||
persisted: PersistedConfig;
|
||||
providers: RequestedSpeechProviders;
|
||||
}): Record<string, unknown> {
|
||||
const { env, persisted, providers } = params;
|
||||
function resolveTts(
|
||||
site: TtsSite | undefined,
|
||||
env: NodeJS.ProcessEnv,
|
||||
): Partial<TTSConfig> | undefined {
|
||||
const config = site?.provider === "openai" ? site : undefined;
|
||||
const apiKey = firstString(config?.apiKey, env.OPENAI_TTS_API_KEY, env.OPENAI_API_KEY);
|
||||
if (!apiKey) return undefined;
|
||||
const baseUrl = firstString(config?.baseUrl, env.OPENAI_TTS_BASE_URL, env.OPENAI_BASE_URL);
|
||||
return {
|
||||
ttsVoice: firstDefined<string>([
|
||||
env.TTS_VOICE,
|
||||
pickIfOpenAi(providers.voiceTts, persisted.features?.voiceMode?.tts?.voice),
|
||||
"alloy",
|
||||
]),
|
||||
ttsModel: firstDefined<string>([
|
||||
env.TTS_MODEL,
|
||||
pickIfOpenAi(providers.voiceTts, persisted.features?.voiceMode?.tts?.model),
|
||||
DEFAULT_OPENAI_TTS_MODEL,
|
||||
]),
|
||||
};
|
||||
}
|
||||
|
||||
function buildOpenAiResolutionInput(params: {
|
||||
env: NodeJS.ProcessEnv;
|
||||
persisted: PersistedConfig;
|
||||
providers: RequestedSpeechProviders;
|
||||
}): Record<string, unknown> {
|
||||
const { env } = params;
|
||||
const openai = params.persisted.providers?.openai;
|
||||
return {
|
||||
sttApiKey: firstDefined<string>([
|
||||
openai?.stt?.apiKey,
|
||||
env.OPENAI_STT_API_KEY,
|
||||
openai?.apiKey,
|
||||
env.OPENAI_API_KEY,
|
||||
]),
|
||||
sttBaseUrl: firstDefined<string>([
|
||||
openai?.stt?.baseUrl,
|
||||
env.OPENAI_STT_BASE_URL,
|
||||
openai?.baseUrl,
|
||||
env.OPENAI_BASE_URL,
|
||||
]),
|
||||
ttsApiKey: firstDefined<string>([
|
||||
openai?.tts?.apiKey,
|
||||
env.OPENAI_TTS_API_KEY,
|
||||
openai?.apiKey,
|
||||
env.OPENAI_API_KEY,
|
||||
]),
|
||||
ttsBaseUrl: firstDefined<string>([
|
||||
openai?.tts?.baseUrl,
|
||||
env.OPENAI_TTS_BASE_URL,
|
||||
openai?.baseUrl,
|
||||
env.OPENAI_BASE_URL,
|
||||
]),
|
||||
...buildOpenAiSttInput(params),
|
||||
...buildOpenAiTtsInput(params),
|
||||
apiKey,
|
||||
...(baseUrl ? { baseUrl } : {}),
|
||||
model: firstString(env.TTS_MODEL, config?.model) ?? DEFAULT_OPENAI_TTS_MODEL,
|
||||
voice: firstString(env.TTS_VOICE, config?.voice) ?? "alloy",
|
||||
responseFormat: "pcm",
|
||||
};
|
||||
}
|
||||
|
||||
@@ -155,46 +69,14 @@ export function resolveOpenAiSpeechConfig(params: {
|
||||
persisted: PersistedConfig;
|
||||
providers: RequestedSpeechProviders;
|
||||
}): OpenAiSpeechProviderConfig | undefined {
|
||||
const input = buildOpenAiResolutionInput(params);
|
||||
const keys = OpenAiEndpointKeysSchema.parse(input);
|
||||
|
||||
if (!keys.sttApiKey && !keys.ttsApiKey) {
|
||||
return undefined;
|
||||
}
|
||||
|
||||
const { env, persisted } = params;
|
||||
const dictationStt = resolveStt(persisted.features?.dictation?.stt, env);
|
||||
const stt = resolveStt(persisted.features?.voiceMode?.stt, env);
|
||||
const tts = resolveTts(persisted.features?.voiceMode?.tts, env);
|
||||
if (!dictationStt && !stt && !tts) return undefined;
|
||||
return {
|
||||
...(keys.sttApiKey ? { stt: buildSttConfig(keys.sttApiKey, keys.sttBaseUrl, input) } : {}),
|
||||
...(keys.ttsApiKey ? { tts: buildTtsConfig(keys.ttsApiKey, keys.ttsBaseUrl, input) } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function buildSttConfig(
|
||||
apiKey: string,
|
||||
baseUrl: string | undefined,
|
||||
input: Record<string, unknown>,
|
||||
): OpenAiSpeechProviderConfig["stt"] {
|
||||
const options = OpenAiSttOptionsSchema.parse(input);
|
||||
return {
|
||||
apiKey,
|
||||
...(baseUrl ? { baseUrl } : {}),
|
||||
...(options.sttConfidenceThreshold !== undefined
|
||||
? { confidenceThreshold: options.sttConfidenceThreshold }
|
||||
: {}),
|
||||
...(options.sttModel ? { model: options.sttModel } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
function buildTtsConfig(
|
||||
apiKey: string,
|
||||
baseUrl: string | undefined,
|
||||
input: Record<string, unknown>,
|
||||
): OpenAiSpeechProviderConfig["tts"] {
|
||||
const options = OpenAiTtsOptionsSchema.parse(input);
|
||||
return {
|
||||
apiKey,
|
||||
...(baseUrl ? { baseUrl } : {}),
|
||||
voice: options.ttsVoice,
|
||||
model: options.ttsModel,
|
||||
responseFormat: "pcm",
|
||||
...(dictationStt ? { dictationStt } : {}),
|
||||
...(stt ? { stt } : {}),
|
||||
...(tts ? { tts } : {}),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
import { createServer, type Server } from "node:http";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import pino from "pino";
|
||||
import { describe, expect, test } from "vitest";
|
||||
import { afterEach, describe, expect, test } from "vitest";
|
||||
|
||||
import { mergeSpeechSettingsPatch } from "../../speech-settings.js";
|
||||
import { PersistedConfigSchema } from "../../../persisted-config.js";
|
||||
import type { SpeechToTextProvider } from "../../speech-provider.js";
|
||||
import { resolveOpenAiSpeechConfig } from "./config.js";
|
||||
import { initializeOpenAiSpeechServices } from "./runtime.js";
|
||||
import { OpenAISTT } from "./stt.js";
|
||||
import { OpenAITTS } from "./tts.js";
|
||||
@@ -16,6 +22,7 @@ describe("initializeOpenAiSpeechServices", () => {
|
||||
},
|
||||
openaiConfig: {
|
||||
stt: { apiKey: "sk-test" },
|
||||
dictationStt: { apiKey: "sk-test" },
|
||||
tts: { apiKey: "sk-test" },
|
||||
},
|
||||
existing: {
|
||||
@@ -32,3 +39,116 @@ describe("initializeOpenAiSpeechServices", () => {
|
||||
expect(services.ttsService).toBeInstanceOf(OpenAITTS);
|
||||
});
|
||||
});
|
||||
|
||||
// An OpenAI-compatible transcription endpoint that records the model each request asks for.
|
||||
async function startTranscriptionServer(): Promise<{
|
||||
server: Server;
|
||||
baseUrl: string;
|
||||
models: string[];
|
||||
keys: (string | undefined)[];
|
||||
paths: (string | undefined)[];
|
||||
}> {
|
||||
const models: string[] = [];
|
||||
const keys: (string | undefined)[] = [];
|
||||
const paths: (string | undefined)[] = [];
|
||||
const server = createServer((request, response) => {
|
||||
let body = "";
|
||||
request.on("data", (chunk: Buffer) => {
|
||||
body += chunk.toString("latin1");
|
||||
});
|
||||
request.on("end", () => {
|
||||
const model = /name="model"\r\n\r\n([^\r]*)/.exec(body)?.[1];
|
||||
models.push(model ?? "<none>");
|
||||
keys.push(request.headers.authorization);
|
||||
paths.push(request.url);
|
||||
response.writeHead(200, { "content-type": "application/json" });
|
||||
response.end(JSON.stringify({ text: "hello" }));
|
||||
});
|
||||
});
|
||||
await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
|
||||
const { port } = server.address() as AddressInfo;
|
||||
return { server, baseUrl: `http://127.0.0.1:${port}/v1`, models, keys, paths };
|
||||
}
|
||||
|
||||
async function transcribeOnce(stt: SpeechToTextProvider | null): Promise<void> {
|
||||
if (!stt) throw new Error("STT service was not created");
|
||||
const session = stt.createSession({ logger: pino({ level: "silent" }), language: "en" });
|
||||
const transcript = new Promise<void>((resolve, reject) => {
|
||||
session.on("transcript", (event) => {
|
||||
if (event.isFinal) resolve();
|
||||
});
|
||||
session.on("error", (error) =>
|
||||
reject(error instanceof Error ? error : new Error(String(error))),
|
||||
);
|
||||
});
|
||||
await session.connect();
|
||||
session.appendPcm16(Buffer.alloc(3200));
|
||||
session.commit();
|
||||
await transcript;
|
||||
session.close();
|
||||
}
|
||||
|
||||
describe("OpenAI speech-to-text models", () => {
|
||||
let server: Server | null = null;
|
||||
|
||||
afterEach(async () => {
|
||||
await new Promise<void>((resolve) => (server ? server.close(() => resolve()) : resolve()));
|
||||
server = null;
|
||||
});
|
||||
|
||||
test("dictation and voice mode each transcribe with the model chosen for them", async () => {
|
||||
const endpoint = await startTranscriptionServer();
|
||||
server = endpoint.server;
|
||||
let persisted = PersistedConfigSchema.parse({
|
||||
features: {
|
||||
dictation: {
|
||||
stt: {
|
||||
provider: "openai",
|
||||
apiKey: "dictation-key",
|
||||
baseUrl: `${endpoint.baseUrl}/dictation`,
|
||||
model: "gpt-4o-mini-transcribe",
|
||||
},
|
||||
},
|
||||
voiceMode: {
|
||||
enabled: true,
|
||||
stt: {
|
||||
provider: "openai",
|
||||
apiKey: "listening-key",
|
||||
baseUrl: `${endpoint.baseUrl}/listening`,
|
||||
model: "whisper-1",
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
persisted = mergeSpeechSettingsPatch(persisted, {
|
||||
voiceMode: { stt: { provider: "openai", model: "whisper-1" } },
|
||||
});
|
||||
const providers = {
|
||||
dictationStt: { provider: "openai", explicit: true },
|
||||
voiceTurnDetection: { provider: "local", explicit: false, enabled: false },
|
||||
voiceStt: { provider: "openai", explicit: true },
|
||||
voiceTts: { provider: "local", explicit: false },
|
||||
} as const;
|
||||
|
||||
const services = initializeOpenAiSpeechServices({
|
||||
providers,
|
||||
openaiConfig: resolveOpenAiSpeechConfig({ env: {}, persisted, providers }),
|
||||
existing: {
|
||||
turnDetectionService: null,
|
||||
sttService: null,
|
||||
ttsService: null,
|
||||
dictationSttService: null,
|
||||
},
|
||||
logger: pino({ level: "silent" }),
|
||||
});
|
||||
await transcribeOnce(services.dictationSttService);
|
||||
await transcribeOnce(services.sttService);
|
||||
|
||||
expect(endpoint.models).toEqual(["gpt-4o-mini-transcribe", "whisper-1"]);
|
||||
expect(endpoint.keys).toEqual(["Bearer dictation-key", "Bearer listening-key"]);
|
||||
expect(endpoint.paths).toEqual([
|
||||
"/v1/dictation/audio/transcriptions",
|
||||
"/v1/listening/audio/transcriptions",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -3,7 +3,11 @@ import type { Logger } from "pino";
|
||||
import type { SpeechToTextProvider, TextToSpeechProvider } from "../../speech-provider.js";
|
||||
import type { RequestedSpeechProviders } from "../../speech-types.js";
|
||||
import type { TurnDetectionProvider } from "../../turn-detection-provider.js";
|
||||
import { DEFAULT_OPENAI_TTS_MODEL, type OpenAiSpeechProviderConfig } from "./config.js";
|
||||
import {
|
||||
DEFAULT_OPENAI_TTS_MODEL,
|
||||
type OpenAiSpeechProviderConfig,
|
||||
type OpenAiSttFeature,
|
||||
} from "./config.js";
|
||||
import { OpenAISTT } from "./stt.js";
|
||||
import { OpenAITTS } from "./tts.js";
|
||||
|
||||
@@ -33,7 +37,7 @@ function resolveOpenAiCredentials(
|
||||
return {
|
||||
openaiSttApiKey: sttApiKey,
|
||||
openaiTtsApiKey: openaiConfig?.tts?.apiKey,
|
||||
openaiDictationApiKey: sttApiKey,
|
||||
openaiDictationApiKey: openaiConfig?.dictationStt?.apiKey,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -94,13 +98,16 @@ export function validateOpenAiCredentialRequirements(params: {
|
||||
}
|
||||
}
|
||||
|
||||
function createOpenAiStt(
|
||||
apiKey: string,
|
||||
openaiConfig: OpenAiSpeechProviderConfig | undefined,
|
||||
logger: Logger,
|
||||
): SpeechToTextProvider {
|
||||
const { apiKey: _sttApiKey, ...sttConfig } = openaiConfig?.stt ?? {};
|
||||
return new OpenAISTT({ apiKey, ...sttConfig }, logger);
|
||||
function createOpenAiStt(params: {
|
||||
apiKey: string;
|
||||
openaiConfig: OpenAiSpeechProviderConfig | undefined;
|
||||
feature: OpenAiSttFeature;
|
||||
logger: Logger;
|
||||
}): SpeechToTextProvider {
|
||||
const site =
|
||||
params.feature === "dictation" ? params.openaiConfig?.dictationStt : params.openaiConfig?.stt;
|
||||
const { apiKey: _sttApiKey, ...sttConfig } = site ?? {};
|
||||
return new OpenAISTT({ apiKey: params.apiKey, ...sttConfig }, params.logger);
|
||||
}
|
||||
|
||||
function createOpenAiTts(
|
||||
@@ -154,7 +161,12 @@ export function initializeOpenAiSpeechServices(params: {
|
||||
logger.info("OpenAI speech provider initialized");
|
||||
|
||||
if (needsOpenAiStt && openAiCredentials.openaiSttApiKey) {
|
||||
sttService = createOpenAiStt(openAiCredentials.openaiSttApiKey, openaiConfig, logger);
|
||||
sttService = createOpenAiStt({
|
||||
apiKey: openAiCredentials.openaiSttApiKey,
|
||||
openaiConfig,
|
||||
feature: "voiceMode",
|
||||
logger,
|
||||
});
|
||||
}
|
||||
|
||||
if (needsOpenAiTts && openAiCredentials.openaiTtsApiKey) {
|
||||
@@ -162,11 +174,12 @@ export function initializeOpenAiSpeechServices(params: {
|
||||
}
|
||||
|
||||
if (needsOpenAiDictation && openAiCredentials.openaiDictationApiKey) {
|
||||
dictationSttService = createOpenAiStt(
|
||||
openAiCredentials.openaiDictationApiKey,
|
||||
dictationSttService = createOpenAiStt({
|
||||
apiKey: openAiCredentials.openaiDictationApiKey,
|
||||
openaiConfig,
|
||||
feature: "dictation",
|
||||
logger,
|
||||
);
|
||||
});
|
||||
}
|
||||
} else if (needsAnyOpenAi) {
|
||||
// validateOpenAiCredentialRequirements already warned about missing credentials
|
||||
|
||||
@@ -6,7 +6,7 @@ import { PersistedConfigSchema } from "../persisted-config.js";
|
||||
import { resolveSpeechConfig } from "./speech-config-resolver.js";
|
||||
|
||||
describe("resolveSpeechConfig", () => {
|
||||
test("resolves local-first defaults without env overrides", () => {
|
||||
test("resolves local-first defaults with voice mode off until turned on", () => {
|
||||
const paseoHome = "/tmp/paseo-home";
|
||||
const persisted = PersistedConfigSchema.parse({});
|
||||
const env = {} as NodeJS.ProcessEnv;
|
||||
@@ -26,17 +26,17 @@ describe("resolveSpeechConfig", () => {
|
||||
expect(result.speech.providers.voiceTurnDetection).toEqual({
|
||||
provider: "local",
|
||||
explicit: false,
|
||||
enabled: true,
|
||||
enabled: false,
|
||||
});
|
||||
expect(result.speech.providers.voiceStt).toEqual({
|
||||
provider: "local",
|
||||
explicit: false,
|
||||
enabled: true,
|
||||
enabled: false,
|
||||
});
|
||||
expect(result.speech.providers.voiceTts).toEqual({
|
||||
provider: "local",
|
||||
explicit: false,
|
||||
enabled: true,
|
||||
enabled: false,
|
||||
});
|
||||
expect(result.speech.local).toEqual({
|
||||
modelsDir: path.join(paseoHome, "models", "local-speech"),
|
||||
@@ -61,13 +61,11 @@ describe("resolveSpeechConfig", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
features: {
|
||||
voiceMode: {
|
||||
enabled: true,
|
||||
turnDetection: { provider: "local" },
|
||||
stt: { provider: "openai", model: "gpt-4o-transcribe" },
|
||||
stt: { provider: "openai", apiKey: "persisted-key", model: "gpt-4o-transcribe" },
|
||||
},
|
||||
},
|
||||
providers: {
|
||||
openai: { apiKey: "persisted-key" },
|
||||
},
|
||||
});
|
||||
const env = {
|
||||
PASEO_DICTATION_LOCAL_STT_MODEL: "parakeet-tdt-0.6b-v2-int8",
|
||||
@@ -130,40 +128,45 @@ describe("resolveSpeechConfig", () => {
|
||||
voice: "pt",
|
||||
});
|
||||
expect(result.openai?.stt?.apiKey).toBe("persisted-key");
|
||||
expect(result.openai?.tts?.apiKey).toBe("persisted-key");
|
||||
expect(result.openai?.tts?.apiKey).toBe("env-key");
|
||||
expect(result.openai?.stt?.model).toBe("gpt-4o-transcribe");
|
||||
});
|
||||
|
||||
test("resolves STT language from env, settings, and voice-to-dictation fallback", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
features: {
|
||||
dictation: {
|
||||
stt: {
|
||||
language: "fr",
|
||||
test.each(["local", "openai"])(
|
||||
"resolves STT language from env, settings, and voice-to-dictation fallback (%s)",
|
||||
(provider) => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
features: {
|
||||
dictation: {
|
||||
stt: {
|
||||
provider,
|
||||
language: "fr",
|
||||
},
|
||||
},
|
||||
voiceMode: {
|
||||
stt: {
|
||||
provider,
|
||||
language: "de",
|
||||
},
|
||||
},
|
||||
},
|
||||
voiceMode: {
|
||||
stt: {
|
||||
language: "de",
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
const result = resolveSpeechConfig({
|
||||
paseoHome: "/tmp/paseo-home",
|
||||
env: {
|
||||
PASEO_DICTATION_LANGUAGE: "es",
|
||||
PASEO_VOICE_LANGUAGE: " ",
|
||||
} as NodeJS.ProcessEnv,
|
||||
persisted,
|
||||
});
|
||||
const result = resolveSpeechConfig({
|
||||
paseoHome: "/tmp/paseo-home",
|
||||
env: {
|
||||
PASEO_DICTATION_LANGUAGE: "es",
|
||||
PASEO_VOICE_LANGUAGE: " ",
|
||||
} as NodeJS.ProcessEnv,
|
||||
persisted,
|
||||
});
|
||||
|
||||
expect(result.speech.sttLanguages).toEqual({
|
||||
dictation: "es",
|
||||
voice: "es",
|
||||
});
|
||||
});
|
||||
expect(result.speech.sttLanguages).toEqual({
|
||||
dictation: "es",
|
||||
voice: "es",
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
test("respects disabled dictation and voice mode feature flags", () => {
|
||||
const persisted = PersistedConfigSchema.parse({
|
||||
|
||||
@@ -43,8 +43,13 @@ const RequestedSpeechProvidersSchema = z.object({
|
||||
voiceTts: OptionalSpeechProviderSchema.default("local"),
|
||||
});
|
||||
|
||||
function resolveOptionalBooleanFlag(value: unknown): boolean {
|
||||
return OptionalBooleanFlagSchema.parse(value) ?? true;
|
||||
// Dictation is on unless turned off. Voice mode is opt-in: it downloads its own models and
|
||||
// runs a hidden agent, so a host only pays for it once someone turns it on.
|
||||
export const DICTATION_ENABLED_BY_DEFAULT = true;
|
||||
export const VOICE_MODE_ENABLED_BY_DEFAULT = false;
|
||||
|
||||
function resolveOptionalBooleanFlag(value: unknown, fallback: boolean): boolean {
|
||||
return OptionalBooleanFlagSchema.parse(value) ?? fallback;
|
||||
}
|
||||
|
||||
interface FeatureProviderInputs {
|
||||
@@ -70,6 +75,7 @@ function buildFeatureProviderInputs(params: {
|
||||
params.env.PASEO_VOICE_MODE_ENABLED,
|
||||
params.persisted.features?.voiceMode?.enabled,
|
||||
]),
|
||||
VOICE_MODE_ENABLED_BY_DEFAULT,
|
||||
);
|
||||
return {
|
||||
dictationStt: {
|
||||
@@ -82,6 +88,7 @@ function buildFeatureProviderInputs(params: {
|
||||
params.env.PASEO_DICTATION_ENABLED,
|
||||
params.persisted.features?.dictation?.enabled,
|
||||
]),
|
||||
DICTATION_ENABLED_BY_DEFAULT,
|
||||
),
|
||||
},
|
||||
voiceTurnDetection: {
|
||||
@@ -119,7 +126,7 @@ function buildRequestedFeatureProvider(
|
||||
};
|
||||
}
|
||||
|
||||
function resolveRequestedSpeechProviders(params: {
|
||||
export function resolveRequestedSpeechProviders(params: {
|
||||
env: NodeJS.ProcessEnv;
|
||||
persisted: PersistedConfig;
|
||||
}): RequestedSpeechProviders {
|
||||
@@ -173,6 +180,7 @@ export function resolveSpeechConfig(params: {
|
||||
openai,
|
||||
speech: {
|
||||
providers,
|
||||
localModelsDir: local.modelsDir,
|
||||
sttLanguages: local.sttLanguages,
|
||||
...(local.local ? { local: local.local } : {}),
|
||||
},
|
||||
|
||||
@@ -2,13 +2,14 @@ import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import pino from "pino";
|
||||
|
||||
import type { PaseoSpeechConfig } from "../bootstrap.js";
|
||||
import type { LocalSpeechModelId } from "./providers/local/models.js";
|
||||
import type { InitializedLocalSpeech } from "./providers/local/runtime.js";
|
||||
import type { SpeechToTextProvider, TextToSpeechProvider } from "./speech-provider.js";
|
||||
import type { TurnDetectionProvider } from "./turn-detection-provider.js";
|
||||
import { createSpeechService } from "./speech-runtime.js";
|
||||
import { createSpeechService, LocalSpeechModelInUseError } from "./speech-runtime.js";
|
||||
|
||||
const { ensureLocalSpeechModelsMock, initializeLocalSpeechServicesMock } = vi.hoisted(() => ({
|
||||
ensureLocalSpeechModelsMock: vi.fn(async () => ({})),
|
||||
const { ensureSherpaOnnxModelMock, initializeLocalSpeechServicesMock } = vi.hoisted(() => ({
|
||||
ensureSherpaOnnxModelMock: vi.fn(async (_options: { signal?: AbortSignal }) => ""),
|
||||
initializeLocalSpeechServicesMock: vi.fn<(args: unknown) => Promise<InitializedLocalSpeech>>(),
|
||||
}));
|
||||
|
||||
@@ -34,9 +35,10 @@ vi.mock("./providers/openai/runtime.js", () => ({
|
||||
validateOpenAiCredentialRequirements: () => {},
|
||||
}));
|
||||
|
||||
vi.mock("./providers/local/models.js", () => ({
|
||||
ensureLocalSpeechModels: ensureLocalSpeechModelsMock,
|
||||
listMissingLocalSpeechModels: vi.fn(async ({ modelIds }: { modelIds: string[] }) => modelIds),
|
||||
vi.mock("./providers/local/sherpa/model-downloader.js", () => ({
|
||||
ensureSherpaOnnxModel: ensureSherpaOnnxModelMock,
|
||||
isSherpaOnnxModelInstalled: vi.fn(async () => false),
|
||||
deleteSherpaOnnxModel: vi.fn(async () => {}),
|
||||
}));
|
||||
|
||||
function createStubStt(id: string): SpeechToTextProvider {
|
||||
@@ -75,6 +77,7 @@ function rejectWhenAborted(signal?: AbortSignal): Promise<never> {
|
||||
function createSpeechConfig(providers: PaseoSpeechConfig["providers"]): PaseoSpeechConfig {
|
||||
return {
|
||||
providers,
|
||||
localModelsDir: "/tmp/missing-local-speech-models",
|
||||
sttLanguages: {
|
||||
dictation: "en",
|
||||
voice: "en",
|
||||
@@ -84,8 +87,8 @@ function createSpeechConfig(providers: PaseoSpeechConfig["providers"]): PaseoSpe
|
||||
|
||||
describe("createSpeechService readiness", () => {
|
||||
beforeEach(() => {
|
||||
ensureLocalSpeechModelsMock.mockReset();
|
||||
ensureLocalSpeechModelsMock.mockResolvedValue({});
|
||||
ensureSherpaOnnxModelMock.mockReset();
|
||||
ensureSherpaOnnxModelMock.mockResolvedValue("");
|
||||
initializeLocalSpeechServicesMock.mockReset();
|
||||
});
|
||||
|
||||
@@ -169,7 +172,7 @@ describe("createSpeechService readiness", () => {
|
||||
|
||||
it("aborts and joins an in-flight model download when stopped", async () => {
|
||||
let downloadSignal: AbortSignal | undefined;
|
||||
ensureLocalSpeechModelsMock.mockImplementation(({ signal }: { signal?: AbortSignal }) => {
|
||||
ensureSherpaOnnxModelMock.mockImplementation(({ signal }: { signal?: AbortSignal }) => {
|
||||
downloadSignal = signal;
|
||||
return rejectWhenAborted(signal);
|
||||
});
|
||||
@@ -201,10 +204,104 @@ describe("createSpeechService readiness", () => {
|
||||
});
|
||||
runtime.start();
|
||||
await runtime.ready;
|
||||
expect(ensureLocalSpeechModelsMock).toHaveBeenCalledOnce();
|
||||
expect(ensureSherpaOnnxModelMock).toHaveBeenCalledOnce();
|
||||
|
||||
await runtime.stop();
|
||||
|
||||
expect(downloadSignal?.aborted).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("createSpeechService settings changes", () => {
|
||||
const dictationOnly = createSpeechConfig({
|
||||
dictationStt: { provider: "local", enabled: true, explicit: true },
|
||||
voiceTurnDetection: { provider: "local", enabled: false, explicit: true },
|
||||
voiceStt: { provider: "local", enabled: false, explicit: true },
|
||||
voiceTts: { provider: "local", enabled: false, explicit: true },
|
||||
});
|
||||
const nothingEnabled = createSpeechConfig({
|
||||
dictationStt: { provider: "local", enabled: false, explicit: true },
|
||||
voiceTurnDetection: { provider: "local", enabled: false, explicit: true },
|
||||
voiceStt: { provider: "local", enabled: false, explicit: true },
|
||||
voiceTts: { provider: "local", enabled: false, explicit: true },
|
||||
});
|
||||
|
||||
function localSpeech(input: {
|
||||
dictationStt: SpeechToTextProvider | null;
|
||||
defaultModelIds: LocalSpeechModelId[];
|
||||
}): InitializedLocalSpeech {
|
||||
return {
|
||||
turnDetectionService: null,
|
||||
sttService: null,
|
||||
ttsService: null,
|
||||
dictationSttService: input.dictationStt,
|
||||
localVoiceTtsProvider: null,
|
||||
localModelConfig: {
|
||||
modelsDir: "/tmp/missing-local-speech-models",
|
||||
defaultModelIds: input.defaultModelIds,
|
||||
},
|
||||
availability: { configured: true, modelsDir: "/tmp/missing-local-speech-models" },
|
||||
cleanup: () => {},
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
ensureSherpaOnnxModelMock.mockReset();
|
||||
ensureSherpaOnnxModelMock.mockResolvedValue("");
|
||||
initializeLocalSpeechServicesMock.mockReset();
|
||||
});
|
||||
|
||||
it("applies new providers without a restart", async () => {
|
||||
initializeLocalSpeechServicesMock.mockResolvedValue(
|
||||
localSpeech({ dictationStt: createStubStt("dictation-local"), defaultModelIds: [] }),
|
||||
);
|
||||
const runtime = createSpeechService({
|
||||
logger: pino({ level: "silent" }),
|
||||
speechConfig: dictationOnly,
|
||||
});
|
||||
runtime.start();
|
||||
await runtime.ready;
|
||||
expect(runtime.getReadiness().dictation.available).toBe(true);
|
||||
|
||||
const disabled = new Promise<void>((resolve) => {
|
||||
runtime.onReadinessChange((snapshot) => {
|
||||
if (snapshot.dictation.reasonCode === "disabled") resolve();
|
||||
});
|
||||
});
|
||||
initializeLocalSpeechServicesMock.mockResolvedValue(
|
||||
localSpeech({ dictationStt: null, defaultModelIds: [] }),
|
||||
);
|
||||
runtime.reconfigure({ speechConfig: nothingEnabled });
|
||||
await disabled;
|
||||
|
||||
expect(runtime.resolveDictationStt()).toBeNull();
|
||||
await runtime.stop();
|
||||
});
|
||||
|
||||
it("refuses to delete a model an enabled feature uses", async () => {
|
||||
initializeLocalSpeechServicesMock.mockResolvedValue(
|
||||
localSpeech({
|
||||
dictationStt: createStubStt("dictation-local"),
|
||||
defaultModelIds: ["parakeet-tdt-0.6b-v2-int8"],
|
||||
}),
|
||||
);
|
||||
ensureSherpaOnnxModelMock.mockImplementation(({ signal }) => rejectWhenAborted(signal));
|
||||
const runtime = createSpeechService({
|
||||
logger: pino({ level: "silent" }),
|
||||
speechConfig: dictationOnly,
|
||||
});
|
||||
runtime.start();
|
||||
await runtime.ready;
|
||||
|
||||
const models = await runtime.listLocalModels();
|
||||
expect(models.filter((model) => model.inUse).map((model) => model.spec.id)).toEqual([
|
||||
"parakeet-tdt-0.6b-v2-int8",
|
||||
]);
|
||||
await expect(runtime.deleteLocalModel("parakeet-tdt-0.6b-v2-int8")).rejects.toBeInstanceOf(
|
||||
LocalSpeechModelInUseError,
|
||||
);
|
||||
await expect(runtime.deleteLocalModel("kokoro-int8-en-v0_19")).resolves.toBeUndefined();
|
||||
|
||||
await runtime.stop();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -2,7 +2,10 @@ import type { Logger } from "pino";
|
||||
|
||||
import type { PaseoOpenAIConfig, PaseoSpeechConfig } from "../bootstrap.js";
|
||||
import type { LocalSpeechModelId } from "./providers/local/config.js";
|
||||
import { ensureLocalSpeechModels, listMissingLocalSpeechModels } from "./providers/local/models.js";
|
||||
import {
|
||||
LocalSpeechModelLibrary,
|
||||
type LocalSpeechModelEntry,
|
||||
} from "./providers/local/model-library.js";
|
||||
import { initializeLocalSpeechServices } from "./providers/local/runtime.js";
|
||||
import {
|
||||
getOpenAiSpeechAvailability,
|
||||
@@ -92,7 +95,7 @@ function buildRealtimeVoiceReadiness(params: {
|
||||
enabled: false,
|
||||
available: false,
|
||||
reasonCode: "disabled",
|
||||
message: "Realtime voice is disabled in daemon config.",
|
||||
message: "Voice mode is off. Turn it on in Settings > Voice.",
|
||||
retryable: false,
|
||||
missingModelIds: [],
|
||||
};
|
||||
@@ -147,7 +150,7 @@ function buildDictationReadiness(params: {
|
||||
enabled: false,
|
||||
available: false,
|
||||
reasonCode: "disabled",
|
||||
message: "Dictation is disabled in daemon config.",
|
||||
message: "Dictation is off. Turn it on in Settings > Voice.",
|
||||
retryable: false,
|
||||
missingModelIds: [],
|
||||
};
|
||||
@@ -292,6 +295,23 @@ function resolveEffectiveProviderIds(params: {
|
||||
};
|
||||
}
|
||||
|
||||
export interface LocalSpeechModelStatus extends LocalSpeechModelEntry {
|
||||
// Selected by an enabled feature, so the runtime keeps it installed.
|
||||
inUse: boolean;
|
||||
}
|
||||
|
||||
export class LocalSpeechModelInUseError extends Error {
|
||||
constructor(public readonly modelId: LocalSpeechModelId) {
|
||||
super(`${modelId} is selected for dictation or voice mode. Choose another model first.`);
|
||||
this.name = "LocalSpeechModelInUseError";
|
||||
}
|
||||
}
|
||||
|
||||
export interface SpeechServiceConfig {
|
||||
openaiConfig?: PaseoOpenAIConfig;
|
||||
speechConfig?: PaseoSpeechConfig;
|
||||
}
|
||||
|
||||
export interface SpeechService {
|
||||
resolveStt: () => SpeechToTextProvider | null;
|
||||
resolveSttLanguage: () => string;
|
||||
@@ -301,37 +321,49 @@ export interface SpeechService {
|
||||
resolveDictationSttLanguage: () => string;
|
||||
getReadiness: () => SpeechReadinessSnapshot;
|
||||
onReadinessChange: (listener: (snapshot: SpeechReadinessSnapshot) => void) => () => void;
|
||||
// Applies new provider/model settings without restarting the daemon.
|
||||
reconfigure: (config: SpeechServiceConfig) => void;
|
||||
listLocalModels: () => Promise<LocalSpeechModelStatus[]>;
|
||||
downloadLocalModel: (modelId: LocalSpeechModelId) => void;
|
||||
deleteLocalModel: (modelId: LocalSpeechModelId) => Promise<void>;
|
||||
start: () => void;
|
||||
stop: () => Promise<void>;
|
||||
ready: Promise<void>;
|
||||
}
|
||||
|
||||
export function createSpeechService(params: {
|
||||
logger: Logger;
|
||||
openaiConfig?: PaseoOpenAIConfig;
|
||||
speechConfig?: PaseoSpeechConfig;
|
||||
}): SpeechService {
|
||||
export function createSpeechService(
|
||||
params: SpeechServiceConfig & {
|
||||
logger: Logger;
|
||||
},
|
||||
): SpeechService {
|
||||
const logger = params.logger.child({ module: "speech-runtime" });
|
||||
const speechConfig = params.speechConfig ?? null;
|
||||
const openaiConfig = params.openaiConfig;
|
||||
const providers = resolveRequestedSpeechProviders(speechConfig);
|
||||
const requestedProviders = describeRequestedProviders(providers);
|
||||
let speechConfig = params.speechConfig ?? null;
|
||||
let openaiConfig = params.openaiConfig;
|
||||
let providers = resolveRequestedSpeechProviders(speechConfig);
|
||||
let requestedProviders = describeRequestedProviders(providers);
|
||||
// The models directory is fixed for the daemon's lifetime; settings changes only pick
|
||||
// which of its models are used.
|
||||
const modelLibrary = speechConfig
|
||||
? new LocalSpeechModelLibrary(speechConfig.localModelsDir, logger)
|
||||
: null;
|
||||
|
||||
validateOpenAiCredentialRequirements({
|
||||
providers,
|
||||
openaiConfig,
|
||||
logger,
|
||||
});
|
||||
|
||||
logger.info(
|
||||
{
|
||||
requestedProviders,
|
||||
availability: {
|
||||
openai: getOpenAiSpeechAvailability(openaiConfig),
|
||||
const logReconciliationStart = (): void => {
|
||||
validateOpenAiCredentialRequirements({
|
||||
providers,
|
||||
openaiConfig,
|
||||
logger,
|
||||
});
|
||||
logger.info(
|
||||
{
|
||||
requestedProviders,
|
||||
availability: {
|
||||
openai: getOpenAiSpeechAvailability(openaiConfig),
|
||||
},
|
||||
},
|
||||
},
|
||||
"Speech provider reconciliation started",
|
||||
);
|
||||
"Speech provider reconciliation started",
|
||||
);
|
||||
};
|
||||
logReconciliationStart();
|
||||
|
||||
let sttService: SpeechToTextProvider | null = null;
|
||||
let ttsService: TextToSpeechProvider | null = null;
|
||||
@@ -346,12 +378,11 @@ export function createSpeechService(params: {
|
||||
|
||||
let missingLocalModelIds: LocalSpeechModelId[] = [];
|
||||
let backgroundDownloadInProgress = false;
|
||||
let backgroundDownloadAbortController: AbortController | null = null;
|
||||
let backgroundDownloadPromise: Promise<void> | null = null;
|
||||
let backgroundDownloadError: string | null = null;
|
||||
let stopped = false;
|
||||
let monitorTimeout: ReturnType<typeof setTimeout> | null = null;
|
||||
let reconcileInFlight: Promise<void> | null = null;
|
||||
let reconcileQueue: Promise<void> = Promise.resolve();
|
||||
const readinessListeners = new Set<(snapshot: SpeechReadinessSnapshot) => void>();
|
||||
let lastReadinessFingerprint: string | null = null;
|
||||
let lastPublishedReadinessSnapshot: SpeechReadinessSnapshot | null = null;
|
||||
@@ -445,12 +476,10 @@ export function createSpeechService(params: {
|
||||
};
|
||||
|
||||
const refreshMissingLocalModels = async (): Promise<void> => {
|
||||
missingLocalModelIds = localModelConfig
|
||||
? await listMissingLocalSpeechModels({
|
||||
modelsDir: localModelConfig.modelsDir,
|
||||
modelIds: localModelConfig.defaultModelIds,
|
||||
})
|
||||
: [];
|
||||
missingLocalModelIds =
|
||||
localModelConfig && modelLibrary
|
||||
? await modelLibrary.listMissing(localModelConfig.defaultModelIds)
|
||||
: [];
|
||||
};
|
||||
|
||||
const reconcileServices = async (): Promise<void> => {
|
||||
@@ -520,16 +549,12 @@ export function createSpeechService(params: {
|
||||
}
|
||||
};
|
||||
|
||||
// Reconciles run one after another, and each starts after it was requested, so a
|
||||
// reconfigure that lands mid-reconcile is applied instead of joining the stale run.
|
||||
const runReconcile = async (): Promise<void> => {
|
||||
if (reconcileInFlight) {
|
||||
await reconcileInFlight;
|
||||
publishReadinessIfChanged();
|
||||
return;
|
||||
}
|
||||
reconcileInFlight = reconcileServices().finally(() => {
|
||||
reconcileInFlight = null;
|
||||
});
|
||||
await reconcileInFlight;
|
||||
const next = reconcileQueue.catch(() => undefined).then(reconcileServices);
|
||||
reconcileQueue = next;
|
||||
await next;
|
||||
publishReadinessIfChanged();
|
||||
};
|
||||
|
||||
@@ -544,12 +569,12 @@ export function createSpeechService(params: {
|
||||
};
|
||||
|
||||
const startBackgroundDownload = (): void => {
|
||||
if (stopped || backgroundDownloadInProgress) {
|
||||
if (stopped || backgroundDownloadInProgress || !modelLibrary) {
|
||||
return;
|
||||
}
|
||||
const modelsDir = localModelConfig?.modelsDir ?? null;
|
||||
const modelsDir = modelLibrary.getModelsDir();
|
||||
const modelIds = [...missingLocalModelIds];
|
||||
if (!modelsDir || modelIds.length === 0) {
|
||||
if (modelIds.length === 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -565,16 +590,9 @@ export function createSpeechService(params: {
|
||||
"Starting background download for missing local speech models",
|
||||
);
|
||||
|
||||
const abortController = new AbortController();
|
||||
backgroundDownloadAbortController = abortController;
|
||||
backgroundDownloadPromise = (async () => {
|
||||
try {
|
||||
await ensureLocalSpeechModels({
|
||||
modelsDir,
|
||||
modelIds,
|
||||
logger,
|
||||
signal: abortController.signal,
|
||||
});
|
||||
await Promise.all(modelIds.map((modelId) => modelLibrary.download(modelId)));
|
||||
await runReconcile();
|
||||
backgroundDownloadError = null;
|
||||
} catch (error) {
|
||||
@@ -588,7 +606,6 @@ export function createSpeechService(params: {
|
||||
"Background local speech model download failed",
|
||||
);
|
||||
} finally {
|
||||
backgroundDownloadAbortController = null;
|
||||
backgroundDownloadInProgress = false;
|
||||
await refreshMissingLocalModels().catch((error) => {
|
||||
logger.warn({ err: error }, "Failed to refresh local speech model status after download");
|
||||
@@ -630,6 +647,62 @@ export function createSpeechService(params: {
|
||||
}
|
||||
};
|
||||
|
||||
const reconcileAndFetchMissingModels = async (): Promise<void> => {
|
||||
await runReconcile();
|
||||
const snapshot = computeReadinessSnapshot();
|
||||
if (snapshot.voiceFeature.enabled && !snapshot.voiceFeature.available) {
|
||||
if (missingLocalModelIds.length > 0) {
|
||||
startBackgroundDownload();
|
||||
}
|
||||
scheduleMonitor();
|
||||
}
|
||||
};
|
||||
|
||||
const reconfigure = (next: SpeechServiceConfig): void => {
|
||||
speechConfig = next.speechConfig ?? null;
|
||||
openaiConfig = next.openaiConfig;
|
||||
providers = resolveRequestedSpeechProviders(speechConfig);
|
||||
requestedProviders = describeRequestedProviders(providers);
|
||||
// A failure for the previous selection must not block downloading the new one.
|
||||
backgroundDownloadError = null;
|
||||
logReconciliationStart();
|
||||
if (!started || stopped) {
|
||||
return;
|
||||
}
|
||||
void reconcileAndFetchMissingModels().catch((error) => {
|
||||
logger.error({ err: error }, "Speech runtime failed to apply new configuration");
|
||||
});
|
||||
};
|
||||
|
||||
const requireModelLibrary = (): LocalSpeechModelLibrary => {
|
||||
if (!modelLibrary) {
|
||||
throw new Error("Local speech models are unavailable on this daemon");
|
||||
}
|
||||
return modelLibrary;
|
||||
};
|
||||
|
||||
const listLocalModels = async (): Promise<LocalSpeechModelStatus[]> => {
|
||||
const inUse = new Set(localModelConfig?.defaultModelIds ?? []);
|
||||
const entries = await requireModelLibrary().list();
|
||||
return entries.map(({ spec, state }) => ({ spec, state, inUse: inUse.has(spec.id) }));
|
||||
};
|
||||
|
||||
const downloadLocalModel = (modelId: LocalSpeechModelId): void => {
|
||||
void requireModelLibrary()
|
||||
.download(modelId)
|
||||
.then(() => runMonitorTick())
|
||||
.catch((error) => {
|
||||
logger.warn({ err: error, modelId }, "Local speech model download failed");
|
||||
});
|
||||
};
|
||||
|
||||
const deleteLocalModel = async (modelId: LocalSpeechModelId): Promise<void> => {
|
||||
if (localModelConfig?.defaultModelIds.includes(modelId)) {
|
||||
throw new LocalSpeechModelInUseError(modelId);
|
||||
}
|
||||
await requireModelLibrary().delete(modelId);
|
||||
};
|
||||
|
||||
const start = (): void => {
|
||||
if (started || stopped) {
|
||||
return;
|
||||
@@ -637,14 +710,7 @@ export function createSpeechService(params: {
|
||||
started = true;
|
||||
void (async () => {
|
||||
try {
|
||||
await runReconcile();
|
||||
const snapshot = computeReadinessSnapshot();
|
||||
if (snapshot.voiceFeature.enabled && !snapshot.voiceFeature.available) {
|
||||
if (missingLocalModelIds.length > 0) {
|
||||
startBackgroundDownload();
|
||||
}
|
||||
scheduleMonitor();
|
||||
}
|
||||
await reconcileAndFetchMissingModels();
|
||||
if (!readySettled) {
|
||||
readySettled = true;
|
||||
resolveReady();
|
||||
@@ -666,7 +732,7 @@ export function createSpeechService(params: {
|
||||
monitorTimeout = null;
|
||||
}
|
||||
localCleanup();
|
||||
backgroundDownloadAbortController?.abort();
|
||||
await modelLibrary?.stop();
|
||||
await backgroundDownloadPromise?.catch(() => undefined);
|
||||
backgroundDownloadPromise = null;
|
||||
};
|
||||
@@ -680,6 +746,10 @@ export function createSpeechService(params: {
|
||||
resolveDictationSttLanguage: () => speechConfig?.sttLanguages?.dictation ?? "en",
|
||||
getReadiness: () => lastPublishedReadinessSnapshot ?? computeReadinessSnapshot(),
|
||||
onReadinessChange: subscribeSpeechReadiness,
|
||||
reconfigure,
|
||||
listLocalModels,
|
||||
downloadLocalModel,
|
||||
deleteLocalModel,
|
||||
start,
|
||||
stop,
|
||||
ready,
|
||||
|
||||
@@ -0,0 +1,202 @@
|
||||
import type { SpeechSettings, SpeechSettingsPatch } from "@getpaseo/protocol/speech/rpc-schemas";
|
||||
|
||||
import {
|
||||
PersistedConfigSchema,
|
||||
SpeechSttConfigSchema,
|
||||
SpeechTtsConfigSchema,
|
||||
type PersistedConfig,
|
||||
} from "../persisted-config.js";
|
||||
import { resolveRequestedSpeechProviders, resolveSpeechConfig } from "./speech-config-resolver.js";
|
||||
import type { SpeechService } from "./speech-runtime.js";
|
||||
import { LocalSttModelIdSchema, LocalTtsModelIdSchema } from "./providers/local/models.js";
|
||||
|
||||
interface SpeechRead {
|
||||
features: SpeechSettings;
|
||||
envOverrides: Record<string, string>;
|
||||
}
|
||||
|
||||
// Owns the per-site wire boundary: credential redaction, env provenance and patch
|
||||
// semantics. The store never assembles or merges provider objects itself.
|
||||
export interface SpeechSettingsOwner {
|
||||
read(persisted: PersistedConfig): SpeechRead;
|
||||
apply(persisted: PersistedConfig): void;
|
||||
}
|
||||
|
||||
export function createSpeechSettingsOwner(params: {
|
||||
paseoHome: string;
|
||||
env: NodeJS.ProcessEnv;
|
||||
speechService: Pick<SpeechService, "reconfigure">;
|
||||
}): SpeechSettingsOwner {
|
||||
const { paseoHome, env, speechService } = params;
|
||||
return {
|
||||
read: (persisted) => readSpeechSettings(persisted, env),
|
||||
apply: (persisted) => {
|
||||
const resolved = resolveSpeechConfig({ paseoHome, env, persisted });
|
||||
speechService.reconfigure({ openaiConfig: resolved.openai, speechConfig: resolved.speech });
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function readSpeechSettings(persisted: PersistedConfig, env: NodeJS.ProcessEnv): SpeechRead {
|
||||
const dictation = persisted.features?.dictation;
|
||||
const voiceMode = persisted.features?.voiceMode;
|
||||
const providers = resolveRequestedSpeechProviders({ persisted, env });
|
||||
const envOverrides: Record<string, string> = {};
|
||||
const recordOverride = (path: string, names: string[], fallback?: unknown) => {
|
||||
if (fallback !== undefined) return;
|
||||
const name = names.find((candidate) => env[candidate]?.trim());
|
||||
if (name) envOverrides[path] = name;
|
||||
};
|
||||
recordOverride("features.dictation.enabled", ["PASEO_DICTATION_ENABLED"]);
|
||||
recordOverride("features.voiceMode.enabled", ["PASEO_VOICE_MODE_ENABLED"]);
|
||||
recordOverride("features.voiceMode.turnDetection.provider", [
|
||||
"PASEO_VOICE_TURN_DETECTION_PROVIDER",
|
||||
]);
|
||||
|
||||
function readSite(
|
||||
site: Record<string, unknown> | undefined,
|
||||
provider: string,
|
||||
path: string,
|
||||
kind: "stt" | "tts",
|
||||
providerEnv: string,
|
||||
localModelEnv: string,
|
||||
) {
|
||||
recordOverride(`${path}.provider`, [providerEnv]);
|
||||
const configured = site?.provider === provider ? site : undefined;
|
||||
const { apiKey, ...fields } = configured ?? {};
|
||||
const openaiModelEnv = kind === "stt" ? "STT_MODEL" : "TTS_MODEL";
|
||||
recordOverride(`${path}.model`, [provider === "openai" ? openaiModelEnv : localModelEnv]);
|
||||
if (kind === "stt") {
|
||||
const languageVars =
|
||||
path === "features.dictation.stt"
|
||||
? ["PASEO_DICTATION_LANGUAGE"]
|
||||
: ["PASEO_VOICE_LANGUAGE", "PASEO_DICTATION_LANGUAGE"];
|
||||
recordOverride(`${path}.language`, languageVars);
|
||||
}
|
||||
if (provider === "openai") {
|
||||
for (const [field, suffix] of [
|
||||
["apiKey", "API_KEY"],
|
||||
["baseUrl", "BASE_URL"],
|
||||
] as const) {
|
||||
recordOverride(
|
||||
`${path}.${field}`,
|
||||
[`OPENAI_${kind.toUpperCase()}_${suffix}`, `OPENAI_${suffix}`],
|
||||
configured?.[field],
|
||||
);
|
||||
}
|
||||
if (kind === "stt") {
|
||||
recordOverride(`${path}.confidenceThreshold`, ["STT_CONFIDENCE_THRESHOLD"]);
|
||||
} else recordOverride(`${path}.voice`, ["TTS_VOICE"]);
|
||||
} else if (kind === "tts") {
|
||||
recordOverride(`${path}.speakerId`, ["PASEO_VOICE_LOCAL_TTS_SPEAKER_ID"]);
|
||||
recordOverride(`${path}.speed`, ["PASEO_VOICE_LOCAL_TTS_SPEED"]);
|
||||
}
|
||||
return {
|
||||
...fields,
|
||||
provider,
|
||||
...(provider === "openai"
|
||||
? { hasApiKey: typeof apiKey === "string" && apiKey.trim().length > 0 }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
features: {
|
||||
dictation: {
|
||||
enabled: providers.dictationStt.enabled !== false,
|
||||
stt: readSite(
|
||||
dictation?.stt,
|
||||
providers.dictationStt.provider,
|
||||
"features.dictation.stt",
|
||||
"stt",
|
||||
"PASEO_DICTATION_STT_PROVIDER",
|
||||
"PASEO_DICTATION_LOCAL_STT_MODEL",
|
||||
),
|
||||
},
|
||||
voiceMode: {
|
||||
enabled: providers.voiceStt.enabled !== false,
|
||||
...(voiceMode?.llm ? { llm: voiceMode.llm } : {}),
|
||||
...(voiceMode?.turnDetection
|
||||
? {
|
||||
turnDetection: {
|
||||
...voiceMode.turnDetection,
|
||||
provider: providers.voiceTurnDetection.provider,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
stt: readSite(
|
||||
voiceMode?.stt,
|
||||
providers.voiceStt.provider,
|
||||
"features.voiceMode.stt",
|
||||
"stt",
|
||||
"PASEO_VOICE_STT_PROVIDER",
|
||||
"PASEO_VOICE_LOCAL_STT_MODEL",
|
||||
),
|
||||
tts: readSite(
|
||||
voiceMode?.tts,
|
||||
providers.voiceTts.provider,
|
||||
"features.voiceMode.tts",
|
||||
"tts",
|
||||
"PASEO_VOICE_TTS_PROVIDER",
|
||||
"PASEO_VOICE_LOCAL_TTS_MODEL",
|
||||
),
|
||||
},
|
||||
},
|
||||
envOverrides,
|
||||
};
|
||||
}
|
||||
|
||||
function mergeFields(current: Record<string, unknown> | undefined, patch: Record<string, unknown>) {
|
||||
const next = { ...current };
|
||||
for (const [key, value] of Object.entries(patch)) {
|
||||
if (value === undefined) continue;
|
||||
if (value === null || (typeof value === "string" && !value.trim())) delete next[key];
|
||||
else next[key] = typeof value === "string" ? value.trim() : value;
|
||||
}
|
||||
return next;
|
||||
}
|
||||
|
||||
function mergeSite(
|
||||
current: Record<string, unknown> | undefined,
|
||||
patch: Record<string, unknown>,
|
||||
kind: "stt" | "tts",
|
||||
) {
|
||||
const providerChanged =
|
||||
patch.provider !== undefined && patch.provider !== (current?.provider ?? "local");
|
||||
const next = mergeFields(providerChanged ? undefined : current, patch);
|
||||
next.provider ??= "local";
|
||||
const site = (kind === "stt" ? SpeechSttConfigSchema : SpeechTtsConfigSchema).parse(next);
|
||||
if (site.provider === "local" && site.model !== undefined) {
|
||||
const schema = kind === "stt" ? LocalSttModelIdSchema : LocalTtsModelIdSchema;
|
||||
if (!schema.safeParse(site.model).success) {
|
||||
throw new Error(
|
||||
`Unknown local ${kind === "stt" ? "speech-to-text" : "voice"} model: ${site.model}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return site;
|
||||
}
|
||||
|
||||
export function mergeSpeechSettingsPatch(
|
||||
persisted: PersistedConfig,
|
||||
patch: SpeechSettingsPatch,
|
||||
): PersistedConfig {
|
||||
const features = { ...persisted.features };
|
||||
if (patch.dictation) {
|
||||
const { stt, ...rest } = patch.dictation;
|
||||
const next = mergeFields(features.dictation, rest);
|
||||
if (stt) next.stt = mergeSite(features.dictation?.stt, stt, "stt");
|
||||
features.dictation = PersistedConfigSchema.shape.features.unwrap().shape.dictation.parse(next);
|
||||
}
|
||||
if (patch.voiceMode) {
|
||||
const { stt, tts, llm, turnDetection, ...rest } = patch.voiceMode;
|
||||
const next = mergeFields(features.voiceMode, rest);
|
||||
if (stt) next.stt = mergeSite(features.voiceMode?.stt, stt, "stt");
|
||||
if (tts) next.tts = mergeSite(features.voiceMode?.tts, tts, "tts");
|
||||
if (llm) next.llm = mergeFields(features.voiceMode?.llm, llm);
|
||||
if (turnDetection)
|
||||
next.turnDetection = mergeFields(features.voiceMode?.turnDetection, turnDetection);
|
||||
features.voiceMode = PersistedConfigSchema.shape.features.unwrap().shape.voiceMode.parse(next);
|
||||
}
|
||||
return { ...persisted, features };
|
||||
}
|
||||
@@ -39,7 +39,8 @@ interface TestPaseoDaemonOptions {
|
||||
staticDir?: string;
|
||||
cleanup?: boolean;
|
||||
openai?: PaseoOpenAIConfig;
|
||||
speech?: PaseoSpeechConfig;
|
||||
// localModelsDir defaults to the local provider's modelsDir, then to a dir under paseoHome.
|
||||
speech?: Omit<PaseoSpeechConfig, "localModelsDir"> & { localModelsDir?: string };
|
||||
voiceLlmProvider?: PaseoDaemonConfig["voiceLlmProvider"];
|
||||
voiceLlmProviderExplicit?: boolean;
|
||||
voiceLlmModel?: string | null;
|
||||
@@ -209,7 +210,15 @@ async function prepareTestDaemonConfig(
|
||||
webUi: options.webUi,
|
||||
trustedProxies: options.trustedProxies,
|
||||
openai: options.openai,
|
||||
speech: options.speech,
|
||||
speech: options.speech
|
||||
? {
|
||||
...options.speech,
|
||||
localModelsDir:
|
||||
options.speech.localModelsDir ??
|
||||
options.speech.local?.modelsDir ??
|
||||
path.join(paseoHome, "models", "local-speech"),
|
||||
}
|
||||
: undefined,
|
||||
voiceLlmProvider: options.voiceLlmProvider ?? null,
|
||||
voiceLlmProviderExplicit: options.voiceLlmProviderExplicit ?? false,
|
||||
voiceLlmModel: options.voiceLlmModel ?? null,
|
||||
|
||||
@@ -1620,6 +1620,7 @@ export class VoiceAssistantWebSocketServer {
|
||||
stt: () => this.speech?.resolveStt() ?? null,
|
||||
sttLanguage: this.speech?.resolveSttLanguage() ?? "en",
|
||||
tts: () => this.speech?.resolveTts() ?? null,
|
||||
speechModels: this.speech,
|
||||
terminalManager: this.terminalManager,
|
||||
providerSnapshotManager: this.providerSnapshotManager,
|
||||
hubExecutionAgents: options.hubExecutionAgents,
|
||||
@@ -1991,6 +1992,8 @@ export class VoiceAssistantWebSocketServer {
|
||||
...(this.advertiseDaemonStatusRpc ? { daemonStatusRpc: true } : {}),
|
||||
// COMPAT(daemonConfigReload): added in v0.4.0, remove gate after 2027-02-14.
|
||||
daemonConfigReload: true,
|
||||
// COMPAT(speechSettings): added in v0.12.0, remove gate after 2027-04-18.
|
||||
speechSettings: true,
|
||||
// COMPAT(relayConfig): added in v0.2.6, remove gate after 2027-01-31.
|
||||
...(this.advertiseRelayConfig ? { relayConfig: true } : {}),
|
||||
// COMPAT(pushTokenRevocation): added in v0.3.2, remove gate after 2027-02-10.
|
||||
|
||||
@@ -257,48 +257,6 @@
|
||||
"providers": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"openai": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"apiKey": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"baseUrl": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"stt": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"apiKey": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"baseUrl": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"tts": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"apiKey": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"baseUrl": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
},
|
||||
"local": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
@@ -639,24 +597,57 @@
|
||||
"type": "boolean"
|
||||
},
|
||||
"stt": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string"
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"const": "local"
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"language": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"required": ["provider"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"language": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"confidenceThreshold": {
|
||||
"type": "number"
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"const": "openai"
|
||||
},
|
||||
"apiKey": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"baseUrl": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"language": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"confidenceThreshold": {
|
||||
"type": "number"
|
||||
}
|
||||
},
|
||||
"required": ["provider"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
]
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
@@ -681,21 +672,57 @@
|
||||
"additionalProperties": false
|
||||
},
|
||||
"stt": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string"
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"const": "local"
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"language": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"required": ["provider"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"language": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"const": "openai"
|
||||
},
|
||||
"apiKey": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"baseUrl": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"language": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"confidenceThreshold": {
|
||||
"type": "number"
|
||||
}
|
||||
},
|
||||
"required": ["provider"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
]
|
||||
},
|
||||
"turnDetection": {
|
||||
"type": "object",
|
||||
@@ -707,29 +734,58 @@
|
||||
"additionalProperties": false
|
||||
},
|
||||
"tts": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string"
|
||||
"oneOf": [
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"const": "local"
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"speakerId": {
|
||||
"type": "integer",
|
||||
"minimum": -9007199254740991,
|
||||
"maximum": 9007199254740991
|
||||
},
|
||||
"speed": {
|
||||
"type": "number"
|
||||
}
|
||||
},
|
||||
"required": ["provider"],
|
||||
"additionalProperties": false
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"voice": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"speakerId": {
|
||||
"type": "integer",
|
||||
"minimum": -9007199254740991,
|
||||
"maximum": 9007199254740991
|
||||
},
|
||||
"speed": {
|
||||
"type": "number"
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"provider": {
|
||||
"type": "string",
|
||||
"const": "openai"
|
||||
},
|
||||
"apiKey": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"baseUrl": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"model": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
},
|
||||
"voice": {
|
||||
"type": "string",
|
||||
"minLength": 1
|
||||
}
|
||||
},
|
||||
"required": ["provider"],
|
||||
"additionalProperties": false
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
]
|
||||
}
|
||||
},
|
||||
"additionalProperties": false
|
||||
|
||||
@@ -116,7 +116,7 @@ Relative paths are resolved against `PASEO_HOME`. Existing worktrees remain wher
|
||||
|
||||
## Voice
|
||||
|
||||
Voice is configured through `features.dictation` and `features.voiceMode`, with provider credentials under `providers`.
|
||||
Voice is configured through `features.dictation` and `features.voiceMode`. Provider credentials and options live inline at `features.dictation.stt`, `features.voiceMode.stt`, and `features.voiceMode.tts`.
|
||||
|
||||
For voice philosophy, architecture, and complete local/OpenAI setup examples, see [Voice docs](/docs/voice).
|
||||
|
||||
@@ -269,9 +269,9 @@ Set the persisted value in `config.json`:
|
||||
- `PASEO_LOG_FILE_ROTATE_SIZE`, override `log.file.rotate.maxSize`
|
||||
- `PASEO_LOG_FILE_ROTATE_COUNT`, override `log.file.rotate.maxFiles`
|
||||
- `PASEO_LOG`, `PASEO_LOG_FORMAT`, legacy log overrides (still supported)
|
||||
- `OPENAI_API_KEY`, override OpenAI provider key
|
||||
- `OPENAI_STT_API_KEY`, `OPENAI_STT_BASE_URL`, OpenAI speech-to-text endpoint (dictation + voice mode STT)
|
||||
- `OPENAI_TTS_API_KEY`, `OPENAI_TTS_BASE_URL`, OpenAI text-to-speech endpoint (voice mode TTS)
|
||||
- `OPENAI_API_KEY`, `OPENAI_BASE_URL`, fallback speech credentials and base URL when neither a site value nor an endpoint env value is set
|
||||
- `OPENAI_STT_API_KEY`, `OPENAI_STT_BASE_URL`, speech-to-text key and base URL fallbacks when no value is saved at the dictation or voice-mode STT site
|
||||
- `OPENAI_TTS_API_KEY`, `OPENAI_TTS_BASE_URL`, text-to-speech key and base URL fallbacks when no value is saved at the voice-mode TTS site
|
||||
- `PASEO_VOICE_LLM_PROVIDER`, override voice LLM provider (`claude`, `codex`, `opencode`)
|
||||
- `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER`, override voice provider selection (`local` or `openai`)
|
||||
- `PASEO_LOCAL_MODELS_DIR`, control local model directory
|
||||
|
||||
+48
-28
@@ -27,27 +27,24 @@ This keeps credentials and execution in your environment and avoids introducing
|
||||
|
||||
Local speech defaults to model IDs `parakeet-tdt-0.6b-v2-int8` (STT) and `kokoro-en-v0_19` (TTS, speaker 0 / voice 00).
|
||||
|
||||
Missing models are downloaded at daemon startup into `$PASEO_HOME/models/local-speech`. Downloads happen only for missing files.
|
||||
Dictation is on by default. Voice mode is off until you turn it on in **Settings > Voice** or set `features.voiceMode.enabled: true`.
|
||||
|
||||
Missing models for enabled features are downloaded at daemon startup into `$PASEO_HOME/models/local-speech`. Downloads happen only for missing files. **Settings > Voice** lists every local model with its size and languages, and lets you download or delete each one.
|
||||
|
||||
### Local STT models and language support
|
||||
|
||||
| Model ID | Languages |
|
||||
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `parakeet-tdt-0.6b-v2-int8` | English only (default). Includes punctuation and capitalization. |
|
||||
| `parakeet-tdt-0.6b-v3-int8` | 25 European languages, auto-detected: Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Spanish, Swedish, Ukrainian. |
|
||||
|
||||
**To use a non-English language, switch the local STT model to `parakeet-tdt-0.6b-v3-int8`.** v3 detects the spoken language automatically — there is no per-language setting for it. The `language` field below does **not** steer the local Parakeet model (v2 is English-only, v3 auto-detects); it only applies to the OpenAI STT provider.
|
||||
**Settings > Voice** lists every local STT model with the languages it supports. The default, `parakeet-tdt-0.6b-v2-int8`, is English-only. For other languages, pick a multilingual model such as `parakeet-tdt-0.6b-v3-int8` (25 European languages) or `whisper-base` (99 languages). The `language` field is passed to STT sessions for both providers. Local recognizers currently detect the spoken language themselves; setting `language` does not expand a model’s supported languages.
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"features": {
|
||||
"dictation": {
|
||||
"stt": { "provider": "local", "model": "parakeet-tdt-0.6b-v2-int8", "language": "en" }
|
||||
"stt": { "provider": "local", "model": "parakeet-tdt-0.6b-v2-int8" }
|
||||
},
|
||||
"voiceMode": {
|
||||
"llm": { "provider": "claude", "model": "haiku" },
|
||||
"stt": { "provider": "local", "model": "parakeet-tdt-0.6b-v2-int8", "language": "en" },
|
||||
"stt": { "provider": "local", "model": "parakeet-tdt-0.6b-v2-int8" },
|
||||
"tts": { "provider": "local", "model": "kokoro-en-v0_19", "speakerId": 0 }
|
||||
}
|
||||
},
|
||||
@@ -59,7 +56,7 @@ Missing models are downloaded at daemon startup into `$PASEO_HOME/models/local-s
|
||||
}
|
||||
```
|
||||
|
||||
For multilingual local dictation, set the model to v3 — it auto-detects the language, so no `language` field is needed:
|
||||
For multilingual local dictation, set a multilingual model. No `language` field is needed:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -72,38 +69,53 @@ For multilingual local dictation, set the model to v3 — it auto-detects the la
|
||||
}
|
||||
```
|
||||
|
||||
The `language` field applies only to the OpenAI STT provider: set `features.dictation.stt.language` for dictation and `features.voiceMode.stt.language` for voice mode. If voice language is omitted, Paseo uses the dictation language before falling back to `en`. It has no effect on the local Parakeet models.
|
||||
For either STT provider, set `features.dictation.stt.language` for dictation and `features.voiceMode.stt.language` for voice mode. If voice language is omitted, Paseo uses the dictation language before falling back to `en`.
|
||||
|
||||
## OpenAI Voice Option
|
||||
|
||||
You can switch dictation, voice STT, and voice TTS to OpenAI by setting provider fields to `openai` and providing OpenAI credentials.
|
||||
Set `provider` to `openai` and enter credentials at each use site that needs them.
|
||||
Dictation, listening and speaking each own their key, base URL and model:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"features": {
|
||||
"dictation": { "stt": { "provider": "openai" } },
|
||||
"dictation": {
|
||||
"stt": { "provider": "openai", "apiKey": "sk-...", "model": "gpt-4o-mini-transcribe" }
|
||||
},
|
||||
"voiceMode": {
|
||||
"stt": { "provider": "openai" },
|
||||
"tts": { "provider": "openai" }
|
||||
}
|
||||
},
|
||||
"providers": {
|
||||
"openai": {
|
||||
"enabled": true,
|
||||
"stt": {
|
||||
"apiKey": "...",
|
||||
"baseUrl": "https://api.openai.com/v1"
|
||||
"provider": "openai",
|
||||
"apiKey": "sk-...",
|
||||
"baseUrl": "https://api.openai.com/v1",
|
||||
"model": "gpt-4o-transcribe",
|
||||
"language": "en"
|
||||
},
|
||||
"tts": {
|
||||
"apiKey": "...",
|
||||
"baseUrl": "https://api.openai.com/v1"
|
||||
}
|
||||
"tts": { "provider": "openai", "apiKey": "sk-...", "model": "tts-1", "voice": "alloy" }
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`providers.openai.stt` covers dictation and voice mode speech-to-text, and `providers.openai.tts` covers voice mode text-to-speech. Because they resolve independently, you can point STT and TTS at different endpoints. Each falls back to `providers.openai.apiKey`/`baseUrl`, then `OPENAI_API_KEY`/`OPENAI_BASE_URL`, when unset. These settings configure only Paseo OpenAI speech traffic, without changing Codex or other OpenAI-backed tools.
|
||||
| Provider | Speech-to-text fields | Text-to-speech fields |
|
||||
| -------- | --------------------------------------------------------------- | ------------------------------------- |
|
||||
| `local` | `model`, `language` | `model`, `speakerId`, `speed` |
|
||||
| `openai` | `apiKey`, `baseUrl`, `model`, `language`, `confidenceThreshold` | `apiKey`, `baseUrl`, `model`, `voice` |
|
||||
|
||||
Fields other than `provider` are optional. Omit `model` to use the provider's default.
|
||||
OpenAI model and voice names accept custom IDs from compatible servers. Keys saved
|
||||
on the host are write-only in the app.
|
||||
|
||||
A saved `apiKey` or `baseUrl` wins over env settings. Without a saved value, STT
|
||||
uses `OPENAI_STT_API_KEY` / `OPENAI_STT_BASE_URL`, TTS uses
|
||||
`OPENAI_TTS_API_KEY` / `OPENAI_TTS_BASE_URL`, and both fall back to
|
||||
`OPENAI_API_KEY` / `OPENAI_BASE_URL`. These settings affect Paseo speech traffic.
|
||||
|
||||
Existing `providers.openai` settings load automatically into OpenAI use sites that
|
||||
lack inline values. Endpoint settings take precedence over the old shared settings.
|
||||
The next save writes the inline shape. A legacy key without an OpenAI site to
|
||||
receive it is dropped with a warning; enter a key when you later select OpenAI.
|
||||
|
||||
Paseo uses these paths under the configured OpenAI base URL:
|
||||
|
||||
@@ -113,6 +125,14 @@ Paseo uses these paths under the configured OpenAI base URL:
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Provider, enablement, model, language, voice and tuning env vars override config.
|
||||
Keys and base URLs follow the fallback rule above.
|
||||
|
||||
- `PASEO_DICTATION_ENABLED`, `PASEO_VOICE_MODE_ENABLED`, feature enablement
|
||||
- `STT_MODEL`, model override for both OpenAI STT sites
|
||||
- `STT_CONFIDENCE_THRESHOLD`, confidence threshold for both OpenAI STT sites
|
||||
- `TTS_MODEL`, `TTS_VOICE`, OpenAI speaking model and voice
|
||||
|
||||
- `PASEO_VOICE_LLM_PROVIDER`, voice agent provider override
|
||||
- `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER`, speech provider selection (`local` or `openai`)
|
||||
- `OPENAI_STT_API_KEY`, `OPENAI_STT_BASE_URL`, OpenAI speech-to-text endpoint (dictation + voice mode STT)
|
||||
@@ -120,8 +140,8 @@ Paseo uses these paths under the configured OpenAI base URL:
|
||||
- `PASEO_LOCAL_MODELS_DIR`, local model storage directory
|
||||
- `PASEO_DICTATION_LOCAL_STT_MODEL`, local dictation STT model ID
|
||||
- `PASEO_VOICE_LOCAL_STT_MODEL`, `PASEO_VOICE_LOCAL_TTS_MODEL`, local voice STT/TTS model IDs
|
||||
- `PASEO_DICTATION_LANGUAGE`, dictation STT language (OpenAI STT only; ignored by local Parakeet)
|
||||
- `PASEO_VOICE_LANGUAGE`, voice mode STT language; falls back to `PASEO_DICTATION_LANGUAGE` when unset (OpenAI STT only; ignored by local Parakeet)
|
||||
- `PASEO_DICTATION_LANGUAGE`, dictation STT language
|
||||
- `PASEO_VOICE_LANGUAGE`, voice mode STT language; falls back to `PASEO_DICTATION_LANGUAGE` when unset
|
||||
- `PASEO_VOICE_LOCAL_TTS_SPEAKER_ID`, `PASEO_VOICE_LOCAL_TTS_SPEED`, optional local voice TTS tuning
|
||||
|
||||
## Operational Notes
|
||||
|
||||
Reference in New Issue
Block a user