puter/paseo
1
0
Fork 0
mirror of https://github.com/getpaseo/paseo.git synced 2026-07-29 12:01:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use separate OpenAI endpoints for speech-to-text and text-to-speech (#1823)

Browse Source 
* feat(voice): configure OpenAI STT and TTS endpoints separately

Replace providers.openai.voice (a single apiKey/baseUrl shared by
speech-to-text and text-to-speech) with independent providers.openai.stt
and providers.openai.tts, each carrying its own apiKey/baseUrl. STT and
TTS now resolve fully independently, so they can point at different
OpenAI-compatible endpoints. The env equivalents OPENAI_VOICE_API_KEY /
OPENAI_VOICE_BASE_URL split into OPENAI_STT_* and OPENAI_TTS_*.

No backcompat: the voice key is removed and no longer read. Each feature
still falls back to providers.openai.apiKey/baseUrl, then OPENAI_API_KEY/
OPENAI_BASE_URL. Composer dictation resolves from the STT endpoint.

* fix(voice): keep daemon bootable and respect global OpenAI key

Two issues from review of the STT/TTS endpoint split:

- A config from an older release that still sets providers.openai.voice
  crashed daemon startup, because the strict schema rejects the now-unknown
  key. Strip it before parsing (alongside the existing local.autoDownload
  strip) so the daemon boots; the value is discarded, not migrated.
- An empty endpoint env var (e.g. a copied .env.example leaving
  OPENAI_STT_API_KEY= blank) shadowed the OPENAI_API_KEY fallback, so
  speech was reported as missing credentials despite a configured global
  key. firstDefined now skips empty/whitespace strings.

* fix(voice): isolate STT and TTS option parsing per endpoint

An STT-only OpenAI setup could be broken by a stale or invalid TTS env
var (e.g. a leftover TTS_VOICE/TTS_MODEL), because the single resolution
schema validated both endpoints' option groups before the per-endpoint
gate. Split into endpoint-key, STT-option, and TTS-option schemas and
parse each option group only when that endpoint has credentials, so an
unused endpoint's bad env can no longer take down the configured one.

* fix(voice): tag voice-config shim and update direct-daemon test callers

- Mark the providers.openai.voice strip with a COMPAT(openaiVoiceConfig)
  comment + removal date so it shows up in the back-compat cleanup
  inventory, per repo convention.
- Update the tests that build the daemon directly with a resolved OpenAI
  config (bootstrap smoke + the real-API voice/daemon e2e suites) to the
  new { stt, tts } shape; the old top-level { apiKey } is no longer read,
  and these files are excluded from typecheck so the break was silent.

* fix(voice): update voice-roundtrip debug script to new OpenAI config shape

Last direct daemon caller still passing the removed top-level
openai: { apiKey }; the debug script lives outside tsconfig.scripts.json
so the stale shape wasn't caught by typecheck. Use { stt, tts }.
This commit is contained in:
Mohamed Boudra
2026-06-30 10:00:27 +02:00
committed by GitHub
parent e9431517a0
commit f91806defe
17 changed files with 359 additions and 157 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
docs/data-model.md
View File

@@ -164,7 +164,12 @@ Single file, validated with `PersistedConfigSchema`.
root?: string // optional root for new worktrees; defaults to $PASEO_HOME/worktrees
},
providers: {
openai: { voice: { apiKey: string, baseUrl: string } },
openai: {
apiKey?: string,
baseUrl?: string,
stt?: { apiKey?: string, baseUrl?: string },
tts?: { apiKey?: string, baseUrl?: string }
},
local: { modelsDir: string }
},
agents: {
@@ -202,13 +207,17 @@ Set these to select OpenAI instead of local speech:
| `PASEO_DICTATION_STT_PROVIDER` | Composer dictation STT provider |
| `PASEO_VOICE_TTS_PROVIDER` | Voice mode TTS provider |
OpenAI voice can be configured under `providers.openai`:
OpenAI speech can be configured under `providers.openai`. STT and TTS resolve independently, so they can point at different endpoints:
```json
{
"providers": {
"openai": {
"voice": {
"stt": {
"apiKey": "sk-...",
"baseUrl": "https://stt.example.com/v1"
},
"tts": {
"apiKey": "sk-...",
"baseUrl": "https://api.openai.com/v1"
}
@@ -217,7 +226,7 @@ OpenAI voice can be configured under `providers.openai`:
}
```
`providers.openai.voice.apiKey` and `providers.openai.voice.baseUrl` apply only to Paseo OpenAI voice features.
`providers.openai.stt` is used for both composer dictation and voice mode speech-to-text; `providers.openai.tts` is used for voice mode text-to-speech. The equivalent env vars are `OPENAI_STT_API_KEY`/`OPENAI_STT_BASE_URL` and `OPENAI_TTS_API_KEY`/`OPENAI_TTS_BASE_URL`. Each feature falls back to `providers.openai.apiKey`/`providers.openai.baseUrl`, then `OPENAI_API_KEY`/`OPENAI_BASE_URL`, when its own fields are unset. These settings apply only to Paseo OpenAI speech features, not to Codex or other OpenAI-backed tools.
Paseo uses these paths under the configured OpenAI base URL:

8
packages/server/.env.example
View File

@@ -1,6 +1,14 @@
# OpenAI API Key (for GPT-4, Whisper STT, and TTS)
OPENAI_API_KEY=
# Optional: point speech-to-text and text-to-speech at different OpenAI-compatible
# endpoints. Each falls back to OPENAI_API_KEY / OPENAI_BASE_URL when unset.
# STT covers composer dictation + voice mode STT; TTS covers voice mode TTS.
OPENAI_STT_API_KEY=
OPENAI_STT_BASE_URL=
OPENAI_TTS_API_KEY=
OPENAI_TTS_BASE_URL=
# TTS Configuration (optional - defaults shown)
TTS_VOICE=alloy
# Available voices: alloy, echo, fable, onyx, nova, shimmer

2
packages/server/scripts/voice-roundtrip-debug.ts
View File

@@ -26,7 +26,7 @@ async function main(): Promise<void> {
const daemon = await createTestPaseoDaemon({
logger,
agentClients: {},
openai: { apiKey },
openai: { stt: { apiKey }, tts: { apiKey } },
speech: {
providers: {
dictationStt: { provider: "openai", explicit: true },

2
packages/server/src/server/bootstrap.smoke.test.ts
View File

@@ -21,7 +21,7 @@ describe("paseo daemon bootstrap", () => {
test("starts and serves health endpoint", async () => {
const daemonHandle = await createTestPaseoDaemon({
openai: { apiKey: "test-openai-api-key" },
openai: { stt: { apiKey: "test-openai-api-key" }, tts: { apiKey: "test-openai-api-key" } },
speech: {
providers: {
dictationStt: { provider: "openai", explicit: true },

4
packages/server/src/server/daemon-client.e2e.test.ts
View File

@@ -547,7 +547,9 @@ beforeAll(async () => {
ctx = await createDaemonTestContext({
dictationFinalTimeoutMs: 5000,
...(openaiApiKey ? { openai: { apiKey: openaiApiKey } } : {}),
...(openaiApiKey
? { openai: { stt: { apiKey: openaiApiKey }, tts: { apiKey: openaiApiKey } } }
: {}),
...(speechConfig ? { speech: speechConfig } : {}),
});
}, 60000);

50
packages/server/src/server/persisted-config.test.ts
View File

@@ -116,20 +116,26 @@ describe("PersistedConfigSchema worktrees config", () => {
});
describe("PersistedConfigSchema provider credentials", () => {
test("accepts OpenAI voice credentials", () => {
test("accepts separate OpenAI STT and TTS credentials", () => {
const parsed = PersistedConfigSchema.parse({
providers: {
openai: {
voice: {
apiKey: " voice-secret ",
baseUrl: " https://voice.example.com/v1 ",
stt: {
apiKey: " stt-secret ",
baseUrl: " https://stt.example.com/v1 ",
},
tts: {
apiKey: " tts-secret ",
baseUrl: " https://tts.example.com/v1 ",
},
},
},
});
expect(parsed.providers?.openai?.voice?.apiKey).toBe("voice-secret");
expect(parsed.providers?.openai?.voice?.baseUrl).toBe("https://voice.example.com/v1");
expect(parsed.providers?.openai?.stt?.apiKey).toBe("stt-secret");
expect(parsed.providers?.openai?.stt?.baseUrl).toBe("https://stt.example.com/v1");
expect(parsed.providers?.openai?.tts?.apiKey).toBe("tts-secret");
expect(parsed.providers?.openai?.tts?.baseUrl).toBe("https://tts.example.com/v1");
});
});
@@ -650,6 +656,38 @@ describe("loadPersistedConfig", () => {
rmSync(home, { recursive: true, force: true });
}
});
test("loads a config that still uses the removed providers.openai.voice block", () => {
const home = createTempHome();
const configPath = path.join(home, "config.json");
try {
writeFileSync(
configPath,
`${JSON.stringify(
{
version: 1,
providers: {
openai: {
apiKey: "global-key",
voice: { apiKey: "voice-key", baseUrl: "https://voice.example.com/v1" },
},
},
},
null,
2,
)}\n`,
);
const config = loadPersistedConfig(home);
expect(config.providers?.openai?.apiKey).toBe("global-key");
expect((config.providers?.openai as Record<string, unknown>)?.voice).toBeUndefined();
expect(config.providers?.openai?.stt).toBeUndefined();
expect(config.providers?.openai?.tts).toBeUndefined();
} finally {
rmSync(home, { recursive: true, force: true });
}
});
});
describe.skipIf(process.platform === "win32")("persisted config file permissions", () => {

36
packages/server/src/server/persisted-config.ts
View File

@@ -45,7 +45,7 @@ const LogConfigSchema = z
})
.strict();
const OpenAiVoiceProviderSchema = z
const OpenAiSpeechEndpointSchema = z
.object({
apiKey: z.string().trim().min(1).optional(),
baseUrl: z.string().trim().min(1).optional(),
@@ -55,8 +55,9 @@ const OpenAiVoiceProviderSchema = z
const OpenAiProviderSchema = z
.object({
apiKey: z.string().min(1).optional(),
voice: OpenAiVoiceProviderSchema.optional(),
baseUrl: z.string().trim().min(1).optional(),
stt: OpenAiSpeechEndpointSchema.optional(),
tts: OpenAiSpeechEndpointSchema.optional(),
})
.strict();
@@ -346,7 +347,11 @@ function getLogger(logger: LoggerLike | undefined): LoggerLike | undefined {
return logger?.child({ module: "config" });
}
function stripDeprecatedLocalSpeechConfigFields(parsed: unknown): unknown {
// Removed config fields are stripped before parsing so the strict schema does not
// reject a config written by an older release. The stripped values are discarded,
// not migrated — there is no back-compat for the removed `providers.openai.voice`
// block (use `providers.openai.stt` / `providers.openai.tts`).
function stripRemovedConfigFields(parsed: unknown): unknown {
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
return parsed;
}
@@ -358,18 +363,25 @@ function stripDeprecatedLocalSpeechConfigFields(parsed: unknown): unknown {
}
const providersRecord = { ...(providers as Record<string, unknown>) };
const local = providersRecord.local;
if (!local || typeof local !== "object" || Array.isArray(local)) {
root.providers = providersRecord;
return root;
}
const localRecord = { ...(local as Record<string, unknown>) };
if ("autoDownload" in localRecord) {
if (local && typeof local === "object" && !Array.isArray(local)) {
const localRecord = { ...(local as Record<string, unknown>) };
delete localRecord.autoDownload;
providersRecord.local = localRecord;
}
const openai = providersRecord.openai;
if (openai && typeof openai === "object" && !Array.isArray(openai)) {
const openaiRecord = { ...(openai as Record<string, unknown>) };
// COMPAT(openaiVoiceConfig): added 2026-06-30, remove after 2026-12-30.
// Drop a `providers.openai.voice` block left by an older release so the strict
// schema doesn't reject it. The value is discarded, not migrated — there is no
// back-compat; configure `providers.openai.stt` / `providers.openai.tts` instead.
delete openaiRecord.voice;
providersRecord.openai = openaiRecord;
}
providersRecord.local = localRecord;
root.providers = providersRecord;
return root;
}
@@ -412,7 +424,7 @@ export function loadPersistedConfig(paseoHome: string, logger?: LoggerLike): Per
});
}
const migrated = stripDeprecatedLocalSpeechConfigFields(parsed);
const migrated = stripRemovedConfigFields(parsed);
const result = PersistedConfigSchema.safeParse(migrated);
if (!result.success) {
const issues = result.error.issues

218
packages/server/src/server/speech/providers/openai/config.test.ts
View File

@@ -2,6 +2,14 @@ import { describe, expect, test } from "vitest";
import { PersistedConfigSchema } from "../../../persisted-config.js";
import { resolveOpenAiSpeechConfig } from "./config.js";
import type { RequestedSpeechProviders } from "../../speech-types.js";
const ALL_OPENAI: RequestedSpeechProviders = {
dictationStt: { provider: "openai", explicit: true },
voiceTurnDetection: { provider: "local", explicit: false },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
};
describe("resolveOpenAiSpeechConfig", () => {
test("treats empty OPENAI_API_KEY as unset", () => {
@@ -14,6 +22,7 @@ describe("resolveOpenAiSpeechConfig", () => {
env,
persisted,
providers: {
...ALL_OPENAI,
dictationStt: { provider: "local", explicit: false },
voiceStt: { provider: "local", explicit: false },
voiceTts: { provider: "local", explicit: false },
@@ -23,138 +32,175 @@ describe("resolveOpenAiSpeechConfig", () => {
expect(resolved).toBeUndefined();
});
test("uses trimmed OPENAI_API_KEY when configured", () => {
test("applies trimmed OPENAI_API_KEY to both STT and TTS", () => {
const persisted = PersistedConfigSchema.parse({});
const env = {
OPENAI_API_KEY: " sk-test ",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({
env,
persisted,
providers: {
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
},
});
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.apiKey).toBe("sk-test");
expect(resolved?.stt?.apiKey).toBe("sk-test");
expect(resolved?.tts?.apiKey).toBe("sk-test");
});
test("uses nested voice config before env and non-voice fallbacks", () => {
test("resolves distinct endpoints for STT and TTS", () => {
const persisted = PersistedConfigSchema.parse({
providers: {
openai: {
apiKey: "fallback-config-key",
voice: {
apiKey: "voice-config-key",
baseUrl: " https://voice.example.com/v1 ",
stt: {
apiKey: "stt-key",
baseUrl: " https://stt.example.com/v1 ",
},
tts: {
apiKey: "tts-key",
baseUrl: " https://tts.example.com/v1 ",
},
baseUrl: "https://legacy-config.example.com/v1",
},
},
});
const env = {
OPENAI_API_KEY: "env-key",
OPENAI_VOICE_API_KEY: "voice-env-key",
OPENAI_VOICE_BASE_URL: "https://voice-env.example.com/v1",
OPENAI_BASE_URL: "https://env.example.com/v1",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({
env,
env: {} as NodeJS.ProcessEnv,
persisted,
providers: {
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
},
providers: ALL_OPENAI,
});
expect(resolved?.apiKey).toBe("voice-config-key");
expect(resolved?.baseUrl).toBe("https://voice.example.com/v1");
expect(resolved?.stt?.apiKey).toBe("voice-config-key");
expect(resolved?.stt?.baseUrl).toBe("https://voice.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("voice-config-key");
expect(resolved?.tts?.baseUrl).toBe("https://voice.example.com/v1");
expect(resolved?.stt?.apiKey).toBe("stt-key");
expect(resolved?.stt?.baseUrl).toBe("https://stt.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("tts-key");
expect(resolved?.tts?.baseUrl).toBe("https://tts.example.com/v1");
});
test("uses voice env config when nested voice config is unset", () => {
const persisted = PersistedConfigSchema.parse({});
const env = {
OPENAI_API_KEY: "sk-test",
OPENAI_VOICE_API_KEY: "voice-env-key",
OPENAI_VOICE_BASE_URL: " https://voice-env.example.com/v1 ",
OPENAI_BASE_URL: "https://env.example.com/v1",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({
env,
persisted,
providers: {
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
},
});
expect(resolved?.apiKey).toBe("voice-env-key");
expect(resolved?.stt?.apiKey).toBe("voice-env-key");
expect(resolved?.tts?.apiKey).toBe("voice-env-key");
expect(resolved?.baseUrl).toBe("https://voice-env.example.com/v1");
expect(resolved?.stt?.baseUrl).toBe("https://voice-env.example.com/v1");
expect(resolved?.tts?.baseUrl).toBe("https://voice-env.example.com/v1");
});
test("falls back to non-voice OpenAI config", () => {
test("prefers nested STT/TTS config over env and global fallbacks", () => {
const persisted = PersistedConfigSchema.parse({
providers: {
openai: {
apiKey: "fallback-config-key",
baseUrl: " https://legacy-config.example.com/v1 ",
baseUrl: "https://global-config.example.com/v1",
stt: { apiKey: "stt-config-key", baseUrl: " https://stt.example.com/v1 " },
tts: { apiKey: "tts-config-key", baseUrl: " https://tts.example.com/v1 " },
},
},
});
const env = {
OPENAI_API_KEY: "sk-test",
OPENAI_BASE_URL: " https://env.example.com/v1 ",
OPENAI_API_KEY: "env-key",
OPENAI_STT_API_KEY: "stt-env-key",
OPENAI_STT_BASE_URL: "https://stt-env.example.com/v1",
OPENAI_TTS_API_KEY: "tts-env-key",
OPENAI_TTS_BASE_URL: "https://tts-env.example.com/v1",
OPENAI_BASE_URL: "https://env.example.com/v1",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({
env,
persisted,
providers: {
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
},
});
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.apiKey).toBe("fallback-config-key");
expect(resolved?.baseUrl).toBe("https://legacy-config.example.com/v1");
expect(resolved?.stt?.apiKey).toBe("stt-config-key");
expect(resolved?.stt?.baseUrl).toBe("https://stt.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("tts-config-key");
expect(resolved?.tts?.baseUrl).toBe("https://tts.example.com/v1");
});
test("falls back to global OpenAI env config when voice-specific inputs are unset", () => {
test("uses STT/TTS env config when nested config is unset", () => {
const persisted = PersistedConfigSchema.parse({});
const env = {
OPENAI_API_KEY: "sk-test",
OPENAI_STT_API_KEY: "stt-env-key",
OPENAI_STT_BASE_URL: " https://stt-env.example.com/v1 ",
OPENAI_TTS_API_KEY: "tts-env-key",
OPENAI_TTS_BASE_URL: " https://tts-env.example.com/v1 ",
OPENAI_BASE_URL: "https://env.example.com/v1",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("stt-env-key");
expect(resolved?.stt?.baseUrl).toBe("https://stt-env.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("tts-env-key");
expect(resolved?.tts?.baseUrl).toBe("https://tts-env.example.com/v1");
});
test("falls back to global OpenAI config for both STT and TTS", () => {
const persisted = PersistedConfigSchema.parse({
providers: {
openai: {
apiKey: "fallback-config-key",
baseUrl: " https://global-config.example.com/v1 ",
},
},
});
const env = {} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("fallback-config-key");
expect(resolved?.stt?.baseUrl).toBe("https://global-config.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("fallback-config-key");
expect(resolved?.tts?.baseUrl).toBe("https://global-config.example.com/v1");
});
test("falls back to global OpenAI env config when feature inputs are unset", () => {
const persisted = PersistedConfigSchema.parse({});
const env = {
OPENAI_API_KEY: "env-key",
OPENAI_BASE_URL: " https://env.example.com/v1 ",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({
env,
persisted,
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("env-key");
expect(resolved?.stt?.baseUrl).toBe("https://env.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("env-key");
expect(resolved?.tts?.baseUrl).toBe("https://env.example.com/v1");
});
test("ignores empty endpoint env vars and falls back to OPENAI_API_KEY", () => {
const persisted = PersistedConfigSchema.parse({});
const env = {
OPENAI_API_KEY: "global-key",
OPENAI_STT_API_KEY: "",
OPENAI_STT_BASE_URL: " ",
OPENAI_TTS_API_KEY: "",
OPENAI_TTS_BASE_URL: "",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("global-key");
expect(resolved?.tts?.apiKey).toBe("global-key");
});
test("omits TTS when only an STT key is configured", () => {
const persisted = PersistedConfigSchema.parse({
providers: {
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
openai: {
stt: { apiKey: "stt-only-key" },
},
},
});
expect(resolved?.apiKey).toBe("env-key");
expect(resolved?.baseUrl).toBe("https://env.example.com/v1");
const resolved = resolveOpenAiSpeechConfig({
env: {} as NodeJS.ProcessEnv,
persisted,
providers: ALL_OPENAI,
});
expect(resolved?.stt?.apiKey).toBe("stt-only-key");
expect(resolved?.tts).toBeUndefined();
});
test("resolves STT even when an unused TTS env var is invalid", () => {
const persisted = PersistedConfigSchema.parse({
providers: {
openai: {
stt: { apiKey: "stt-only-key" },
},
},
});
const env = { TTS_VOICE: "not-a-real-voice", TTS_MODEL: "bogus-model" } as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("stt-only-key");
expect(resolved?.tts).toBeUndefined();
});
});

119
packages/server/src/server/speech/providers/openai/config.ts
View File

@@ -8,8 +8,6 @@ import type { TTSConfig } from "./tts.js";
export const DEFAULT_OPENAI_TTS_MODEL = "tts-1";
export interface OpenAiSpeechProviderConfig {
apiKey?: string;
baseUrl?: string;
stt?: Partial<STTConfig> & { apiKey?: string };
tts?: Partial<TTSConfig> & { apiKey?: string };
}
@@ -30,11 +28,23 @@ const OptionalTrimmedStringSchema = z
.optional()
.transform((value) => (value && value.length > 0 ? value : undefined));
const OpenAiSpeechResolutionSchema = z.object({
apiKey: OptionalTrimmedStringSchema,
baseUrl: OptionalTrimmedStringSchema,
// Endpoint credentials only — plain trimmed strings, so this never throws on a
// malformed value. The STT/TTS option groups parse separately and only for the
// endpoint that is actually configured, so a stale env var for an unused endpoint
// (e.g. a leftover TTS_VOICE in an STT-only setup) can't break the other one.
const OpenAiEndpointKeysSchema = z.object({
sttApiKey: OptionalTrimmedStringSchema,
sttBaseUrl: OptionalTrimmedStringSchema,
ttsApiKey: OptionalTrimmedStringSchema,
ttsBaseUrl: OptionalTrimmedStringSchema,
});
const OpenAiSttOptionsSchema = z.object({
sttConfidenceThreshold: OptionalFiniteNumberSchema,
sttModel: OptionalTrimmedStringSchema,
});
const OpenAiTtsOptionsSchema = z.object({
ttsVoice: z.string().trim().toLowerCase().pipe(OpenAiTtsVoiceSchema).default("alloy"),
ttsModel: z
.string()
@@ -57,9 +67,15 @@ function pickIfOpenAi<T>(
function firstDefined<T>(values: Array<T | null | undefined>): T | undefined {
for (const value of values) {
if (value !== undefined && value !== null) {
return value;
if (value === undefined || value === null) {
continue;
}
// Empty/whitespace env vars (e.g. a copied .env.example with OPENAI_STT_API_KEY=)
// must not shadow a later fallback such as OPENAI_API_KEY.
if (typeof value === "string" && value.trim().length === 0) {
continue;
}
return value;
}
return undefined;
}
@@ -108,18 +124,32 @@ function buildOpenAiResolutionInput(params: {
persisted: PersistedConfig;
providers: RequestedSpeechProviders;
}): Record<string, unknown> {
const { env } = params;
const openai = params.persisted.providers?.openai;
return {
apiKey: firstDefined<string>([
params.persisted.providers?.openai?.voice?.apiKey,
params.env.OPENAI_VOICE_API_KEY,
params.persisted.providers?.openai?.apiKey,
params.env.OPENAI_API_KEY,
sttApiKey: firstDefined<string>([
openai?.stt?.apiKey,
env.OPENAI_STT_API_KEY,
openai?.apiKey,
env.OPENAI_API_KEY,
]),
baseUrl: firstDefined<string>([
params.persisted.providers?.openai?.voice?.baseUrl,
params.env.OPENAI_VOICE_BASE_URL,
params.persisted.providers?.openai?.baseUrl,
params.env.OPENAI_BASE_URL,
sttBaseUrl: firstDefined<string>([
openai?.stt?.baseUrl,
env.OPENAI_STT_BASE_URL,
openai?.baseUrl,
env.OPENAI_BASE_URL,
]),
ttsApiKey: firstDefined<string>([
openai?.tts?.apiKey,
env.OPENAI_TTS_API_KEY,
openai?.apiKey,
env.OPENAI_API_KEY,
]),
ttsBaseUrl: firstDefined<string>([
openai?.tts?.baseUrl,
env.OPENAI_TTS_BASE_URL,
openai?.baseUrl,
env.OPENAI_BASE_URL,
]),
...buildOpenAiSttInput(params),
...buildOpenAiTtsInput(params),
@@ -131,29 +161,46 @@ export function resolveOpenAiSpeechConfig(params: {
persisted: PersistedConfig;
providers: RequestedSpeechProviders;
}): OpenAiSpeechProviderConfig | undefined {
const parsed = OpenAiSpeechResolutionSchema.parse(buildOpenAiResolutionInput(params));
const input = buildOpenAiResolutionInput(params);
const keys = OpenAiEndpointKeysSchema.parse(input);
if (!parsed.apiKey) {
if (!keys.sttApiKey && !keys.ttsApiKey) {
return undefined;
}
return {
apiKey: parsed.apiKey,
...(parsed.baseUrl ? { baseUrl: parsed.baseUrl } : {}),
stt: {
apiKey: parsed.apiKey,
...(parsed.baseUrl ? { baseUrl: parsed.baseUrl } : {}),
...(parsed.sttConfidenceThreshold !== undefined
? { confidenceThreshold: parsed.sttConfidenceThreshold }
: {}),
...(parsed.sttModel ? { model: parsed.sttModel } : {}),
},
tts: {
apiKey: parsed.apiKey,
...(parsed.baseUrl ? { baseUrl: parsed.baseUrl } : {}),
voice: parsed.ttsVoice,
model: parsed.ttsModel,
responseFormat: "pcm",
},
...(keys.sttApiKey ? { stt: buildSttConfig(keys.sttApiKey, keys.sttBaseUrl, input) } : {}),
...(keys.ttsApiKey ? { tts: buildTtsConfig(keys.ttsApiKey, keys.ttsBaseUrl, input) } : {}),
};
}
function buildSttConfig(
apiKey: string,
baseUrl: string | undefined,
input: Record<string, unknown>,
): OpenAiSpeechProviderConfig["stt"] {
const options = OpenAiSttOptionsSchema.parse(input);
return {
apiKey,
...(baseUrl ? { baseUrl } : {}),
...(options.sttConfidenceThreshold !== undefined
? { confidenceThreshold: options.sttConfidenceThreshold }
: {}),
...(options.sttModel ? { model: options.sttModel } : {}),
};
}
function buildTtsConfig(
apiKey: string,
baseUrl: string | undefined,
input: Record<string, unknown>,
): OpenAiSpeechProviderConfig["tts"] {
const options = OpenAiTtsOptionsSchema.parse(input);
return {
apiKey,
...(baseUrl ? { baseUrl } : {}),
voice: options.ttsVoice,
model: options.ttsModel,
responseFormat: "pcm",
};
}

1
packages/server/src/server/speech/providers/openai/runtime.test.ts
View File

@@ -15,7 +15,6 @@ describe("initializeOpenAiSpeechServices", () => {
voiceTts: { provider: "openai", explicit: true },
},
openaiConfig: {
apiKey: "sk-test",
stt: { apiKey: "sk-test" },
tts: { apiKey: "sk-test" },
},

8
packages/server/src/server/speech/providers/openai/runtime.ts
View File

@@ -29,11 +29,11 @@ export interface SpeechServices {
function resolveOpenAiCredentials(
openaiConfig: OpenAiSpeechProviderConfig | undefined,
): OpenAiCredentialState {
const openaiApiKey = openaiConfig?.apiKey;
const sttApiKey = openaiConfig?.stt?.apiKey;
return {
openaiSttApiKey: openaiConfig?.stt?.apiKey ?? openaiApiKey,
openaiTtsApiKey: openaiConfig?.tts?.apiKey ?? openaiApiKey,
openaiDictationApiKey: openaiApiKey,
openaiSttApiKey: sttApiKey,
openaiTtsApiKey: openaiConfig?.tts?.apiKey,
openaiDictationApiKey: sttApiKey,
};
}

3
packages/server/src/server/speech/speech-config-resolver.test.ts
View File

@@ -129,7 +129,8 @@ describe("resolveSpeechConfig", () => {
dictation: "es",
voice: "pt",
});
expect(result.openai?.apiKey).toBe("persisted-key");
expect(result.openai?.stt?.apiKey).toBe("persisted-key");
expect(result.openai?.tts?.apiKey).toBe("persisted-key");
expect(result.openai?.stt?.model).toBe("gpt-4o-transcribe");
});

2
packages/server/src/server/voice-local-agent.e2e.test.ts
View File

@@ -79,7 +79,7 @@ function waitForSignal<T>(
beforeAll(async () => {
ctx = await createDaemonTestContext({
agentClients: {},
openai: { apiKey: openaiApiKey! },
openai: { stt: { apiKey: openaiApiKey! }, tts: { apiKey: openaiApiKey! } },
speech: {
providers: {
dictationStt: { provider: "openai", explicit: true },

2
packages/server/src/server/voice-roundtrip.e2e.test.ts
View File

@@ -161,7 +161,7 @@ let ctx: DaemonTestContext;
beforeAll(async () => {
ctx = await createDaemonTestContext({
agentClients: {},
openai: { apiKey: openaiApiKey! },
openai: { stt: { apiKey: openaiApiKey! }, tts: { apiKey: openaiApiKey! } },
speech: {
providers: {
dictationStt: { provider: "openai", explicit: true },

32
packages/website/public/schemas/paseo.config.v1.json
View File

@@ -174,6 +174,38 @@
"apiKey": {
"type": "string",
"minLength": 1
},
"baseUrl": {
"type": "string",
"minLength": 1
},
"stt": {
"type": "object",
"properties": {
"apiKey": {
"type": "string",
"minLength": 1
},
"baseUrl": {
"type": "string",
"minLength": 1
}
},
"additionalProperties": false
},
"tts": {
"type": "object",
"properties": {
"apiKey": {
"type": "string",
"minLength": 1
},
"baseUrl": {
"type": "string",
"minLength": 1
}
},
"additionalProperties": false
}
},
"additionalProperties": false

2
public-docs/configuration.md
View File

@@ -204,6 +204,8 @@ In the mobile app, enter the password in the direct connection setup screen.
- `PASEO_LOG_FILE_ROTATE_COUNT`, override `log.file.rotate.maxFiles`
- `PASEO_LOG`, `PASEO_LOG_FORMAT`, legacy log overrides (still supported)
- `OPENAI_API_KEY`, override OpenAI provider key
- `OPENAI_STT_API_KEY`, `OPENAI_STT_BASE_URL`, OpenAI speech-to-text endpoint (dictation + voice mode STT)
- `OPENAI_TTS_API_KEY`, `OPENAI_TTS_BASE_URL`, OpenAI text-to-speech endpoint (voice mode TTS)
- `PASEO_VOICE_LLM_PROVIDER`, override voice LLM provider (`claude`, `codex`, `opencode`)
- `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER`, override voice provider selection (`local` or `openai`)
- `PASEO_LOCAL_MODELS_DIR`, control local model directory

10
public-docs/voice.md
View File

@@ -90,7 +90,11 @@ You can switch dictation, voice STT, and voice TTS to OpenAI by setting provider
},
"providers": {
"openai": {
"voice": {
"stt": {
"apiKey": "...",
"baseUrl": "https://api.openai.com/v1"
},
"tts": {
"apiKey": "...",
"baseUrl": "https://api.openai.com/v1"
}
@@ -99,7 +103,7 @@ You can switch dictation, voice STT, and voice TTS to OpenAI by setting provider
}
```
`providers.openai.voice.apiKey` and `providers.openai.voice.baseUrl` configure only Paseo OpenAI voice traffic, without changing Codex or other OpenAI-backed tools.
`providers.openai.stt` covers dictation and voice mode speech-to-text, and `providers.openai.tts` covers voice mode text-to-speech. Because they resolve independently, you can point STT and TTS at different endpoints. Each falls back to `providers.openai.apiKey`/`baseUrl`, then `OPENAI_API_KEY`/`OPENAI_BASE_URL`, when unset. These settings configure only Paseo OpenAI speech traffic, without changing Codex or other OpenAI-backed tools.
Paseo uses these paths under the configured OpenAI base URL:
@@ -111,6 +115,8 @@ Paseo uses these paths under the configured OpenAI base URL:
- `PASEO_VOICE_LLM_PROVIDER`, voice agent provider override
- `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER`, speech provider selection (`local` or `openai`)
- `OPENAI_STT_API_KEY`, `OPENAI_STT_BASE_URL`, OpenAI speech-to-text endpoint (dictation + voice mode STT)
- `OPENAI_TTS_API_KEY`, `OPENAI_TTS_BASE_URL`, OpenAI text-to-speech endpoint (voice mode TTS)
- `PASEO_LOCAL_MODELS_DIR`, local model storage directory
- `PASEO_DICTATION_LOCAL_STT_MODEL`, local dictation STT model ID
- `PASEO_VOICE_LOCAL_STT_MODEL`, `PASEO_VOICE_LOCAL_TTS_MODEL`, local voice STT/TTS model IDs