Use separate OpenAI endpoints for speech-to-text and text-to-speech (#1823)

* feat(voice): configure OpenAI STT and TTS endpoints separately

Replace providers.openai.voice (a single apiKey/baseUrl shared by
speech-to-text and text-to-speech) with independent providers.openai.stt
and providers.openai.tts, each carrying its own apiKey/baseUrl. STT and
TTS now resolve fully independently, so they can point at different
OpenAI-compatible endpoints. The env equivalents OPENAI_VOICE_API_KEY /
OPENAI_VOICE_BASE_URL split into OPENAI_STT_* and OPENAI_TTS_*.

No backcompat: the voice key is removed and no longer read. Each feature
still falls back to providers.openai.apiKey/baseUrl, then OPENAI_API_KEY/
OPENAI_BASE_URL. Composer dictation resolves from the STT endpoint.

* fix(voice): keep daemon bootable and respect global OpenAI key

Two issues from review of the STT/TTS endpoint split:

- A config from an older release that still sets providers.openai.voice
  crashed daemon startup, because the strict schema rejects the now-unknown
  key. Strip it before parsing (alongside the existing local.autoDownload
  strip) so the daemon boots; the value is discarded, not migrated.
- An empty endpoint env var (e.g. a copied .env.example leaving
  OPENAI_STT_API_KEY= blank) shadowed the OPENAI_API_KEY fallback, so
  speech was reported as missing credentials despite a configured global
  key. firstDefined now skips empty/whitespace strings.

* fix(voice): isolate STT and TTS option parsing per endpoint

An STT-only OpenAI setup could be broken by a stale or invalid TTS env
var (e.g. a leftover TTS_VOICE/TTS_MODEL), because the single resolution
schema validated both endpoints' option groups before the per-endpoint
gate. Split into endpoint-key, STT-option, and TTS-option schemas and
parse each option group only when that endpoint has credentials, so an
unused endpoint's bad env can no longer take down the configured one.

* fix(voice): tag voice-config shim and update direct-daemon test callers

- Mark the providers.openai.voice strip with a COMPAT(openaiVoiceConfig)
  comment + removal date so it shows up in the back-compat cleanup
  inventory, per repo convention.
- Update the tests that build the daemon directly with a resolved OpenAI
  config (bootstrap smoke + the real-API voice/daemon e2e suites) to the
  new { stt, tts } shape; the old top-level { apiKey } is no longer read,
  and these files are excluded from typecheck so the break was silent.

* fix(voice): update voice-roundtrip debug script to new OpenAI config shape

Last direct daemon caller still passing the removed top-level
openai: { apiKey }; the debug script lives outside tsconfig.scripts.json
so the stale shape wasn't caught by typecheck. Use { stt, tts }.
This commit is contained in:
Mohamed Boudra
2026-06-30 10:00:27 +02:00
committed by GitHub
parent e9431517a0
commit f91806defe
17 changed files with 359 additions and 157 deletions
+13 -4
View File
@@ -164,7 +164,12 @@ Single file, validated with `PersistedConfigSchema`.
root?: string // optional root for new worktrees; defaults to $PASEO_HOME/worktrees root?: string // optional root for new worktrees; defaults to $PASEO_HOME/worktrees
}, },
providers: { providers: {
openai: { voice: { apiKey: string, baseUrl: string } }, openai: {
apiKey?: string,
baseUrl?: string,
stt?: { apiKey?: string, baseUrl?: string },
tts?: { apiKey?: string, baseUrl?: string }
},
local: { modelsDir: string } local: { modelsDir: string }
}, },
agents: { agents: {
@@ -202,13 +207,17 @@ Set these to select OpenAI instead of local speech:
| `PASEO_DICTATION_STT_PROVIDER` | Composer dictation STT provider | | `PASEO_DICTATION_STT_PROVIDER` | Composer dictation STT provider |
| `PASEO_VOICE_TTS_PROVIDER` | Voice mode TTS provider | | `PASEO_VOICE_TTS_PROVIDER` | Voice mode TTS provider |
OpenAI voice can be configured under `providers.openai`: OpenAI speech can be configured under `providers.openai`. STT and TTS resolve independently, so they can point at different endpoints:
```json ```json
{ {
"providers": { "providers": {
"openai": { "openai": {
"voice": { "stt": {
"apiKey": "sk-...",
"baseUrl": "https://stt.example.com/v1"
},
"tts": {
"apiKey": "sk-...", "apiKey": "sk-...",
"baseUrl": "https://api.openai.com/v1" "baseUrl": "https://api.openai.com/v1"
} }
@@ -217,7 +226,7 @@ OpenAI voice can be configured under `providers.openai`:
} }
``` ```
`providers.openai.voice.apiKey` and `providers.openai.voice.baseUrl` apply only to Paseo OpenAI voice features. `providers.openai.stt` is used for both composer dictation and voice mode speech-to-text; `providers.openai.tts` is used for voice mode text-to-speech. The equivalent env vars are `OPENAI_STT_API_KEY`/`OPENAI_STT_BASE_URL` and `OPENAI_TTS_API_KEY`/`OPENAI_TTS_BASE_URL`. Each feature falls back to `providers.openai.apiKey`/`providers.openai.baseUrl`, then `OPENAI_API_KEY`/`OPENAI_BASE_URL`, when its own fields are unset. These settings apply only to Paseo OpenAI speech features, not to Codex or other OpenAI-backed tools.
Paseo uses these paths under the configured OpenAI base URL: Paseo uses these paths under the configured OpenAI base URL:
+8
View File
@@ -1,6 +1,14 @@
# OpenAI API Key (for GPT-4, Whisper STT, and TTS) # OpenAI API Key (for GPT-4, Whisper STT, and TTS)
OPENAI_API_KEY= OPENAI_API_KEY=
# Optional: point speech-to-text and text-to-speech at different OpenAI-compatible
# endpoints. Each falls back to OPENAI_API_KEY / OPENAI_BASE_URL when unset.
# STT covers composer dictation + voice mode STT; TTS covers voice mode TTS.
OPENAI_STT_API_KEY=
OPENAI_STT_BASE_URL=
OPENAI_TTS_API_KEY=
OPENAI_TTS_BASE_URL=
# TTS Configuration (optional - defaults shown) # TTS Configuration (optional - defaults shown)
TTS_VOICE=alloy TTS_VOICE=alloy
# Available voices: alloy, echo, fable, onyx, nova, shimmer # Available voices: alloy, echo, fable, onyx, nova, shimmer
@@ -26,7 +26,7 @@ async function main(): Promise<void> {
const daemon = await createTestPaseoDaemon({ const daemon = await createTestPaseoDaemon({
logger, logger,
agentClients: {}, agentClients: {},
openai: { apiKey }, openai: { stt: { apiKey }, tts: { apiKey } },
speech: { speech: {
providers: { providers: {
dictationStt: { provider: "openai", explicit: true }, dictationStt: { provider: "openai", explicit: true },
@@ -21,7 +21,7 @@ describe("paseo daemon bootstrap", () => {
test("starts and serves health endpoint", async () => { test("starts and serves health endpoint", async () => {
const daemonHandle = await createTestPaseoDaemon({ const daemonHandle = await createTestPaseoDaemon({
openai: { apiKey: "test-openai-api-key" }, openai: { stt: { apiKey: "test-openai-api-key" }, tts: { apiKey: "test-openai-api-key" } },
speech: { speech: {
providers: { providers: {
dictationStt: { provider: "openai", explicit: true }, dictationStt: { provider: "openai", explicit: true },
@@ -547,7 +547,9 @@ beforeAll(async () => {
ctx = await createDaemonTestContext({ ctx = await createDaemonTestContext({
dictationFinalTimeoutMs: 5000, dictationFinalTimeoutMs: 5000,
...(openaiApiKey ? { openai: { apiKey: openaiApiKey } } : {}), ...(openaiApiKey
? { openai: { stt: { apiKey: openaiApiKey }, tts: { apiKey: openaiApiKey } } }
: {}),
...(speechConfig ? { speech: speechConfig } : {}), ...(speechConfig ? { speech: speechConfig } : {}),
}); });
}, 60000); }, 60000);
@@ -116,20 +116,26 @@ describe("PersistedConfigSchema worktrees config", () => {
}); });
describe("PersistedConfigSchema provider credentials", () => { describe("PersistedConfigSchema provider credentials", () => {
test("accepts OpenAI voice credentials", () => { test("accepts separate OpenAI STT and TTS credentials", () => {
const parsed = PersistedConfigSchema.parse({ const parsed = PersistedConfigSchema.parse({
providers: { providers: {
openai: { openai: {
voice: { stt: {
apiKey: " voice-secret ", apiKey: " stt-secret ",
baseUrl: " https://voice.example.com/v1 ", baseUrl: " https://stt.example.com/v1 ",
},
tts: {
apiKey: " tts-secret ",
baseUrl: " https://tts.example.com/v1 ",
}, },
}, },
}, },
}); });
expect(parsed.providers?.openai?.voice?.apiKey).toBe("voice-secret"); expect(parsed.providers?.openai?.stt?.apiKey).toBe("stt-secret");
expect(parsed.providers?.openai?.voice?.baseUrl).toBe("https://voice.example.com/v1"); expect(parsed.providers?.openai?.stt?.baseUrl).toBe("https://stt.example.com/v1");
expect(parsed.providers?.openai?.tts?.apiKey).toBe("tts-secret");
expect(parsed.providers?.openai?.tts?.baseUrl).toBe("https://tts.example.com/v1");
}); });
}); });
@@ -650,6 +656,38 @@ describe("loadPersistedConfig", () => {
rmSync(home, { recursive: true, force: true }); rmSync(home, { recursive: true, force: true });
} }
}); });
test("loads a config that still uses the removed providers.openai.voice block", () => {
const home = createTempHome();
const configPath = path.join(home, "config.json");
try {
writeFileSync(
configPath,
`${JSON.stringify(
{
version: 1,
providers: {
openai: {
apiKey: "global-key",
voice: { apiKey: "voice-key", baseUrl: "https://voice.example.com/v1" },
},
},
},
null,
2,
)}\n`,
);
const config = loadPersistedConfig(home);
expect(config.providers?.openai?.apiKey).toBe("global-key");
expect((config.providers?.openai as Record<string, unknown>)?.voice).toBeUndefined();
expect(config.providers?.openai?.stt).toBeUndefined();
expect(config.providers?.openai?.tts).toBeUndefined();
} finally {
rmSync(home, { recursive: true, force: true });
}
});
}); });
describe.skipIf(process.platform === "win32")("persisted config file permissions", () => { describe.skipIf(process.platform === "win32")("persisted config file permissions", () => {
+24 -12
View File
@@ -45,7 +45,7 @@ const LogConfigSchema = z
}) })
.strict(); .strict();
const OpenAiVoiceProviderSchema = z const OpenAiSpeechEndpointSchema = z
.object({ .object({
apiKey: z.string().trim().min(1).optional(), apiKey: z.string().trim().min(1).optional(),
baseUrl: z.string().trim().min(1).optional(), baseUrl: z.string().trim().min(1).optional(),
@@ -55,8 +55,9 @@ const OpenAiVoiceProviderSchema = z
const OpenAiProviderSchema = z const OpenAiProviderSchema = z
.object({ .object({
apiKey: z.string().min(1).optional(), apiKey: z.string().min(1).optional(),
voice: OpenAiVoiceProviderSchema.optional(),
baseUrl: z.string().trim().min(1).optional(), baseUrl: z.string().trim().min(1).optional(),
stt: OpenAiSpeechEndpointSchema.optional(),
tts: OpenAiSpeechEndpointSchema.optional(),
}) })
.strict(); .strict();
@@ -346,7 +347,11 @@ function getLogger(logger: LoggerLike | undefined): LoggerLike | undefined {
return logger?.child({ module: "config" }); return logger?.child({ module: "config" });
} }
function stripDeprecatedLocalSpeechConfigFields(parsed: unknown): unknown { // Removed config fields are stripped before parsing so the strict schema does not
// reject a config written by an older release. The stripped values are discarded,
// not migrated — there is no back-compat for the removed `providers.openai.voice`
// block (use `providers.openai.stt` / `providers.openai.tts`).
function stripRemovedConfigFields(parsed: unknown): unknown {
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) { if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
return parsed; return parsed;
} }
@@ -358,18 +363,25 @@ function stripDeprecatedLocalSpeechConfigFields(parsed: unknown): unknown {
} }
const providersRecord = { ...(providers as Record<string, unknown>) }; const providersRecord = { ...(providers as Record<string, unknown>) };
const local = providersRecord.local; const local = providersRecord.local;
if (!local || typeof local !== "object" || Array.isArray(local)) { if (local && typeof local === "object" && !Array.isArray(local)) {
root.providers = providersRecord; const localRecord = { ...(local as Record<string, unknown>) };
return root;
}
const localRecord = { ...(local as Record<string, unknown>) };
if ("autoDownload" in localRecord) {
delete localRecord.autoDownload; delete localRecord.autoDownload;
providersRecord.local = localRecord;
}
const openai = providersRecord.openai;
if (openai && typeof openai === "object" && !Array.isArray(openai)) {
const openaiRecord = { ...(openai as Record<string, unknown>) };
// COMPAT(openaiVoiceConfig): added 2026-06-30, remove after 2026-12-30.
// Drop a `providers.openai.voice` block left by an older release so the strict
// schema doesn't reject it. The value is discarded, not migrated — there is no
// back-compat; configure `providers.openai.stt` / `providers.openai.tts` instead.
delete openaiRecord.voice;
providersRecord.openai = openaiRecord;
} }
providersRecord.local = localRecord;
root.providers = providersRecord; root.providers = providersRecord;
return root; return root;
} }
@@ -412,7 +424,7 @@ export function loadPersistedConfig(paseoHome: string, logger?: LoggerLike): Per
}); });
} }
const migrated = stripDeprecatedLocalSpeechConfigFields(parsed); const migrated = stripRemovedConfigFields(parsed);
const result = PersistedConfigSchema.safeParse(migrated); const result = PersistedConfigSchema.safeParse(migrated);
if (!result.success) { if (!result.success) {
const issues = result.error.issues const issues = result.error.issues
@@ -2,6 +2,14 @@ import { describe, expect, test } from "vitest";
import { PersistedConfigSchema } from "../../../persisted-config.js"; import { PersistedConfigSchema } from "../../../persisted-config.js";
import { resolveOpenAiSpeechConfig } from "./config.js"; import { resolveOpenAiSpeechConfig } from "./config.js";
import type { RequestedSpeechProviders } from "../../speech-types.js";
const ALL_OPENAI: RequestedSpeechProviders = {
dictationStt: { provider: "openai", explicit: true },
voiceTurnDetection: { provider: "local", explicit: false },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
};
describe("resolveOpenAiSpeechConfig", () => { describe("resolveOpenAiSpeechConfig", () => {
test("treats empty OPENAI_API_KEY as unset", () => { test("treats empty OPENAI_API_KEY as unset", () => {
@@ -14,6 +22,7 @@ describe("resolveOpenAiSpeechConfig", () => {
env, env,
persisted, persisted,
providers: { providers: {
...ALL_OPENAI,
dictationStt: { provider: "local", explicit: false }, dictationStt: { provider: "local", explicit: false },
voiceStt: { provider: "local", explicit: false }, voiceStt: { provider: "local", explicit: false },
voiceTts: { provider: "local", explicit: false }, voiceTts: { provider: "local", explicit: false },
@@ -23,138 +32,175 @@ describe("resolveOpenAiSpeechConfig", () => {
expect(resolved).toBeUndefined(); expect(resolved).toBeUndefined();
}); });
test("uses trimmed OPENAI_API_KEY when configured", () => { test("applies trimmed OPENAI_API_KEY to both STT and TTS", () => {
const persisted = PersistedConfigSchema.parse({}); const persisted = PersistedConfigSchema.parse({});
const env = { const env = {
OPENAI_API_KEY: " sk-test ", OPENAI_API_KEY: " sk-test ",
} as NodeJS.ProcessEnv; } as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
env,
persisted,
providers: {
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
},
});
expect(resolved?.apiKey).toBe("sk-test");
expect(resolved?.stt?.apiKey).toBe("sk-test"); expect(resolved?.stt?.apiKey).toBe("sk-test");
expect(resolved?.tts?.apiKey).toBe("sk-test"); expect(resolved?.tts?.apiKey).toBe("sk-test");
}); });
test("uses nested voice config before env and non-voice fallbacks", () => { test("resolves distinct endpoints for STT and TTS", () => {
const persisted = PersistedConfigSchema.parse({ const persisted = PersistedConfigSchema.parse({
providers: { providers: {
openai: { openai: {
apiKey: "fallback-config-key", stt: {
voice: { apiKey: "stt-key",
apiKey: "voice-config-key", baseUrl: " https://stt.example.com/v1 ",
baseUrl: " https://voice.example.com/v1 ", },
tts: {
apiKey: "tts-key",
baseUrl: " https://tts.example.com/v1 ",
}, },
baseUrl: "https://legacy-config.example.com/v1",
}, },
}, },
}); });
const env = {
OPENAI_API_KEY: "env-key",
OPENAI_VOICE_API_KEY: "voice-env-key",
OPENAI_VOICE_BASE_URL: "https://voice-env.example.com/v1",
OPENAI_BASE_URL: "https://env.example.com/v1",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ const resolved = resolveOpenAiSpeechConfig({
env, env: {} as NodeJS.ProcessEnv,
persisted, persisted,
providers: { providers: ALL_OPENAI,
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
},
}); });
expect(resolved?.apiKey).toBe("voice-config-key"); expect(resolved?.stt?.apiKey).toBe("stt-key");
expect(resolved?.baseUrl).toBe("https://voice.example.com/v1"); expect(resolved?.stt?.baseUrl).toBe("https://stt.example.com/v1");
expect(resolved?.stt?.apiKey).toBe("voice-config-key"); expect(resolved?.tts?.apiKey).toBe("tts-key");
expect(resolved?.stt?.baseUrl).toBe("https://voice.example.com/v1"); expect(resolved?.tts?.baseUrl).toBe("https://tts.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("voice-config-key");
expect(resolved?.tts?.baseUrl).toBe("https://voice.example.com/v1");
}); });
test("uses voice env config when nested voice config is unset", () => { test("prefers nested STT/TTS config over env and global fallbacks", () => {
const persisted = PersistedConfigSchema.parse({});
const env = {
OPENAI_API_KEY: "sk-test",
OPENAI_VOICE_API_KEY: "voice-env-key",
OPENAI_VOICE_BASE_URL: " https://voice-env.example.com/v1 ",
OPENAI_BASE_URL: "https://env.example.com/v1",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({
env,
persisted,
providers: {
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
},
});
expect(resolved?.apiKey).toBe("voice-env-key");
expect(resolved?.stt?.apiKey).toBe("voice-env-key");
expect(resolved?.tts?.apiKey).toBe("voice-env-key");
expect(resolved?.baseUrl).toBe("https://voice-env.example.com/v1");
expect(resolved?.stt?.baseUrl).toBe("https://voice-env.example.com/v1");
expect(resolved?.tts?.baseUrl).toBe("https://voice-env.example.com/v1");
});
test("falls back to non-voice OpenAI config", () => {
const persisted = PersistedConfigSchema.parse({ const persisted = PersistedConfigSchema.parse({
providers: { providers: {
openai: { openai: {
apiKey: "fallback-config-key", apiKey: "fallback-config-key",
baseUrl: " https://legacy-config.example.com/v1 ", baseUrl: "https://global-config.example.com/v1",
stt: { apiKey: "stt-config-key", baseUrl: " https://stt.example.com/v1 " },
tts: { apiKey: "tts-config-key", baseUrl: " https://tts.example.com/v1 " },
}, },
}, },
}); });
const env = { const env = {
OPENAI_API_KEY: "sk-test", OPENAI_API_KEY: "env-key",
OPENAI_BASE_URL: " https://env.example.com/v1 ", OPENAI_STT_API_KEY: "stt-env-key",
OPENAI_STT_BASE_URL: "https://stt-env.example.com/v1",
OPENAI_TTS_API_KEY: "tts-env-key",
OPENAI_TTS_BASE_URL: "https://tts-env.example.com/v1",
OPENAI_BASE_URL: "https://env.example.com/v1",
} as NodeJS.ProcessEnv; } as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
env,
persisted,
providers: {
dictationStt: { provider: "openai", explicit: true },
voiceStt: { provider: "openai", explicit: true },
voiceTts: { provider: "openai", explicit: true },
},
});
expect(resolved?.apiKey).toBe("fallback-config-key"); expect(resolved?.stt?.apiKey).toBe("stt-config-key");
expect(resolved?.baseUrl).toBe("https://legacy-config.example.com/v1"); expect(resolved?.stt?.baseUrl).toBe("https://stt.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("tts-config-key");
expect(resolved?.tts?.baseUrl).toBe("https://tts.example.com/v1");
}); });
test("falls back to global OpenAI env config when voice-specific inputs are unset", () => { test("uses STT/TTS env config when nested config is unset", () => {
const persisted = PersistedConfigSchema.parse({});
const env = {
OPENAI_API_KEY: "sk-test",
OPENAI_STT_API_KEY: "stt-env-key",
OPENAI_STT_BASE_URL: " https://stt-env.example.com/v1 ",
OPENAI_TTS_API_KEY: "tts-env-key",
OPENAI_TTS_BASE_URL: " https://tts-env.example.com/v1 ",
OPENAI_BASE_URL: "https://env.example.com/v1",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("stt-env-key");
expect(resolved?.stt?.baseUrl).toBe("https://stt-env.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("tts-env-key");
expect(resolved?.tts?.baseUrl).toBe("https://tts-env.example.com/v1");
});
test("falls back to global OpenAI config for both STT and TTS", () => {
const persisted = PersistedConfigSchema.parse({
providers: {
openai: {
apiKey: "fallback-config-key",
baseUrl: " https://global-config.example.com/v1 ",
},
},
});
const env = {} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("fallback-config-key");
expect(resolved?.stt?.baseUrl).toBe("https://global-config.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("fallback-config-key");
expect(resolved?.tts?.baseUrl).toBe("https://global-config.example.com/v1");
});
test("falls back to global OpenAI env config when feature inputs are unset", () => {
const persisted = PersistedConfigSchema.parse({}); const persisted = PersistedConfigSchema.parse({});
const env = { const env = {
OPENAI_API_KEY: "env-key", OPENAI_API_KEY: "env-key",
OPENAI_BASE_URL: " https://env.example.com/v1 ", OPENAI_BASE_URL: " https://env.example.com/v1 ",
} as NodeJS.ProcessEnv; } as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
env,
persisted, expect(resolved?.stt?.apiKey).toBe("env-key");
expect(resolved?.stt?.baseUrl).toBe("https://env.example.com/v1");
expect(resolved?.tts?.apiKey).toBe("env-key");
expect(resolved?.tts?.baseUrl).toBe("https://env.example.com/v1");
});
test("ignores empty endpoint env vars and falls back to OPENAI_API_KEY", () => {
const persisted = PersistedConfigSchema.parse({});
const env = {
OPENAI_API_KEY: "global-key",
OPENAI_STT_API_KEY: "",
OPENAI_STT_BASE_URL: " ",
OPENAI_TTS_API_KEY: "",
OPENAI_TTS_BASE_URL: "",
} as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("global-key");
expect(resolved?.tts?.apiKey).toBe("global-key");
});
test("omits TTS when only an STT key is configured", () => {
const persisted = PersistedConfigSchema.parse({
providers: { providers: {
dictationStt: { provider: "openai", explicit: true }, openai: {
voiceStt: { provider: "openai", explicit: true }, stt: { apiKey: "stt-only-key" },
voiceTts: { provider: "openai", explicit: true }, },
}, },
}); });
expect(resolved?.apiKey).toBe("env-key"); const resolved = resolveOpenAiSpeechConfig({
expect(resolved?.baseUrl).toBe("https://env.example.com/v1"); env: {} as NodeJS.ProcessEnv,
persisted,
providers: ALL_OPENAI,
});
expect(resolved?.stt?.apiKey).toBe("stt-only-key");
expect(resolved?.tts).toBeUndefined();
});
test("resolves STT even when an unused TTS env var is invalid", () => {
const persisted = PersistedConfigSchema.parse({
providers: {
openai: {
stt: { apiKey: "stt-only-key" },
},
},
});
const env = { TTS_VOICE: "not-a-real-voice", TTS_MODEL: "bogus-model" } as NodeJS.ProcessEnv;
const resolved = resolveOpenAiSpeechConfig({ env, persisted, providers: ALL_OPENAI });
expect(resolved?.stt?.apiKey).toBe("stt-only-key");
expect(resolved?.tts).toBeUndefined();
}); });
}); });
@@ -8,8 +8,6 @@ import type { TTSConfig } from "./tts.js";
export const DEFAULT_OPENAI_TTS_MODEL = "tts-1"; export const DEFAULT_OPENAI_TTS_MODEL = "tts-1";
export interface OpenAiSpeechProviderConfig { export interface OpenAiSpeechProviderConfig {
apiKey?: string;
baseUrl?: string;
stt?: Partial<STTConfig> & { apiKey?: string }; stt?: Partial<STTConfig> & { apiKey?: string };
tts?: Partial<TTSConfig> & { apiKey?: string }; tts?: Partial<TTSConfig> & { apiKey?: string };
} }
@@ -30,11 +28,23 @@ const OptionalTrimmedStringSchema = z
.optional() .optional()
.transform((value) => (value && value.length > 0 ? value : undefined)); .transform((value) => (value && value.length > 0 ? value : undefined));
const OpenAiSpeechResolutionSchema = z.object({ // Endpoint credentials only — plain trimmed strings, so this never throws on a
apiKey: OptionalTrimmedStringSchema, // malformed value. The STT/TTS option groups parse separately and only for the
baseUrl: OptionalTrimmedStringSchema, // endpoint that is actually configured, so a stale env var for an unused endpoint
// (e.g. a leftover TTS_VOICE in an STT-only setup) can't break the other one.
const OpenAiEndpointKeysSchema = z.object({
sttApiKey: OptionalTrimmedStringSchema,
sttBaseUrl: OptionalTrimmedStringSchema,
ttsApiKey: OptionalTrimmedStringSchema,
ttsBaseUrl: OptionalTrimmedStringSchema,
});
const OpenAiSttOptionsSchema = z.object({
sttConfidenceThreshold: OptionalFiniteNumberSchema, sttConfidenceThreshold: OptionalFiniteNumberSchema,
sttModel: OptionalTrimmedStringSchema, sttModel: OptionalTrimmedStringSchema,
});
const OpenAiTtsOptionsSchema = z.object({
ttsVoice: z.string().trim().toLowerCase().pipe(OpenAiTtsVoiceSchema).default("alloy"), ttsVoice: z.string().trim().toLowerCase().pipe(OpenAiTtsVoiceSchema).default("alloy"),
ttsModel: z ttsModel: z
.string() .string()
@@ -57,9 +67,15 @@ function pickIfOpenAi<T>(
function firstDefined<T>(values: Array<T | null | undefined>): T | undefined { function firstDefined<T>(values: Array<T | null | undefined>): T | undefined {
for (const value of values) { for (const value of values) {
if (value !== undefined && value !== null) { if (value === undefined || value === null) {
return value; continue;
} }
// Empty/whitespace env vars (e.g. a copied .env.example with OPENAI_STT_API_KEY=)
// must not shadow a later fallback such as OPENAI_API_KEY.
if (typeof value === "string" && value.trim().length === 0) {
continue;
}
return value;
} }
return undefined; return undefined;
} }
@@ -108,18 +124,32 @@ function buildOpenAiResolutionInput(params: {
persisted: PersistedConfig; persisted: PersistedConfig;
providers: RequestedSpeechProviders; providers: RequestedSpeechProviders;
}): Record<string, unknown> { }): Record<string, unknown> {
const { env } = params;
const openai = params.persisted.providers?.openai;
return { return {
apiKey: firstDefined<string>([ sttApiKey: firstDefined<string>([
params.persisted.providers?.openai?.voice?.apiKey, openai?.stt?.apiKey,
params.env.OPENAI_VOICE_API_KEY, env.OPENAI_STT_API_KEY,
params.persisted.providers?.openai?.apiKey, openai?.apiKey,
params.env.OPENAI_API_KEY, env.OPENAI_API_KEY,
]), ]),
baseUrl: firstDefined<string>([ sttBaseUrl: firstDefined<string>([
params.persisted.providers?.openai?.voice?.baseUrl, openai?.stt?.baseUrl,
params.env.OPENAI_VOICE_BASE_URL, env.OPENAI_STT_BASE_URL,
params.persisted.providers?.openai?.baseUrl, openai?.baseUrl,
params.env.OPENAI_BASE_URL, env.OPENAI_BASE_URL,
]),
ttsApiKey: firstDefined<string>([
openai?.tts?.apiKey,
env.OPENAI_TTS_API_KEY,
openai?.apiKey,
env.OPENAI_API_KEY,
]),
ttsBaseUrl: firstDefined<string>([
openai?.tts?.baseUrl,
env.OPENAI_TTS_BASE_URL,
openai?.baseUrl,
env.OPENAI_BASE_URL,
]), ]),
...buildOpenAiSttInput(params), ...buildOpenAiSttInput(params),
...buildOpenAiTtsInput(params), ...buildOpenAiTtsInput(params),
@@ -131,29 +161,46 @@ export function resolveOpenAiSpeechConfig(params: {
persisted: PersistedConfig; persisted: PersistedConfig;
providers: RequestedSpeechProviders; providers: RequestedSpeechProviders;
}): OpenAiSpeechProviderConfig | undefined { }): OpenAiSpeechProviderConfig | undefined {
const parsed = OpenAiSpeechResolutionSchema.parse(buildOpenAiResolutionInput(params)); const input = buildOpenAiResolutionInput(params);
const keys = OpenAiEndpointKeysSchema.parse(input);
if (!parsed.apiKey) { if (!keys.sttApiKey && !keys.ttsApiKey) {
return undefined; return undefined;
} }
return { return {
apiKey: parsed.apiKey, ...(keys.sttApiKey ? { stt: buildSttConfig(keys.sttApiKey, keys.sttBaseUrl, input) } : {}),
...(parsed.baseUrl ? { baseUrl: parsed.baseUrl } : {}), ...(keys.ttsApiKey ? { tts: buildTtsConfig(keys.ttsApiKey, keys.ttsBaseUrl, input) } : {}),
stt: { };
apiKey: parsed.apiKey, }
...(parsed.baseUrl ? { baseUrl: parsed.baseUrl } : {}),
...(parsed.sttConfidenceThreshold !== undefined function buildSttConfig(
? { confidenceThreshold: parsed.sttConfidenceThreshold } apiKey: string,
: {}), baseUrl: string | undefined,
...(parsed.sttModel ? { model: parsed.sttModel } : {}), input: Record<string, unknown>,
}, ): OpenAiSpeechProviderConfig["stt"] {
tts: { const options = OpenAiSttOptionsSchema.parse(input);
apiKey: parsed.apiKey, return {
...(parsed.baseUrl ? { baseUrl: parsed.baseUrl } : {}), apiKey,
voice: parsed.ttsVoice, ...(baseUrl ? { baseUrl } : {}),
model: parsed.ttsModel, ...(options.sttConfidenceThreshold !== undefined
responseFormat: "pcm", ? { confidenceThreshold: options.sttConfidenceThreshold }
}, : {}),
...(options.sttModel ? { model: options.sttModel } : {}),
};
}
function buildTtsConfig(
apiKey: string,
baseUrl: string | undefined,
input: Record<string, unknown>,
): OpenAiSpeechProviderConfig["tts"] {
const options = OpenAiTtsOptionsSchema.parse(input);
return {
apiKey,
...(baseUrl ? { baseUrl } : {}),
voice: options.ttsVoice,
model: options.ttsModel,
responseFormat: "pcm",
}; };
} }
@@ -15,7 +15,6 @@ describe("initializeOpenAiSpeechServices", () => {
voiceTts: { provider: "openai", explicit: true }, voiceTts: { provider: "openai", explicit: true },
}, },
openaiConfig: { openaiConfig: {
apiKey: "sk-test",
stt: { apiKey: "sk-test" }, stt: { apiKey: "sk-test" },
tts: { apiKey: "sk-test" }, tts: { apiKey: "sk-test" },
}, },
@@ -29,11 +29,11 @@ export interface SpeechServices {
function resolveOpenAiCredentials( function resolveOpenAiCredentials(
openaiConfig: OpenAiSpeechProviderConfig | undefined, openaiConfig: OpenAiSpeechProviderConfig | undefined,
): OpenAiCredentialState { ): OpenAiCredentialState {
const openaiApiKey = openaiConfig?.apiKey; const sttApiKey = openaiConfig?.stt?.apiKey;
return { return {
openaiSttApiKey: openaiConfig?.stt?.apiKey ?? openaiApiKey, openaiSttApiKey: sttApiKey,
openaiTtsApiKey: openaiConfig?.tts?.apiKey ?? openaiApiKey, openaiTtsApiKey: openaiConfig?.tts?.apiKey,
openaiDictationApiKey: openaiApiKey, openaiDictationApiKey: sttApiKey,
}; };
} }
@@ -129,7 +129,8 @@ describe("resolveSpeechConfig", () => {
dictation: "es", dictation: "es",
voice: "pt", voice: "pt",
}); });
expect(result.openai?.apiKey).toBe("persisted-key"); expect(result.openai?.stt?.apiKey).toBe("persisted-key");
expect(result.openai?.tts?.apiKey).toBe("persisted-key");
expect(result.openai?.stt?.model).toBe("gpt-4o-transcribe"); expect(result.openai?.stt?.model).toBe("gpt-4o-transcribe");
}); });
@@ -79,7 +79,7 @@ function waitForSignal<T>(
beforeAll(async () => { beforeAll(async () => {
ctx = await createDaemonTestContext({ ctx = await createDaemonTestContext({
agentClients: {}, agentClients: {},
openai: { apiKey: openaiApiKey! }, openai: { stt: { apiKey: openaiApiKey! }, tts: { apiKey: openaiApiKey! } },
speech: { speech: {
providers: { providers: {
dictationStt: { provider: "openai", explicit: true }, dictationStt: { provider: "openai", explicit: true },
@@ -161,7 +161,7 @@ let ctx: DaemonTestContext;
beforeAll(async () => { beforeAll(async () => {
ctx = await createDaemonTestContext({ ctx = await createDaemonTestContext({
agentClients: {}, agentClients: {},
openai: { apiKey: openaiApiKey! }, openai: { stt: { apiKey: openaiApiKey! }, tts: { apiKey: openaiApiKey! } },
speech: { speech: {
providers: { providers: {
dictationStt: { provider: "openai", explicit: true }, dictationStt: { provider: "openai", explicit: true },
@@ -174,6 +174,38 @@
"apiKey": { "apiKey": {
"type": "string", "type": "string",
"minLength": 1 "minLength": 1
},
"baseUrl": {
"type": "string",
"minLength": 1
},
"stt": {
"type": "object",
"properties": {
"apiKey": {
"type": "string",
"minLength": 1
},
"baseUrl": {
"type": "string",
"minLength": 1
}
},
"additionalProperties": false
},
"tts": {
"type": "object",
"properties": {
"apiKey": {
"type": "string",
"minLength": 1
},
"baseUrl": {
"type": "string",
"minLength": 1
}
},
"additionalProperties": false
} }
}, },
"additionalProperties": false "additionalProperties": false
+2
View File
@@ -204,6 +204,8 @@ In the mobile app, enter the password in the direct connection setup screen.
- `PASEO_LOG_FILE_ROTATE_COUNT`, override `log.file.rotate.maxFiles` - `PASEO_LOG_FILE_ROTATE_COUNT`, override `log.file.rotate.maxFiles`
- `PASEO_LOG`, `PASEO_LOG_FORMAT`, legacy log overrides (still supported) - `PASEO_LOG`, `PASEO_LOG_FORMAT`, legacy log overrides (still supported)
- `OPENAI_API_KEY`, override OpenAI provider key - `OPENAI_API_KEY`, override OpenAI provider key
- `OPENAI_STT_API_KEY`, `OPENAI_STT_BASE_URL`, OpenAI speech-to-text endpoint (dictation + voice mode STT)
- `OPENAI_TTS_API_KEY`, `OPENAI_TTS_BASE_URL`, OpenAI text-to-speech endpoint (voice mode TTS)
- `PASEO_VOICE_LLM_PROVIDER`, override voice LLM provider (`claude`, `codex`, `opencode`) - `PASEO_VOICE_LLM_PROVIDER`, override voice LLM provider (`claude`, `codex`, `opencode`)
- `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER`, override voice provider selection (`local` or `openai`) - `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER`, override voice provider selection (`local` or `openai`)
- `PASEO_LOCAL_MODELS_DIR`, control local model directory - `PASEO_LOCAL_MODELS_DIR`, control local model directory
+8 -2
View File
@@ -90,7 +90,11 @@ You can switch dictation, voice STT, and voice TTS to OpenAI by setting provider
}, },
"providers": { "providers": {
"openai": { "openai": {
"voice": { "stt": {
"apiKey": "...",
"baseUrl": "https://api.openai.com/v1"
},
"tts": {
"apiKey": "...", "apiKey": "...",
"baseUrl": "https://api.openai.com/v1" "baseUrl": "https://api.openai.com/v1"
} }
@@ -99,7 +103,7 @@ You can switch dictation, voice STT, and voice TTS to OpenAI by setting provider
} }
``` ```
`providers.openai.voice.apiKey` and `providers.openai.voice.baseUrl` configure only Paseo OpenAI voice traffic, without changing Codex or other OpenAI-backed tools. `providers.openai.stt` covers dictation and voice mode speech-to-text, and `providers.openai.tts` covers voice mode text-to-speech. Because they resolve independently, you can point STT and TTS at different endpoints. Each falls back to `providers.openai.apiKey`/`baseUrl`, then `OPENAI_API_KEY`/`OPENAI_BASE_URL`, when unset. These settings configure only Paseo OpenAI speech traffic, without changing Codex or other OpenAI-backed tools.
Paseo uses these paths under the configured OpenAI base URL: Paseo uses these paths under the configured OpenAI base URL:
@@ -111,6 +115,8 @@ Paseo uses these paths under the configured OpenAI base URL:
- `PASEO_VOICE_LLM_PROVIDER`, voice agent provider override - `PASEO_VOICE_LLM_PROVIDER`, voice agent provider override
- `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER`, speech provider selection (`local` or `openai`) - `PASEO_DICTATION_STT_PROVIDER`, `PASEO_VOICE_STT_PROVIDER`, `PASEO_VOICE_TTS_PROVIDER`, speech provider selection (`local` or `openai`)
- `OPENAI_STT_API_KEY`, `OPENAI_STT_BASE_URL`, OpenAI speech-to-text endpoint (dictation + voice mode STT)
- `OPENAI_TTS_API_KEY`, `OPENAI_TTS_BASE_URL`, OpenAI text-to-speech endpoint (voice mode TTS)
- `PASEO_LOCAL_MODELS_DIR`, local model storage directory - `PASEO_LOCAL_MODELS_DIR`, local model storage directory
- `PASEO_DICTATION_LOCAL_STT_MODEL`, local dictation STT model ID - `PASEO_DICTATION_LOCAL_STT_MODEL`, local dictation STT model ID
- `PASEO_VOICE_LOCAL_STT_MODEL`, `PASEO_VOICE_LOCAL_TTS_MODEL`, local voice STT/TTS model IDs - `PASEO_VOICE_LOCAL_STT_MODEL`, `PASEO_VOICE_LOCAL_TTS_MODEL`, local voice STT/TTS model IDs