Compare commits

..
Author SHA1 Message Date
Mohamed BoudraandElsa Granger 95cc77e8ba test(app): give the lightbox test theme the opacity scale window chrome now loads
Co-authored-by: Elsa Granger <zeyugao@outlook.com>
2026-11-08 03:20:08 +01:00
Mohamed BoudraandElsa Granger 4230593d1e fix(app): keep the fullscreen composer's exit button clear of the window's top-right
Co-authored-by: Elsa Granger <zeyugao@outlook.com>
2026-11-08 02:54:02 +01:00
Mohamed BoudraandElsa Granger 843ce6ebff fix(app): dodge the Duo status region like window controls
The status region beside the iPhone Duo's top band is the window's top-right
obstruction, the same one Windows and Linux window controls publish. Every row
in the band (workspace header in both widths, left sidebar's first row,
explorer tab rail) starts at the band's top, sits in the top bar's height, and
the row owning the top-right corner pads past the status region. Nothing
starts below it. The docked explorer takes the corner from the center column.

Removes the per-row clearance (useTopBandRow) and its start-below fallback.

Co-authored-by: Elsa Granger <zeyugao@outlook.com>
2026-11-08 02:46:56 +01:00
Mohamed BoudraandElsa Granger 5bf7f8b7e7 fix(app): keep the header in the top band whichever sidebar is open
With the left sidebar pinned open on the iPhone Duo inner screen, the
workspace header dropped below the status region, so it jumped 56pt
between sidebar states. Headers now always trim their trailing side to
the status region and let the title give up width instead.

Co-authored-by: Elsa Granger <zeyugao@outlook.com>
2026-11-06 15:19:10 +01:00
Mohamed BoudraandElsa Granger 142b2d2bc4 fix(app): keep explorer tab labels whole beside the Duo status region
Measured the iPhone Duo inner screen in portrait (669×951, iOS 27.1
simulator unfolded in DeviceHub): system top 82, top bar guide
{16, 26, 509, 44}, status region occlusion {525, 0, 144, 82}. The tests
now use those values instead of estimates.

On that screen the explorer tab rail, trimmed to the status region,
cut the Changes tab down to "C". The rail now starts below an occlusion
it meets, like toasts; the workspace pane keeps the band.

Co-authored-by: Elsa Granger <zeyugao@outlook.com>
2026-11-06 10:14:37 +01:00
Mohamed BoudraandElsa Granger 7bafd0fdad fix(app): use the top band beside the iPhone Duo status region
On iOS 27.1+, the window layout probe publishes the top bar guide frame
and active occlusion rects. When the top bar starts above the system top
inset, the safe-area top moves up to it, and each header row that lands
in the band trims its trailing padding to the status region. A row
whose controls cannot fit beside it starts below the region on its own.

When the guide is missing or starts at or below the system top, the
override is a no-op, which is the case on every iPhone and on the Duo
outer screen.

Co-authored-by: Elsa Granger <zeyugao@outlook.com>
2026-11-06 05:10:28 +01:00
18 changed files with 329 additions and 145 deletions
@@ -18,6 +18,8 @@ final class WindowLayoutView: ExpoView {
private var barGuide: UILayoutGuide?
private var barEdge = "none"
private var bottomBarGuide: UILayoutGuide?
private var topBarGuide: UILayoutGuide?
private var occlusions: [CGRect] = []
private var measuredBounds = CGRect.zero
required init(appContext: AppContext? = nil) {
@@ -38,6 +40,7 @@ final class WindowLayoutView: ExpoView {
super.didMoveToWindow()
previousPayload = nil
bottomBarGuide = nil
topBarGuide = nil
setNeedsUpdateConstraints()
scheduleUpdate()
}
@@ -62,6 +65,8 @@ final class WindowLayoutView: ExpoView {
if #available(iOS 27.1, *) {
// UIKit tracks these reads and invalidates constraints when regions change.
divisions = reservedRegions(kind: .division).compactMap { $0.isActive ? $0.frame : nil }
// Frames already include the system's interactive margins.
occlusions = reservedRegions(kind: .occlusion).compactMap { $0.isActive ? $0.frame : nil }
let edge: String
switch traitCollection.verticalBarEdge {
case .leading: edge = "leading"
@@ -73,6 +78,11 @@ final class WindowLayoutView: ExpoView {
// screen's corners rather than the rectangular safe area.
bottomBarGuide = layoutGuide(for: .bar(onEdge: UIRectEdge.bottom, extent: 44))
}
if topBarGuide == nil {
// Where a top bar rests. Beside a partial status region it can start above the
// rectangular safe area; content still keeps out of the occlusions.
topBarGuide = layoutGuide(for: .bar(onEdge: UIRectEdge.top, extent: 44))
}
if edge != barEdge {
barEdge = edge
barGuide = edge == "none" ? nil : layoutGuide(
@@ -106,11 +116,17 @@ final class WindowLayoutView: ExpoView {
if let bottomBar = bottomBarGuide?.layoutFrame, bottomBar.height > 0 {
bottomClearance = max(0, bounds.maxY - bottomBar.maxY)
}
var topBar: Any = NSNull()
if let frame = topBarGuide?.layoutFrame, frame.width > 0, frame.height > 0 {
topBar = rect(frame)
}
let payload: [String: Any] = [
"bounds": rect(bounds),
"verticalBarEdge": barEdge,
"verticalBar": barPayload,
"bottomClearance": bottomClearance,
"topBar": topBar,
"occlusions": occlusions.map(rect),
"divisions": divisions.map(rect)
]
guard !NSDictionary(dictionary: payload).isEqual(to: previousPayload ?? [:]) else { return }
@@ -11,6 +11,7 @@ const { theme, imageMetadata, useAttachmentPreviewUrlMock } = vi.hoisted(() => {
spacing: { 1: 4, 2: 8, 3: 12, 4: 16 },
iconSize: { sm: 14, md: 18, lg: 22 },
borderWidth: { 1: 1 },
opacity: { 50: 0.5 },
borderRadius: { full: 999, md: 6, lg: 8 },
fontSize: { xs: 11, sm: 13, base: 15 },
fontWeight: { normal: "400" },
@@ -12,10 +12,19 @@ import {
iconButtonChromeGlyphSize,
mutedIconColorMapping,
} from "@/components/ui/icon-button-chrome";
import { HEADER_INNER_HEIGHT_MOBILE, HEADER_TOP_PADDING_MOBILE } from "@/constants/layout";
import {
HEADER_INNER_HEIGHT,
HEADER_INNER_HEIGHT_MOBILE,
HEADER_TOP_PADDING_MOBILE,
} from "@/constants/layout";
import { WindowChromeSafeArea } from "@/utils/desktop-window";
import type { ShortcutKey } from "@/utils/format-shortcut";
import { useTopBand } from "@/window-layout";
const ThemedArrowLeft = withUnistyles(ArrowLeft);
const HeaderSafeArea = withUnistyles(WindowChromeSafeArea, (theme) => ({
horizontalPadding: theme.spacing[3],
}));
const NO_SHORTCUT: ShortcutKey[] = [];
@@ -39,7 +48,8 @@ interface CompactHeaderProps {
* left-aligned between them. Neither the header nor its islands draw a background, so it reads the
* same in flow and floating over content (see `FloatingHeaderLayout`, whose fade gives the icons
* contrast). Its height matches the old flat header, so anything
* offset by the header height still lines up.
* offset by the header height still lines up. In a window's top band it takes the top bar's
* place, like the full header, and keeps clear of the status region as it does of window controls.
*/
export function CompactHeader({
navigation,
@@ -51,22 +61,20 @@ export function CompactHeader({
actions,
}: CompactHeaderProps) {
const insets = useSafeAreaInsets();
const inTopBand = useTopBand() !== null;
const rootStyle = useMemo(
() => [
styles.root,
{
paddingTop: insets.top + HEADER_TOP_PADDING_MOBILE,
marginLeft: insets.left,
marginRight: insets.right,
},
],
[insets.top, insets.left, insets.right],
() => ({
paddingTop: insets.top + (inTopBand ? 0 : HEADER_TOP_PADDING_MOBILE),
marginLeft: insets.left,
marginRight: insets.right,
}),
[inTopBand, insets.top, insets.left, insets.right],
);
return (
<IconButtonChromeProvider value="header-island">
<View style={rootStyle}>
<View style={styles.row}>
<HeaderSafeArea placement="inline" style={rootStyle}>
<View style={inTopBand ? styles.topBandRow : styles.row}>
<View style={styles.island}>
{navigation === "menu" ? <SidebarMenuToggle /> : <BackButton onBack={onBack} />}
</View>
@@ -80,7 +88,7 @@ export function CompactHeader({
</View>
{actions ? <View style={styles.island}>{actions}</View> : null}
</View>
</View>
</HeaderSafeArea>
</IconButtonChromeProvider>
);
}
@@ -139,15 +147,18 @@ function BackButton({ onBack }: { onBack?: () => void }) {
}
const styles = StyleSheet.create((theme) => ({
root: {
paddingHorizontal: theme.spacing[3],
},
row: {
height: HEADER_INNER_HEIGHT_MOBILE,
flexDirection: "row",
alignItems: "center",
gap: theme.spacing[2],
},
topBandRow: {
height: HEADER_INNER_HEIGHT,
flexDirection: "row",
alignItems: "center",
gap: theme.spacing[2],
},
titleGroup: {
flex: 1,
minWidth: 0,
+6 -4
View File
@@ -29,7 +29,7 @@ import {
resolveDesktopSidebarInsetLeft,
resolveDesktopSidebarWidth,
} from "@/components/desktop-sidebar-layout";
import { useWindowLayout, resolveDesktopDivisionLayout } from "@/window-layout";
import { useTopBand, useWindowLayout, resolveDesktopDivisionLayout } from "@/window-layout";
import {
SIDEBAR_RESIZE_ACTIVATION_OFFSET,
SIDEBAR_RESIZE_FAIL_OFFSET,
@@ -748,9 +748,11 @@ function DesktopSidebar({
],
[insetsTop, insets.bottom, insets.left, overlay, divisionLayout],
);
// Below the window controls' row, or in the top band's place, the first row starts flush.
const inTopBand = useTopBand() !== null;
const sidebarHeaderGroupStyle = useMemo(
() => [styles.sidebarHeaderGroup, ownsTopLeft && styles.sidebarHeaderGroupBelowChrome],
[ownsTopLeft],
() => [styles.sidebarHeaderGroup, (ownsTopLeft || inTopBand) && styles.sidebarHeaderGroupFlush],
[ownsTopLeft, inTopBand],
);
return (
<MobilePanelOverlay
@@ -875,7 +877,7 @@ const styles = StyleSheet.create((theme) => ({
borderBottomWidth: 1,
borderBottomColor: theme.colors.border,
},
sidebarHeaderGroupBelowChrome: {
sidebarHeaderGroupFlush: {
paddingTop: 0,
},
// The nav block caps at half the sidebar; a long item list scrolls inside the
@@ -13,6 +13,7 @@ import {
type ControlSizing,
} from "@/components/ui/control-geometry";
import { StyleSheet } from "react-native-unistyles";
import { WindowChromeSafeArea } from "@/utils/desktop-window";
import { useComposerFullscreenExit } from "./fullscreen-exit";
import { resolveGlyphRailOffset } from "./state";
@@ -90,7 +91,7 @@ export function ComposerInputPresentation({
testID={fullscreen ? "composer-fullscreen" : undefined}
>
{fullscreen ? (
<View style={styles.exitRow}>
<WindowChromeSafeArea placement="inline" style={styles.exitRow}>
<Button
variant="ghost"
size="sm"
@@ -103,7 +104,7 @@ export function ComposerInputPresentation({
>
{null}
</Button>
</View>
</WindowChromeSafeArea>
) : null}
<ComposerFullscreenBody fullscreen={fullscreen}>
<View
@@ -163,7 +163,7 @@ import {
WorkspaceExplorerSidebarToggle,
WorkspaceHeaderExplorerToggle,
} from "@/screens/workspace/workspace-explorer-toggle";
import { useHasWindowChromeObstruction } from "@/utils/desktop-window";
import { useHasWindowChromeObstruction, WindowChromeRegion } from "@/utils/desktop-window";
import {
resolveWorkspaceHeaderRenderState,
type WorkspaceHeaderCheckoutState,
@@ -1024,6 +1024,23 @@ function useLastMainPane(input: {
return lastMainPaneRef;
}
/** The docked explorer beside the center column takes the top-right window-chrome corner. */
function DesktopCenterColumnChrome({
explorerOpen,
focusModeEnabled,
children,
}: {
explorerOpen: boolean;
focusModeEnabled: boolean;
children: ReactNode;
}) {
return (
<WindowChromeRegion corners={explorerOpen && !focusModeEnabled ? "top-left" : "both"}>
<View style={styles.centerColumn}>{children}</View>
</WindowChromeRegion>
);
}
function WorkspaceScreenContent({
serverId,
workspaceId,
@@ -3635,7 +3652,10 @@ function WorkspaceScreenContent({
<SheetTabs onCloseTab={handleCloseTabById} />
</View>
) : (
<View style={styles.centerColumn}>
<DesktopCenterColumnChrome
explorerOpen={isExplorerSidebarShowing}
focusModeEnabled={desktopFocusModeEnabled}
>
{rendersDesktopSplitContent ? null : (
<AdaptiveWorkspaceChrome compactHeader={compactWorkspaceHeader}>
{fullWorkspaceActions}
@@ -3674,7 +3694,7 @@ function WorkspaceScreenContent({
<View style={styles.centerContent}>
<View style={styles.content}>{desktopContent}</View>
</View>
</View>
</DesktopCenterColumnChrome>
);
const [workspaceViewportWidth, setWorkspaceViewportWidth] = useState(0);
+38 -7
View File
@@ -9,30 +9,61 @@ import {
describe("window chrome", () => {
it("has no corner obstruction outside Electron or in fullscreen", () => {
expect(resolveWindowChromeObstruction({ mode: null, isFullscreen: false })).toEqual({
expect(
resolveWindowChromeObstruction({ mode: null, isFullscreen: false, topBand: null }),
).toEqual({
topLeft: null,
topRight: null,
});
expect(resolveWindowChromeObstruction({ mode: "native-mac", isFullscreen: true })).toEqual({
expect(
resolveWindowChromeObstruction({ mode: "native-mac", isFullscreen: true, topBand: null }),
).toEqual({
topLeft: null,
topRight: null,
});
});
it("places native controls in their physical top corner", () => {
expect(resolveWindowChromeObstruction({ mode: "native-mac", isFullscreen: false })).toEqual({
expect(
resolveWindowChromeObstruction({ mode: "native-mac", isFullscreen: false, topBand: null }),
).toEqual({
topLeft: { width: 78, height: 45 },
topRight: null,
});
expect(resolveWindowChromeObstruction({ mode: "custom-windows", isFullscreen: false })).toEqual(
{ topLeft: null, topRight: { width: 138, height: 36 } },
);
expect(resolveWindowChromeObstruction({ mode: "custom-linux", isFullscreen: false })).toEqual({
expect(
resolveWindowChromeObstruction({
mode: "custom-windows",
isFullscreen: false,
topBand: null,
}),
).toEqual({ topLeft: null, topRight: { width: 138, height: 36 } });
expect(
resolveWindowChromeObstruction({ mode: "custom-linux", isFullscreen: false, topBand: null }),
).toEqual({
topLeft: null,
topRight: { width: 108, height: 36 },
});
});
// Measured: iPhone Duo inner screen, portrait. The top bar ends at x 525, where the status
// region (0..82) starts; the band starts at y 26.
it("treats the status region beside a native top band as top-right window controls", () => {
expect(
resolveWindowChromeObstruction({
mode: null,
isFullscreen: false,
topBand: { top: 26, trailingObstruction: { width: 144, height: 56 } },
}),
).toEqual({ topLeft: null, topRight: { width: 144, height: 56 } });
expect(
resolveWindowChromeObstruction({
mode: null,
isFullscreen: false,
topBand: { top: 26, trailingObstruction: null },
}),
).toEqual({ topLeft: null, topRight: null });
});
it("insets and reserves only claimed corners", () => {
const obstruction = { topLeft: { width: 80, height: 28 }, topRight: { width: 48, height: 32 } };
expect(
+9 -1
View File
@@ -12,6 +12,7 @@ import {
getDesktopWindowControlsWidth,
} from "@/desktop/window-chrome-presentation";
import { isNative } from "@/constants/platform";
import { useTopBand, type TopBand } from "@/window-layout";
export type WindowChromeCorners = "none" | "top-left" | "top-right" | "both";
type WindowChromeSafeAreaPlacement = "inline" | "below";
@@ -105,10 +106,15 @@ export function removeWindowChromeCorner(
return corner === "top-left" ? "top-right" : "top-left";
}
/** Desktop window controls, or the system region a native top band gives way to. */
export function resolveWindowChromeObstruction(input: {
mode: DesktopWindowChromeMode | null;
isFullscreen: boolean;
topBand: TopBand | null;
}): WindowChromeObstruction {
if (input.topBand?.trailingObstruction) {
return { topLeft: null, topRight: input.topBand.trailingObstruction };
}
if (!input.mode || input.isFullscreen) return EMPTY_OBSTRUCTION;
if (input.mode === "native-mac") {
return {
@@ -217,13 +223,15 @@ export function WindowChromeProvider({ children }: { children: ReactNode }) {
};
}, []);
const topBand = useTopBand();
const obstruction = useMemo(
() =>
resolveWindowChromeObstruction({
mode: isElectronReady ? getDesktopWindowChromeMode() : null,
isFullscreen: windowState.isFullscreen,
topBand,
}),
[isElectronReady, windowState.isFullscreen],
[isElectronReady, windowState.isFullscreen, topBand],
);
const desktopWindowChrome = useMemo(
() => ({
+9 -1
View File
@@ -1,9 +1,12 @@
import { createContext, useContext } from "react";
import { useWindowDimensions } from "react-native";
import type { WindowLayout } from "./model";
import type { TopBand, WindowLayout } from "./model";
export const WindowLayoutContext = createContext<WindowLayout | null>(null);
/** Published by `WindowLayoutProvider` beside the safe-area insets it derives from it. */
export const TopBandContext = createContext<TopBand | null>(null);
export function useWindowLayout(): WindowLayout | null {
const layout = useContext(WindowLayoutContext);
const { width, height } = useWindowDimensions();
@@ -13,3 +16,8 @@ export function useWindowLayout(): WindowLayout | null {
? layout
: null;
}
/** The top band beside the system's status region, or null where the window has none. */
export function useTopBand(): TopBand | null {
return useContext(TopBandContext);
}
+17 -5
View File
@@ -2,18 +2,19 @@ import { useCallback, useMemo, useState, type ReactNode } from "react";
import { StyleSheet, View, type NativeSyntheticEvent, type ViewProps } from "react-native";
import { requireNativeViewManager } from "expo-modules-core";
import { SafeAreaInsetsContext, useSafeAreaInsets } from "react-native-safe-area-context";
import { WindowLayoutContext, useWindowLayout } from "./context";
import { resolveSafeAreaInsets, type WindowLayout } from "./model";
import { TopBandContext, WindowLayoutContext, useWindowLayout } from "./context";
import { resolveSafeAreaInsets, resolveTopBand, type WindowLayout } from "./model";
export { useWindowLayout } from "./context";
export { WindowLayoutActions, WindowLayoutActionsHost } from "./actions";
export { WindowLayoutSurface, WindowLayoutDivisionGap } from "./surface";
export { useTopBand } from "./context";
export {
resolveDesktopDivisionLayout,
resolveWorkspaceActionPlacement,
resolveWorkspaceBarActions,
} from "./model";
export type { DesktopDivisionLayout, WindowLayout, WindowLayoutRect } from "./model";
export type { DesktopDivisionLayout, TopBand, WindowLayout, WindowLayoutRect } from "./model";
const GeometryProbe = requireNativeViewManager<
ViewProps & {
@@ -41,12 +42,23 @@ export function WindowLayoutProvider({ children }: { children: ReactNode }) {
);
}
/** The one bottom inset every descendant reads through `useSafeAreaInsets()`. */
/**
* The one top and bottom inset every descendant reads through `useSafeAreaInsets()`, and
* the top band that `WindowChromeProvider` reads through `useTopBand()`.
*/
function WindowSafeAreaInsets({ children }: { children: ReactNode }) {
const systemInsets = useSafeAreaInsets();
const layout = useWindowLayout();
const insets = useMemo(() => resolveSafeAreaInsets(systemInsets, layout), [systemInsets, layout]);
return <SafeAreaInsetsContext.Provider value={insets}>{children}</SafeAreaInsetsContext.Provider>;
const topBand = useMemo(
() => resolveTopBand(systemInsets.top, layout),
[systemInsets.top, layout],
);
return (
<TopBandContext.Provider value={topBand}>
<SafeAreaInsetsContext.Provider value={insets}>{children}</SafeAreaInsetsContext.Provider>
</TopBandContext.Provider>
);
}
const styles = StyleSheet.create({ root: { flex: 1 } });
+2 -1
View File
@@ -3,12 +3,13 @@ import type { ReactNode } from "react";
export { useWindowLayout } from "./context";
export { WindowLayoutActions, WindowLayoutActionsHost } from "./actions";
export { WindowLayoutSurface, WindowLayoutDivisionGap } from "./surface";
export { useTopBand } from "./context";
export {
resolveDesktopDivisionLayout,
resolveWorkspaceActionPlacement,
resolveWorkspaceBarActions,
} from "./model";
export type { DesktopDivisionLayout, WindowLayout, WindowLayoutRect } from "./model";
export type { DesktopDivisionLayout, TopBand, WindowLayout, WindowLayoutRect } from "./model";
export function WindowLayoutProvider({ children }: { children: ReactNode }) {
return children;
+126 -6
View File
@@ -1,35 +1,155 @@
import { describe, expect, it } from "vitest";
import { resolveSafeAreaInsets, type WindowLayout } from "./model";
import {
resolveSafeAreaInsets,
resolveTopBand,
type WindowLayout,
type WindowLayoutRect,
} from "./model";
const systemInsets = { top: 59, right: 0, bottom: 34, left: 0 };
function layout(bottomClearance: number | null | undefined): WindowLayout {
function layout(input: Partial<WindowLayout> = {}): WindowLayout {
return {
bounds: { x: 0, y: 0, width: 402, height: 874 },
verticalBarEdge: "none",
verticalBar: null,
bottomClearance,
divisions: [],
...input,
};
}
describe("resolveSafeAreaInsets", () => {
it("replaces only the bottom with the measured bar clearance", () => {
expect(resolveSafeAreaInsets(systemInsets, layout(24))).toEqual({
expect(resolveSafeAreaInsets(systemInsets, layout({ bottomClearance: 24 }))).toEqual({
...systemInsets,
bottom: 24,
});
});
it("keeps a measured zero instead of falling back to the system inset", () => {
expect(resolveSafeAreaInsets(systemInsets, layout(0)).bottom).toBe(0);
expect(resolveSafeAreaInsets(systemInsets, layout({ bottomClearance: 0 })).bottom).toBe(0);
});
it.each([null, undefined])("leaves system insets alone without a clearance (%s)", (value) => {
expect(resolveSafeAreaInsets(systemInsets, layout(value))).toBe(systemInsets);
expect(resolveSafeAreaInsets(systemInsets, layout({ bottomClearance: value }))).toBe(
systemInsets,
);
});
it("leaves system insets alone before the native layout arrives", () => {
expect(resolveSafeAreaInsets(systemInsets, null)).toBe(systemInsets);
});
it("starts the top where the top bar rests when it sits above the system top", () => {
const insets = resolveSafeAreaInsets(
systemInsets,
layout({ topBar: { x: 0, y: 12, width: 402, height: 44 } }),
);
expect(insets).toEqual({ ...systemInsets, top: 12 });
});
it("keeps the system top when the top bar rests at or below it", () => {
const insets = resolveSafeAreaInsets(
systemInsets,
layout({ topBar: { x: 0, y: 59, width: 402, height: 44 } }),
);
expect(insets).toBe(systemInsets);
});
});
// Measured: iPhone Duo inner screen, portrait (669×951), iOS 27.1 simulator unfolded in
// DeviceHub.
const duo = {
systemTop: 82,
topBar: { x: 16, y: 26, width: 509, height: 44 },
statusRegion: { x: 525, y: 0, width: 144, height: 82 },
};
function duoLayout(occlusions: WindowLayoutRect[]): WindowLayout {
return layout({
bounds: { x: 0, y: 0, width: 669, height: 951 },
topBar: duo.topBar,
occlusions,
});
}
describe("resolveTopBand", () => {
it("publishes the top bar's start and the status region past its trailing end", () => {
expect(resolveTopBand(duo.systemTop, duoLayout([duo.statusRegion]))).toEqual({
top: 26,
trailingObstruction: { width: 144, height: 56 },
});
});
it("reaches as low as the deepest occlusion beside the bar", () => {
const deeper = { x: 600, y: 0, width: 69, height: 96 };
expect(
resolveTopBand(duo.systemTop, duoLayout([duo.statusRegion, deeper]))?.trailingObstruction,
).toEqual({ width: 144, height: 70 });
});
it("rounds a fractional obstruction up so controls never touch the status region", () => {
const fractional = layout({
bounds: { x: 0, y: 0, width: 669, height: 951 },
topBar: { x: 16, y: 26, width: 509.25, height: 44 },
occlusions: [{ x: 525.25, y: 0, width: 143.75, height: 82.5 }],
});
expect(resolveTopBand(duo.systemTop, fractional)?.trailingObstruction).toEqual({
width: 144,
height: 57,
});
});
it("has no trailing obstruction when the bar reaches the window's trailing edge", () => {
const fullWidth = layout({
bounds: { x: 0, y: 0, width: 669, height: 951 },
topBar: { x: 0, y: 26, width: 669, height: 44 },
});
expect(resolveTopBand(duo.systemTop, fullWidth)).toEqual({
top: 26,
trailingObstruction: null,
});
});
it("ignores occlusions without a top bar guide (before iOS 27.1 or unresolved)", () => {
const withoutGuide = layout({ topBar: null, occlusions: [duo.statusRegion] });
expect(resolveTopBand(duo.systemTop, withoutGuide)).toBeNull();
expect(resolveTopBand(duo.systemTop, layout({ occlusions: [duo.statusRegion] }))).toBeNull();
});
it("ignores occlusions when the top bar guide sits below the status region", () => {
const below = layout({
topBar: { x: 16, y: 82, width: 637, height: 44 },
occlusions: [duo.statusRegion],
});
expect(resolveTopBand(duo.systemTop, below)).toBeNull();
});
// Measured: iPhone 17 Pro, iOS 27.2 simulator, portrait.
it("is null on a regular iPhone, whose top bar starts at the status bar's bottom", () => {
const iphone = layout({
topBar: { x: 18, y: 62, width: 366, height: 44 },
occlusions: [{ x: 0, y: 0, width: 402, height: 54 }],
});
expect(resolveTopBand(62, iphone)).toBeNull();
});
// Measured: iPhone Duo outer screen, iOS 27.1 simulator. The status region is a trailing
// column, which the vertical bar layout already owns.
it("is null on the Duo outer screen, whose top bar starts below a zero system top", () => {
const outer = layout({
bounds: { x: 0, y: 0, width: 466, height: 678 },
verticalBarEdge: "trailing",
topBar: { x: 7 / 3, y: 26, width: 1139 / 3, height: 44 },
occlusions: [
{ x: 1199 / 3, y: 88 / 3, width: 37, height: 37 },
{ x: 382, y: 0, width: 84, height: 174 },
],
});
expect(resolveTopBand(0, outer)).toBeNull();
});
it("is null before the native layout arrives", () => {
expect(resolveTopBand(duo.systemTop, null)).toBeNull();
});
});
+43 -5
View File
@@ -14,18 +14,56 @@ export interface WindowLayout {
* the system has no bottom bar guide. Zero is a measured value.
*/
bottomClearance?: number | null;
/** Where a top bar rests, or null until the system resolves its top bar guide. */
topBar?: WindowLayoutRect | null;
/** Active system occlusions (status region, camera, Live Activities), margins included. */
occlusions?: WindowLayoutRect[];
divisions: WindowLayoutRect[];
}
/** The window's measured bottom clearance replaces the rectangular safe-area bottom. */
export function resolveSafeAreaInsets<Insets extends { bottom: number }>(
/**
* The band above the rectangular safe area that a top bar may use because the system
* occupies only its trailing end, like window controls on a desktop title bar.
*/
export interface TopBand {
/** Where the top bar starts, in window points. */
top: number;
/**
* The system region past the bar's trailing end, measured from the band's top, or null
* when the bar reaches the window's trailing edge.
*/
trailingObstruction: { width: number; height: number } | null;
}
/** Null unless the top bar guide starts above the system's rectangular safe area. */
export function resolveTopBand(systemTop: number, layout: WindowLayout | null): TopBand | null {
const bar = layout?.topBar;
if (!layout || !bar || bar.y >= systemTop) return null;
const top = Math.max(0, bar.y);
const barEnd = bar.x + bar.width;
const width = Math.ceil(layout.bounds.width - barEnd);
if (width <= 0) return { top, trailingObstruction: null };
const beside = (layout.occlusions ?? []).filter(
(frame) => frame.x + frame.width > barEnd && frame.y < bar.y + bar.height,
);
const bottom = Math.max(bar.y + bar.height, ...beside.map((frame) => frame.y + frame.height));
return { top, trailingObstruction: { width, height: Math.ceil(bottom - top) } };
}
/**
* The window's measured bars replace the rectangular safe area: the bottom bar's clearance,
* and the top bar's start when it sits above the system top.
*/
export function resolveSafeAreaInsets<Insets extends { top: number; bottom: number }>(
systemInsets: Insets,
layout: WindowLayout | null,
): Insets {
const clearance = layout?.bottomClearance;
return typeof clearance === "number" && clearance !== systemInsets.bottom
? { ...systemInsets, bottom: clearance }
: systemInsets;
const bottom = typeof clearance === "number" ? clearance : systemInsets.bottom;
const top = resolveTopBand(systemInsets.top, layout)?.top ?? systemInsets.top;
return top === systemInsets.top && bottom === systemInsets.bottom
? systemInsets
: { ...systemInsets, top, bottom };
}
export interface DesktopDivisionLayout {
@@ -33,9 +33,9 @@ async function main(): Promise<void> {
const apiKey = requireEnv("OPENAI_API_KEY");
const transcriptionModel = process.env.STT_MODEL ?? "gpt-4o-transcribe";
// Mirrors the daemon: no built-in prompt, only an explicit override. Whisper
// treats `prompt` as preceding transcript, so a default would bias the baseline.
const prompt = process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT?.trim() || undefined;
const prompt =
process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT ??
"Transcribe only what the speaker says. Do not add words. Preserve punctuation and casing. If the audio is silence or non-speech noise, return an empty transcript.";
const openai = new OpenAI({ apiKey });
@@ -43,7 +43,7 @@ async function main(): Promise<void> {
file: await import("node:fs").then((fs) => fs.createReadStream(audioPath)),
language: "en",
model: transcriptionModel,
...(prompt ? { prompt } : {}),
prompt,
response_format: "json",
// Aim for determinism.
temperature: 0,
@@ -57,13 +57,11 @@ class FakeRealtimeSession extends EventEmitter implements StreamingTranscription
class FakeSttProvider implements SpeechToTextProvider {
public readonly id = "fake";
public lastLanguage?: string;
public lastPrompt?: string;
constructor(private readonly session: FakeRealtimeSession) {}
createSession(
params: Parameters<SpeechToTextProvider["createSession"]>[0],
): StreamingTranscriptionSession {
this.lastLanguage = params.language;
this.lastPrompt = params.prompt;
return this.session;
}
}
@@ -596,58 +594,3 @@ it("closes every dictation stream when one provider cleanup fails", async () =>
expect(second.closed).toBe(true);
expect(manager.hasDemand).toBe(false);
});
describe("DictationStreamManager (transcription prompt)", () => {
const originalPromptEnv = process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT;
afterEach(() => {
if (originalPromptEnv === undefined) {
delete process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT;
} else {
process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT = originalPromptEnv;
}
});
const startDictation = async (
options: { language?: string } = {},
): Promise<{ provider: FakeSttProvider; manager: DictationStreamManager }> => {
const session = new FakeRealtimeSession();
const provider = new FakeSttProvider(session);
const manager = new DictationStreamManager({
logger: pino({ level: "silent" }),
emit: () => {},
sessionId: "prompt",
stt: provider,
...(options.language ? { language: options.language } : {}),
});
await manager.handleStart("d1", "audio/pcm;rate=24000;bits=16");
manager.cleanupAll();
return { provider, manager };
};
it("sends no prompt by default", async () => {
delete process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT;
const { provider } = await startDictation();
expect(provider.lastPrompt).toBeUndefined();
});
it("sends no prompt for non-English dictation by default", async () => {
delete process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT;
const { provider } = await startDictation({ language: "zh" });
expect(provider.lastLanguage).toBe("zh");
expect(provider.lastPrompt).toBeUndefined();
});
it("passes an explicit prompt override through", async () => {
process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT =
" Vocabulary: Paseo, Dokploy, Kubernetes. ";
const { provider } = await startDictation();
expect(provider.lastPrompt).toBe("Vocabulary: Paseo, Dokploy, Kubernetes.");
});
it.each(["", " "])("treats a blank override (%j) as no prompt", async (value) => {
process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT = value;
const { provider } = await startDictation();
expect(provider.lastPrompt).toBeUndefined();
});
});
@@ -38,12 +38,6 @@ function parseNonNegativeNumber(value: string | undefined): number | null {
return parsed;
}
/** Unset, empty, and whitespace-only overrides all mean "send no prompt". */
function readTranscriptionPromptOverride(env: NodeJS.ProcessEnv): string | undefined {
const value = env.PASEO_DICTATION_TRANSCRIPTION_PROMPT?.trim();
return value ? value : undefined;
}
function convertPCMToWavBuffer(
pcmBuffer: Buffer,
sampleRate: number,
@@ -191,16 +185,16 @@ export class DictationStreamManager {
return;
}
// Whisper uses the prompt as transcript context, not silence instructions.
// Keep context opt-in so it does not bias speech or introduce unspoken text.
const transcriptionPrompt = readTranscriptionPromptOverride(process.env);
const transcriptionPrompt =
process.env.PASEO_DICTATION_TRANSCRIPTION_PROMPT ??
"Transcribe only what the speaker says. Do not add words. Preserve punctuation and casing. If the audio is silence or non-speech noise, return an empty transcript.";
let stt: ReturnType<SpeechToTextProvider["createSession"]>;
try {
stt = sttProvider.createSession({
logger: this.logger.child({ dictationId }),
language: this.language,
...(transcriptionPrompt ? { prompt: transcriptionPrompt } : {}),
prompt: transcriptionPrompt,
});
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
-11
View File
@@ -142,19 +142,8 @@ Keys and base URLs follow the fallback rule above.
- `PASEO_VOICE_LOCAL_STT_MODEL`, `PASEO_VOICE_LOCAL_TTS_MODEL`, local voice STT/TTS model IDs
- `PASEO_DICTATION_LANGUAGE`, dictation STT language
- `PASEO_VOICE_LANGUAGE`, voice mode STT language; falls back to `PASEO_DICTATION_LANGUAGE` when unset
- `PASEO_DICTATION_TRANSCRIPTION_PROMPT`, optional OpenAI dictation transcription prompt (including compatible servers). Unset by default; empty or whitespace-only is treated as unset
- `PASEO_VOICE_LOCAL_TTS_SPEAKER_ID`, `PASEO_VOICE_LOCAL_TTS_SPEED`, optional local voice TTS tuning
## Transcription Prompt
With the OpenAI provider, including compatible servers, Paseo sends no transcription prompt by default. Whisper-family models use `prompt` as transcript context, so instruction text can distort the transcript or bias its language. Omitting it does not prevent every hallucination on silence or noise.
For OpenAI dictation, set `PASEO_DICTATION_TRANSCRIPTION_PROMPT` to send a prompt explicitly, for example to bias toward project vocabulary. An empty or whitespace-only value means no prompt. Local providers ignore this setting, and voice mode never sends a prompt.
```bash
PASEO_DICTATION_TRANSCRIPTION_PROMPT="Vocabulary: Paseo, TypeScript."
```
## Operational Notes
Voice mode can launch and control agents. Treat voice prompts with the same care as direct agent instructions, especially when specifying working directories or destructive operations.
-11
View File
@@ -120,19 +120,8 @@ Paseo 在配置的 OpenAI base URL 下使用以下路径:
- `PASEO_VOICE_LOCAL_STT_MODEL`、`PASEO_VOICE_LOCAL_TTS_MODEL`:本地语音 STT/TTS 模型 ID
- `PASEO_DICTATION_LANGUAGE`:口述 STT 语言(仅限 OpenAI STT;本地 Parakeet 会忽略)
- `PASEO_VOICE_LANGUAGE`:语音模式 STT 语言;未设置时退回到 `PASEO_DICTATION_LANGUAGE`(仅限 OpenAI STT;本地 Parakeet 会忽略)
- `PASEO_DICTATION_TRANSCRIPTION_PROMPT`:可选的 OpenAI 口述转录提示词(包括兼容服务器)。默认不设置;空值或仅含空白字符的值视为未设置
- `PASEO_VOICE_LOCAL_TTS_SPEAKER_ID`、`PASEO_VOICE_LOCAL_TTS_SPEED`:可选的本地语音 TTS 调优
## 转录提示词 {#transcription-prompt}
使用 OpenAI Provider(包括兼容服务器)进行口述时,Paseo 默认不发送转录提示词。Whisper 系列模型把 `prompt` 当作转录上下文,因此指令文本可能使转录失真或影响输出语言。不发送提示词也无法消除静音或噪声中的所有幻觉。
对于 OpenAI 口述,可以设置 `PASEO_DICTATION_TRANSCRIPTION_PROMPT` 来显式发送提示词,例如提示项目术语。空值或仅含空白字符的值表示不发送提示词。本地 Provider 会忽略此设置,语音模式也不会发送提示词。
```bash
PASEO_DICTATION_TRANSCRIPTION_PROMPT="Vocabulary: Paseo, TypeScript."
```
## 使用须知 {#operational-notes}
语音模式可以启动和控制 Agent。请像对待直接的 Agent 指令一样谨慎对待语音提示词,尤其是在指定工作目录或执行破坏性操作时。