fix(app): restore Android tablet model picker interactions (#4845)

Restore provider/model selection in Android tablet modal windows by giving both native modal hosts their own gesture-handler root. Preserve phone sheet and web portal behavior.

Include native regression scripts, test dependency resolution, and verified CI test corrections. Generated media and logs are excluded. Refs #4475 and #4377; consolidates #4721 and #4771.

Co-authored-by: Mike P. Kuhl <mkuhl@softmachine.at>
Co-authored-by: Cameron Chin <cjcrjc@users.noreply.github.com>
This commit is contained in:
Mohamed Boudra
2026-09-14 19:10:32 +02:00
committed by GitHub
co-authored by Mike P. Kuhl Cameron Chin
parent d63132a37d
commit 799c91b39d
10 changed files with 149 additions and 16 deletions
+4 -1
View File
@@ -58,7 +58,10 @@ Android touches sailed straight through to the chat scroll view behind it.
Two escape hatches in the codebase:
- **`Modal`** (combobox, dropdown menu and tooltip on native) — opens a new Android window, so
hit-testing starts fresh in that window. Side effect: a Modal opening on
hit-testing starts fresh in that window. If its children use gesture-handler,
the Modal owner must wrap its content in a full-height `GestureHandlerRootView`:
the app's root does not span Android windows. React context alone does not
establish a native gesture root. Side effect: a Modal opening on
Android can detach the IME from an underlying TextInput. Fine for combobox
(it has its own input) and tooltip (no input). **Not** fine for autocomplete
(the composer's input must stay focused so the user keeps typing).
+47 -2
View File
@@ -1,4 +1,6 @@
# Native sheet gestures
# Native modal and sheet gestures
## Compact sheet gestures
Use an Android emulator at 1080×1920, density 420, with the checkout's native app bundle loaded.
Connect the app to an isolated daemon, enable plugins, and install `plugin-examples/modal-ui` there.
@@ -22,7 +24,7 @@ its parent without selecting a model or submitting a prompt. It does not assert
These scripts live outside the default mobile suite because they require the installed example and
an explicit connected host. See [mobile testing](../../../../../docs/mobile-testing.md) for device setup.
## Recorded verification
### Recorded verification
Android API 35, 1080×1920 at density 420, development binary 0.7.2 with this checkout's JavaScript.
Plugin tests used an isolated daemon. Recordings show the same body drag
@@ -45,3 +47,46 @@ Root typecheck, lint and format: passed
The model form's catalog stayed on “Loading…” in the debug app even though the isolated daemon's
provider API returned models. Its sheet dismissal/reopen was exercised; populated model selection
was not. iOS and Electron were not exercised.
## Tablet model selection
`model-tablet.android.ad` exercises the New workspace popover. `model-tablet-agent.android.ad`
exercises the modal used by an existing agent's narrow composer on a wide device. Both drill into
a populated provider, select a real model, assert the updated trigger, reopen, and dismiss with
Android Back. New workspace also pans beyond the short list viewport and flings back without
selecting a row or dismissing the popover. They do not submit a prompt.
Use an English-language Android tablet at 1600×2560, density 320 (800dp), with this checkout's
JavaScript and a connected isolated daemon. On a Pixel Tablet AVD, rotate to portrait explicitly;
its natural landscape orientation letterboxes this portrait-locked app into the compact layout:
```sh
adb -s "$ANDROID_SERIAL" shell settings put system accelerometer_rotation 0
adb -s "$ANDROID_SERIAL" shell settings put system user_rotation 1
adb -s "$ANDROID_SERIAL" shell wm density 320
```
Prepare an idle agent through the daemon's public `createAgent` API with a real provider/model and
no initial prompt. Set `MODEL_QA_AGENT_ID` and `MODEL_QA_AGENT_TITLE` to that agent. In New workspace,
remember a model first (the phone layout works on the broken baseline), then close the picker.
Choose a model visible on the provider's first page and set its catalog ID and displayed label.
Set `MODEL_QA_LAST_MODEL_ID` to its initially offscreen last model; use a short overflowing catalog
(six rows during this verification) for the coordinate-based pan/fling regression:
```sh
agent-device test packages/app/e2e/mobile/modal-sheet/model-tablet.android.ad \
packages/app/e2e/mobile/modal-sheet/model-tablet-agent.android.ad \
--serial "$ANDROID_SERIAL" --metro-port "$MODEL_QA_METRO_PORT" \
--env FORM_URL="paseo://new?serverId=${SHEET_QA_SERVER_ID}" \
--env AGENT_URL="paseo://h/${SHEET_QA_SERVER_ID}/agent/${MODEL_QA_AGENT_ID}" \
--env AGENT_TITLE="$MODEL_QA_AGENT_TITLE" \
--env PROVIDER_ID=codex --env MODEL_ID="$MODEL_QA_MODEL_ID" \
--env MODEL_LABEL="$MODEL_QA_MODEL_LABEL" --env LAST_MODEL_ID="$MODEL_QA_LAST_MODEL_ID" \
--artifacts-dir /tmp/paseo-model-tablet-qa
```
The existing-agent journey needs the sidebar pinned so its composer stays narrow. Start each run with no modal open;
a failed provider tap leaves its modal open, so dismiss it before the next run. These host-dependent
journeys remain outside the default mobile suite, alongside the compact sheet journeys above.
Keep generated screenshots, recordings, and run logs outside the repository.
@@ -0,0 +1,23 @@
# Idle existing agent in a narrow composer on a wide Android device (AdaptiveModalSheet).
# AGENT_URL=paseo://h/<serverId>/agent/<agentId>; AGENT_TITLE identifies the loaded route.
context platform=android timeout=60000 retries=0
env APP_ID=sh.paseo.debug
env PROVIDER_ID=codex
open "${APP_ID}" "${AGENT_URL}"
wait "label=\"${AGENT_TITLE}\"" 20000
wait "id=\"combined-model-selector\"" 10000
press "id=\"combined-model-selector\""
wait "id=\"model-provider-${PROVIDER_ID}\"" 10000
press "id=\"model-provider-${PROVIDER_ID}\""
wait "id=\"compact-model-list\"" 3000
press "id=\"model-row-${PROVIDER_ID}-${MODEL_ID}\""
wait "id=\"combined-model-selector\" label=\"Select model (${MODEL_LABEL})\"" 3000
press "id=\"combined-model-selector\""
wait "id=\"model-provider-${PROVIDER_ID}\"" 10000
press "id=\"model-provider-${PROVIDER_ID}\""
wait "id=\"model-row-${PROVIDER_ID}-${MODEL_ID}\"" 3000
back
wait "id=\"combined-model-selector\" label=\"Select model (${MODEL_LABEL})\"" 3000
close
@@ -0,0 +1,30 @@
# Connected host with a selected model, 1600x2560 at density 320. See README.md.
# FORM_URL=paseo://new?serverId=<serverId>; MODEL_ID and MODEL_LABEL name a real catalog model.
context platform=android timeout=60000 retries=0
env APP_ID=sh.paseo.debug
env PROVIDER_ID=codex
open "${APP_ID}" "${FORM_URL}"
wait "label=\"Workspace project\"" 20000
press "id=\"combined-model-selector\""
wait "id=\"combobox-desktop-container\"" 10000
# A remembered model opens its provider's list. Return to the provider page first.
press "label=\"Back\""
wait "id=\"model-provider-${PROVIDER_ID}\"" 10000
press "id=\"model-provider-${PROVIDER_ID}\""
wait "id=\"compact-model-list\"" 3000
press "id=\"model-row-${PROVIDER_ID}-${MODEL_ID}\""
wait "id=\"combined-model-selector\" label=\"Select model (${MODEL_LABEL})\"" 3000
# A drag that leaves the short viewport must scroll without selecting or dismissing.
press "id=\"combined-model-selector\""
wait "id=\"model-row-${PROVIDER_ID}-${MODEL_ID}\"" 3000
gesture pan 1000 1850 0 -700 600
wait "id=\"compact-model-list\"" 3000
wait "id=\"model-row-${PROVIDER_ID}-${LAST_MODEL_ID}\"" 3000
gesture fling down 1000 1750 500
wait "id=\"model-row-${PROVIDER_ID}-${MODEL_ID}\"" 3000
# Android Back dismisses the whole popover without changing the remembered selection.
back
wait "id=\"combined-model-selector\" label=\"Select model (${MODEL_LABEL})\"" 3000
close
@@ -293,11 +293,14 @@ export async function holdRewindCompletion(
}
if (
!released &&
sessionMessage?.type === "agent.rewind.response" &&
(sessionMessage?.type === "agent.rewind.response" ||
sessionMessage?.type === "agent.timeline.replacement") &&
payload?.agentId === agentId
) {
// The replacement also signals completion and removes the rewound row/menu.
// Hold it with the response so the pending-state assertion observes a pending rewind.
delayedForwards.push(() => ws.send(message));
resolveDelayedResponse?.();
if (sessionMessage.type === "agent.rewind.response") resolveDelayedResponse?.();
return;
}
ws.send(message);
+2 -1
View File
@@ -165,7 +165,8 @@ export async function createCreationScenario(page: Page) {
await pressSubmitBeforeTheNextRender(page, button);
},
async expectPromptVisible(prompt?: string) {
const rows = page.getByTestId("user-message");
// Inactive agent tabs retain their timeline DOM beside the visible draft.
const rows = page.getByTestId("user-message").filter({ visible: true });
await expect(prompt ? rows.filter({ hasText: prompt }) : rows.first()).toBeVisible();
},
async expectWorkspaceReadyBeforeAgentCompletion() {
@@ -19,6 +19,7 @@ import {
type BottomSheetBackgroundProps,
} from "@gorhom/bottom-sheet";
import Animated, { useAnimatedStyle } from "react-native-reanimated";
import { GestureHandlerRootView } from "react-native-gesture-handler";
import { ArrowLeft, Search, X } from "lucide-react-native";
import {
IsolatedBottomSheetModal,
@@ -71,6 +72,9 @@ const SCROLL_CONTENT_GROW = { flexGrow: 1 };
const ABSOLUTE_FILL_STYLE = { ...StyleSheet.absoluteFillObject };
const styles = StyleSheet.create((theme) => ({
nativeModalRoot: {
flex: 1,
},
desktopOverlay: {
...StyleSheet.absoluteFillObject,
backgroundColor: "rgba(0,0,0,0.55)",
@@ -724,7 +728,10 @@ export function AdaptiveModalSheet({
onDismiss={notifyNativeModalDismiss}
hardwareAccelerated
>
{desktopContent}
{/* Android Modal opens a separate window outside the app's gesture root. */}
<GestureHandlerRootView style={styles.nativeModalRoot}>
{desktopContent}
</GestureHandlerRootView>
</Modal>
);
}
+3 -1
View File
@@ -34,6 +34,7 @@ import {
BottomSheetBackgroundProps,
} from "@gorhom/bottom-sheet";
import Animated, { FadeIn, FadeOut } from "react-native-reanimated";
import { GestureHandlerRootView } from "react-native-gesture-handler";
import { Check, File, Folder, Search } from "lucide-react-native";
import {
flip,
@@ -1266,7 +1267,8 @@ function DesktopComboboxBody(props: DesktopBodyProps): ReactElement {
visible={props.isOpen}
onRequestClose={props.handleClose}
>
{overlay}
{/* Android Modal opens a separate window outside the app's gesture root. */}
<GestureHandlerRootView style={styles.desktopOverlay}>{overlay}</GestureHandlerRootView>
</Modal>
);
}
+23 -5
View File
@@ -56,17 +56,27 @@ export default defineConfig({
server: {
deps: {
fallbackCJS: true,
inline: ["zustand", "@tanstack/react-query", "react-native-web"],
inline: [
"zustand",
"@tanstack/react-query",
"react-native-web",
"react-native-gesture-handler",
],
},
},
},
// Reanimated ships one file per platform and picks between them by extension
// (`findHostInstance.web.js`). Vite's dependency optimizer does not apply `resolve.extensions`,
// Reanimated and gesture-handler pick platform files by extension
// (e.g. `GestureHandlerRootView.web.js`). Vite's optimizer does not apply `resolve.extensions`,
// so it scans the native files and dies on imports react-native-web has no answer for.
// Unbundled, the same imports go through the resolver below and land on the web files.
optimizeDeps: {
include: ["react/jsx-runtime"],
exclude: ["react-native-reanimated"],
// Bundle the CJS dependencies of the excluded gesture-handler package for the browser.
include: [
"react/jsx-runtime",
"react-native-gesture-handler > hoist-non-react-statics",
"react-native-gesture-handler > invariant",
],
exclude: ["react-native-reanimated", "react-native-gesture-handler"],
},
// The globals a React Native bundler defines, which esbuild is no longer there to supply for
// the package excluded above.
@@ -101,6 +111,14 @@ export default defineConfig({
replacement: path.resolve(__dirname, "../relay/src/index.ts"),
},
{ find: "@", replacement: path.resolve(__dirname, "src") },
// The CJS entry bypasses Vite's React Native alias and web-extension resolution.
{
find: /^react-native-gesture-handler$/,
replacement: path.resolve(
rootNodeModules,
"react-native-gesture-handler/lib/module/index.js",
),
},
// Must precede the `react-native` alias: a string `find` matches by prefix, so this subpath
// would otherwise resolve inside a react-native-web *file* and break the dependency scan.
// Reanimated only imports it on the native path, which no test takes.
@@ -76,11 +76,12 @@ async function fixture() {
.poll(
async () => {
status = await ok(["status", "--home", home], overrides);
return status.connectedDaemon;
return status;
},
{ timeout: 30_000 },
)
.toBe("reachable");
// A connected socket can outlive a status RPC timeout without reporting the worker.
.toMatchObject({ connectedDaemon: "reachable", workerPid: expect.any(Number) });
return status;
}
async function configure(home: string, listen: string) {
@@ -132,7 +133,7 @@ test("managed two-home restart retains its supervisor and never routes ordinary
expect(launchA.listen).toBe(`127.0.0.1:${portA}`);
expect(launchB.listen).toBe(`127.0.0.1:${portB}`);
const beforeA = await f.liveStatus(a);
const beforeB = await f.ok(["--home", b, "daemon", "status"], poisoned);
const beforeB = await f.liveStatus(b, poisoned);
if (process.platform !== "win32") {
for (const home of [a, b, path.join(f.root, ".paseo")])
expect((await stat(home)).mode & 0o777).toBe(0o700);