puter/paseo
1
0
Fork 0
mirror of https://github.com/getpaseo/paseo.git synced 2026-11-05 16:04:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
78669b1f5b2fadbeb2fbe2a0114b07946a1cad1c
paseo/docs/mobile-testing.md
T
Mohamed Boudra 4f7813d3aa Keep the tapped line in place when chat disclosures expand or collapse, with the native timeline on LegendList (#8717) 
* Keep a disclosure's trigger in place on web

Expanding or collapsing a turn summary, tool-call group, tool call details
or todo badge now holds the row that owns the trigger. Rows inside the
timeline get a disclosure context; each toggle runs its state change through
`useDisclosureHold()`, and the viewport's new `holdRow(rowId, apply)` replaces
`holdReadingPosition()`.

On web the hold pins the reading anchor to that row, stops following output,
and resumes following when the change leaves the list at its end. Native keeps
its previous behaviour (detach, then apply) until the anchored toggle lands.

* Render the native chat timeline with LegendList in forward order

The native timeline was an inverted FlatList. Disclosure toggles moved their
trigger, and a cold open settled visibly. It is now a forward-order LegendList
that stays virtualized, and one owner moves the scroll offset at a time:

- Before placement: LegendList's initial scroll, behind its hidden reveal.
- While following: the end pin (native-end-pin.tsx). This is a zero-height first
  child at the bottom of the content, which native maintainVisibleContentPosition
  keeps in place. A content shrink or growth never scrolls from JS; only a
  viewport resize or an explicit jump does.
- During a disclosure hold and while detached: LegendList MVCP with
  shouldRestorePosition (native-disclosure-hold.ts), so a toggle keeps its
  trigger at the same y and paints one layout change.
- Every newly mounted row starts at a per-kind estimate (new-row-estimates.ts).

PaseoScrollView defers React Native's content clamp while MVCP is set. A shrink
below an unmoved anchor no longer draws a frame past the end (RN #58578,
PR #58695). VisibleContentPositionClampTest covers this.

LegendList 3.7.1 patch, one "Paseo patch" comment per hunk:
- The content height comes from React state on the new architecture.
- Rows get a per-row size estimate.
- The MVCP anchor lock runs on native too. Upstream gated it to web in d24a9d93
  without a native reason. Without the lock, the size pass for prepended history
  re-anchors on a prepended row and moves the rows on screen. Native ends the
  lock when its row leaves the view, not on the 300 ms clock.
- Header and footer reveal together with the rows.
- Only lists that keep their end preserve the initial end.
- A remeasure within one pixel plus the 1/8 dp floor of the known size is
  noise. Upstream compares the raw size with the floored one, so the live
  turn's assistant row (1380 -> 1381 px) read as a resize and moved every row
  a frame after the reveal.
- The Android silent initial scroll retry sends its 1 dp nudge and the return
  to the target in one task. Upstream returns on the next frame, but its 16 ms
  check can finish the scroll first and cancel that frame, so the list revealed
  3 px short of its end and moved into place a frame later.
- The top-padding shrink hold applies on iOS only.

Android, emulator. Evidence is in .artifacts/legend-impl.
- J1, J2 at the end, J3, J4, J5, J6, J7: pass.
- J2 while streaming: not reachable on the Ten second stream fixture; its turns
  never fold.
- J8: no blank frames over 10 fresh flings, at host load 32-37.

J4: still. Before these two hunks, 6 of 7 cold opens moved every row one frame
after the reveal. After them, 9 valid cold opens show no shift. 5 of those
were traced through the nudge path. Runs that hit the dev client's startup
SIGSEGV in MountingCoordinator::pullTransaction were left out. That crash is
in the emulator's dropbox from before this work.

* Revise a history row when its turn summary line changes

LegendList renders a row again only when its item changes. The forward-order
timeline in d1f2ae352 folds a finished turn into its host user message, but the
host's item does not change when the fold appears, so a turn that folded live
lost its summary line until something else revised the row.

collapseTurns now reports the hosts whose summary line appeared, changed or went
away since the previous fold (updatedHostIds), and the history row revision
treats them as changed content.

* Stop following the end while a user scroll owns the viewport

The bottom anchor controller kept sticky-bottom for the whole of a drag and its momentum and only
decided at the end. While it is sticky the native list mounts the end pin, so a drag started during
a stream had every new line move the rows under the finger: on the iOS simulator the offset jumped
by each line's height (+22, +35, +22 pt) mid-drag, and Android does the same with the same code.

beginUserScroll now detaches. endUserScroll resumes following when the gesture ends at the end
(scrolling there without animation) and otherwise stays detached and drops any pending anchor, as
before. The re-stick is not animated: on Android a line that mounts during a smooth scroll cancels
it, and the list then followed 40 pt short of the end until the controller detached it.

Measured after the change, frame by frame:
- iOS and Android, drag up mid-stream: the content tracks the finger and momentum with no line
  steps; the stream grows below; tapping the pill re-sticks and follows again.
- A short drag at the end, mid-stream: following resumes; the release lands on the end in one
  frame (the drag distance, here 8 pt on iOS and 21 dp on Android).

A line already mounting when the drag begins can still move the rows once in its first frame,
because the detach reaches native one commit after the touch.

* Correct the iOS scroll view under maintainVisibleContentPosition

Phase 2 of the LegendList timeline. The native timeline relies on maintainVisibleContentPosition
for the end pin and the disclosure hold, and React Native 0.86 on iOS gets two cases wrong.
modules/paseo-scroll now registers PaseoScrollViewComponentView, a subclass of
RCTScrollViewComponentView, for the iOS scroll view, like PaseoScrollView on Android.

- A mount that removes its own anchor: detaching removes the end pin in the same mount that opens
  the tapped disclosure, and the offset jumped by the removed view's frame. The subclass keeps the
  offset when the anchor's tag is no longer registered, the check React Native 0.88 adds in
  https://github.com/react/react-native/pull/57294. That PR's extra superview check is not copied
  because it skips anchors detached by removeClippedSubviews
  (https://github.com/react/react-native/issues/58910).
- A shrink at the end: UIKit clamps the offset to the new end during the mount, then React Native
  adds the anchor's movement, so a turn folding while following landed above the end
  (https://github.com/react/react-native/issues/58578). The subclass restores the pre-mount offset
  for the adjustment and clamps after it, as
  https://github.com/react/react-native/pull/58695 does.

The dev client was rebuilt with the module. Verified on the iPhone 16 Pro simulator, frame by frame:
the summary far up and the latest turn's summary while following each keep the trigger at 0 px
with one layout change per toggle, the todo card the same, the cold open lands still, older
history pages keep the visible rows in place, and the keyboard, send-time shrink and streaming
stay pinned to the end.

Freshly mounted markdown on iOS paints one frame after its layout, because enriched-markdown
renders asynchronously (https://github.com/software-mansion/enriched-markdown/issues/1131). A
synchronous first render removed that frame but raised fling frame time at p90 and p99, so it is
not part of this change.

docs/agent-stream-disclosure.md now describes the clamp correction on both platforms.

* Keep the rows under the finger at a drag's start and its release

Two one-frame row movements remained around a user scroll while the timeline streams.

A drag's first frames. The end pin stays mounted until JS stops following, one round trip after
the drag begins, so a line mounting in those frames scrolled the rows under the finger by its
height. Both scroll view subclasses now skip the content-position correction while the user drags
and the anchor is the end pin, which they recognise by its native ID ("paseo-end-pin"). Android
overrides scrollToPreservingMomentum, the position keeper's only scroll call, from the begin-drag
event until the finger lifts; iOS skips the adjustment while the scroll view is dragging. The skip
is limited to the pin: LegendList's own anchoring for the detached list also uses the native
correction, and a history page landing mid-drag needs it. It starts at the drag rather than at
touch-down: a tap or long press while following would otherwise leave the list stranded short of
its end, still following, once the finger lifts.

The release. A gesture ending within 32 pt of the end re-stuck with an instant scroll, a snap of up
to 56 px. Following now resumes only when the finger lifts at or past the end, or momentum stops
there, within 1 px, the distance web resumes at (strategy-web.tsx). Anything else stays detached with the
pill. The check runs when the gesture is decided, so output that arrived below meanwhile counts. A
release past the end counts as at the end: on iOS it bounces back, and a line landing during the
bounce scrolls in as following would. endUserScroll takes isAtEnd, like releaseDisclosure.

Tests: VisibleContentPositionClampTest gains growth at the end during a drag (red before the
change: the offset moved by the 120 px growth, 352 -> 472) and growth without a touch (the pin
still follows); the module suite passes on emulator-5558, 11 tests.
bottom-anchor-controller.test.ts: a release near but short of the end stays detached.

Device, frame scans while streaming (dev clients rebuilt):
- Android, 8 slow drags with a pill tap between: no frame moves against the finger once a drag
  starts; the stream follows between drags. Three short releases 50 px short of the end stay put
  until the pill; three drags back past the end resume following.
- iOS simulator, 6 drags: no line step during any drag or its momentum. Two short releases stay
  detached with no snap; three drags past the end resume following within 0.5 s of the bounce.

* Document the LegendList timeline and remove what the inverted list left

The native section of docs/agent-stream-disclosure.md described the FlatList hold (anchor
neighbour, native child index, frame spacing, render window). It now covers why the timeline is
forward order, who owns the offset in each state, the hold and its sync layout trigger, user
scroll and release, draw distance numbers, the LegendList patch and the scroll view corrections.

Draw distance stays at 2000. The spike measured it on a chat with a 15,000 px turn expanded: 500
blanked the viewport 1.0 to 1.5 s per fling and 2000 not at all. A re-measure on the collapsed chat
(500, 1000, 1500, 2000, interleaved) blanked at no value and its jank followed host load, which
stayed between 19 and 40, so it could not separate 1000 or 1500 from 2000.

The patch file now opens with a header listing its eight hunks by purpose and linking
LegendApp/legend-list#593, which reports each one upstream. The markdown doc records the iOS
first-paint delay (software-mansion/enriched-markdown#1131).

Removed with the inverted list: PaseoScrollView's child-reveal override (a no-op without the
inverting transform) and ScrollSelectionTest, the invertedWrapper style, and the inverted-list
passages in android.md, native-text-selection.md, mobile-testing.md, floating-panels.md and
agent-stream-performance.md. model.test, layout.test and footer-spacing.test still asserted the
inverted native order and failed; they now assert forward order, and footer-spacing uses the
real strategies instead of an inverted stub.

* fix(app): show a reply's footer when the next prompt arrives on native

LegendList re-renders a mounted row only when its item or extraData changes.
When a new prompt arrives, the previous reply's footer moves from the live
auxiliary slot into that reply's layout item (completedFooter). The reply's
StreamItem keeps its identity, and renderItem is a stable event, so the row never
rendered its new layout and the footer (copy, timing, fork) disappeared until the
row remounted. FlatList had the same gap.

The history renderer is now LegendList's extraData. Its identity changes with the
history layout map, and StreamRow's memo limits the pass to rows whose layout
item changed.

* test(app): run the web disclosure hold cases in a real browser

The three hold cases simulated layout in jsdom: hand-written row heights,
scroll clamping and resize callbacks. They could pass while Chromium paints
the trigger somewhere else. They now run in Vitest browser mode with real
layout, ResizeObserver and scroll clamping. The two hold cases fail when the
hold is bypassed.

The browser project pre-bundles @tanstack/react-virtual, so its first import no
longer reloads the test mid-run. The lucide stub gains ArrowDown, which
pane-find imports.

* fix(app): let a drag end a disclosure hold on native

A drag detached the bottom controller but kept the disclosure hold. Every item
size change restarts the hold's quiet frames, including a streaming row's, so
the hold can outlive the toggle. While it lasts, shouldRestorePosition accepts
only the held row. Once the drag scrolls that row away, a history page arriving
mid-gesture gets no anchor, and the rows the reader was on move by the page's
height.

The drag now cancels the hold without releasing it. The controller keeps no hold
state of its own beyond detached, and the gesture's end decides whether following
resumes.

* perf(app): keep the scroll-to-bottom button's state out of the timeline

Opening a tool call or turn at the end of the chat now holds its trigger in
place, so the timeline detaches and the scroll-to-bottom button appears. That
visibility was useState in AgentStreamView, so showing the button re-rendered
the whole timeline view for a change only the button reads. The viewport now
writes near-bottom into a per-agent store that only the button subscribes to.

* fix(app): show the scroll-to-bottom button on native whenever following has stopped

A gesture now resumes following only when it ends within 1 px of the end, but
the button still hid within 32 px. A release 2-32 px short of the end left the
list detached with no button to return. While detached the button now hides
only at the end; while following it keeps the 32 px tolerance so content
growth that the offset trails for a frame does not flash it. The button also
updates when following starts or stops without a scroll.

* fix(app): stop the timeline jumping when the keyboard opens or closes on native

The keyboard's settled height was committed to the list after the motion, as
paddingTop on Android and contentInset.top on iOS, to keep the top of the
history reachable under the translated dock. While following, the end pin
absorbed it in the same mount. While detached or holding a disclosure, nothing
did: Android's native anchor is LegendList's ScrollAdjust view, which padding
does not move, and LegendList compensates a padding change only on iOS. Every
row jumped by the keyboard height 0.2-1 s after the keyboard settled, and back
after it closed. On send, the same commit raced the jump to the end.

The keyboard now only translates the list with the composer. The cost: while
the keyboard is open, the composer viewport clips the top keyboard-height of
the list, so the start of a short conversation is hidden until it closes.

* chore(app): describe the native timeline as LegendList in stream comments

* fix(app): keep a disclosure's trigger in place when its row is cut off at the top on native

LegendList's idsInView starts at the first fully visible row. When a turn host's prompt bubble
sat above the viewport, a disclosure hold found no anchor in the data pass (shouldRestorePosition
rejects every other row), and the size pass anchored on the row below the host instead, so the
trigger moved by the host's size change. The patch appends the partly visible rows to idsInView
after the fully visible ones, so the held row anchors while every pass that does not restrict the
anchor keeps the first fully visible row.

The browser test renders the patched native build, which vitest now aliases instead of the
package's unpatched web build.

* fix(app): let a turn's summary line collapse again after it opens

The React Compiler cached StreamRow's `renderStreamItem(layoutItem)` on the layout item and the
renderer. The revised item that useRevisedHistoryRows hands a row whose display state changed only
fed the row id, so the row reused its cached content. Opening a turn usually inserts its rows
directly below the host, which also gives the host a new layout item. When another row sits
between them, such as a compaction marker, the host kept its layout item, its summary line stayed
collapsed, and every later tap asked to open the turn again.

The renderer now takes the item it renders, so the compiler's cache keys on the revision. StreamRow
moves to its own module, which the tests compile the way Expo does.

* test(app): alias the patched LegendList build in browser tests only

Unit tests load @legendapp/list outside Vite, so the native build's react-native import skipped
the react-native-web alias and reached React Native's Flow source, failing every unit suite that
imports the native strategy.

* fix(app): keep a size-only disclosure's trigger in place when its row is cut off at the top on native

LegendList's prepareMVCP picks the anchor for a size change with no data change as the first row
in view, without asking shouldRestorePosition. A disclosure that only grows its own row (a todo
card) while that row's top was above the viewport therefore held the row below it, and the trigger
moved by the row's size change. The patch now applies shouldRestorePosition to size changes as it
already does to data changes.

* test(app): render the timeline row revision test in a real browser

The repo's unit tests do not mount components in JSDOM. The test still compiles stream-row.tsx with
the React Compiler, so it fails when a revised row ignores the item it was handed.
2026-11-04 17:21:31 +07:00

579 lines
27 KiB
Markdown
Raw Blame History

# Mobile Testing
## Agent Device
Agent Device `.ad` scripts are the primary mobile E2E format. An agent discovers a working flow interactively, saves the successful commands, then the replay runner executes the same typed plan locally or in CI.
Record a flow while driving the app normally:
```bash
agent-device open sh.paseo.debug \
--platform ios \
--session terminal-author \
--save-script ./packages/app/e2e/mobile/agent-device/terminal.ios.ad
agent-device snapshot -i --session terminal-author
agent-device press 'id="workspace-header-menu-trigger"' --session terminal-author
# Continue the flow and verify its result with wait/get/is/find.
agent-device close --session terminal-author
```
`close` writes the script. Keep selectors based on stable app IDs. Keep assertions as `wait`, `get`, `is`, or `find` commands; screenshots are evidence, not assertions.
Run the Paseo mobile suite:
```bash
npm run test:e2e:mobile
```
The runner uses an isolated Agent Device state directory, verifies or starts Metro for this checkout, prewarms the iOS runner, discovers each script's platform from its `context` header, and cleans its sessions, runner lease, daemon, and any Metro process it started. Attempt results, timings, logs, and failure artifacts go under `.dev/agent-device-artifacts`.
Set `PASEO_MOBILE_E2E_METRO_PORT` when this worktree already has Metro on a non-default port:
```bash
PASEO_MOBILE_E2E_METRO_PORT=62093 npm run test:e2e:mobile
```
[native-terminal-basic.ios.ad](../packages/app/e2e/mobile/agent-device/native-terminal-basic.ios.ad) and [native-terminal-basic.android.ad](../packages/app/e2e/mobile/agent-device/native-terminal-basic.android.ad) are the smallest examples. Each opens a fresh terminal, types a command at zero delay, submits it, and asserts its distinct output. The app must be connected to a daemon with an active workspace.
For Android background fold recovery, install the release APK on a foldable AVD, open a
connected workspace, close the keyboard, and run:
```bash
ANDROID_SERIAL=emulator-5574 node packages/app/e2e/mobile/fold/android.mjs
```
The AVD must expose `CLOSED` and `OPENED` device states. The test backgrounds the app,
changes posture, resumes immediately, and asserts compact/wide workspace controls without
restarting the process. It writes snapshots and native failure state under
`.dev/agent-device-artifacts/android-fold`. Set `PASEO_MOBILE_APP_ID=sh.paseo` for a
production package. This needs a native rebuild when the Unistyles patch changes; a Metro
reload cannot update it.
For Android keyboard continuity, run the current checkout in the app, open an idle terminal
with an empty prompt, hide its keyboard, and run:
```bash
ANDROID_SERIAL=emulator-5554 node packages/app/e2e/mobile/terminal-keyboard/android.mjs
```
Use a real docked software keyboard. The harness taps Ctrl, Esc, and Enter and checks Android's
focused input identity and IME hide/show events. It saves screenshots and logs under
`.dev/agent-device-artifacts/terminal-keyboard-android`. Set `PASEO_TERMINAL_KEYBOARD_APP_ID=sh.paseo`
to test an installed production build. It never submits a chat message.
`npm run test:e2e:composer-keyboard:android` preserves the chat control and
keyboard regression flow, then runs the same growth, retained-height, bounds, and
hold-to-delete and background-tap dismissal scenario on chat, a workspace draft
tab, and New workspace. It checks empty-content taps and header taps at the cap
without changing the input height. Each
host also opens command/file autocomplete, the model selector, the attachment
menu, and the forge picker from both keyboard states. Autocomplete opens with
the keyboard closed, before the keyboard opens, and after it settles. Each time
the expected best match must be drawn whole between the header and the composer,
above the IME frame, and tapping it there must produce the exact composer text.
Accessibility reports rows behind the keyboard too, so
`packages/app/e2e/mobile/composer-keyboard/autocomplete-placement.mjs` compares
bounds. Android leaves the chat and composer out of the accessibility tree while
suggestions are open, so the composer bounds come from a snapshot with an empty
input in the same keyboard state. The queries are partial names with known best
matches: in chat `/mo` finds the mock provider's `/mock` command, in drafts
`/release` finds this repository's `release-beta` skill for the first Claude model,
and `@app` finds `packages/app`. It then submits a long
fixture from each host and checks native stream displacement twice: after the
response settles, and after returning to the bottom and letting the scrollbar
fade. The image comparison excludes the scrollbar; keyboard dismissal alone
does not prove that native scrolling receives touches.
Artifacts are grouped by host under
`.dev/agent-device-artifacts/composer-keyboard-android`. Configure its daemon,
Metro, and device through the `PASEO_COMPOSER_KEYBOARD_*` variables in
`packages/app/e2e/mobile/composer-keyboard/android.sh`. Use a software keyboard;
the headless input helper cannot verify the
[visible composer constraints](floating-panels.md#gotcha-3--keyboard-layout-and-portal-anchors).
To rerun only the autocomplete, model selector, attachment menu, and forge picker
checks on each host, skipping the chat baseline and the growth, dismissal, and
stream scenarios:
```bash
PASEO_COMPOSER_KEYBOARD_SCENARIO=overlays npm run test:e2e:composer-keyboard:android
```
`node packages/app/e2e/mobile/composer-keyboard/ios.mjs` runs the same
autocomplete checks on iOS for chat, a workspace draft tab, and New workspace,
with commands, files, and files under a grown draft. Suggestions open while the
keyboard opens (`fill` taps and types at once), after it settles, with it closed,
and before it reopens. `test:e2e:mobile` cannot host this: its `.ad` flows assert
accessibility state and cannot compare bounds. Run it where `agent-device`
reaches the simulator, with the development app connected to the fixture host
and its Claude provider available. Set `PASEO_COMPOSER_KEYBOARD_SERVER_ID`,
`_AGENT_ID` (an idle mock chat), and `_WORKSPACE_ID` (its workspace); `_DEVICE`,
`_SESSION`, and `_DRAFT_ID` are optional. Artifacts go to
`.dev/agent-device-artifacts/composer-keyboard-ios`.
With a Codex draft selected, set `_SCENARIO=growth` on a 402pt-wide iPhone to
check live wrapping, shrinking, the 12pt anchor gap, and command insertion. Set
`_SCENARIO=previews` on an SE-sized iPhone to check long and short descriptions
under an eight-line draft, selection with the keyboard shown and hidden, and
file mentions. These scenarios only need `_SERVER_ID` and `_WORKSPACE_ID`;
`_UDID` selects an exact simulator and `_ARTIFACTS_DIR` overrides the output path.
The Android sidebar scroll-retention flow requires a connected host with enough
workspaces to overflow the compact sidebar. Pass `TOP_WORKSPACE` and
`SCROLLED_WORKSPACE` as the `serverId:workspaceId` suffixes of their sidebar row
IDs. Choose one row near the top and one initially below the viewport. Replace
the quoted example values below with those IDs. Start in a workspace with the
sidebar closed and keep the list stable during the run. The same flow checks
reopening by swipe and button:
```bash
agent-device replay packages/app/e2e/mobile/agent-device/native-sidebar-scroll-retention.android.yaml \
--maestro --platform android --serial emulator-5556 --metro-port 8099 \
--env TOP_WORKSPACE='server-id:top-workspace-id' \
--env SCROLLED_WORKSPACE='server-id:scrolled-workspace-id'
```
Pass this checkout's Metro port so the debug app uses the intended bundle.
For iOS, compare the scrolled row's screen bounds and screenshots across reopening.
Agent Device's iOS visibility lookup includes offscreen scroll descendants, so
its `assertNotVisible` cannot establish the starting scroll position for this flow.
On a Play-image emulator, disable Gmail and Calendar first. Their welcome
screens launch on their own, take input focus, and the run fails with
`IME did not become visible` while the app still looks focused:
```bash
adb shell pm disable-user --user 0 com.google.android.gm
adb shell pm disable-user --user 0 com.google.android.calendar
```
When replay diverges, read its ranked selector suggestions. Edit the script deliberately and rerun it from the beginning. `--update` is retained for compatibility but no longer rewrites scripts.
## Maestro compatibility
Existing Maestro flows live in `packages/app/maestro/`. Agent Device can execute its supported subset with `agent-device test <path> --maestro`, which provides a migration path while `.ad` coverage replaces these flows.
### Existing flows
Maestro flows live in `packages/app/maestro/`. Reusable sub-flows live in `packages/app/maestro/flows/`.
Run a flow:
```bash
maestro test packages/app/maestro/my-flow.yaml
```
### Screenshots
`takeScreenshot` writes to the **current working directory** — there's no way to configure the output path in the YAML. To keep screenshots out of the checkout, `cd` into a temp directory and use an absolute path for the flow:
```bash
FLOW="$(pwd)/packages/app/maestro/my-flow.yaml"
mkdir -p /tmp/maestro-out
cd /tmp/maestro-out && maestro test "$FLOW"
```
`packages/app/maestro/.gitignore` excludes `*.png` as a safety net.
### Element targeting
Use `testID` or `nativeID` on components, then target with `id:` in flows. Prefer this over text matching — text breaks on copy changes.
```tsx
// Component
<Pressable testID="sidebar-sessions" onPress={onPress}>
```
```yaml
# Flow
- tapOn:
id: "sidebar-sessions"
- assertVisible:
id: "sidebar-sessions"
```
### Conditional steps
Use `runFlow:when:visible` for steps that should only execute when a specific element is on screen:
```yaml
- runFlow:
when:
visible:
id: "sidebar-sessions"
commands:
- swipe:
direction: LEFT
duration: 300
```
This is how `flows/dev-client.yaml` handles Expo dev client screens that only appear in dev builds.
### Don't use launchApp against a running dev app
`launchApp` kills and restarts the app, disrupting Expo dev client state and host connections. For flows that test against an already-running dev app, **omit launchApp entirely** — just interact with whatever is on screen.
Use `launchApp` only in flows that need a clean start (e.g., onboarding tests).
### Swipe gestures
Use `start`/`end` with percentage coordinates for precise control:
```yaml
# Edge swipe from left to open sidebar
- swipe:
start: "5%,50%"
end: "80%,50%"
duration: 300
```
`direction: RIGHT` is simpler but less precise — use it for generic swipes, use coordinates when the start position matters (edge gestures, avoiding specific UI regions).
### Assertions
`assertVisible` checks **actual screen visibility**, not just view tree presence. An element that exists in the tree but is off-screen (e.g., `translateX: -400`) will correctly fail `assertVisible`. This makes it reliable for catching animation bugs where state says "open" but the view is visually hidden.
For async elements, use `extendedWaitUntil`:
```yaml
- extendedWaitUntil:
visible: ".*Online.*"
timeout: 90000
```
### Dev client handling
Two reusable flows handle Expo dev client screens after launch:
- `flows/launch.yaml` — handles dev launcher, dismisses dev menu, asserts "Welcome to Paseo"
- `flows/dev-client.yaml` — same but without asserting a particular app route
### Reach the composer
`flows/land-in-chat.yaml` is the canonical "get into a chat" primitive. It `clearState`s, runs `launch.yaml`, taps the welcome screen's direct-connection option, types `127.0.0.1:6767`, submits, and waits for `message-input-root`. Compose any composer-level fixture on top of it:
```yaml
appId: sh.paseo
---
- runFlow: flows/land-in-chat.yaml
# ...your scenario here, starting from a ready composer
```
See `image-picker-repro.yaml` for an example.
**Prefer direct connection over relay pairing for local E2E.** Relay needs a 400+ character pairing URL typed into an input; direct needs `127.0.0.1:6767`. The daemon listens on 6767 and the simulator can reach it directly.
### New Workspace Creation
The Android workspace-creation regression has a dedicated harness:
```bash
bash packages/app/maestro/test-workspace-create-android-crash.sh
```
For a short recording that starts after launch/connection/sidebar setup:
```bash
bash packages/app/maestro/record-workspace-create-android-focus.sh
```
The flow details are documented in `packages/app/maestro/README.md`. The important rule is that a valid new-workspace assertion must prove the redirect completed: select a real model, tap `Create`, wait for `workspace-header-title`, wait for `message-input-root`, assert `New workspace` is gone, and assert the Android redbox strings are absent. Waiting for the composer alone is too weak because it can still be the `/new` route after a validation error.
New workspace scenarios should compose the reusable subflows in `packages/app/maestro/flows/`:
- `android-dev-client.yaml`
- `connect-direct-if-welcome.yaml`
- `open-prepared-project-sidebar.yaml`
- `new-workspace-open-from-sidebar.yaml`
- `new-workspace-select-codex-gpt54.yaml`
- `new-workspace-submit-and-assert-created.yaml`
The workspace-create shell scripts render those subflows into a temp directory before running Maestro, which keeps nested `runFlow` paths and `${PASEO_MAESTRO_*}` placeholders working together.
### Inputs that Maestro types into
Maestro `inputText` fires one character at a time. React Native's **controlled** `TextInput` re-renders per keystroke; if a controlled input's state update lags or re-mounts mid-type, characters are dropped silently — the final value on screen is a truncated/scrambled version of what was "typed."
For inputs that E2E flows type into (host endpoint, pairing URL, etc.), use an **uncontrolled ref-backed input**: `defaultValue` + `onChangeText` writes into a `useRef`, reads via the ref on submit. No per-keystroke re-render, no dropped characters.
See `pair-link-modal.tsx` for the pattern (`useRef`-backed `onChangeText`, no `value=` prop). Always pair the source change with a Maestro `assertVisible` on the input's `id + text` after `inputText`, so regressions are caught immediately.
### Dropdowns that launch native presenters (iOS)
On iOS, when a dropdown menu (`DropdownMenu` / RN `Modal`) item needs to launch a native presenter like `PHPickerViewController` (image picker) or a `UIDocumentPicker`, the callback **must not fire while the `Modal` is still dismissing**. UIKit dismissal completion spans multiple frames beyond React unmount; launching a native presenter mid-dismissal leaves an invisible backdrop mounted that traps every subsequent touch.
`DropdownMenu` handles this by deferring the selected item's `onSelect` until `Modal.onDismiss` fires (UIKit-level dismissal complete), then adds a small extra buffer before invoking it. See `components/ui/dropdown-menu.tsx`'s `selectItem` / `flushPendingSelect`.
When building a new component that composes a dropdown with a native presenter, reuse this dropdown — do not invent a new timing shim.
## Self-verification loops
Maestro can only interact with the app UI — it can't toggle iOS appearance, change locale, or simulate network conditions. For bugs that depend on system-level state, wrap Maestro in a bash script that handles the system changes between Maestro runs.
This pattern also lets agents self-verify fixes without manual user testing.
### Pattern
1. Run baseline Maestro flow (confirm feature works)
2. Make system-level change via `xcrun simctl` (toggle appearance, etc.)
3. Re-run Maestro flow (confirm feature still works)
4. Repeat N iterations to catch intermittent failures
Scripts run `maestro test` from inside a temp directory so screenshots don't dirty the checkout.
See `packages/app/maestro/test-sidebar-theme.sh` for the canonical example:
```bash
bash packages/app/maestro/test-sidebar-theme.sh 6 1
# Args: iterations=6, wait_seconds=1 between toggle and test
```
Key elements of the script pattern:
```bash
set -euo pipefail
ITERATIONS="${1:-3}"
for i in $(seq 1 "$ITERATIONS"); do
# Toggle system state
xcrun simctl ui booted appearance light
# Wait for change to propagate
sleep 1
# Run Maestro flow and capture result
if maestro test "$FLOW" 2>&1 | tee "$ITER_DIR/test.log"; then
echo "PASS"
else
echo "FAIL"
xcrun simctl io booted screenshot "$ITER_DIR/failure-state.png"
fi
done
```
### Android audio focus interruptions
Use native instrumentation for audio interruptions. JS-only tests cannot prove Android focus,
input silencing, or route recovery. The module's `android/src/androidTest` suite runs through
Gradle's `connectedDebugAndroidTest` task and uses real `AudioManager` and `AudioRecord` instances.
Run the microphone-privacy cases on Android 12 or newer.
Verify these distinct outcomes:
- Dictation continues when another media owner takes permanent, temporary, or ducking focus.
- Disabling microphone access pauses audio delivery without discarding the dictation. Enabling
access resumes it. Starting dictation while access is disabled waits for the same recovery.
- Returning from music controls resumes dictation without pausing music again. Cancelling a
backgrounded recording must prevent the lifecycle callback from restarting it.
- Two-way voice still stops coherently on focus loss. Keep the incoming-call check below: it
exercises denied focus during lifecycle resume, beyond an ordinary media takeover.
```bash
# Start voice mode in an existing composer, then background Paseo.
adb emu gsm call 5551234
# Foreground Paseo while the call is still ringing, then end the simulated call.
adb emu gsm cancel 5551234
```
Expected result: no `RuntimeException: Audio focus request failed`, and voice mode stops or pauses
coherently. Repeat with dictation: it must retain captured audio and recover microphone access.
Use browser/device automation to operate the app; the commands above only control emulator telephony.
This fix requires a rebuilt native app. An OTA update on an older binary retains legacy capture.
### Releasing the audio session when idle
Paseo must not hold the OS audio session once it is neither capturing nor playing, or the user's
background music stays paused. On iOS this is not just a "while recording" problem: the
`.playAndRecord`/`.voiceChat` category is non-mixing and survives backgrounding, and iOS re-asserts
it every time the app returns to the foreground — so one dictation turn kills music for the life of
the process, on every open, until the app is force-quit. Android holds
`AUDIOFOCUS_GAIN_TRANSIENT_EXCLUSIVE` with the same effect. It must also reset
`MODE_IN_COMMUNICATION` and clear the selected communication device; abandoning focus alone can
leave Bluetooth earbuds on their narrow-band call route without an active microphone indicator.
`createAudioEngine` (`packages/app/src/voice/audio-engine.native.ts`) calls
`releaseAudioSession()` whenever capture stops and the playback queue drains, and the iOS module
also releases on `OnAppEntersBackground`. The native side re-guards on `isRecording` /
`speechPlayer.isPlaying`, because the native engine is a singleton shared by multiple JS engine
wrappers (voice provider + dictation) and only it knows the true state.
This cannot be validated by a JS test — verify on a device: play music through Bluetooth earbuds,
open Paseo, use dictation once, stop, and confirm the music resumes at full quality; then
background/foreground the app and confirm it keeps playing at full quality.
## Unistyles + Reanimated
### The crash
Applying Unistyles theme-reactive styles (`StyleSheet.create((theme) => ...)`) directly to `Animated.View` causes **"Unable to find node on an unmounted component"** on theme change.
Unistyles wraps styled components in `<UnistylesComponent>` and patches native view properties via C++. Reanimated also manages the same native node for animated transforms. When the theme changes, both systems try to update the node simultaneously and the view crashes.
### The fix
Use plain React Native `StyleSheet.create` for static positioning on `Animated.View`. Pass theme-dependent values as inline styles from `useUnistyles()`:
```tsx
// BAD: Unistyles dynamic style on Animated.View
const styles = StyleSheet.create((theme) => ({
sidebar: {
position: "absolute",
top: 0,
left: 0,
bottom: 0,
backgroundColor: theme.colors.surfaceSidebar, // theme-reactive
overflow: "hidden",
},
}));
<Animated.View style={[styles.sidebar, animatedStyle]} />;
```
```tsx
// GOOD: static stylesheet + inline theme values
import { StyleSheet as RNStyleSheet } from "react-native";
const staticStyles = RNStyleSheet.create({
sidebar: {
position: "absolute",
top: 0,
left: 0,
bottom: 0,
overflow: "hidden",
},
});
const { theme } = useUnistyles();
<Animated.View
style={[staticStyles.sidebar, animatedStyle, { backgroundColor: theme.colors.surfaceSidebar }]}
/>;
```
Regular `View` components can safely use Unistyles dynamic styles — the conflict is specific to `Animated.View`.
## Native Chat Stream Layout
Do not compute stream neighbors, history/live-head seams, turn footer ownership, assistant block spacing, or tool sequence endings inside React render loops. Those policies live in `packages/app/src/agent-stream/layout.ts` and are unit-tested without React Native rendering.
Platform-specific stream edges belong on `StreamStrategy`. Web and native both render in chronological order, so both use the last history item as the history/live-head boundary and render content before a footer.
If a chat footer looks duplicated or appears above the assistant message on mobile, start with `packages/app/src/agent-stream/layout.test.ts`. Do not add a React Native renderer test for this class of bug; make the pure layout invariant fail first.
## Local iOS device builds
Native changes (anything under `packages/expo-two-way-audio/ios/`, or any new Expo module) **cannot be
tested over Metro or EAS Update**. Those ship JS only. A stale dev-client binary silently lacks the new
native functions, so the feature no-ops and the test proves nothing. Rebuild the binary.
Symptom of a stale dev client: `Cannot find native module 'ExpoDocumentPicker'` at startup, followed by a
cascade of `Route "./X.tsx" is missing the required default export` warnings. The route warnings are not a
router bug — the module-level throw aborts `_layout.tsx` evaluation, so its default export never gets
assigned. One missing native module, N confusing warnings.
**Prerequisite: Xcode needs a signed-in Apple ID for the signing team _with a valid keychain token_.** An
account that merely appears in Xcode → Settings → Accounts is not enough — if its `Xcode-Token` is missing,
the build below fails at code signing and blames a _certificate_ instead. See
[Signing needs a working Apple ID token in Xcode](#signing-needs-a-working-apple-id-token-in-xcode) before
you start, not after it fails.
```bash
cd packages/app
npm --prefix ../.. run build:client
APP_VARIANT=development npx expo prebuild --platform ios
APP_VARIANT=development npx expo run:ios --device
```
`APP_VARIANT=development` is required. The `ios` npm script does not set it, and `app.config.js` defaults
to `production` — so a bare `npm run ios` builds `sh.paseo` and collides with the App Store install instead
of the `sh.paseo.debug` dev client. Ignore prebuild's `--non-interactive is not supported` warning; use
`CI=1` if you need non-interactive.
### Signing needs a working Apple ID token in Xcode
Device builds fail with a trio of errors that all point away from the real cause:
```
error: No Accounts: Add a new account in Accounts settings.
error: Provisioning profile "iOS Team Provisioning Profile: *" doesn't include
signing certificate "Apple Development: <name>".
error: Provisioning profile "iOS Team Provisioning Profile: *" doesn't include
the aps-environment entitlement.
```
**`No Accounts` does not reliably mean no account is configured.** Check the line _above_ it in the build
log before believing it:
```
DVTDeveloperAccountManager: Failed to load credentials for <apple-id>:
Invalid credentials in keychain for <apple-id>, missing Xcode-Token
```
That is the actual failure: the Apple ID is registered in Xcode, but its `Xcode-Token` keychain item is gone
(typically after an Apple ID password change or a keychain reset). Xcode reports it as `No Accounts`, then
falls back to whatever stale profile is cached — and if the machine holds a dev cert newer than that
profile, you get the _certificate_ mismatch as a second-order symptom. Chasing the certificate error is a
dead end.
Verify what is actually configured rather than trusting the message:
```bash
# Accounts Xcode knows about
defaults read com.apple.dt.Xcode DVTDeveloperAccountManagerAppleIDLists
# The credential that is actually missing (error => no token)
security find-generic-password -s "Xcode-Token"
```
Fix: Xcode → Settings → Accounts, re-sign-in to the listed Apple ID (removing and re-adding forces a fresh
token). That restores `-allowProvisioningUpdates`, profile regeneration, and push.
Signing an Xcode-_managed_ profile via `CODE_SIGN_STYLE=Manual` is not a workaround; Xcode rejects it with
`is Xcode managed, but signing settings require a manually managed profile`.
If you need a build on the device before the token is fixed, build unsigned and sign by hand with a cert the
cached profile already embeds (list them with `security cms -D -i <profile>.mobileprovision`, then compare
against `security find-identity -v -p codesigning`):
```bash
xcodebuild ... CODE_SIGNING_ALLOWED=NO CODE_SIGNING_REQUIRED=NO CODE_SIGN_IDENTITY="" build
cp <profile>.mobileprovision "$APP/embedded.mobileprovision"
# sign nested frameworks/dylibs first, then the bundle, then:
xcrun devicectl device install app --device <udid> "$APP"
```
This is a stopgap, not a fix: `expo prebuild` regenerates `packages/app/ios/` and discards it. It also
requires stripping entitlements the cached profile lacks (e.g. `aps-environment`), so push is dead in such a
build.
To verify only that native Swift compiles, skip signing entirely — the pods have their own schemes:
```bash
cd packages/app/ios
xcodebuild -workspace PaseoDebug.xcworkspace -scheme ExpoTwoWayAudio \
-sdk iphoneos -destination 'generic/platform=iOS' CODE_SIGNING_ALLOWED=NO build
```
Confirm the log compiled _your_ file and not a cached copy: the `SwiftCompile` line should reference
`packages/expo-two-way-audio/ios/...`, not a path inside DerivedData.
### Headless launch cannot verify runtime behavior
`xcrun devicectl device process launch` reports `The app terminated with the exit code 0` about a second
after launch when the phone's screen is off — the app is suspended and killed before the JS bundle loads.
This is **not** a build defect. Confirm with a control: launch `com.apple.Preferences` the same way and
watch it exit identically. Headless install and launch prove the binary is valid and that startup reaches
RN init; anything about on-screen behavior needs a human holding the phone.
## iOS Simulator
```bash
# Screenshot
xcrun simctl io booted screenshot /tmp/screenshot.png
# Dark/light mode
xcrun simctl ui booted appearance # check current
xcrun simctl ui booted appearance dark # set dark
xcrun simctl ui booted appearance light # set light
```
Expo dev server logs are in the tmux pane running `npm run dev`. Daemon logs are at `$PASEO_HOME/daemon.log` (see [development.md](development.md)).
Reference in New Issue View Git Blame Copy Permalink