diff --git a/CLAUDE.md b/CLAUDE.md index 9a2c70de29..71682af7e9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -52,6 +52,7 @@ At the start of non-trivial work, list `docs/` and skim anything relevant to the | [docs/connected-hosts.md](docs/connected-hosts.md) | Daemon-owned host registrations, routed SDK scopes, identity and authority boundaries | | [docs/permissions.md](docs/permissions.md) | Semantic daemon permissions, principals, credentials, pairing invitations, and Hub authority | | [docs/terminal-performance.md](docs/terminal-performance.md) | Terminal latency pipeline, coalescing/backpressure invariants, benchmark + perf spec usage | +| [docs/terminal-rendering.md](docs/terminal-rendering.md) | Native terminal view: xterm owns the grid, cluster placement, emoji widths, mixed-direction limits, render regression | | [docs/agent-stream-performance.md](docs/agent-stream-performance.md) | Assistant text pipeline — coalescing window, paced reveal, why arrival lumps are smoothed at render | | [docs/markdown-renderer.md](docs/markdown-renderer.md) | Shared Markdown engine, native selection, links, images and library constraints | | [docs/native-text-selection.md](docs/native-text-selection.md) | Native diagram grouping, selection boundaries and leading inline image limitations | diff --git a/docs/terminal-rendering.md b/docs/terminal-rendering.md new file mode 100644 index 0000000000..fd304aa144 --- /dev/null +++ b/docs/terminal-rendering.md @@ -0,0 +1,65 @@ +# Terminal rendering + +Web and Electron draw terminals with xterm.js. iOS and Android keep an `@xterm/headless` buffer in JS and draw it with the `PaseoTerminal` native view in `packages/app/modules/paseo-terminal`: CoreText on iOS, `Canvas.drawTextRun` on Android. `src/terminal/native-renderer/terminal-frame.ts` turns the visible buffer into frames, and both native views take the same props. + +## xterm owns the grid + +xterm decides which column every cell occupies. The native views never let text advance move a cell. Each row is shaped once, one run per font face, so Hangul jamo, ligatures, combining marks, conjuncts and fallback fonts see their neighbours. Each shaped cluster is then drawn at its first cell's column, centred in the columns it owns, and scaled down when its glyphs are wider. Cursor, selection, hit-testing and copy all use xterm's columns, so they stay correct whatever the font does. + +A cluster is the run of cells the shaper merged, and it is drawn once from the shaped row. Drawing part of a cluster on its own reshapes it: a half-form conjunct such as `स्ते` comes apart into `स्` with a visible virama. So colour and selection never split a cluster. A cluster whose cells differ in colour is drawn once per colour, clipped to that colour's columns. + +Android also groups cells with `android.icu.text.BreakIterator`, because Minikin's advances and cursor boundaries alone split some conjuncts. Grapheme rules follow the OS ICU version. Android 10 (API 29) splits `स्ते` after the virama and renders it incorrectly. The tested corpus passed on API 33, 34 and 35; API 30–32 have not been tested. We accept that version-dependent limitation to avoid bundling approximately 14 MB of ICU4J data. CoreText keeps this conjunct together on iOS. This renderer does not guarantee identical shaping across OS versions. + +`TerminalRowLayoutTest.kt` is the Android regression test. It draws the rows of `grapheme-rows.json` and compares each with its grapheme clusters drawn one at a time, and with a wrong clustering that must not match. `terminal-frame.test.ts` checks that those rows are what xterm produces and that the clusters are UAX #29 grapheme clusters. Run it with `./gradlew :paseo-terminal:connectedDebugAndroidTest` from `packages/app/android`, with `ANDROID_SERIAL` set. + +The view reports ready only after it has measured its grid size. The pane attaches with that size, so the daemon replays the terminal at the client's size. Before this, the replay arrived at the PTY's old size, the resize came after it, and the shell's redraw landed on the wrong rows. + +Emoji widths come from the headless buffer, which uses xterm's default Unicode 6 table. `👍` gets one column, and `👋🏽` and `🇰🇷` get two cells drawn as one cluster. Web loads the Unicode 11 addon, so the same output can wrap differently on web and native. + +## Ink outside the row + +Marks stacked above or below a letter, Arabic letter tails and some fallback glyphs ink into the rows next to theirs. Neither view clips a row to its own bounds, because a row clip also cuts marks and Arabic dots. Rows draw top to bottom, so a later row's cell backgrounds cover the ink that reaches into it. + +Android redraws every row on each frame, so overhanging ink cannot go stale. Its row `RenderNode`s are created with `setClipToBounds(false)`. The default would clip only unselected rows, because selected rows are drawn straight onto the view's canvas. + +iOS repaints only the rows that changed. Each `TerminalRowLayout` records how far its glyphs ink above and below the row. A changed row invalidates its old and new ink extent, and `draw(_:)` redraws every row whose ink extent meets the dirty rect. Invalidating only the row's own rect left an erased row's marks in the row next to it, erased a neighbour's marks when a row was rewritten, and showed or hid overhang depending on which rows changed together. + +## Mixed-direction text + +Rows draw left to right in logical order on both platforms. Terminal bidi escape modes are not implemented. On iOS the row needs both the `kCTWritingDirection` override and an LTR paragraph base direction. Without the base direction, bidi rule L1 moves trailing spaces ahead of a row that starts with Arabic or Hebrew, and the whole row shifts. + +Arabic still gets the contextual form its logical neighbours select, but in left-to-right order, so letters join on the wrong side. Correct visual Arabic needs implicit bidi per row: reorder RTL runs for display and map cursor, selection and hit-testing through the visual-to-logical index. That changes which column a character appears in, so it has to be decided as product behaviour first. Shaping alone cannot fix it. + +## Render regression + +The scripts live in `packages/app/e2e/mobile/terminal-rendering`. Run them from `packages/app` on the daemon's host, starting from an idle, focused terminal with an empty prompt. + +`render.mjs` prints a mixed-script fixture, screenshots it with agent-device, and checks the geometry with `check.mjs`. The check rebuilds the expected buffer with `@xterm/headless` and measures the grid from a ruler row. It checks that: + +- every grapheme cluster has ink inside its own columns, and blank cells stay blank; +- on the accent rows, every cell is drawn in its own colour, including cells inside one cluster; +- the cursor is drawn after the text; +- every row is exposed as `terminal-row-`; +- selecting one cell of a conjunct highlights only that cell. + +On iOS the screenshot is taken at 3 pixels per point (`PASEO_TERMINAL_RENDERING_PIXEL_DENSITY`), because the simulator's default is one pixel per point and a glyph is then about 12 pixels. Focus the terminal before a run: with the simulator's hardware keyboard connected the key bar shows without a focused input, and `type` fails. In a debug build, the dev client's floating gear button covers the right end of the first row, so ruler cells there fail. + +```bash +PLATFORM=android ANDROID_SERIAL=emulator-5554 node e2e/mobile/terminal-rendering/render.mjs +PLATFORM=ios IOS_DEVICE="iPhone 17" node e2e/mobile/terminal-rendering/render.mjs +``` + +Geometry passes when a cell holds the wrong glyph. `identity.mjs` judges each cluster of a `render.mjs` run against pango's rendering of the same characters in the device's fonts. The reference is cut at the edges of the cluster's columns, placed as the renderer places it, because glyphs such as Devanagari letters ink past their advance and the device cuts them there too. A shape passes when its coverage correlates with its own reference at 0.6 or more, and no other cluster of the corpus matches it better. Box and block characters are judged against their Unicode definition. Shades are judged by even density over a 4×4 grid, because the device and pango fall back to different shade fonts. Clusters that pango draws without ink are reported as unjudged instead: flags on both platforms and Apple Color Emoji. The `rect` and `swap` controls fill every cluster with white, or judge each one against another cluster. Both must fail every cluster, which shows the oracle refuses ink that is present but wrong. An Arabic letter is compared with all four of its positional forms, so the oracle shows the right letter is drawn, not that it has the right form. Ink that a neighbouring row draws into a cluster's cells counts as that cluster's ink. On iOS the tail of `م` in the Arabic row reaches one pixel into the `न` below it, so `न` fails although both rows are drawn correctly. + +```bash +FONTCONFIG_FILE=fonts.conf node e2e/mobile/terminal-rendering/identity.mjs [none|rect|swap] +``` + +`fonts.conf` points fontconfig at the device's font files and lists, for `monospace`, the terminal font followed by the device's fallbacks for Hangul, Arabic, Devanagari, CJK, emoji and symbols: + +- **Android:** pull the fonts from `/system/fonts` with `adb pull`. The terminal font is Droid Sans Mono. +- **iOS:** copy the simulator runtime's fonts from `RuntimeRoot/System/Library/Fonts`. The terminal font is Menlo. PingFang stores its outlines in Apple's `hvgl` table, which FreeType cannot draw, so the CJK references come from Apple SD Gothic Neo instead. `語` then scores below the threshold although the device draws it correctly. + +`overhang.mjs` prints rows whose marks reach the rows next to them. In one update it writes into two rows that hold a neighbour's marks and erases two rows whose marks reach a blank row. The marks must be drawn before the update, stay drawn in the written rows, and leave no ink behind the erased rows. Run it like `render.mjs`. + +`reattach.mjs` checks the restore size. It changes the PTY size by opening the keyboard (`RESIZE=keyboard`) or folding an Android foldable (`RESIZE=fold`), relaunches the app, and compares the client's rows with `paseo terminal capture`. A trial where no terminal rows appear, for example after a launch crash, counts as never attached and fails the run. diff --git a/packages/app/e2e/mobile/terminal-rendering/check.mjs b/packages/app/e2e/mobile/terminal-rendering/check.mjs new file mode 100644 index 0000000000..fbf4529bb3 --- /dev/null +++ b/packages/app/e2e/mobile/terminal-rendering/check.mjs @@ -0,0 +1,311 @@ +import assert from "node:assert/strict"; +import { writeFileSync } from "node:fs"; +import xterm from "@xterm/headless"; +import { PNG } from "pngjs"; +import { ACCENT, fixtureLines, fixtureOutput, RULER_CELLS, SELECT_LINE } from "./fixture.mjs"; + +/** A pixel is ink when one channel differs from the terminal background by more than this. */ +const INK_THRESHOLD = 70; +/** Ink pixels a printed grapheme cluster needs inside its own columns. */ +const MIN_CLUSTER_INK = 3; +/** How far a pixel may sit from the background-to-color blend line and still count as that color. */ +const BLEND_TOLERANCE = 40; +/** Pixels of another cell's color tolerated in a cell, for antialiasing at the cell edge. */ +const MAX_FOREIGN_INK = 2; + +const graphemes = new Intl.Segmenter("und", { granularity: "grapheme" }); + +/** + * Checks a screenshot of the fixture against the cells xterm assigns: every printed grapheme + * cluster has ink inside its own columns, blank cells stay blank, and the cursor sits right after + * the text. A cluster that spans cells, such as a Devanagari conjunct, is shaped as one unit, so + * it is judged across all its columns. On rows that use the fixture's accent color, every cell + * must be drawn in its own color, including cells that share a shaped cluster. + * + * These are geometry and color checks: they cannot tell a correct glyph from a wrong one. + * + * `rows` maps a terminal row index to its rect in screenshot pixels. + */ +export async function checkTerminalScreenshot({ png, size, rows }) { + const terminal = await fixtureTerminal(size); + assert.ok(rows.get(0), "The ruler row is not exposed as terminal-row-0"); + const grid = measureGrid(png, rows.get(0)); + const { failures, slots } = checkRows({ png, terminal, rows, grid }); + terminal.dispose(); + return { grid, failures, slots }; +} + +/** Where the harness selects: the first cell of the conjunct that starts the `select` line. */ +export async function selectionTarget({ size }) { + const terminal = await fixtureTerminal(size); + const row = lineNames(terminal.buffer.active, terminal.buffer.active.cursorY + 1).indexOf( + SELECT_LINE, + ); + terminal.dispose(); + return { row, col: 0, clusterColumns: 2 }; +} + +/** + * Checks a screenshot taken with only the first cell of the `select` conjunct selected: that cell + * has the selection background and the conjunct's second, unselected cell keeps its accent color. + */ +export function checkPartialSelection({ png, rows, grid, target }) { + const rect = rows.get(target.row); + if (!rect) return [{ row: target.row, kind: "missing-row" }]; + const cell = (col) => ({ + left: grid.left + col * grid.cellWidth + 1, + right: grid.left + (col + 1) * grid.cellWidth - 1, + top: rect.y, + bottom: rect.y + rect.height, + }); + const selected = cell(target.col); + const unselected = cell(target.col + 1); + const selectionBackground = pixelAt(png, selected.left + 1, selected.top + 1); + const unselectedBackground = pixelAt(png, unselected.left + 1, unselected.top + 1); + if ( + !differs(selectionBackground, grid.background) || + differs(unselectedBackground, grid.background) + ) { + return [{ row: target.row, col: target.col, kind: "selection-not-set-up" }]; + } + const failures = []; + const ownInk = countColor(png, unselected, grid.background, ACCENT); + if (ownInk < MIN_CLUSTER_INK) { + failures.push({ row: target.row, col: target.col + 1, kind: "unselected-cell-lost-color" }); + } + return failures; +} + +async function fixtureTerminal(size) { + const terminal = new xterm.Terminal({ + cols: size.columns, + rows: size.rows, + allowProposedApi: true, + }); + await new Promise((done) => + terminal.write(fixtureOutput(size.columns).replaceAll("\n", "\r\n"), done), + ); + return terminal; +} + +export function writeOverlay(png, slots, path) { + const out = new PNG({ width: png.width, height: png.height }); + png.data.copy(out.data); + for (const slot of slots) { + const color = slot.ok ? [0, 200, 0] : [255, 0, 0]; + const { left, right, top, bottom } = slot.box; + for (let x = Math.ceil(left); x < right; x += 1) { + for (const y of [Math.ceil(top), Math.floor(bottom) - 1]) { + const offset = (out.width * y + x) * 4; + out.data.set(color, offset); + } + } + if (!slot.ok) { + for (let y = Math.ceil(top); y < bottom; y += 1) { + for (const x of [Math.ceil(left), Math.floor(right) - 1]) { + out.data.set(color, (out.width * y + x) * 4); + } + } + } + } + writeFileSync(path, PNG.sync.write(out)); +} + +function inkAt(png, x, y, background) { + const offset = (png.width * Math.round(y) + Math.round(x)) * 4; + return ( + Math.abs(png.data[offset] - background[0]) > INK_THRESHOLD || + Math.abs(png.data[offset + 1] - background[1]) > INK_THRESHOLD || + Math.abs(png.data[offset + 2] - background[2]) > INK_THRESHOLD + ); +} + +/** Pixels in `box` that differ from `background` in some channel by more than the ink threshold. */ +export function countInk(png, box, background) { + let count = 0; + for (let y = Math.ceil(box.top); y < Math.floor(box.bottom); y += 1) { + for (let x = Math.ceil(box.left); x < Math.floor(box.right); x += 1) { + if (inkAt(png, x, y, background)) count += 1; + } + } + return count; +} + +function differs(a, b) { + return a.some((channel, index) => Math.abs(channel - b[index]) > INK_THRESHOLD); +} + +/** Distance from `pixel` to the nearest blend of `background` toward `color`. */ +function blendDistance(pixel, background, color) { + const direction = color.map((channel, index) => channel - background[index]); + const length = direction.reduce((sum, value) => sum + value * value, 0); + const offset = pixel.map((channel, index) => channel - background[index]); + const along = offset.reduce((sum, value, index) => sum + value * direction[index], 0); + const alpha = length === 0 ? 0 : Math.min(1, Math.max(0, along / length)); + return Math.hypot(...offset.map((value, index) => value - alpha * direction[index])); +} + +/** Ink pixels in `box` that are a blend of the background toward `color`. */ +function countColor(png, box, background, color, other) { + let count = 0; + for (let y = Math.ceil(box.top); y < Math.floor(box.bottom); y += 1) { + for (let x = Math.ceil(box.left); x < Math.floor(box.right); x += 1) { + if (!inkAt(png, x, y, background)) continue; + const pixel = pixelAt(png, x, y); + const distance = blendDistance(pixel, background, color); + if (distance > BLEND_TOLERANCE) continue; + if (other && blendDistance(pixel, background, other) <= distance) continue; + count += 1; + } + } + return count; +} + +function pixelAt(png, x, y) { + const offset = (png.width * Math.round(y) + Math.round(x)) * 4; + return [png.data[offset], png.data[offset + 1], png.data[offset + 2]]; +} + +/** Reads the grid origin and cell width from a ruler row of `RULER_CELLS` full blocks. */ +export function measureGrid(png, rulerRect) { + const y = rulerRect.y + rulerRect.height / 2; + const background = pixelAt(png, rulerRect.x + rulerRect.width - 2, y); + let left = -1; + let right = -1; + for (let x = Math.ceil(rulerRect.x); x < rulerRect.x + rulerRect.width; x += 1) { + if (!inkAt(png, x, y, background)) continue; + if (left < 0) left = x; + right = x + 1; + } + assert.ok(left >= 0, "The ruler row has no ink"); + const foreground = pixelAt(png, Math.round((left + right) / 2), y); + return { left, cellWidth: (right - left) / RULER_CELLS, background, foreground }; +} + +function lineNames(buffer, count) { + const names = fixtureLines(0).map((line) => line.name); + const result = []; + let index = -1; + for (let row = 0; row < count; row += 1) { + if (!buffer.getLine(row).isWrapped) index += 1; + result.push(names[index] ?? `line-${index}`); + } + return result; +} + +/** The row's printed cells, consecutive cells of one grapheme cluster grouped together. */ +function rowClusters(line, columns, cell) { + const cells = []; + let text = ""; + for (let col = 0; col < columns; col += 1) { + line.getCell(col, cell); + const width = cell.getWidth(); + if (width === 0) continue; + const chars = cell.getChars() || " "; + cells.push({ + col, + width, + chars, + offset: text.length, + painted: cell.isInverse() || !cell.isBgDefault(), + accent: cell.isFgRGB() && cell.getFgColor() === rgbValue(ACCENT), + }); + text += chars; + } + const clusterEnds = [...graphemes.segment(text)].map( + (segment) => segment.index + segment.segment.length, + ); + const clusters = []; + let end = 0; + for (const entry of cells) { + while (entry.offset >= clusterEnds[end]) end += 1; + const last = clusters.at(-1); + if (last && last.end === end) last.cells.push(entry); + else clusters.push({ end, cells: [entry] }); + } + return clusters.map((cluster) => cluster.cells); +} + +function rgbValue([red, green, blue]) { + return (red << 16) | (green << 8) | blue; +} + +/** Each cell of a cluster that uses the accent color holds ink of its own color only. */ +function checkClusterColors({ png, cluster, rect, grid }) { + const kinds = []; + for (const entry of cluster) { + const box = { + left: grid.left + entry.col * grid.cellWidth + 1, + right: grid.left + (entry.col + entry.width) * grid.cellWidth - 1, + top: rect.y, + bottom: rect.y + rect.height, + }; + const [color, other] = entry.accent ? [ACCENT, grid.foreground] : [grid.foreground, ACCENT]; + const own = countColor(png, box, grid.background, color, other); + const foreign = countColor(png, box, grid.background, other, color); + if (own < MIN_CLUSTER_INK || foreign > MAX_FOREIGN_INK) { + kinds.push({ col: entry.col, kind: entry.accent ? "accent-color-lost" : "color-leaked" }); + } + } + return kinds; +} + +function checkRows({ png, terminal, rows, grid }) { + const buffer = terminal.buffer.active; + const lastRow = buffer.cursorY; + const names = lineNames(buffer, lastRow + 1); + const failures = []; + const slots = []; + const cell = buffer.getNullCell(); + for (let row = 0; row <= lastRow; row += 1) { + const rect = rows.get(row); + if (!rect) { + failures.push({ row, line: names[row], kind: "missing-row" }); + continue; + } + const clusters = rowClusters(buffer.getLine(row), terminal.cols, cell); + // Color is judged on whole rows that use the accent, so a color leaking into a neighbouring + // default-colored cell fails too. + const checksColor = clusters.some((cluster) => cluster.some((entry) => entry.accent)); + for (const cluster of clusters) { + const first = cluster[0]; + const columns = cluster.reduce((sum, entry) => sum + entry.width, 0); + const left = grid.left + first.col * grid.cellWidth; + const right = left + columns * grid.cellWidth; + const box = { left: left + 1, right: right - 1, top: rect.y, bottom: rect.y + rect.height }; + const chars = cluster.map((entry) => entry.chars).join(""); + const isCursor = row === lastRow && first.col === buffer.cursorX; + let kind = null; + if (isCursor) { + const covered = countInk(png, box, grid.background); + const area = (box.right - box.left) * (box.bottom - box.top); + if (covered < area * 0.25) kind = "cursor-not-drawn"; + } else if (chars.trim() !== "") { + if (countInk(png, box, grid.background) < MIN_CLUSTER_INK) kind = "missing-ink"; + } else if (!cluster.some((entry) => entry.painted)) { + const quarter = (right - left) / 4; + const center = { ...box, left: left + quarter, right: right - quarter }; + if (countInk(png, center, grid.background) > 0) kind = "ink-in-blank"; + } + const colorFailures = + kind === null && checksColor && chars.trim() !== "" + ? checkClusterColors({ png, cluster, rect, grid }) + : []; + slots.push({ + row, + line: names[row], + col: first.col, + width: columns, + chars, + isCursor, + box, + ok: kind === null && colorFailures.length === 0, + }); + if (kind) failures.push({ row, line: names[row], col: first.col, char: chars, kind }); + for (const failure of colorFailures) { + failures.push({ row, line: names[row], char: chars, ...failure }); + } + } + } + return { failures, slots }; +} diff --git a/packages/app/e2e/mobile/terminal-rendering/fixture.mjs b/packages/app/e2e/mobile/terminal-rendering/fixture.mjs new file mode 100644 index 0000000000..e4ff5c2bb4 --- /dev/null +++ b/packages/app/e2e/mobile/terminal-rendering/fixture.mjs @@ -0,0 +1,59 @@ +// Terminal output covering the scripts and styles the native renderer must place on the grid. +// Each entry is one line of output; `ruler` is drawn with full blocks so the harness can read +// the grid origin and cell width from pixels, independent of the renderer under test. + +const ESC = "\u001b"; +const sgr = (codes, text) => `${ESC}[${codes}m${text}${ESC}[0m`; + +export const RULER_CELLS = 20; +/** A foreground no theme uses, so its pixels can be told apart from default text. */ +export const ACCENT = [255, 64, 160]; +const accent = (text) => sgr(`38;2;${ACCENT.join(";")}`, text); +/** + * The `select` line starts with a conjunct whose first cell the harness selects on its own. Its + * second cell uses the accent, which no theme's selection foreground matches. + */ +export const SELECT_LINE = "select"; + +export function fixtureLines(columns) { + const wrapSyllables = Math.ceil(columns / 2) + 6; + return [ + { name: "ruler", text: "█".repeat(RULER_CELLS) }, + { name: "ascii", text: "ASCII abcdefghij 0123456789 end" }, + { name: "hangul-mixed", text: "한글 문장 Hello end" }, + { name: "hangul-color-boundary", text: `안녕하세요${sgr(31, "NEXT")} end` }, + { name: "hangul-bold-boundary", text: `${sgr(1, "안녕하세요")}NEXT end` }, + { name: "hangul-decomposed", text: "한글 end" }, + { name: "cjk", text: "日本語テキスト 中文字符 end" }, + { name: "combining", text: "é ä ñ x̣̂ end" }, + { name: "arabic", text: "مرحبا لا 123 abc end" }, + { name: "devanagari", text: "नमस्ते क्षत्रिय end" }, + { name: "emoji", text: "👍 ❤️ 👋🏽 👩‍💻 🇰🇷 end" }, + // One shaped cluster over two cells, colored per cell: the color must follow the cell. + { name: "split-color", text: `क्${accent("ष")} ${accent("क्")}ष ल${accent("ا")} end` }, + { name: SELECT_LINE, text: `क्${accent("ष")} end` }, + { name: "box", text: "╭──┬──╮ │ab│cd│ ╰──┴──╯ ▀▄▌▐░▒▓ end" }, + { + name: "styles", + text: [ + sgr(1, "bold"), + sgr(3, "italic"), + sgr(4, "under"), + sgr(7, "inverse"), + sgr(42, "bg"), + sgr(9, "strike"), + "end", + ].join(" "), + }, + { name: "hangul-wrap", text: "가".repeat(wrapSyllables) }, + // The last line has no newline: the shell sleeps with the cursor right after it. + { name: "cursor", text: "> 안녕하세요" }, + ]; +} + +/** The bytes `cat` writes; the tty turns each newline into CR LF. */ +export function fixtureOutput(columns) { + return fixtureLines(columns) + .map((line) => line.text) + .join("\n"); +} diff --git a/packages/app/e2e/mobile/terminal-rendering/identity.mjs b/packages/app/e2e/mobile/terminal-rendering/identity.mjs new file mode 100644 index 0000000000..a0a6246a3d --- /dev/null +++ b/packages/app/e2e/mobile/terminal-rendering/identity.mjs @@ -0,0 +1,423 @@ +import { execFileSync } from "node:child_process"; +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { resolve } from "node:path"; +import { PNG } from "pngjs"; +import { checkTerminalScreenshot } from "./check.mjs"; + +// Checks that every printed cluster of a `render.mjs` run is the right glyph. `check.mjs` proves +// where ink lands and what color it has; this compares each cluster's ink with pango's (HarfBuzz) +// rendering of the same characters in the device's own fonts, so a wrong glyph, tofu, a solid +// block, a split conjunct or a cluster from elsewhere fails. +// +// FONTCONFIG_FILE= node e2e/mobile/terminal-rendering/identity.mjs [control] +// +// The fonts.conf lists the device's font files and prefers, for `monospace`, the terminal font +// followed by the device's fallbacks; `docs/terminal-rendering.md` says how to build one. Needs +// `pango-view` on the PATH. +// +// control: +// none judge the screenshot (default); exits 1 when a cluster fails. +// rect first fill every judged cluster's box with white; exits 1 unless every cluster fails. +// swap judge each cluster against a different cluster of the corpus; exits 1 unless every +// cluster fails. +// The two controls prove the oracle refuses ink that is present but wrong. + +/** Coverage grid side used to compare shapes. */ +const GRID = 16; +/** A pixel at or above this ink strength (0..1) is part of a glyph's bounding box. */ +const INK = 0.25; +/** Lowest shape correlation a cluster may have with its own reference. */ +const THRESHOLD = 0.6; +/** How much better another cluster may match before the cluster's own match stops counting. */ +const MARGIN = 0.02; +/** Lines whose cells carry decorations or a custom-drawn ruler; they stay geometry-only. */ +const SKIPPED_LINES = new Set(["ruler", "styles"]); +const ZWJ = "‍"; + +/** + * The renderer draws box and block characters to fill the cell, so they are judged against their + * Unicode definition. Shades come from the font, whose fallback differs between the device and + * pango, so a shade is judged by how evenly and densely it inks its own bounding box. Fonts dither + * differently: ░ ▒ ▓ measure 0.16, 0.31, 0.75 in Droid Sans Mono and 0.19, 0.50, 0.81 in Menlo. The + * bands split between those, and do not overlap, so one shade never passes as another. + */ +const CELL_GLYPHS = { + "─": { arms: "LR" }, + "│": { arms: "UD" }, + "┬": { arms: "LRD" }, + "┴": { arms: "LRU" }, + "╭": { arms: "DR" }, + "╮": { arms: "DL" }, + "╰": { arms: "UR" }, + "╯": { arms: "UL" }, + "▀": { fill: "top" }, + "▄": { fill: "bottom" }, + "▌": { fill: "left" }, + "▐": { fill: "right" }, + "░": { shade: [0.08, 0.25] }, + "▒": { shade: [0.25, 0.625] }, + "▓": { shade: [0.625, 0.9] }, +}; +/** A shade must ink each region of this grid over its bounding box within its band. */ +const SHADE_GRID = 4; +/** Regions of the cell, as fractions [left, top, right, bottom], that a line to each edge crosses. */ +const ARM_REGIONS = { + U: [0.3, 0, 0.7, 0.2], + D: [0.3, 0.8, 0.7, 1], + L: [0, 0.3, 0.2, 0.7], + R: [0.8, 0.3, 1, 0.7], +}; +/** [filled, empty] halves of each half block. */ +const HALVES = { + top: [ + [0, 0, 1, 0.5], + [0, 0.5, 1, 1], + ], + bottom: [ + [0, 0.5, 1, 1], + [0, 0, 1, 0.5], + ], + left: [ + [0, 0, 0.5, 1], + [0.5, 0, 1, 1], + ], + right: [ + [0.5, 0, 1, 1], + [0, 0, 0.5, 1], + ], +}; + +const [dirArg, control = "none"] = process.argv.slice(2); +if (!dirArg || !["none", "rect", "swap"].includes(control)) { + throw new Error("Usage: identity.mjs [none|rect|swap]"); +} +const dir = resolve(dirArg); +const report = JSON.parse(readFileSync(resolve(dir, "report.json"), "utf8")); +const png = PNG.sync.read(readFileSync(resolve(dir, "terminal.png"))); +const rows = new Map(report.rows.map((row) => [row.index, row.rect])); +const { grid, slots } = await checkTerminalScreenshot({ png, size: report.size, rows }); + +const printed = slots.filter( + (slot) => !SKIPPED_LINES.has(slot.line) && !slot.isCursor && slot.chars.trim() !== "", +); +/** The columns xterm gives each printed cluster. */ +const columnsOf = new Map(printed.map((slot) => [slot.chars, slot.width])); + +if (control === "rect") { + for (const { box } of printed) { + for (let y = Math.ceil(box.top); y < Math.floor(box.bottom); y += 1) { + for (let x = Math.ceil(box.left); x < Math.floor(box.right); x += 1) { + png.data.set([255, 255, 255, 255], (png.width * y + x) * 4); + } + } + } +} + +/** Ink strength 0..1 of each pixel in a box: the largest channel distance from the background. */ +function screenshotInk(box) { + const left = Math.ceil(box.left); + const top = Math.ceil(box.top); + const width = Math.floor(box.right) - left; + const height = Math.floor(box.bottom) - top; + const ink = new Float32Array(width * height); + for (let y = 0; y < height; y += 1) { + for (let x = 0; x < width; x += 1) { + const offset = (png.width * (top + y) + left + x) * 4; + let distance = 0; + for (let channel = 0; channel < 3; channel += 1) { + distance = Math.max( + distance, + Math.abs(png.data[offset + channel] - grid.background[channel]), + ); + } + ink[y * width + x] = distance / 255; + } + } + return { ink, width, height }; +} + +/** Ink of the slot's whole cells, scaled so the foreground color is 1. */ +function cellInk(slot) { + const left = Math.round(grid.left + slot.col * grid.cellWidth); + const right = Math.round(grid.left + (slot.col + slot.width) * grid.cellWidth); + const cells = screenshotInk({ left, right, top: slot.box.top, bottom: slot.box.bottom }); + const full = + Math.max(...[0, 1, 2].map((c) => Math.abs(grid.foreground[c] - grid.background[c]))) / 255; + for (let i = 0; i < cells.ink.length; i += 1) cells.ink[i] = Math.min(1, cells.ink[i] / full); + return cells; +} + +const scratch = mkdtempSync(resolve(tmpdir(), "terminal-identity-")); +/** pango-view sizes its image to the text's advance plus this margin on each side. */ +const REFERENCE_MARGIN = 48; +function pangoInk(text) { + const out = resolve(scratch, "reference.png"); + execFileSync("pango-view", [ + "--no-display", + "--dpi=72", + "--font=monospace 96px", + `--text=${text}`, + "--background=black", + "--foreground=white", + `--margin=${REFERENCE_MARGIN}`, + "-o", + out, + ]); + const image = PNG.sync.read(readFileSync(out)); + const ink = new Float32Array(image.width * image.height); + for (let i = 0; i < ink.length; i += 1) { + const o = i * 4; + ink[i] = Math.max(image.data[o], image.data[o + 1], image.data[o + 2]) / 255; + } + return { + ink, + width: image.width, + height: image.height, + advance: image.width - 2 * REFERENCE_MARGIN, + }; +} + +/** The terminal font's cell advance at the reference size. */ +const referenceCell = pangoInk("M").advance; + +/** + * The text as the renderer places it in `columns` cells: centred, scaled down when its advance is + * wider, and cut at the cells' edges. Glyphs whose ink overhangs their advance, such as Devanagari + * headlines, are cut on the device too. + */ +function referenceInk(text, columns) { + const image = pangoInk(text); + const box = Math.max(image.advance, columns * referenceCell); + const left = Math.round(REFERENCE_MARGIN + (image.advance - box) / 2); + const right = Math.round(REFERENCE_MARGIN + (image.advance + box) / 2); + for (let y = 0; y < image.height; y += 1) { + for (let x = 0; x < image.width; x += 1) { + if (x < left || x >= right) image.ink[y * image.width + x] = 0; + } + } + return image; +} + +function regionInk({ ink, width, height }, [x0, y0, x1, y1]) { + let [sum, max, count] = [0, 0, 0]; + for (let y = Math.floor(y0 * height); y < Math.ceil(y1 * height); y += 1) { + for (let x = Math.floor(x0 * width); x < Math.ceil(x1 * width); x += 1) { + const value = ink[y * width + x] ?? 0; + sum += value; + max = Math.max(max, value); + count += 1; + } + } + return { mean: sum / count, max }; +} + +function inkBox({ ink, width, height }) { + let [minX, minY, maxX, maxY] = [width, height, -1, -1]; + for (let y = 0; y < height; y += 1) { + for (let x = 0; x < width; x += 1) { + if (ink[y * width + x] < INK) continue; + minX = Math.min(minX, x); + maxX = Math.max(maxX, x); + minY = Math.min(minY, y); + maxY = Math.max(maxY, y); + } + } + return maxX < 0 ? null : { x: minX, y: minY, width: maxX - minX + 1, height: maxY - minY + 1 }; +} + +/** Whether an unbroken line crosses the region to the cell edge: a row for L/R, a column for U/D. */ +function lineCrosses({ ink, width, height }, arm) { + const [x0, y0, x1, y1] = ARM_REGIONS[arm]; + const [left, top, right, bottom] = [ + Math.floor(x0 * width), + Math.floor(y0 * height), + Math.ceil(x1 * width), + Math.ceil(y1 * height), + ]; + const horizontal = arm === "L" || arm === "R"; + const [outerFrom, outerTo, innerFrom, innerTo] = horizontal + ? [top, bottom, left, right] + : [left, right, top, bottom]; + for (let outer = outerFrom; outer < outerTo; outer += 1) { + let unbroken = true; + for (let inner = innerFrom; inner < innerTo && unbroken; inner += 1) { + const [x, y] = horizontal ? [inner, outer] : [outer, inner]; + unbroken = (ink[y * width + x] ?? 0) >= 0.5; + } + if (unbroken) return true; + } + return false; +} + +/** 1 when the cell matches the character's definition, 0 otherwise. */ +function cellGlyphScore(slot, spec) { + const ink = cellInk(slot); + if (spec.arms) { + const matches = Object.keys(ARM_REGIONS).every((arm) => + spec.arms.includes(arm) ? lineCrosses(ink, arm) : regionInk(ink, ARM_REGIONS[arm]).max < 0.3, + ); + return matches ? 1 : 0; + } + if (spec.fill) { + const [filled, empty] = HALVES[spec.fill].map((region) => regionInk(ink, region).mean); + return filled >= 0.8 && empty <= 0.2 ? 1 : 0; + } + const box = inkBox(ink); + if (!box || box.width < 0.7 * ink.width || box.height < 0.5 * ink.height) return 0; + const [x0, y0] = [box.x / ink.width, box.y / ink.height]; + const [x1, y1] = [(box.x + box.width) / ink.width, (box.y + box.height) / ink.height]; + const [w, h] = [(x1 - x0) / SHADE_GRID, (y1 - y0) / SHADE_GRID]; + const regions = Array.from({ length: SHADE_GRID * SHADE_GRID }, (_, index) => { + const [gx, gy] = [index % SHADE_GRID, Math.floor(index / SHADE_GRID)]; + return [x0 + gx * w, y0 + gy * h, x0 + (gx + 1) * w, y0 + (gy + 1) * h]; + }); + const means = regions.map((region) => regionInk(ink, region).mean); + return means.every((mean) => mean >= spec.shade[0] && mean < spec.shade[1]) ? 1 : 0; +} + +/** Coverage of the ink's bounding box, centered in a square that keeps its aspect, at GRID². */ +function coverage(image) { + const box = inkBox(image); + if (!box) return null; + const side = Math.max(box.width, box.height); + const originX = box.x - (side - box.width) / 2; + const originY = box.y - (side - box.height) / 2; + const vector = new Float32Array(GRID * GRID); + const step = side / GRID; + const samples = 4; + for (let gy = 0; gy < GRID; gy += 1) { + for (let gx = 0; gx < GRID; gx += 1) { + let sum = 0; + for (let sample = 0; sample < samples * samples; sample += 1) { + const x = Math.floor(originX + (gx + ((sample % samples) + 0.5) / samples) * step); + const y = Math.floor( + originY + (gy + (Math.floor(sample / samples) + 0.5) / samples) * step, + ); + sum += + x >= 0 && y >= 0 && x < image.width && y < image.height + ? image.ink[y * image.width + x] + : 0; + } + vector[gy * GRID + gx] = sum / (samples * samples); + } + } + return vector; +} + +/** Pearson correlation of two coverage vectors; -1 when either has no ink or no contrast. */ +function correlation(a, b) { + if (!a || !b) return -1; + const n = a.length; + let [ma, mb] = [0, 0]; + for (let i = 0; i < n; i += 1) { + ma += a[i]; + mb += b[i]; + } + ma /= n; + mb /= n; + let [num, da, db] = [0, 0, 0]; + for (let i = 0; i < n; i += 1) { + num += (a[i] - ma) * (b[i] - mb); + da += (a[i] - ma) ** 2; + db += (b[i] - mb) ** 2; + } + return da === 0 || db === 0 ? -1 : num / Math.sqrt(da * db); +} + +const references = new Map(); +/** + * The cluster alone and with a zero-width joiner before, after and on both sides, so any Arabic + * positional form of the right letter matches. + */ +function referenceVectors(chars) { + if (!references.has(chars)) { + const forms = [chars, ZWJ + chars, chars + ZWJ, ZWJ + chars + ZWJ]; + references.set( + chars, + forms.map((text) => coverage(referenceInk(text, columnsOf.get(chars)))), + ); + } + return references.get(chars); +} + +function shapeScore(vector, chars) { + return Math.max(...referenceVectors(chars).map((reference) => correlation(vector, reference))); +} + +/** + * pango draws no ink for Apple Color Emoji, and draws flags on neither device. Those clusters have + * no reference and are reported instead of judged. + */ +const hasReference = (chars) => + CELL_GLYPHS[chars] || + (!/^\p{Regional_Indicator}{2}$/u.test(chars) && referenceVectors(chars).some(Boolean)); +const judged = printed.filter((slot) => hasReference(slot.chars)); +const unjudged = printed + .filter((slot) => !hasReference(slot.chars)) + .map((slot) => ({ + line: slot.line, + col: slot.col, + chars: slot.chars, + reason: "no reference ink", + })); + +const distinct = [...new Set(judged.map((slot) => slot.chars))]; +const shapes = distinct.filter((chars) => !CELL_GLYPHS[chars]); + +/** The cluster a slot is judged against: its own, or under `swap` a different one. */ +function expectedChars(slot, index) { + if (control !== "swap") return slot.chars; + const own = distinct.indexOf(slot.chars); + const other = distinct[(own + 1 + (index % (distinct.length - 1))) % distinct.length]; + return other === slot.chars ? distinct[(own + 1) % distinct.length] : other; +} + +function judge(slot, index) { + const expected = expectedChars(slot, index); + const result = { line: slot.line, row: slot.row, col: slot.col, chars: slot.chars, expected }; + const spec = CELL_GLYPHS[expected]; + if (spec) { + result.score = cellGlyphScore(slot, spec); + return result; + } + // The shape must also resemble its own cluster at least as much as any other cluster in the + // corpus, within MARGIN, so a near-lookalike such as g judged as 8 fails. + const vector = coverage(screenshotInk(slot.box)); + const score = shapeScore(vector, expected); + let nearest = { chars: expected, score }; + for (const other of shapes) { + if (other === expected) continue; + const otherScore = shapeScore(vector, other); + if (otherScore > nearest.score) nearest = { chars: other, score: otherScore }; + } + const distinctive = nearest.chars === expected || nearest.score - score <= MARGIN; + result.score = distinctive ? score : Math.min(score, 0); + result.nearest = nearest; + return result; +} + +const results = judged.map(judge); +rmSync(scratch, { recursive: true, force: true }); + +const passed = results.filter((result) => result.score >= THRESHOLD); +const failed = results.filter((result) => result.score < THRESHOLD); +const outDir = resolve(dir, "identity"); +mkdirSync(outDir, { recursive: true }); +writeFileSync( + resolve(outDir, `${control}.json`), + `${JSON.stringify({ control, threshold: THRESHOLD, unjudged, results }, null, 2)}\n`, +); + +const format = (result) => + `${result.line}@${result.col}(${result.chars}${result.expected === result.chars ? "" : `→${result.expected}`})=${result.score.toFixed(2)}${result.nearest && result.nearest.chars !== result.expected ? ` nearest ${result.nearest.chars}` : ""}`; +console.log( + `${report.label} ${control}: ${results.length} clusters judged, ${passed.length} pass, ${failed.length} fail, ${unjudged.length} unjudged (${outDir})`, +); +if (control === "none") { + for (const result of failed) console.log(` fail ${format(result)}`); + process.exitCode = failed.length === 0 ? 0 : 1; +} else { + for (const result of passed) console.log(` accepted ${format(result)}`); + process.exitCode = passed.length === 0 ? 0 : 1; +} diff --git a/packages/app/e2e/mobile/terminal-rendering/overhang.mjs b/packages/app/e2e/mobile/terminal-rendering/overhang.mjs new file mode 100644 index 0000000000..da977d78d3 --- /dev/null +++ b/packages/app/e2e/mobile/terminal-rendering/overhang.mjs @@ -0,0 +1,236 @@ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { homedir, tmpdir } from "node:os"; +import { resolve } from "node:path"; +import { setTimeout } from "node:timers/promises"; +import { PNG } from "pngjs"; +import { countInk, measureGrid } from "./check.mjs"; +import { RULER_CELLS } from "./fixture.mjs"; + +// Checks ink that leaves its row: stacked marks above and below a row reach into the rows next to +// it. The script prints such rows, then in one update writes into two rows that hold another row's +// overhang and erases two rows whose overhang reaches a blank row. The overhang must be drawn +// before the update, stay drawn in the written rows, and leave no ink behind the erased rows. +// +// Preconditions are the same as `render.mjs`: the app runs the build being verified against a +// daemon on the same host as this script, and shows an idle, focused terminal with an empty prompt. +// +// PLATFORM=android ANDROID_SERIAL=emulator-5554 node e2e/mobile/terminal-rendering/overhang.mjs +// PLATFORM=ios IOS_DEVICE="iPhone 17" node e2e/mobile/terminal-rendering/overhang.mjs +const platform = process.env.PLATFORM ?? "android"; +const appId = process.env.PASEO_TERMINAL_RENDERING_APP_ID ?? "sh.paseo.debug"; +const label = process.env.PASEO_TERMINAL_RENDERING_LABEL ?? "overhang"; +const session = process.env.AGENT_DEVICE_SESSION ?? `terminal-rendering-${platform}`; +const artifacts = resolve( + process.env.PASEO_TERMINAL_RENDERING_ARTIFACTS ?? + resolve(homedir(), ".paseo/artifacts/terminal-rendering", platform), + label, +); +const fixtureDir = process.env.PASEO_TERMINAL_RENDERING_FIXTURE_DIR ?? tmpdir(); +const screenshotArgs = + platform === "ios" + ? ["--pixel-density", process.env.PASEO_TERMINAL_RENDERING_PIXEL_DENSITY ?? "3"] + : []; +const env = { + ...process.env, + AGENT_DEVICE_STATE_DIR: + process.env.AGENT_DEVICE_STATE_DIR ?? + resolve(homedir(), ".paseo/artifacts/terminal-rendering/device-state"), +}; +mkdirSync(artifacts, { recursive: true }); + +/** Marks stacked below the row: Arabic dots, rings below `g`, a vowel sign below a consonant. */ +const BELOW = "پپپ ييي ججج g̥̥̥̥ कृ"; +/** Marks stacked above the row: accents over `e` and `s`, candrabindu, om. */ +const ABOVE = "é́́́ ŝ̂̂ कँ ॐ"; +/** The marks rows stay left of this column; the update writes at it, outside the checked boxes. */ +const CHECKED_COLUMNS = 26; +const WRITE_COLUMN = 30; +/** 0-based rows. Each source row's overhang reaches the row next to it. */ +const SOURCES = [ + { row: 2, text: BELOW, overhangRow: 3, update: "write" }, + { row: 6, text: ABOVE, overhangRow: 5, update: "erase" }, + { row: 9, text: BELOW, overhangRow: 10, update: "erase" }, + { row: 13, text: ABOVE, overhangRow: 12, update: "write" }, +]; +const PARK_ROW = 16; + +function ad(...args) { + return execFileSync("agent-device", [...args, "--session", session], { + env, + encoding: "utf8", + timeout: 60_000, + }); +} + +function deviceArgs() { + if (platform === "android") { + return [ + "--platform", + "android", + ...(process.env.ANDROID_SERIAL ? ["--serial", process.env.ANDROID_SERIAL] : []), + ]; + } + return [ + "--platform", + "ios", + ...(process.env.IOS_DEVICE ? ["--device", process.env.IOS_DEVICE] : []), + ]; +} + +function snapshotNodes() { + return JSON.parse(ad("snapshot", "--json")).data.nodes; +} + +function pressControl(id) { + const node = JSON.parse(ad("snapshot", "-i", "--json")).data.nodes.find( + (candidate) => candidate.identifier === id, + ); + assert.ok(node?.rect, `No ${id} control on screen`); + const { x, y, width, height } = node.rect; + ad("press", String(Math.round(x + width / 2)), String(Math.round(y + height / 2))); +} + +async function pressEnter() { + pressControl("terminal-key-enter"); + await setTimeout(2_000); +} + +async function runCommand(command) { + ad("type", command, "--delay-ms", "0"); + await pressEnter(); +} + +async function terminalSize() { + await runCommand("clear; stty size"); + for (const node of snapshotNodes()) { + const match = /^(\d+) (\d+)$/.exec((node.label ?? "").trim()); + if (match && (node.identifier ?? "").startsWith("terminal-row-")) { + return { rows: Number(match[1]), columns: Number(match[2]) }; + } + } + throw new Error("No terminal row showed the `stty size` output"); +} + +function shellQuote(text) { + return `'${text.replaceAll("'", "'\\''")}'`; +} + +/** Prints the source rows, waits for Enter, applies the update, and waits for Enter again. */ +function fixtureScript() { + const at = (row, col) => `\\033[${row + 1};${col + 1}H`; + const print = (row, text) => `printf '${at(row, 0)}%s' ${shellQuote(text)}`; + const update = SOURCES.map((source) => + source.update === "write" + ? `${at(source.overhangRow, WRITE_COLUMN)}x` + : `${at(source.row, 0)}\\033[2K`, + ).join(""); + return [ + "stty -echo", + "printf '\\033[2J\\033[3J'", + print(0, "█".repeat(RULER_CELLS)), + ...SOURCES.map((source) => print(source.row, source.text)), + `printf '${at(PARK_ROW, 0)}'`, + "read -r _", + `printf '${update}${at(PARK_ROW, 0)}'`, + "read -r _", + "stty echo", + "", + ].join("\n"); +} + +function screenshot(name) { + const path = resolve(artifacts, name); + ad("screenshot", path, ...screenshotArgs); + return PNG.sync.read(readFileSync(path)); +} + +/** Pixels in `box` where the two screenshots differ by more than the ink threshold. */ +function changedPixels(before, after, box) { + const difference = new PNG({ width: before.width, height: before.height }); + for (let index = 0; index < before.data.length; index += 1) { + difference.data[index] = Math.abs(before.data[index] - after.data[index]); + } + return countInk(difference, box, [0, 0, 0]); +} + +ad("open", appId, ...deviceArgs()); +const controls = JSON.parse(ad("snapshot", "-i", "--json")).data.nodes; +const hasControl = (id) => controls.some((node) => node.identifier === id); +assert.ok(hasControl("terminal-keyboard-toggle"), "Open a terminal first"); +if (!hasControl("terminal-key-enter")) pressControl("terminal-keyboard-toggle"); + +const size = await terminalSize(); +assert.ok( + size.rows > PARK_ROW && size.columns > WRITE_COLUMN, + `Terminal too small: ${JSON.stringify(size)}`, +); +const scriptPath = resolve(fixtureDir, "paseo-terminal-overhang.sh"); +writeFileSync(scriptPath, fixtureScript()); +await runCommand(`sh ${scriptPath}`); + +const nodes = snapshotNodes(); +const before = screenshot("before.png"); +await pressEnter(); +const after = screenshot("after.png"); +await pressEnter(); + +// Snapshot rects are in points on iOS and pixels on Android; the checks work in screenshot pixels. +const scale = before.width / Math.max(...nodes.map((node) => node.rect?.width ?? 0)); +const ruler = nodes.find((node) => node.identifier === "terminal-row-0")?.rect; +assert.ok(ruler, "The ruler row is not exposed as terminal-row-0"); +const rulerRect = { + x: ruler.x * scale, + y: ruler.y * scale, + width: ruler.width * scale, + height: ruler.height * scale, +}; +const grid = measureGrid(before, rulerRect); +const rowBox = (row) => ({ + left: grid.left, + right: grid.left + CHECKED_COLUMNS * grid.cellWidth, + top: rulerRect.y + row * rulerRect.height, + bottom: rulerRect.y + (row + 1) * rulerRect.height, +}); + +/** Measures one source row's overhang before and after the update. */ +function measure(source) { + const box = rowBox(source.overhangRow); + const overhangBefore = countInk(before, box, grid.background); + if (source.update === "write") { + const changedAfterWrite = changedPixels(before, after, box); + const failures = [ + ...(overhangBefore === 0 ? ["overhang-not-drawn"] : []), + ...(changedAfterWrite > 0 ? ["overhang-lost-after-write"] : []), + ]; + return { ...source, overhangBefore, changedAfterWrite, failures }; + } + const inkAfterErase = countInk(after, box, grid.background); + const sourceInkAfterErase = countInk(after, rowBox(source.row), grid.background); + const failures = [ + ...(overhangBefore === 0 ? ["overhang-not-drawn"] : []), + ...(inkAfterErase > 0 ? ["overhang-left-after-erase"] : []), + ...(sourceInkAfterErase > 0 ? ["source-left-after-erase"] : []), + ]; + return { ...source, overhangBefore, inkAfterErase, sourceInkAfterErase, failures }; +} + +const results = SOURCES.map(measure); + +const report = { platform, label, size, grid, rulerRect, results }; +writeFileSync(resolve(artifacts, "report.json"), `${JSON.stringify(report, null, 2)}\n`); +const failures = results.flatMap((result) => + result.failures.map((failure) => `row ${result.overhangRow}: ${failure}`), +); +console.log(`${label}: ${failures.length} failures (${artifacts})`); +for (const result of results) { + console.log( + ` row ${result.row} -> ${result.overhangRow} (${result.update}): overhang ${result.overhangBefore}` + + (result.update === "write" + ? `, changed after write ${result.changedAfterWrite}` + : `, left after erase ${result.inkAfterErase}, source left ${result.sourceInkAfterErase}`), + ); +} +for (const failure of failures) console.log(` ${failure}`); +process.exitCode = failures.length === 0 ? 0 : 1; diff --git a/packages/app/e2e/mobile/terminal-rendering/reattach.mjs b/packages/app/e2e/mobile/terminal-rendering/reattach.mjs new file mode 100644 index 0000000000..c5cb7734e9 --- /dev/null +++ b/packages/app/e2e/mobile/terminal-rendering/reattach.mjs @@ -0,0 +1,149 @@ +import { homedir } from "node:os"; +import { execFileSync } from "node:child_process"; +import { mkdirSync, writeFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { setTimeout } from "node:timers/promises"; + +// Reattaches to a terminal whose PTY has a different size than the app will measure, and checks +// that the client rows equal the daemon's rows. RESIZE=keyboard opens the keyboard, and the +// relaunched app starts with it closed. RESIZE=fold folds an Android foldable emulator and +// relaunches the app unfolded. Either way the shell redraws its prompt on attach. +// +// Preconditions: the app shows terminal TERMINAL_ID, idle at an empty shell prompt, against the +// daemon at PASEO_HOST. +// +// PLATFORM=android TERMINAL_ID=511fd9f0 PASEO_HOST=localhost:6870 \ +// node e2e/mobile/terminal-rendering/reattach.mjs +// +// The client used to be sized after the replay arrived, so the shell's redraw landed on the +// wrong rows in most trials. TRIALS sets how many reattaches to check. +const platform = process.env.PLATFORM ?? "android"; +const appId = process.env.PASEO_TERMINAL_RENDERING_APP_ID ?? "sh.paseo.debug"; +const appUrl = process.env.PASEO_TERMINAL_RENDERING_APP_URL; +const terminalId = process.env.TERMINAL_ID; +const host = process.env.PASEO_HOST; +const trials = Number(process.env.TRIALS ?? 5); +const resize = process.env.RESIZE ?? "keyboard"; +const label = process.env.PASEO_TERMINAL_RENDERING_LABEL ?? "reattach"; +const session = process.env.AGENT_DEVICE_SESSION ?? `terminal-rendering-${platform}`; +const artifacts = resolve( + process.env.PASEO_TERMINAL_RENDERING_ARTIFACTS ?? + resolve(homedir(), ".paseo/artifacts/terminal-rendering", platform), + label, +); +const env = { + ...process.env, + AGENT_DEVICE_STATE_DIR: + process.env.AGENT_DEVICE_STATE_DIR ?? + resolve(homedir(), ".paseo/artifacts/terminal-rendering/device-state"), +}; +if (!terminalId || !host) throw new Error("Set TERMINAL_ID and PASEO_HOST"); +mkdirSync(artifacts, { recursive: true }); + +function ad(...args) { + return execFileSync("agent-device", [...args, "--session", session], { + env, + encoding: "utf8", + timeout: 120_000, + }); +} + +function paseo(...args) { + return execFileSync("npm", ["run", "-s", "cli", "--", ...args, "--host", host], { + cwd: resolve(import.meta.dirname, "../../../../.."), + encoding: "utf8", + }); +} + +function pressControl(id) { + const node = JSON.parse(ad("snapshot", "-i", "--json")).data.nodes.find( + (candidate) => candidate.identifier === id, + ); + if (!node?.rect) throw new Error(`No ${id} control on screen`); + const { x, y, width, height } = node.rect; + ad("press", String(Math.round(x + width / 2)), String(Math.round(y + height / 2))); +} + +async function resizePty() { + if (resize === "keyboard") { + pressControl("terminal-keyboard-toggle"); + await setTimeout(2_000); + return; + } + const adb = (...args) => + execFileSync("adb", [ + ...(process.env.ANDROID_SERIAL ? ["-s", process.env.ANDROID_SERIAL] : []), + ...args, + ]); + adb("emu", "fold"); + await setTimeout(5_000); + adb("shell", "am", "force-stop", appId); + adb("emu", "unfold"); + await setTimeout(3_000); +} + +function clientRows() { + const nodes = JSON.parse(ad("snapshot", "--json")).data.nodes; + const byIndex = new Map(nodes.map((node) => [node.index, node])); + const rows = []; + for (const node of nodes) { + const match = /^terminal-row-(\d+)$/.exec(node.identifier ?? ""); + if (!match) continue; + // agent-device leaves the label on the parent when the row inherits it. + const text = node.label ?? (node.inheritsLabel ? byIndex.get(node.parentIndex)?.label : ""); + rows[Number(match[1])] = (text ?? "").trim(); + } + return rows; +} + +/** + * Waits for the relaunched app to show terminal rows, then for the shell's redraw to land. + * Returns null when no row appears, for example when the app crashed on launch. + */ +async function attachedClientRows() { + for (let attempt = 0; attempt < 30; attempt += 1) { + await setTimeout(2_000); + if (clientRows().some((text) => text !== "")) { + await setTimeout(3_000); + return clientRows(); + } + } + return null; +} + +function mismatches(client, server) { + const rows = []; + for (let row = 0; row < Math.max(client.length, server.length); row += 1) { + const expected = (server[row] ?? "").trim(); + const actual = client[row] ?? ""; + if (expected !== actual) rows.push({ row, server: expected, client: actual }); + } + return rows; +} + +const results = []; +for (let trial = 1; trial <= trials; trial += 1) { + paseo("terminal", "send-keys", terminalId, "clear", "Enter"); + await setTimeout(1_500); + await resizePty(); + ad("open", appId, ...(appUrl ? [appUrl] : []), "--relaunch"); + const client = await attachedClientRows(); + ad("screenshot", resolve(artifacts, `trial-${trial}.png`)); + if (!client) { + results.push({ trial, attached: false }); + console.log(`${label} trial ${trial}: no terminal rows appeared`); + continue; + } + const server = paseo("terminal", "capture", terminalId).split("\n"); + const rows = mismatches(client, server); + results.push({ trial, attached: true, mismatchedRows: rows }); + console.log(`${label} trial ${trial}: ${rows.length} mismatched rows`); +} + +writeFileSync(resolve(artifacts, "report.json"), `${JSON.stringify(results, null, 2)}\n`); +const attached = results.filter((result) => result.attached); +const failed = attached.filter((result) => result.mismatchedRows.length > 0).length; +console.log( + `${label}: ${failed} of ${attached.length} reattaches differ from the daemon, ${trials - attached.length} never attached (${artifacts})`, +); +process.exitCode = failed === 0 && attached.length === trials ? 0 : 1; diff --git a/packages/app/e2e/mobile/terminal-rendering/render.mjs b/packages/app/e2e/mobile/terminal-rendering/render.mjs new file mode 100644 index 0000000000..7aacfcaa8e --- /dev/null +++ b/packages/app/e2e/mobile/terminal-rendering/render.mjs @@ -0,0 +1,197 @@ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; +import { homedir, tmpdir } from "node:os"; +import { resolve } from "node:path"; +import { setTimeout } from "node:timers/promises"; +import { PNG } from "pngjs"; +import { + checkPartialSelection, + checkTerminalScreenshot, + selectionTarget, + writeOverlay, +} from "./check.mjs"; +import { fixtureOutput } from "./fixture.mjs"; + +// Renders the fixture in a real Paseo terminal and checks the screenshot with `check.mjs`, then +// selects one cell of a conjunct and checks a second screenshot. +// +// Preconditions: the app runs the build being verified against a daemon on the same host as +// this script, and shows an idle, focused terminal with an empty shell prompt. +// +// PLATFORM=android ANDROID_SERIAL=emulator-5554 node e2e/mobile/terminal-rendering/render.mjs +// PLATFORM=ios IOS_DEVICE="iPhone 17" node e2e/mobile/terminal-rendering/render.mjs +const platform = process.env.PLATFORM ?? "android"; +const appId = process.env.PASEO_TERMINAL_RENDERING_APP_ID ?? "sh.paseo.debug"; +const label = process.env.PASEO_TERMINAL_RENDERING_LABEL ?? "run"; +const session = process.env.AGENT_DEVICE_SESSION ?? `terminal-rendering-${platform}`; +const artifacts = resolve( + process.env.PASEO_TERMINAL_RENDERING_ARTIFACTS ?? + resolve(homedir(), ".paseo/artifacts/terminal-rendering", platform), + label, +); +const fixtureDir = process.env.PASEO_TERMINAL_RENDERING_FIXTURE_DIR ?? tmpdir(); +// iOS simulators screenshot at one pixel per point unless asked; glyph identity needs the +// device's real density. Android screenshots are always in device pixels. +const screenshotArgs = + platform === "ios" + ? ["--pixel-density", process.env.PASEO_TERMINAL_RENDERING_PIXEL_DENSITY ?? "3"] + : []; +const env = { + ...process.env, + AGENT_DEVICE_STATE_DIR: + process.env.AGENT_DEVICE_STATE_DIR ?? + resolve(homedir(), ".paseo/artifacts/terminal-rendering/device-state"), +}; +mkdirSync(artifacts, { recursive: true }); + +function ad(...args) { + return execFileSync("agent-device", [...args, "--session", session], { + env, + encoding: "utf8", + timeout: 60_000, + }); +} + +function deviceArgs() { + if (platform === "android") { + return [ + "--platform", + "android", + ...(process.env.ANDROID_SERIAL ? ["--serial", process.env.ANDROID_SERIAL] : []), + ]; + } + return [ + "--platform", + "ios", + ...(process.env.IOS_DEVICE ? ["--device", process.env.IOS_DEVICE] : []), + ]; +} + +function snapshotNodes() { + return JSON.parse(ad("snapshot", "--json")).data.nodes; +} + +function rowNodes(nodes) { + const rows = new Map(); + for (const node of nodes) { + const match = /^terminal-row-(\d+)$/.exec(node.identifier ?? ""); + if (match && node.rect) rows.set(Number(match[1]), node); + } + return rows; +} + +/** + * Taps a control at its center. On iOS the keyboard toolbar is a full-screen accessibility node + * listed after the key bar, so pressing by selector reports the keys as covered. + */ +function pressControl(id) { + const node = JSON.parse(ad("snapshot", "-i", "--json")).data.nodes.find( + (candidate) => candidate.identifier === id, + ); + assert.ok(node?.rect, `No ${id} control on screen`); + const { x, y, width, height } = node.rect; + ad("press", String(Math.round(x + width / 2)), String(Math.round(y + height / 2))); +} + +async function runCommand(command) { + ad("type", command, "--delay-ms", "0"); + pressControl("terminal-key-enter"); + await setTimeout(2_000); +} + +async function terminalSize() { + await runCommand("clear; stty size"); + for (const node of snapshotNodes()) { + const match = /^(\d+) (\d+)$/.exec((node.label ?? "").trim()); + if (match && (node.identifier ?? "").startsWith("terminal-row-")) { + return { rows: Number(match[1]), columns: Number(match[2]) }; + } + } + throw new Error("No terminal row showed the `stty size` output"); +} + +ad("open", appId, ...deviceArgs()); +const controls = JSON.parse(ad("snapshot", "-i", "--json")).data.nodes; +const hasControl = (id) => controls.some((node) => node.identifier === id); +assert.ok(hasControl("terminal-keyboard-toggle"), "Open a terminal first"); +if (!hasControl("terminal-key-enter")) pressControl("terminal-keyboard-toggle"); + +const size = await terminalSize(); +const fixturePath = resolve(fixtureDir, "paseo-terminal-rendering.txt"); +const scriptPath = resolve(fixtureDir, "paseo-terminal-rendering.sh"); +writeFileSync(fixturePath, fixtureOutput(size.columns)); +writeFileSync(scriptPath, `printf '\\033[2J\\033[3J\\033[H'; cat '${fixturePath}'; sleep 120\n`); +await runCommand(`sh ${scriptPath}`); + +const nodes = snapshotNodes(); +const screenshotPath = resolve(artifacts, "terminal.png"); +ad("screenshot", screenshotPath, ...screenshotArgs); +const png = PNG.sync.read(readFileSync(screenshotPath)); +// Snapshot rects are in points on iOS and pixels on Android; the checks work in screenshot pixels. +const scale = png.width / Math.max(...nodes.map((node) => node.rect?.width ?? 0)); +const toPixels = ({ x, y, width, height }) => ({ + x: x * scale, + y: y * scale, + width: width * scale, + height: height * scale, +}); +const rows = rowNodes(nodes); +const rects = new Map([...rows].map(([index, node]) => [index, toPixels(node.rect)])); +const { grid, failures, slots } = await checkTerminalScreenshot({ png, size, rows: rects }); + +// Long-press selects the conjunct as a word; dragging back from its second cell leaves the first. +const target = await selectionTarget({ size }); +const targetRect = rects.get(target.row); +assert.ok(targetRect, "The select line is not on screen"); +const y = Math.round((targetRect.y + targetRect.height / 2) / scale); +const cellX = (offset) => Math.round((grid.left + (target.col + offset) * grid.cellWidth) / scale); +ad("longpress", String(cellX(0.5)), String(y), "800"); +ad("gesture", "pan", String(cellX(1.75)), String(y), String(cellX(0.5) - cellX(1.75)), "0", "600"); +const selectionPath = resolve(artifacts, "selection.png"); +ad("screenshot", selectionPath, ...screenshotArgs); +const selectionFailures = checkPartialSelection({ + png: PNG.sync.read(readFileSync(selectionPath)), + rows: rects, + grid, + target, +}); +for (const failure of selectionFailures) failures.push({ line: "select", ...failure }); +// A tap clears the selection; it would otherwise survive into the next run's screenshot. +const lastRect = rects.get(size.rows - 1); +if (lastRect) { + ad("press", String(cellX(4)), String(Math.round((lastRect.y + lastRect.height / 2) / scale))); +} +try { + pressControl("terminal-key-ctrl"); + ad("type", "c", "--delay-ms", "0"); +} catch (error) { + console.warn(`Could not interrupt the fixture: ${error.message}`); +} + +writeOverlay(png, slots, resolve(artifacts, "overlay.png")); +const report = { + platform, + label, + size, + grid, + checkedClusters: slots.length, + failures, + rows: [...rows.keys()].map((index) => ({ + index, + label: rows.get(index).label, + rect: rects.get(index), + })), +}; +writeFileSync(resolve(artifacts, "report.json"), `${JSON.stringify(report, null, 2)}\n`); + +const byLine = Map.groupBy(failures, (failure) => failure.line); +console.log( + `${label}: ${slots.length} clusters checked, ${failures.length} failures (${artifacts})`, +); +for (const [line, list] of byLine) { + console.log( + ` ${line}: ${list.map((failure) => `${failure.kind}@${failure.col ?? "-"}${failure.char ? `(${failure.char})` : ""}`).join(" ")}`, + ); +} +process.exitCode = failures.length === 0 ? 0 : 1; diff --git a/packages/app/modules/paseo-terminal/android/build.gradle b/packages/app/modules/paseo-terminal/android/build.gradle new file mode 100644 index 0000000000..3be89253be --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/build.gradle @@ -0,0 +1,25 @@ +apply plugin: 'com.android.library' + +group = 'sh.paseo.terminal' +version = '0.1.0' + +def expoModulesCorePlugin = new File(project(":expo-modules-core").projectDir.absolutePath, "ExpoModulesCorePlugin.gradle") +apply from: expoModulesCorePlugin +applyKotlinExpoModulesCorePlugin() +useCoreDependencies() +useExpoPublishing() +useDefaultAndroidSdkVersions() + +android { + namespace "sh.paseo.terminal" + defaultConfig { + testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner" + } +} + +dependencies { + implementation "androidx.customview:customview:1.1.0" + testImplementation "junit:junit:4.13.2" + androidTestImplementation "androidx.test:runner:1.6.2" + androidTestImplementation "androidx.test.ext:junit:1.2.1" +} diff --git a/packages/app/modules/paseo-terminal/android/src/androidTest/assets/grapheme-rows.json b/packages/app/modules/paseo-terminal/android/src/androidTest/assets/grapheme-rows.json new file mode 100644 index 0000000000..31c2b0c8d7 --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/androidTest/assets/grapheme-rows.json @@ -0,0 +1,38 @@ +[ + { + "output": "स्ते", + "row": "2,1,2,1,1,1,1,1,1,1,1,1,1,1,1,1;8,4294638330,0,0;स्ते ", + "clusterStarts": [0, 2, 3, 4, 5, 6, 7], + "contrastStarts": [0, 1, 2, 3, 4, 5, 6, 7] + }, + { + "output": "क्ष", + "row": "2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1;8,4294638330,0,0;क्ष ", + "clusterStarts": [0, 2, 3, 4, 5, 6, 7], + "contrastStarts": [0, 1, 2, 3, 4, 5, 6, 7] + }, + { + "output": "নক্ষত্র", + "row": "1,1,2,1,1,1,2,1,1,1,1,1,1,1,1,1;8,4294638330,0,0;নক্ষত্র ", + "clusterStarts": [0, 1, 3, 5, 6, 7], + "contrastStarts": [0, 1, 2, 3, 4, 5, 6, 7] + }, + { + "output": "क्अ", + "row": "2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1;8,4294638330,0,0;क्अ ", + "clusterStarts": [0, 1, 2, 3, 4, 5, 6, 7], + "contrastStarts": [0, 2, 3, 4, 5, 6, 7] + }, + { + "output": "ক্অ", + "row": "2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1;8,4294638330,0,0;ক্অ ", + "clusterStarts": [0, 1, 2, 3, 4, 5, 6, 7], + "contrastStarts": [0, 2, 3, 4, 5, 6, 7] + }, + { + "output": "अ्क", + "row": "2,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1;8,4294638330,0,0;अ्क ", + "clusterStarts": [0, 1, 2, 3, 4, 5, 6, 7], + "contrastStarts": [0, 2, 3, 4, 5, 6, 7] + } +] diff --git a/packages/app/modules/paseo-terminal/android/src/androidTest/java/sh/paseo/terminal/TerminalRowLayoutTest.kt b/packages/app/modules/paseo-terminal/android/src/androidTest/java/sh/paseo/terminal/TerminalRowLayoutTest.kt new file mode 100644 index 0000000000..e9b8ab1edf --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/androidTest/java/sh/paseo/terminal/TerminalRowLayoutTest.kt @@ -0,0 +1,105 @@ +package sh.paseo.terminal + +import android.os.Build +import android.graphics.Bitmap +import android.graphics.Canvas +import android.graphics.Paint +import android.util.Log +import androidx.test.ext.junit.runners.AndroidJUnit4 +import androidx.test.platform.app.InstrumentationRegistry +import org.json.JSONArray +import org.json.JSONObject +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith + +private const val COLUMNS = 8 + +/** Rasters that draw the same clusters match exactly; a wrong clustering differs by 0.12 or more. */ +private const val MATCH_TOLERANCE = 0.01f + +@RunWith(AndroidJUnit4::class) +class TerminalRowLayoutTest { + private val font = TerminalFont("monospace", 40f, 1f) + private val paints = TerminalPaints() + + /** + * Draws each row of `grapheme-rows.json` and compares it with its grapheme clusters drawn one at a + * time, each centred in its own columns. `terminal-frame.test.ts` checks that the rows are what + * xterm produces and that the clusters are UAX #29 grapheme clusters. + */ + @Test + fun drawsEachGraphemeClusterInItsOwnColumns() { + val assets = InstrumentationRegistry.getInstrumentation().context.assets + val rows = JSONArray(assets.open("grapheme-rows.json").bufferedReader().readText()) + for (index in 0 until rows.length()) { + val row = rows.getJSONObject(index) + val source = TerminalRowSource.decode(row.getString("row")) + val layout = TerminalRowLayout(source, font, TerminalCustomGlyphs.EMPTY) + // Selection covers row 0 only, so drawing as row 1 draws nothing selected. + val drawn = raster { layout.drawSelected(it, 1, TerminalSelection(TerminalSelectionRecord()), paints) } + val expected = raster { drawClusters(it, source, starts(row, "clusterStarts")) } + val contrast = raster { drawClusters(it, source, starts(row, "contrastStarts")) } + val output = row.getString("output") + val (toExpected, toContrast) = difference(drawn, expected) to difference(drawn, contrast) + Log.i("TerminalRowLayoutTest", "$output expected=$toExpected contrast=$toContrast") + // Android 10's platform ICU splits this conjunct. Keep its observed raster baseline + // explicit rather than claiming this supported OS passes the modern grapheme corpus. + if (Build.VERSION.SDK_INT == 29 && output == "स्ते") { + assertTrue("Android 10 unexpectedly matches modern conjunct clustering", toExpected > MATCH_TOLERANCE) + assertTrue("Android 10 split-conjunct baseline changed", toContrast <= MATCH_TOLERANCE) + continue + } + assertTrue("$output differs from its grapheme clusters by $toExpected", toExpected <= MATCH_TOLERANCE) + assertTrue("$output also matches the wrong clusters", toContrast > MATCH_TOLERANCE) + } + } + + private fun starts(row: JSONObject, key: String): List = + row.getJSONArray(key).let { starts -> List(starts.length()) { starts.getInt(it) } } + + private fun raster(draw: (Canvas) -> Unit): Bitmap { + val bitmap = Bitmap.createBitmap(COLUMNS * font.cellWidthPx, font.cellHeightPx, Bitmap.Config.ARGB_8888) + draw(Canvas(bitmap)) + return bitmap + } + + /** Draws each cluster as its own text, centred in its columns and scaled down when wider. */ + private fun drawClusters(canvas: Canvas, source: TerminalRowSource, starts: List) { + val cellCount = source.cells.size / 2 + val paint = Paint(font.paint(0)).apply { color = source.styles[1].toLong().toInt() } + val offsets = IntArray(cellCount + 1) + val cols = IntArray(cellCount + 1) + for (cell in 0 until cellCount) { + offsets[cell + 1] = offsets[cell] + source.cells[cell * 2] + cols[cell + 1] = cols[cell] + source.cells[cell * 2 + 1] + } + for ((index, first) in starts.withIndex()) { + val end = starts.getOrElse(index + 1) { cellCount } + val text = source.text.substring(offsets[first], offsets[end]) + if (text.isBlank()) continue + val slot = (cols[end] - cols[first]) * font.cellWidthPx.toFloat() + val advance = paint.measureText(text) + val x = cols[first] * font.cellWidthPx + if (advance > slot + 1f) 0f else (slot - advance) / 2f + canvas.save() + if (advance > slot + 1f) canvas.scale(slot / advance, slot / advance, x, font.cellHeightPx / 2f) + canvas.drawText(text, x, font.baselinePx.toFloat(), paint) + canvas.restore() + } + } + + /** Share of inked pixels, in either image, whose coverage differs by more than a quarter. */ + private fun difference(a: Bitmap, b: Bitmap): Float { + var inked = 0 + var differing = 0 + for (y in 0 until a.height) { + for (x in 0 until a.width) { + val (left, right) = (a.getPixel(x, y) ushr 24) to (b.getPixel(x, y) ushr 24) + if (left < 64 && right < 64) continue + inked += 1 + if (kotlin.math.abs(left - right) > 64) differing += 1 + } + } + return if (inked == 0) 0f else differing.toFloat() / inked + } +} diff --git a/packages/app/modules/paseo-terminal/android/src/main/AndroidManifest.xml b/packages/app/modules/paseo-terminal/android/src/main/AndroidManifest.xml new file mode 100644 index 0000000000..94cbbcfc39 --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/main/AndroidManifest.xml @@ -0,0 +1 @@ + diff --git a/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/PaseoTerminalModule.kt b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/PaseoTerminalModule.kt new file mode 100644 index 0000000000..e6250ec280 --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/PaseoTerminalModule.kt @@ -0,0 +1,24 @@ +package sh.paseo.terminal + +import expo.modules.kotlin.modules.Module +import expo.modules.kotlin.modules.ModuleDefinition + +class PaseoTerminalModule : Module() { + override fun definition() = ModuleDefinition { + Name("PaseoTerminal") + + View(PaseoTerminalView::class) { + Events("onCellMetrics") + + Prop("frame") { view: PaseoTerminalView, frame: TerminalFrameRecord -> view.setFrame(frame) } + Prop("theme") { view: PaseoTerminalView, theme: TerminalThemeRecord -> view.setTheme(theme) } + Prop("fontFamily") { view: PaseoTerminalView, family: String -> view.setFontFamily(family) } + Prop("fontSize") { view: PaseoTerminalView, size: Double -> view.setFontSize(size) } + Prop("customGlyphs") { view: PaseoTerminalView, glyphs: List -> + view.setCustomGlyphs(glyphs) + } + + OnViewDidUpdateProps { view: PaseoTerminalView -> view.commit() } + } + } +} diff --git a/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/PaseoTerminalView.kt b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/PaseoTerminalView.kt new file mode 100644 index 0000000000..3fcf012e03 --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/PaseoTerminalView.kt @@ -0,0 +1,196 @@ +package sh.paseo.terminal + +import android.annotation.SuppressLint +import android.content.Context +import android.graphics.Canvas +import android.graphics.Color +import android.graphics.Rect +import android.os.Bundle +import android.view.MotionEvent +import android.widget.TextView +import androidx.core.view.ViewCompat +import androidx.core.view.accessibility.AccessibilityNodeInfoCompat +import androidx.customview.widget.ExploreByTouchHelper +import expo.modules.kotlin.AppContext +import expo.modules.kotlin.viewevent.EventDispatcher +import expo.modules.kotlin.views.ExpoView + +private const val MIN_CACHED_ROWS = 256 +private const val CURSOR_ALPHA = 0.45f + +/** + * Paints terminal frames on the cell grid. Row shaping is cached by row content, so cursor and + * selection changes repaint without reshaping, and unchanged rows replay their display lists. + */ +@SuppressLint("ViewConstructor") +class PaseoTerminalView(context: Context, appContext: AppContext) : ExpoView(context, appContext) { + private val onCellMetrics by EventDispatcher>() + + private var fontFamily = "monospace" + private var fontSize = 12.0 + private var font: TerminalFont? = null + private var customGlyphs = TerminalCustomGlyphs.EMPTY + private var frame: TerminalFrameRecord? = null + private var frameChanged = false + + private var rows: List = emptyList() + private var columns = 0 + private var cursor: TerminalCursorRecord? = null + private var selection: TerminalSelection? = null + private var backgroundColor = Color.BLACK + private var cursorColor = Color.WHITE + private val paints = TerminalPaints() + private val rowCache = object : LinkedHashMap(64, 0.75f, true) { + override fun removeEldestEntry(eldest: MutableMap.MutableEntry): Boolean { + if (size <= maxOf(MIN_CACHED_ROWS, rows.size * 2)) return false + eldest.value.release() + return true + } + } + private val accessibility = RowAccessibility() + + init { + setWillNotDraw(false) + ViewCompat.setAccessibilityDelegate(this, accessibility) + } + + fun setFrame(value: TerminalFrameRecord) { + frame = value + frameChanged = true + } + + fun setTheme(value: TerminalThemeRecord) { + backgroundColor = argb(value.background) + cursorColor = argb(value.cursor) + paints.selectionBackground = argb(value.selectionBackground) + paints.selectionForeground = argb(value.selectionForeground) + invalidate() + } + + fun setFontFamily(value: String) { + fontFamily = value + } + + fun setFontSize(value: Double) { + fontSize = value + } + + fun setCustomGlyphs(value: List) { + customGlyphs = TerminalCustomGlyphs(value) + clearRowCache() + frameChanged = true + } + + /** Applies a batch of prop updates: font first, then the frame laid out with it. */ + fun commit() { + val density = resources.displayMetrics.density + val sizePx = (fontSize * density).toFloat() + val current = font + if (current == null || !current.sameAs(fontFamily, sizePx, density)) { + val next = TerminalFont(fontFamily, sizePx, density) + font = next + clearRowCache() + frameChanged = true + onCellMetrics( + mapOf( + "cellWidth" to next.cellWidthPx / density, + "cellHeight" to next.cellHeightPx / density, + ), + ) + } + if (frameChanged) { + frameChanged = false + layoutFrame() + } + invalidate() + } + + private fun layoutFrame() { + val currentFont = font ?: return + val record = frame ?: return + columns = record.columns + cursor = record.cursor + selection = record.selection?.let(::TerminalSelection) + rows = record.rows.map { encoded -> + rowCache.getOrPut(encoded) { + TerminalRowLayout(TerminalRowSource.decode(encoded), currentFont, customGlyphs) + } + } + accessibility.invalidateRoot() + } + + private fun clearRowCache() { + for (layout in rowCache.values) layout.release() + rowCache.clear() + rows = emptyList() + } + + override fun onDraw(canvas: Canvas) { + super.onDraw(canvas) + canvas.drawColor(backgroundColor) + val currentFont = font ?: return + val cellWidth = currentFont.cellWidthPx + val cellHeight = currentFont.cellHeightPx + val gridWidth = columns * cellWidth + canvas.save() + canvas.clipRect(0, 0, minOf(width, gridWidth), height) + val activeSelection = selection + for ((index, row) in rows.withIndex()) { + canvas.save() + canvas.translate(0f, (index * cellHeight).toFloat()) + if (activeSelection != null && activeSelection.intersectsRow(index)) { + row.drawSelected(canvas, index, activeSelection, paints) + } else { + row.draw(canvas, gridWidth, paints) + } + canvas.restore() + } + cursor?.let { position -> + paints.fill.color = withAlpha(cursorColor, CURSOR_ALPHA) + val left = (position.col * cellWidth).toFloat() + val top = (position.row * cellHeight).toFloat() + canvas.drawRect(left, top, left + cellWidth, top + cellHeight, paints.fill) + } + canvas.restore() + } + + override fun dispatchHoverEvent(event: MotionEvent): Boolean = + accessibility.dispatchHoverEvent(event) || super.dispatchHoverEvent(event) + + override fun onDetachedFromWindow() { + super.onDetachedFromWindow() + for (layout in rowCache.values) layout.release() + } + + private fun withAlpha(color: Int, alpha: Float): Int = + Color.argb((Color.alpha(color) * alpha).toInt(), Color.red(color), Color.green(color), Color.blue(color)) + + /** Exposes each non-blank row as a virtual text node, `terminal-row-`. */ + private inner class RowAccessibility : ExploreByTouchHelper(this@PaseoTerminalView) { + override fun getVirtualViewAt(x: Float, y: Float): Int { + val cellHeight = font?.cellHeightPx ?: return INVALID_ID + val row = (y / cellHeight).toInt() + return if (row in rows.indices && rows[row].describe().isNotEmpty()) row else INVALID_ID + } + + override fun getVisibleVirtualViews(virtualViewIds: MutableList) { + for ((index, row) in rows.withIndex()) { + if (row.describe().isNotEmpty()) virtualViewIds.add(index) + } + } + + @Suppress("DEPRECATION") + override fun onPopulateNodeForVirtualView(virtualViewId: Int, node: AccessibilityNodeInfoCompat) { + val cellHeight = font?.cellHeightPx ?: 1 + val label = rows.getOrNull(virtualViewId)?.describe().orEmpty() + node.className = TextView::class.java.name + node.text = label + node.contentDescription = label + node.viewIdResourceName = "terminal-row-$virtualViewId" + val top = virtualViewId * cellHeight + node.setBoundsInParent(Rect(0, top, minOf(width, columns * (font?.cellWidthPx ?: 1)), top + cellHeight)) + } + + override fun onPerformActionForVirtualView(virtualViewId: Int, action: Int, arguments: Bundle?): Boolean = false + } +} diff --git a/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalCustomGlyphs.kt b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalCustomGlyphs.kt new file mode 100644 index 0000000000..ce1e8d5229 --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalCustomGlyphs.kt @@ -0,0 +1,76 @@ +package sh.paseo.terminal + +import android.graphics.Canvas +import android.graphics.Paint +import android.graphics.Path + +/** + * Box drawing and block elements painted as cell geometry instead of font glyphs, so frames and + * blocks join across cells regardless of the font's own metrics. + */ +internal class TerminalCustomGlyph(record: TerminalCustomGlyphRecord) { + private val rects: FloatArray = FloatArray(record.rects.size) { record.rects[it].toFloat() } + private val commands: DoubleArray = record.path + private val strokeWidth: Float = record.strokeWidth.toFloat() + private var scaledPath: Path? = null + private var scaledWidth = 0f + private var scaledHeight = 0f + + fun draw(canvas: Canvas, x: Float, width: Float, height: Float, density: Float, paint: Paint) { + paint.style = Paint.Style.FILL + var index = 0 + while (index + 3 < rects.size) { + canvas.drawRect( + x + rects[index] * width, + rects[index + 1] * height, + x + (rects[index] + rects[index + 2]) * width, + (rects[index + 1] + rects[index + 3]) * height, + paint, + ) + index += 4 + } + if (commands.isEmpty()) return + val path = pathFor(width, height) + paint.style = Paint.Style.STROKE + paint.strokeWidth = strokeWidth * density + paint.strokeCap = Paint.Cap.BUTT + paint.strokeJoin = Paint.Join.MITER + canvas.save() + canvas.translate(x, 0f) + canvas.drawPath(path, paint) + canvas.restore() + paint.style = Paint.Style.FILL + } + + private fun pathFor(width: Float, height: Float): Path { + val cached = scaledPath + if (cached != null && scaledWidth == width && scaledHeight == height) return cached + val path = Path() + var index = 0 + while (index < commands.size) { + val x = { offset: Int -> commands[index + offset].toFloat() * width } + val y = { offset: Int -> commands[index + offset].toFloat() * height } + when (commands[index].toInt()) { + 0 -> { path.moveTo(x(1), y(2)); index += 3 } + 1 -> { path.lineTo(x(1), y(2)); index += 3 } + 2 -> { path.quadTo(x(1), y(2), x(3), y(4)); index += 5 } + else -> index = commands.size + } + } + scaledPath = path + scaledWidth = width + scaledHeight = height + return path + } +} + +internal class TerminalCustomGlyphs(records: List) { + private val glyphs: Map = + records.associate { it.char to TerminalCustomGlyph(it) } + + operator fun get(text: String): TerminalCustomGlyph? = glyphs[text] + + companion object { + val EMPTY = TerminalCustomGlyphs(emptyList()) + } +} diff --git a/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalFont.kt b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalFont.kt new file mode 100644 index 0000000000..046efd1e18 --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalFont.kt @@ -0,0 +1,61 @@ +package sh.paseo.terminal + +import android.graphics.Paint +import android.graphics.Typeface +import kotlin.math.max +import kotlin.math.roundToInt + +private const val MEASURE_TEXT = "mmmmmmmmmm" + +/** + * The terminal font at one pixel size: a paint per face (regular, bold, italic, bold italic) and + * the cell those faces are laid out on. Cells are whole device pixels so every column origin is + * exact. Platform fallback fonts cover characters the family lacks. + */ +internal class TerminalFont(val family: String, val sizePx: Float, val density: Float) { + private val faces: Array = Array(4) { face -> + Paint(Paint.ANTI_ALIAS_FLAG or Paint.SUBPIXEL_TEXT_FLAG).apply { + typeface = Typeface.create(family, typefaceStyle(face)) + textSize = sizePx + } + } + + val cellWidthPx: Int + val cellHeightPx: Int + val baselinePx: Int + val underlineOffsetPx: Float + val underlineThicknessPx: Float + val strikeOffsetPx: Float + val strikeThicknessPx: Float + + init { + val regular = faces[0] + cellWidthPx = max(1, (regular.measureText(MEASURE_TEXT) / MEASURE_TEXT.length).roundToInt()) + val metrics = regular.fontMetricsInt + cellHeightPx = max(1, metrics.descent - metrics.ascent) + baselinePx = -metrics.ascent + underlineOffsetPx = regular.underlinePosition + underlineThicknessPx = max(1f, regular.underlineThickness) + strikeOffsetPx = regular.strikeThruPosition + strikeThicknessPx = max(1f, regular.strikeThruThickness) + } + + /** Paint for a style's face; callers only change its color. */ + fun paint(flags: Int): Paint = faces[face(flags)] + + fun sameAs(family: String, sizePx: Float, density: Float): Boolean = + this.family == family && this.sizePx == sizePx && this.density == density + + companion object { + fun face(flags: Int): Int = + (if (flags and TerminalStyleFlags.BOLD != 0) 1 else 0) + + (if (flags and TerminalStyleFlags.ITALIC != 0) 2 else 0) + + private fun typefaceStyle(face: Int): Int = when (face) { + 1 -> Typeface.BOLD + 2 -> Typeface.ITALIC + 3 -> Typeface.BOLD_ITALIC + else -> Typeface.NORMAL + } + } +} diff --git a/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalFrame.kt b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalFrame.kt new file mode 100644 index 0000000000..c27be7cc7a --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalFrame.kt @@ -0,0 +1,100 @@ +package sh.paseo.terminal + +import expo.modules.kotlin.records.Field +import expo.modules.kotlin.records.Record + +class TerminalCursorRecord : Record { + @Field val row: Int = 0 + @Field val col: Int = 0 +} + +class TerminalSelectionRecord : Record { + @Field val startRow: Int = 0 + @Field val startCol: Int = 0 + @Field val endRow: Int = 0 + @Field val endCol: Int = 0 +} + +class TerminalFrameRecord : Record { + @Field val columns: Int = 0 + /** Encoded rows, `cells;styles;text`; see `frame-encoding.ts`. */ + @Field val rows: List = emptyList() + @Field val cursor: TerminalCursorRecord? = null + @Field val selection: TerminalSelectionRecord? = null +} + +class TerminalThemeRecord : Record { + @Field val background: Double = 0.0 + @Field val cursor: Double = 0.0 + @Field val selectionBackground: Double = 0.0 + @Field val selectionForeground: Double = 0.0 +} + +class TerminalCustomGlyphRecord : Record { + @Field val char: String = "" + @Field val rects: DoubleArray = DoubleArray(0) + @Field val path: DoubleArray = DoubleArray(0) + @Field val strokeWidth: Double = 0.0 +} + +internal object TerminalStyleFlags { + const val BOLD = 1 + const val ITALIC = 2 + const val UNDERLINE = 4 + const val STRIKETHROUGH = 8 +} + +/** JS sends colors as unsigned ARGB numbers; Android paints signed ARGB ints. */ +internal fun argb(value: Double): Int = value.toLong().toInt() + +/** One row as the caller described it: cell text, column spans and style spans. */ +internal class TerminalRowSource private constructor( + val text: String, + val cells: IntArray, + val styles: DoubleArray, +) { + companion object { + /** Decodes `cells;styles;text`, where both number lists are comma separated. */ + fun decode(encoded: String): TerminalRowSource { + val cellsEnd = encoded.indexOf(';') + val stylesEnd = encoded.indexOf(';', cellsEnd + 1) + val cells = parseNumbers(encoded, 0, cellsEnd) + val styles = parseNumbers(encoded, cellsEnd + 1, stylesEnd) + return TerminalRowSource( + encoded.substring(stylesEnd + 1), + IntArray(cells.size) { cells[it].toInt() }, + DoubleArray(styles.size) { styles[it].toDouble() }, + ) + } + + private fun parseNumbers(encoded: String, start: Int, end: Int): LongArray { + if (end <= start) return LongArray(0) + var count = 1 + for (index in start until end) if (encoded[index] == ',') count += 1 + val numbers = LongArray(count) + var slot = 0 + for (index in start until end) { + val char = encoded[index] + if (char == ',') slot += 1 else numbers[slot] = numbers[slot] * 10 + (char - '0') + } + return numbers + } + } +} + +internal class TerminalSelection(record: TerminalSelectionRecord) { + private val startRow = record.startRow + private val startCol = record.startCol + private val endRow = record.endRow + private val endCol = record.endCol + + fun intersectsRow(row: Int): Boolean = row in startRow..endRow + + /** A cell is selected when any column it covers is inside the inclusive range. */ + fun contains(row: Int, col: Int, span: Int): Boolean { + if (!intersectsRow(row)) return false + val firstCol = if (row == startRow) startCol else 0 + val lastCol = if (row == endRow) endCol else Int.MAX_VALUE + return col + span - 1 >= firstCol && col <= lastCol + } +} diff --git a/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalRowLayout.kt b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalRowLayout.kt new file mode 100644 index 0000000000..d2ee2355bc --- /dev/null +++ b/packages/app/modules/paseo-terminal/android/src/main/java/sh/paseo/terminal/TerminalRowLayout.kt @@ -0,0 +1,306 @@ +package sh.paseo.terminal + +import android.graphics.Canvas +import android.graphics.Paint +import android.graphics.RenderNode +import android.graphics.text.MeasuredText +import android.icu.text.BreakIterator + +/** Shaped glyphs may exceed their slot by this many pixels before they are scaled down. */ +private const val OVERFLOW_TOLERANCE_PX = 1f +/** Stands in for an open edge when a group is clipped on one side only. */ +private const val CLIP_UNBOUNDED_PX = 100_000f + +/** + * One row shaped once and placed on the terminal grid. + * + * Each run of cells with the same font face is shaped as a whole, so contextual forms, ligatures, + * marks and fallback fonts see their neighbours. The shaper reports a cluster's advance on its + * first UTF-16 unit and zero on the units it absorbed. A cell whose first unit has zero advance + * therefore belongs to the cluster of the cell before it: a ligature, conjunct, emoji sequence or + * reordered mark that spans columns. A cell that does not start a grapheme cluster also stays with + * the cell before it: a half-form conjunct such as स्त keeps both consonants' advances, and drawing + * either half alone would show an explicit virama. Each resulting group is drawn from the shaped + * run (in context) at its first cell's column, centred in the columns it owns, and scaled down only when + * its glyphs are wider than those columns. Colors never split groups or reshape the row: a group + * whose cells differ in color or selection is drawn once per color, clipped to those cells' columns. + */ +internal class TerminalRowLayout( + val source: TerminalRowSource, + private val font: TerminalFont, + private val customGlyphs: TerminalCustomGlyphs, +) { + private val chars: CharArray = source.text.toCharArray() + private val cellCount = source.cells.size / 2 + private val cellCol = IntArray(cellCount) + private val cellSpan = IntArray(cellCount) + private val cellStart = IntArray(cellCount) + private val cellEnd = IntArray(cellCount) + private val cellForeground = IntArray(cellCount) + private val cellBackground = IntArray(cellCount) + private val cellFlags = IntArray(cellCount) + private val cellGlyph = arrayOfNulls(cellCount) + private val groups = ArrayList() + private val measured: MeasuredText? + val text: String get() = source.text + + private var node: RenderNode? = null + + private class Group( + val firstCell: Int, + val lastCell: Int, + val col: Int, + val start: Int, + val end: Int, + val contextStart: Int, + val contextEnd: Int, + val offsetPx: Float, + val scale: Float, + val blank: Boolean, + ) + + init { + readCells() + measured = measure() + if (measured != null) buildGroups(measured) + } + + private fun readCells() { + var offset = 0 + var col = 0 + var styleIndex = 0 + var styleEndCol = 0 + for (cell in 0 until cellCount) { + val length = source.cells[cell * 2] + val span = maxOf(1, source.cells[cell * 2 + 1]) + while (col >= styleEndCol && styleIndex + 3 < source.styles.size) { + styleEndCol += source.styles[styleIndex].toInt() + styleIndex += 4 + } + val style = styleIndex - 4 + if (style >= 0) { + cellForeground[cell] = argb(source.styles[style + 1]) + cellBackground[cell] = argb(source.styles[style + 2]) + cellFlags[cell] = source.styles[style + 3].toInt() + } + cellCol[cell] = col + cellSpan[cell] = span + cellStart[cell] = offset + cellEnd[cell] = minOf(chars.size, offset + length) + cellGlyph[cell] = customGlyphs[source.text.substring(cellStart[cell], cellEnd[cell])] + offset = cellEnd[cell] + col += span + } + } + + /** Same face, not custom glyph: cells shaped together. */ + private fun shapesWith(cell: Int, other: Int): Boolean = + cellGlyph[cell] == null && cellGlyph[other] == null && + TerminalFont.face(cellFlags[cell]) == TerminalFont.face(cellFlags[other]) + + private fun measure(): MeasuredText? { + if (chars.isEmpty()) return null + val builder = MeasuredText.Builder(chars) + var cell = 0 + while (cell < cellCount) { + var last = cell + while (last + 1 < cellCount && shapesWith(cell, last + 1)) last += 1 + val length = cellEnd[last] - cellStart[cell] + if (length > 0) { + if (cellGlyph[cell] != null) { + builder.appendReplacementRun(font.paint(cellFlags[cell]), length, 0f) + } else { + builder.appendStyleRun(font.paint(cellFlags[cell]), length, false) + } + } + cell = last + 1 + } + val trailing = chars.size - (if (cellCount > 0) cellEnd[cellCount - 1] else 0) + if (trailing > 0) builder.appendReplacementRun(font.paint(0), trailing, 0f) + return builder.build() + } + + private fun buildGroups(measured: MeasuredText) { + // Keep platform graphemes together even when Minikin reports separate advances/cursors. + // Rules follow the OS ICU version: Android 10 splits Indic conjuncts such as स्ते. + val graphemes = BreakIterator.getCharacterInstance() + graphemes.setText(source.text) + var cell = 0 + while (cell < cellCount) { + if (cellGlyph[cell] != null) { + cell += 1 + continue + } + var last = cell + while (last + 1 < cellCount && shapesWith(cell, last + 1)) last += 1 + val contextStart = cellStart[cell] + val contextEnd = cellEnd[last] + var first = cell + for (next in cell + 1..last) { + val startsCluster = + cellEnd[next] > cellStart[next] && + measured.getCharWidthAt(cellStart[next]) != 0f && + graphemes.isBoundary(cellStart[next]) + if (startsCluster) { + addGroup(measured, first, next - 1, contextStart, contextEnd) + first = next + } + } + addGroup(measured, first, last, contextStart, contextEnd) + cell = last + 1 + } + } + + private fun addGroup(measured: MeasuredText, first: Int, last: Int, contextStart: Int, contextEnd: Int) { + val start = cellStart[first] + val end = cellEnd[last] + if (end <= start) return + var columns = 0 + for (cell in first..last) columns += cellSpan[cell] + val slot = (columns * font.cellWidthPx).toFloat() + val advance = measured.getWidth(start, end) + val scale = if (advance > slot + OVERFLOW_TOLERANCE_PX) slot / advance else 1f + val offset = if (scale < 1f) 0f else (slot - advance) / 2f + var blank = true + for (index in start until end) { + if (chars[index] != ' ') { + blank = false + break + } + } + groups.add(Group(first, last, cellCol[first], start, end, contextStart, contextEnd, offset, scale, blank)) + } + + /** Draws the row with its own colors, replaying a recorded display list when possible. */ + fun draw(canvas: Canvas, width: Int, paints: TerminalPaints) { + if (!canvas.isHardwareAccelerated) { + drawContent(canvas, -1, null, paints) + return + } + // Marks stacked above or below a letter ink outside the row. The node must not clip them: the + // view redraws every row on each frame, so ink outside the row is never left stale. + val recorded = node ?: RenderNode("terminal-row").also { + it.setClipToBounds(false) + node = it + } + if (!recorded.hasDisplayList() || recorded.width != width) { + recorded.setPosition(0, 0, width, font.cellHeightPx) + val recording = recorded.beginRecording() + try { + drawContent(recording, -1, null, paints) + } finally { + recorded.endRecording() + } + } + canvas.drawRenderNode(recorded) + } + + /** Draws the row with selected cells in selection colors. Selection never reshapes the row. */ + fun drawSelected(canvas: Canvas, row: Int, selection: TerminalSelection, paints: TerminalPaints) { + drawContent(canvas, row, selection, paints) + } + + fun release() { + node?.discardDisplayList() + node = null + } + + private fun selected(row: Int, selection: TerminalSelection?, cell: Int): Boolean = + selection?.contains(row, cellCol[cell], cellSpan[cell]) == true + + private fun drawContent(canvas: Canvas, row: Int, selection: TerminalSelection?, paints: TerminalPaints) { + val cellWidth = font.cellWidthPx.toFloat() + val cellHeight = font.cellHeightPx.toFloat() + val fill = paints.fill + for (cell in 0 until cellCount) { + val background = + if (selected(row, selection, cell)) paints.selectionBackground else cellBackground[cell] + if (background ushr 24 == 0) continue + fill.color = background + canvas.drawRect( + cellCol[cell] * cellWidth, + 0f, + (cellCol[cell] + cellSpan[cell]) * cellWidth, + cellHeight, + fill, + ) + } + + val baseline = font.baselinePx.toFloat() + val text = measured + if (text != null) { + for (group in groups) { + if (group.blank) continue + val paint = font.paint(cellFlags[group.firstCell]) + var first = group.firstCell + while (first <= group.lastCell) { + val color = foreground(row, selection, first, paints) + var last = first + while (last < group.lastCell && foreground(row, selection, last + 1, paints) == color) last += 1 + // Only edges between differently colored cells clip; the group's outer edges may overhang. + val clipped = first > group.firstCell || last < group.lastCell + canvas.save() + if (clipped) { + val left = if (first > group.firstCell) cellCol[first] * cellWidth else -CLIP_UNBOUNDED_PX + val right = + if (last < group.lastCell) (cellCol[last] + cellSpan[last]) * cellWidth else CLIP_UNBOUNDED_PX + canvas.clipRect(left, -CLIP_UNBOUNDED_PX, right, CLIP_UNBOUNDED_PX) + } + paint.color = color + drawGroupText(canvas, text, group, paint, cellWidth, cellHeight, baseline) + canvas.restore() + first = last + 1 + } + } + } + + for (cell in 0 until cellCount) { + val glyph = cellGlyph[cell] + val flags = cellFlags[cell] + val foreground = foreground(row, selection, cell, paints) + if (glyph != null) { + fill.color = foreground + glyph.draw(canvas, cellCol[cell] * cellWidth, cellSpan[cell] * cellWidth, cellHeight, font.density, fill) + } + if (flags and (TerminalStyleFlags.UNDERLINE or TerminalStyleFlags.STRIKETHROUGH) == 0) continue + fill.color = foreground + val left = cellCol[cell] * cellWidth + val right = left + cellSpan[cell] * cellWidth + if (flags and TerminalStyleFlags.UNDERLINE != 0) { + val top = baseline + font.underlineOffsetPx + canvas.drawRect(left, top, right, top + font.underlineThicknessPx, fill) + } + if (flags and TerminalStyleFlags.STRIKETHROUGH != 0) { + val top = baseline + font.strikeOffsetPx + canvas.drawRect(left, top, right, top + font.strikeThicknessPx, fill) + } + } + } + + private fun drawGroupText( + canvas: Canvas, + text: MeasuredText, + group: Group, + paint: Paint, + cellWidth: Float, + cellHeight: Float, + baseline: Float, + ) { + val x = group.col * cellWidth + group.offsetPx + if (group.scale < 1f) canvas.scale(group.scale, group.scale, x, cellHeight / 2f) + canvas.drawTextRun(text, group.start, group.end, group.contextStart, group.contextEnd, x, baseline, false, paint) + } + + private fun foreground(row: Int, selection: TerminalSelection?, cell: Int, paints: TerminalPaints): Int = + if (selected(row, selection, cell)) paints.selectionForeground else cellForeground[cell] + + /** The row's text as accessibility reads it. */ + fun describe(): String = source.text.trimEnd() +} + +/** Paint state shared by every row of one view. */ +internal class TerminalPaints { + val fill = Paint(Paint.ANTI_ALIAS_FLAG) + var selectionBackground: Int = 0xFFFFFF00.toInt() + var selectionForeground: Int = 0xFF000000.toInt() +} diff --git a/packages/app/modules/paseo-terminal/contract.ts b/packages/app/modules/paseo-terminal/contract.ts new file mode 100644 index 0000000000..ee4fe45ab0 --- /dev/null +++ b/packages/app/modules/paseo-terminal/contract.ts @@ -0,0 +1,60 @@ +/** + * Native terminal renderer. The caller owns terminal state (xterm), input and gestures; this view + * shapes and paints one frame of terminal cells on the authoritative cell grid and reports the + * cell size its font produces. + * + * Colors are unsigned 32-bit ARGB numbers (`0xAARRGGBB`). Alpha 0 means "not painted". + */ +export interface PaseoTerminalFrame { + /** Columns the frame paints. Cells beyond it are clipped. */ + columns: number; + rows: PaseoTerminalRow[]; + cursor: PaseoTerminalCursor | null; + selection: PaseoTerminalSelection | null; +} + +/** + * One terminal row. Cells start at column 0 and are consecutive. `text` is every cell's text in + * order, so cell `i` owns the next `cells[2i]` UTF-16 code units of `text` and covers the next + * `cells[2i + 1]` columns. Style spans are consecutive from column 0 and cover all columns. + */ +export interface PaseoTerminalRow { + text: string; + /** Flattened `[utf16Length, columnSpan]` per cell. */ + cells: number[]; + /** Flattened `[columnCount, foreground, background, flags]` per style span. */ + styles: number[]; +} + +export interface PaseoTerminalCursor { + row: number; + col: number; +} + +/** Inclusive cell range in frame rows. A cell is selected when any of its columns is. */ +export interface PaseoTerminalSelection { + startRow: number; + startCol: number; + endRow: number; + endCol: number; +} + +export interface PaseoTerminalTheme { + background: number; + cursor: number; + selectionBackground: number; + selectionForeground: number; +} + +export interface PaseoTerminalCellMetrics { + cellWidth: number; + cellHeight: number; +} + +export const PASEO_TERMINAL_STYLE_FLAGS = { + bold: 1, + italic: 2, + underline: 4, + strikethrough: 8, + dim: 16, +} as const; diff --git a/packages/app/src/terminal/native-renderer/terminal-custom-glyph.test.ts b/packages/app/modules/paseo-terminal/custom-glyphs.test.ts similarity index 81% rename from packages/app/src/terminal/native-renderer/terminal-custom-glyph.test.ts rename to packages/app/modules/paseo-terminal/custom-glyphs.test.ts index 95de6c8c54..f3892110c0 100644 --- a/packages/app/src/terminal/native-renderer/terminal-custom-glyph.test.ts +++ b/packages/app/modules/paseo-terminal/custom-glyphs.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; -import { resolveTerminalCustomGlyph } from "./terminal-custom-glyph"; +import { encodeTerminalCustomGlyphs, resolveTerminalCustomGlyph } from "./custom-glyphs"; describe("native terminal custom glyphs", () => { it("fills the complete cell for the full block used by the Claude mascot", () => { @@ -85,4 +85,22 @@ describe("native terminal custom glyphs", () => { expect(resolveTerminalCustomGlyph(char), `missing custom glyph ${char}`).not.toBeNull(); } }); + + it("encodes every glyph for the native renderer in cell fractions", () => { + const encoded = new Map(encodeTerminalCustomGlyphs().map((glyph) => [glyph.char, glyph])); + + expect(encoded.get("▛")).toEqual({ + char: "▛", + rects: [0, 0, 0.5, 0.5, 0.5, 0, 0.5, 0.5, 0, 0.5, 0.5, 0.5], + path: [], + strokeWidth: 0, + }); + expect(encoded.get("╭")).toEqual({ + char: "╭", + rects: [], + path: [0, 0.5, 1, 1, 0.5, 0.75, 2, 0.5, 0.5, 0.75, 0.5, 1, 1, 0.5], + strokeWidth: 1, + }); + expect(encoded.get("┣")?.strokeWidth).toBe(3); + }); }); diff --git a/packages/app/src/terminal/native-renderer/terminal-custom-glyph.ts b/packages/app/modules/paseo-terminal/custom-glyphs.ts similarity index 66% rename from packages/app/src/terminal/native-renderer/terminal-custom-glyph.ts rename to packages/app/modules/paseo-terminal/custom-glyphs.ts index f63de40e24..fd55193f74 100644 --- a/packages/app/src/terminal/native-renderer/terminal-custom-glyph.ts +++ b/packages/app/modules/paseo-terminal/custom-glyphs.ts @@ -82,12 +82,60 @@ const BLOCK_GLYPHS: Readonly> = { "▟": rects(upperRight(), lowerLeft(), lowerRight()), }; +/** + * One custom glyph as the native view consumes it. Coordinates are fractions of one cell. + * `rects` is flattened `[x, y, width, height]`. `path` is flattened commands: + * `0 x y` moves, `1 x y` draws a line, `2 cx cy x y` draws a quadratic curve. Paths are + * stroked with `strokeWidth` points regardless of cell size. + */ +export interface PaseoTerminalCustomGlyph { + char: string; + rects: number[]; + path: number[]; + strokeWidth: number; +} + +const PATH_COMMAND_CODES: Readonly> = { + M: { code: 0, coordinates: 2 }, + L: { code: 1, coordinates: 2 }, + Q: { code: 2, coordinates: 4 }, +}; + export function resolveTerminalCustomGlyph(char: string): TerminalCustomGlyph | null { return LIGHT_BOX_GLYPHS[char] ?? HEAVY_BOX_GLYPHS[char] ?? BLOCK_GLYPHS[char] ?? null; } -export function isTerminalCustomGlyph(char: string): boolean { - return resolveTerminalCustomGlyph(char) !== null; +export function encodeTerminalCustomGlyphs(): PaseoTerminalCustomGlyph[] { + const table = { ...LIGHT_BOX_GLYPHS, ...HEAVY_BOX_GLYPHS, ...BLOCK_GLYPHS }; + return Object.entries(table).map(([char, glyph]) => + glyph.kind === "rects" + ? { + char, + rects: glyph.rects.flatMap((value) => [value.x, value.y, value.width, value.height]), + path: [], + strokeWidth: 0, + } + : { char, rects: [], path: encodePath(glyph.path), strokeWidth: glyph.strokeWidth }, + ); +} + +function encodePath(pathData: string): number[] { + const tokens = pathData.match(/[MLQ]|-?\d*\.?\d+/g) ?? []; + const encoded: number[] = []; + let index = 0; + while (index < tokens.length) { + const command = PATH_COMMAND_CODES[tokens[index]]; + if (!command) { + throw new Error(`Unsupported terminal glyph path command in ${pathData}`); + } + const coordinates = tokens.slice(index + 1, index + 1 + command.coordinates).map(Number); + if (coordinates.length !== command.coordinates || coordinates.some(Number.isNaN)) { + throw new Error(`Malformed terminal glyph path ${pathData}`); + } + encoded.push(command.code, ...coordinates); + index += 1 + command.coordinates; + } + return encoded; } function path(pathData: string, strokeWidth = 1): TerminalPathGlyph { diff --git a/packages/app/modules/paseo-terminal/expo-module.config.json b/packages/app/modules/paseo-terminal/expo-module.config.json new file mode 100644 index 0000000000..6291f992c1 --- /dev/null +++ b/packages/app/modules/paseo-terminal/expo-module.config.json @@ -0,0 +1,9 @@ +{ + "platforms": ["android", "ios"], + "android": { + "modules": ["sh.paseo.terminal.PaseoTerminalModule"] + }, + "ios": { + "modules": ["PaseoTerminalModule"] + } +} diff --git a/packages/app/modules/paseo-terminal/frame-encoding.test.ts b/packages/app/modules/paseo-terminal/frame-encoding.test.ts new file mode 100644 index 0000000000..a40d4302c2 --- /dev/null +++ b/packages/app/modules/paseo-terminal/frame-encoding.test.ts @@ -0,0 +1,34 @@ +import { describe, expect, test } from "vitest"; + +import { encodeTerminalFrame } from "./frame-encoding"; + +describe("encodeTerminalFrame", () => { + test("puts cells and styles before text, so text may contain separators", () => { + const native = encodeTerminalFrame({ + columns: 10, + rows: [ + { + text: "안a;b,", + cells: [1, 2, 1, 1, 1, 1, 1, 1, 1, 1], + styles: [3, 0xffe5e5e5, 0, 0, 3, 0xffff0000, 0, 1], + }, + { text: "", cells: [], styles: [] }, + ], + cursor: { row: 1, col: 0 }, + selection: { startRow: 0, startCol: 0, endRow: 0, endCol: 1 }, + }); + + expect(native).toEqual({ + columns: 10, + rows: ["1,2,1,1,1,1,1,1,1,1;3,4293256677,0,0,3,4294901760,0,1;안a;b,", ";;"], + cursor: { row: 1, col: 0 }, + selection: { startRow: 0, startCol: 0, endRow: 0, endCol: 1 }, + }); + }); + + test("leaves out a missing cursor and selection instead of sending null", () => { + const native = encodeTerminalFrame({ columns: 1, rows: [], cursor: null, selection: null }); + + expect(Object.keys(native)).toEqual(["columns", "rows"]); + }); +}); diff --git a/packages/app/modules/paseo-terminal/frame-encoding.ts b/packages/app/modules/paseo-terminal/frame-encoding.ts new file mode 100644 index 0000000000..fc4d113bd2 --- /dev/null +++ b/packages/app/modules/paseo-terminal/frame-encoding.ts @@ -0,0 +1,27 @@ +import type { PaseoTerminalCursor, PaseoTerminalFrame, PaseoTerminalSelection } from "./contract"; + +/** A frame as it crosses to native. */ +export interface NativePaseoTerminalFrame { + columns: number; + rows: string[]; + cursor?: PaseoTerminalCursor; + selection?: PaseoTerminalSelection; +} + +/** + * Rows cross as `cells;styles;text`, with the numbers comma separated. One string per row keeps + * prop conversion to a single copy instead of one call per number, and the string is also the + * native shaping cache key, so an unchanged row costs one lookup. + * + * A missing cursor or selection is left out rather than sent as `null`: Expo on iOS fails to + * convert `null` into an optional record field and then drops the whole frame. + */ +export function encodeTerminalFrame(frame: PaseoTerminalFrame): NativePaseoTerminalFrame { + const native: NativePaseoTerminalFrame = { + columns: frame.columns, + rows: frame.rows.map((row) => `${row.cells.join(",")};${row.styles.join(",")};${row.text}`), + }; + if (frame.cursor) native.cursor = frame.cursor; + if (frame.selection) native.selection = frame.selection; + return native; +} diff --git a/packages/app/modules/paseo-terminal/index.tsx b/packages/app/modules/paseo-terminal/index.tsx new file mode 100644 index 0000000000..321cf7c30c --- /dev/null +++ b/packages/app/modules/paseo-terminal/index.tsx @@ -0,0 +1,47 @@ +import { requireNativeView } from "expo"; +import { useCallback, useMemo } from "react"; +import type { StyleProp, ViewStyle } from "react-native"; + +import type { PaseoTerminalCellMetrics, PaseoTerminalFrame, PaseoTerminalTheme } from "./contract"; +import { encodeTerminalCustomGlyphs, type PaseoTerminalCustomGlyph } from "./custom-glyphs"; +import { encodeTerminalFrame, type NativePaseoTerminalFrame } from "./frame-encoding"; + +export * from "./contract"; + +export interface PaseoTerminalViewProps { + frame: PaseoTerminalFrame; + theme: PaseoTerminalTheme; + fontFamily: string; + /** Font size in points after the system font scale is applied. */ + fontSize: number; + onCellMetrics?: (metrics: PaseoTerminalCellMetrics) => void; + style?: StyleProp; +} + +interface NativePaseoTerminalViewProps extends Omit< + PaseoTerminalViewProps, + "frame" | "onCellMetrics" +> { + frame: NativePaseoTerminalFrame; + customGlyphs: PaseoTerminalCustomGlyph[]; + onCellMetrics?: (event: { nativeEvent: PaseoTerminalCellMetrics }) => void; +} + +const NativePaseoTerminalView = requireNativeView("PaseoTerminal"); +const CUSTOM_GLYPHS = encodeTerminalCustomGlyphs(); + +export function PaseoTerminalView({ frame, onCellMetrics, ...props }: PaseoTerminalViewProps) { + const nativeFrame = useMemo(() => encodeTerminalFrame(frame), [frame]); + const handleCellMetrics = useCallback( + (event: { nativeEvent: PaseoTerminalCellMetrics }) => onCellMetrics?.(event.nativeEvent), + [onCellMetrics], + ); + return ( + + ); +} diff --git a/packages/app/modules/paseo-terminal/ios/PaseoTerminal.podspec b/packages/app/modules/paseo-terminal/ios/PaseoTerminal.podspec new file mode 100644 index 0000000000..f15570465f --- /dev/null +++ b/packages/app/modules/paseo-terminal/ios/PaseoTerminal.podspec @@ -0,0 +1,22 @@ +Pod::Spec.new do |s| + s.name = 'PaseoTerminal' + s.version = '0.1.0' + s.summary = 'Native terminal renderer for Paseo' + s.description = 'Shapes and paints terminal frames on the terminal cell grid' + s.license = 'Apache-2.0' + s.author = 'Paseo' + s.homepage = 'https://paseo.sh' + s.platforms = { :ios => '15.1' } + s.swift_version = '5.4' + s.source = { :path => '.' } + s.static_framework = true + + s.dependency 'ExpoModulesCore' + + s.pod_target_xcconfig = { + 'DEFINES_MODULE' => 'YES', + 'SWIFT_COMPILATION_MODE' => 'wholemodule' + } + + s.source_files = "**/*.{h,m,swift}" +end diff --git a/packages/app/modules/paseo-terminal/ios/PaseoTerminalModule.swift b/packages/app/modules/paseo-terminal/ios/PaseoTerminalModule.swift new file mode 100644 index 0000000000..ec6d7edb0c --- /dev/null +++ b/packages/app/modules/paseo-terminal/ios/PaseoTerminalModule.swift @@ -0,0 +1,21 @@ +import ExpoModulesCore + +public class PaseoTerminalModule: Module { + public func definition() -> ModuleDefinition { + Name("PaseoTerminal") + + View(PaseoTerminalView.self) { + Events("onCellMetrics") + + Prop("frame") { (view: PaseoTerminalView, frame: TerminalFrameRecord) in view.setFrame(frame) } + Prop("theme") { (view: PaseoTerminalView, theme: TerminalThemeRecord) in view.setTheme(theme) } + Prop("fontFamily") { (view: PaseoTerminalView, family: String) in view.setFontFamily(family) } + Prop("fontSize") { (view: PaseoTerminalView, size: Double) in view.setFontSize(size) } + Prop("customGlyphs") { (view: PaseoTerminalView, glyphs: [TerminalCustomGlyphRecord]) in + view.setCustomGlyphs(glyphs) + } + + OnViewDidUpdateProps { (view: PaseoTerminalView) in view.commit() } + } + } +} diff --git a/packages/app/modules/paseo-terminal/ios/PaseoTerminalView.swift b/packages/app/modules/paseo-terminal/ios/PaseoTerminalView.swift new file mode 100644 index 0000000000..240cddddab --- /dev/null +++ b/packages/app/modules/paseo-terminal/ios/PaseoTerminalView.swift @@ -0,0 +1,179 @@ +import ExpoModulesCore +import UIKit + +private let minCachedRows = 256 +private let cursorAlpha: CGFloat = 0.45 + +/// Paints terminal frames on the cell grid. Row shaping is cached by row content, and a frame +/// repaints only the rows whose layout, selection or cursor changed. A row's glyphs may ink outside +/// the row, so a repaint covers the old and new row's ink and redraws every row whose ink it meets. +final class PaseoTerminalView: ExpoView { + let onCellMetrics = EventDispatcher() + + private var fontFamily = "Menlo" + private var fontSize: CGFloat = 12 + private var font: TerminalFont? + private var customGlyphs = TerminalCustomGlyphs.empty + private var frameRecord: TerminalFrameRecord? + private var frameChanged = false + + private var rows: [TerminalRowLayout] = [] + private var columns = 0 + private var cursor: (row: Int, col: Int)? + private var selection: TerminalSelection? + private var background = TerminalColor(Double(0xff00_0000)) + private var cursorColor = TerminalColor(Double(0xffff_ffff)) + private var rowPaint = TerminalRowPaint( + selectionBackground: TerminalColor(Double(0xffff_ff00)), + selectionForeground: TerminalColor(Double(0xff00_0000))) + private var rowCache: [String: TerminalRowLayout] = [:] + + required init(appContext: AppContext? = nil) { + super.init(appContext: appContext) + isOpaque = true + contentMode = .redraw + isAccessibilityElement = false + } + + func setFrame(_ record: TerminalFrameRecord) { + frameRecord = record + frameChanged = true + } + + func setTheme(_ record: TerminalThemeRecord) { + background = TerminalColor(record.background) + cursorColor = TerminalColor(record.cursor) + rowPaint = TerminalRowPaint( + selectionBackground: TerminalColor(record.selectionBackground), + selectionForeground: TerminalColor(record.selectionForeground)) + backgroundColor = UIColor(cgColor: background.cgColor()) + setNeedsDisplay() + } + + func setFontFamily(_ value: String) { fontFamily = value } + + func setFontSize(_ value: Double) { fontSize = CGFloat(value) } + + func setCustomGlyphs(_ records: [TerminalCustomGlyphRecord]) { + customGlyphs = TerminalCustomGlyphs(records) + rowCache.removeAll() + rows = [] + frameChanged = true + } + + /// Applies a batch of prop updates: font first, then the frame laid out with it. + func commit() { + let displayScale = traitCollection.displayScale > 0 ? traitCollection.displayScale : UIScreen.main.scale + if font?.sameAs(family: fontFamily, size: fontSize, displayScale: displayScale) != true { + let next = TerminalFont(family: fontFamily, size: fontSize, displayScale: displayScale) + font = next + rowCache.removeAll() + rows = [] + frameChanged = true + onCellMetrics(["cellWidth": next.cellWidth, "cellHeight": next.cellHeight]) + setNeedsDisplay() + } + if frameChanged { + frameChanged = false + layoutFrame() + } + } + + override func traitCollectionDidChange(_ previous: UITraitCollection?) { + super.traitCollectionDidChange(previous) + if previous?.displayScale != traitCollection.displayScale { commit() } + } + + override func layoutSubviews() { + super.layoutSubviews() + setNeedsDisplay() + } + + private func layoutFrame() { + guard let font, let record = frameRecord else { return } + let previousRows = rows + let previousSelection = selection + let previousCursor = cursor + let previousColumns = columns + + columns = record.columns + cursor = record.cursor.map { ($0.row, $0.col) } + selection = record.selection.map(TerminalSelection.init) + rows = record.rows.map { encoded in + if let cached = rowCache[encoded] { return cached } + let layout = TerminalRowLayout( + source: TerminalRowSource(encoded: encoded), font: font, customGlyphs: customGlyphs) + rowCache[encoded] = layout + return layout + } + trimRowCache(keeping: record.rows) + updateAccessibility() + + if previousColumns != columns || previousRows.count != rows.count { + setNeedsDisplay() + return + } + for index in rows.indices { + let layoutChanged = previousRows[index] !== rows[index] + let selectionChanged = previousSelection?.columns(onRow: index) != selection?.columns(onRow: index) + let cursorChanged = + (previousCursor?.row == index || cursor?.row == index) + && (previousCursor?.row != cursor?.row || previousCursor?.col != cursor?.col) + if layoutChanged || selectionChanged || cursorChanged { + let previous = paintedRect(previousRows[index], at: index, font: font) + setNeedsDisplay(previous.union(paintedRect(rows[index], at: index, font: font))) + } + } + } + + /// The area row `index` paints, including glyph ink outside the row. + private func paintedRect(_ row: TerminalRowLayout, at index: Int, font: TerminalFont) -> CGRect { + let top = CGFloat(index) * font.cellHeight + return CGRect(x: 0, y: top + row.inkTop, width: bounds.width, height: row.inkBottom - row.inkTop) + } + + private func trimRowCache(keeping currentRows: [String]) { + guard rowCache.count > max(minCachedRows, rows.count * 2) else { return } + let current = Set(currentRows) + rowCache = rowCache.filter { current.contains($0.key) } + } + + override func draw(_ rect: CGRect) { + guard let context = UIGraphicsGetCurrentContext() else { return } + context.setFillColor(background.cgColor()) + context.fill(rect) + guard let font else { return } + let gridWidth = CGFloat(columns) * font.cellWidth + context.saveGState() + context.clip(to: CGRect(x: 0, y: 0, width: min(bounds.width, gridWidth), height: bounds.height)) + for (index, row) in rows.enumerated() where paintedRect(row, at: index, font: font).intersects(rect) { + row.draw(in: context, row: index, top: CGFloat(index) * font.cellHeight, selection: selection, paint: rowPaint) + } + if let cursor { + context.setFillColor(cursorColor.cgColor(alphaScale: cursorAlpha)) + context.fill(CGRect( + x: CGFloat(cursor.col) * font.cellWidth, y: CGFloat(cursor.row) * font.cellHeight, + width: font.cellWidth, height: font.cellHeight)) + } + context.restoreGState() + } + + /// Exposes each non-blank row as a static text element, `terminal-row-`. + private func updateAccessibility() { + guard let font else { return } + var elements: [UIAccessibilityElement] = [] + for (index, row) in rows.enumerated() { + let label = row.accessibilityText + guard !label.isEmpty else { continue } + let element = UIAccessibilityElement(accessibilityContainer: self) + element.accessibilityIdentifier = "terminal-row-\(index)" + element.accessibilityLabel = label + element.accessibilityTraits = .staticText + element.accessibilityFrameInContainerSpace = CGRect( + x: 0, y: CGFloat(index) * font.cellHeight, + width: min(bounds.width, CGFloat(columns) * font.cellWidth), height: font.cellHeight) + elements.append(element) + } + accessibilityElements = elements + } +} diff --git a/packages/app/modules/paseo-terminal/ios/TerminalCustomGlyphs.swift b/packages/app/modules/paseo-terminal/ios/TerminalCustomGlyphs.swift new file mode 100644 index 0000000000..2442131916 --- /dev/null +++ b/packages/app/modules/paseo-terminal/ios/TerminalCustomGlyphs.swift @@ -0,0 +1,73 @@ +import CoreGraphics + +/// Box drawing and block elements painted as cell geometry instead of font glyphs, so frames and +/// blocks join across cells regardless of the font's own metrics. +final class TerminalCustomGlyph { + private let rects: [Double] + private let commands: [Double] + private let strokeWidth: CGFloat + + init(_ record: TerminalCustomGlyphRecord) { + rects = record.rects + commands = record.path + strokeWidth = CGFloat(record.strokeWidth) + } + + func draw(in context: CGContext, rect cell: CGRect, color: CGColor) { + context.saveGState() + context.setFillColor(color) + var index = 0 + while index + 3 < rects.count { + context.fill(CGRect( + x: cell.minX + CGFloat(rects[index]) * cell.width, + y: cell.minY + CGFloat(rects[index + 1]) * cell.height, + width: CGFloat(rects[index + 2]) * cell.width, + height: CGFloat(rects[index + 3]) * cell.height)) + index += 4 + } + if !commands.isEmpty { + let point = { (offset: Int) in + CGPoint( + x: cell.minX + CGFloat(self.commands[offset]) * cell.width, + y: cell.minY + CGFloat(self.commands[offset + 1]) * cell.height) + } + let path = CGMutablePath() + index = 0 + while index < commands.count { + switch Int(commands[index]) { + case 0: + path.move(to: point(index + 1)) + index += 3 + case 1: + path.addLine(to: point(index + 1)) + index += 3 + case 2: + path.addQuadCurve(to: point(index + 3), control: point(index + 1)) + index += 5 + default: + index = commands.count + } + } + context.setStrokeColor(color) + context.setLineWidth(strokeWidth) + context.setLineCap(.butt) + context.setLineJoin(.miter) + context.addPath(path) + context.strokePath() + } + context.restoreGState() + } +} + +final class TerminalCustomGlyphs { + static let empty = TerminalCustomGlyphs([]) + private let glyphs: [String: TerminalCustomGlyph] + + init(_ records: [TerminalCustomGlyphRecord]) { + var glyphs: [String: TerminalCustomGlyph] = [:] + for record in records { glyphs[record.char] = TerminalCustomGlyph(record) } + self.glyphs = glyphs + } + + subscript(text: String) -> TerminalCustomGlyph? { glyphs[text] } +} diff --git a/packages/app/modules/paseo-terminal/ios/TerminalFont.swift b/packages/app/modules/paseo-terminal/ios/TerminalFont.swift new file mode 100644 index 0000000000..d2f52a78da --- /dev/null +++ b/packages/app/modules/paseo-terminal/ios/TerminalFont.swift @@ -0,0 +1,67 @@ +import CoreText +import UIKit + +private let measureText = "mmmmmmmmmm" + +/// The terminal font at one size: a CTFont per face (regular, bold, italic, bold italic) and the +/// cell those faces are laid out on. Cells are whole device pixels so every column origin is +/// exact. CoreText's cascade list covers characters the family lacks. +final class TerminalFont { + let family: String + let size: CGFloat + let displayScale: CGFloat + private let faces: [CTFont] + + /// Cell size and baseline in points, each a whole number of device pixels. + let cellWidth: CGFloat + let cellHeight: CGFloat + let baseline: CGFloat + let underlineOffset: CGFloat + let underlineThickness: CGFloat + let strikeOffset: CGFloat + let strikeThickness: CGFloat + + init(family: String, size: CGFloat, displayScale: CGFloat) { + self.family = family + self.size = size + self.displayScale = displayScale + let regular = TerminalFont.makeFont(family: family, size: size, traits: []) + faces = [ + regular, + TerminalFont.makeFont(family: family, size: size, traits: .traitBold), + TerminalFont.makeFont(family: family, size: size, traits: .traitItalic), + TerminalFont.makeFont(family: family, size: size, traits: [.traitBold, .traitItalic]), + ] + + let pixel = { (value: CGFloat) in max(1, (value * displayScale).rounded()) / displayScale } + let line = CTLineCreateWithAttributedString( + NSAttributedString(string: measureText, attributes: [.font: regular])) + let width = CGFloat(CTLineGetTypographicBounds(line, nil, nil, nil)) + cellWidth = pixel(width / CGFloat(measureText.count)) + let ascent = CTFontGetAscent(regular) + let descent = CTFontGetDescent(regular) + let leading = CTFontGetLeading(regular) + cellHeight = pixel(ascent + descent + leading) + baseline = pixel(ascent) + underlineOffset = -CTFontGetUnderlinePosition(regular) + underlineThickness = max(1 / displayScale, CTFontGetUnderlineThickness(regular)) + strikeOffset = -CTFontGetXHeight(regular) / 2 + strikeThickness = underlineThickness + } + + func font(flags: Int) -> CTFont { faces[TerminalFont.face(flags: flags)] } + + func sameAs(family: String, size: CGFloat, displayScale: CGFloat) -> Bool { + self.family == family && self.size == size && self.displayScale == displayScale + } + + static func face(flags: Int) -> Int { + (flags & TerminalStyleFlags.bold != 0 ? 1 : 0) + (flags & TerminalStyleFlags.italic != 0 ? 2 : 0) + } + + private static func makeFont(family: String, size: CGFloat, traits: UIFontDescriptor.SymbolicTraits) -> CTFont { + let base = UIFontDescriptor(fontAttributes: [.family: family]) + let descriptor = traits.isEmpty ? base : (base.withSymbolicTraits(traits) ?? base) + return UIFont(descriptor: descriptor, size: size) as CTFont + } +} diff --git a/packages/app/modules/paseo-terminal/ios/TerminalFrame.swift b/packages/app/modules/paseo-terminal/ios/TerminalFrame.swift new file mode 100644 index 0000000000..84ba6e8b0c --- /dev/null +++ b/packages/app/modules/paseo-terminal/ios/TerminalFrame.swift @@ -0,0 +1,122 @@ +import CoreGraphics +import ExpoModulesCore + +struct TerminalCursorRecord: Record { + @Field var row: Int = 0 + @Field var col: Int = 0 +} + +struct TerminalSelectionRecord: Record { + @Field var startRow: Int = 0 + @Field var startCol: Int = 0 + @Field var endRow: Int = 0 + @Field var endCol: Int = 0 +} + +struct TerminalFrameRecord: Record { + @Field var columns: Int = 0 + /// Encoded rows, `cells;styles;text`; see `frame-encoding.ts`. + @Field var rows: [String] = [] + @Field var cursor: TerminalCursorRecord? = nil + @Field var selection: TerminalSelectionRecord? = nil +} + +struct TerminalThemeRecord: Record { + @Field var background: Double = 0 + @Field var cursor: Double = 0 + @Field var selectionBackground: Double = 0 + @Field var selectionForeground: Double = 0 +} + +struct TerminalCustomGlyphRecord: Record { + @Field var char: String = "" + @Field var rects: [Double] = [] + @Field var path: [Double] = [] + @Field var strokeWidth: Double = 0 +} + +enum TerminalStyleFlags { + static let bold = 1 + static let italic = 2 + static let underline = 4 + static let strikethrough = 8 +} + +/// JS sends colors as unsigned ARGB numbers. +struct TerminalColor: Equatable { + let argb: UInt32 + + init(_ value: Double) { + argb = UInt32(truncatingIfNeeded: Int64(value)) + } + + var isPainted: Bool { argb >> 24 != 0 } + + func cgColor(alphaScale: CGFloat = 1) -> CGColor { + let channel = { (shift: UInt32) in CGFloat((argb >> shift) & 0xff) / 255 } + return CGColor(srgbRed: channel(16), green: channel(8), blue: channel(0), alpha: channel(24) * alphaScale) + } +} + +/// One row as the caller described it: cell text, column spans and style spans. +struct TerminalRowSource { + let text: String + let cells: [Int] + let styles: [Double] + + /// Decodes `cells;styles;text`, where both number lists are comma separated. + init(encoded: String) { + let utf8 = encoded.utf8 + let separator = UInt8(ascii: ";") + let cellsEnd = utf8.firstIndex(of: separator) ?? utf8.endIndex + let stylesStart = cellsEnd == utf8.endIndex ? cellsEnd : utf8.index(after: cellsEnd) + let stylesEnd = utf8[stylesStart...].firstIndex(of: separator) ?? utf8.endIndex + let textStart = stylesEnd == utf8.endIndex ? stylesEnd : utf8.index(after: stylesEnd) + cells = TerminalRowSource.parseNumbers(utf8[utf8.startIndex.. [Int64] { + guard !bytes.isEmpty else { return [] } + var numbers: [Int64] = [0] + for byte in bytes { + if byte == UInt8(ascii: ",") { + numbers.append(0) + } else { + numbers[numbers.count - 1] = numbers[numbers.count - 1] * 10 + Int64(byte - UInt8(ascii: "0")) + } + } + return numbers + } +} + +struct TerminalSelection: Equatable { + let startRow: Int + let startCol: Int + let endRow: Int + let endCol: Int + + init(_ record: TerminalSelectionRecord) { + startRow = record.startRow + startCol = record.startCol + endRow = record.endRow + endCol = record.endCol + } + + func intersects(row: Int) -> Bool { row >= startRow && row <= endRow } + + /// A cell is selected when any column it covers is inside the inclusive range. + func contains(row: Int, col: Int, span: Int) -> Bool { + guard intersects(row: row) else { return false } + let firstCol = row == startRow ? startCol : 0 + let lastCol = row == endRow ? endCol : Int.max + return col + span - 1 >= firstCol && col <= lastCol + } + + /// The part of the selection on one row, so a row only repaints when its own part changes. + func columns(onRow row: Int) -> ClosedRange? { + guard intersects(row: row) else { return nil } + return (row == startRow ? startCol : 0)...(row == endRow ? endCol : Int.max) + } +} diff --git a/packages/app/modules/paseo-terminal/ios/TerminalRowLayout.swift b/packages/app/modules/paseo-terminal/ios/TerminalRowLayout.swift new file mode 100644 index 0000000000..80050c9040 --- /dev/null +++ b/packages/app/modules/paseo-terminal/ios/TerminalRowLayout.swift @@ -0,0 +1,335 @@ +import CoreText +import UIKit + +/// Shaped glyphs may exceed their slot by this many points before they are scaled down. +private let overflowTolerance: CGFloat = 0.5 +/// Stands in for an open edge when a group is clipped on one side only. +private let clipUnbounded: CGFloat = 100_000 + +/// Lays text out left to right in logical order. The override fixes every character's direction; +/// the base direction also covers trailing spaces, which bidi rule L1 otherwise moves to the +/// direction detected from the first strong character, ahead of an Arabic or Hebrew row. +private let leftToRightAttributes: [NSAttributedString.Key: Any] = { + var base = CTWritingDirection.leftToRight + let paragraph = withUnsafeBytes(of: &base) { bytes in + CTParagraphStyleCreate( + [CTParagraphStyleSetting( + spec: .baseWritingDirection, valueSize: MemoryLayout.size, value: bytes.baseAddress!)], + 1) + } + let overrideLeftToRight = CTWritingDirection.leftToRight.rawValue | 2 // kCTWritingDirectionOverride + return [ + NSAttributedString.Key(kCTWritingDirectionAttributeName as String): [NSNumber(value: overrideLeftToRight)], + NSAttributedString.Key(kCTParagraphStyleAttributeName as String): paragraph, + ] +}() + +/// Colors a row is painted with besides its own styles. +struct TerminalRowPaint { + let selectionBackground: TerminalColor + let selectionForeground: TerminalColor +} + +/// One row shaped once and placed on the terminal grid. +/// +/// Each run of cells with the same font face is shaped as a whole line, in logical order and left +/// to right like the terminal grid, so contextual forms, ligatures, marks and fallback fonts see +/// their neighbours. CoreText reports which UTF-16 offset produced each glyph. Cells whose units +/// produced no advancing glyph, and cells a reordered glyph crosses, join the cluster of the cell +/// before them: a ligature, conjunct, emoji sequence or reordered mark that spans columns. Each +/// group is drawn at its first cell's column with the glyph offsets CoreText produced, centred in +/// the columns it owns and scaled down only when wider than them. Colors never split groups. +final class TerminalRowLayout { + let source: TerminalRowSource + private let font: TerminalFont + private let units: [UInt16] + private let cellCount: Int + private var cellCol: [Int] + private var cellSpan: [Int] + private var cellStart: [Int] + private var cellEnd: [Int] + private var cellForeground: [TerminalColor] + private var cellBackground: [TerminalColor] + private var cellFlags: [Int] + private var cellGlyph: [TerminalCustomGlyph?] + private var groups: [Group] = [] + /// The row's painted extent, relative to its top edge. Marks stacked above or below a + /// letter and some fallback glyphs ink outside the row; the view repaints that area with it. + private(set) var inkTop: CGFloat = 0 + private(set) var inkBottom: CGFloat + + private struct GlyphRun { + let font: CTFont + var glyphs: [CGGlyph] + /// Relative to the group's pen origin, y up from the baseline. + var positions: [CGPoint] + } + + private struct Group { + let firstCell: Int + let lastCell: Int + let col: Int + let offset: CGFloat + let scale: CGFloat + let runs: [GlyphRun] + } + + private struct ShapedGlyph { + let font: CTFont + let glyph: CGGlyph + let position: CGPoint + let advance: CGFloat + let cell: Int + } + + init(source: TerminalRowSource, font: TerminalFont, customGlyphs: TerminalCustomGlyphs) { + self.source = source + self.font = font + units = Array(source.text.utf16) + cellCount = source.cells.count / 2 + cellCol = Array(repeating: 0, count: cellCount) + cellSpan = Array(repeating: 1, count: cellCount) + cellStart = Array(repeating: 0, count: cellCount) + cellEnd = Array(repeating: 0, count: cellCount) + cellForeground = Array(repeating: TerminalColor(0), count: cellCount) + cellBackground = Array(repeating: TerminalColor(0), count: cellCount) + cellFlags = Array(repeating: 0, count: cellCount) + cellGlyph = Array(repeating: nil, count: cellCount) + inkBottom = font.cellHeight + readCells(customGlyphs: customGlyphs) + shapeRuns() + } + + var accessibilityText: String { + var text = source.text + while text.last == " " { text.removeLast() } + return text + } + + private func readCells(customGlyphs: TerminalCustomGlyphs) { + var offset = 0 + var col = 0 + var styleIndex = 0 + var styleEndCol = 0 + for cell in 0..= styleEndCol && styleIndex + 3 < source.styles.count { + styleEndCol += Int(source.styles[styleIndex]) + styleIndex += 4 + } + let style = styleIndex - 4 + if style >= 0 { + cellForeground[cell] = TerminalColor(source.styles[style + 1]) + cellBackground[cell] = TerminalColor(source.styles[style + 2]) + cellFlags[cell] = Int(source.styles[style + 3]) + } + cellCol[cell] = col + cellSpan[cell] = span + cellStart[cell] = offset + cellEnd[cell] = min(units.count, offset + length) + cellGlyph[cell] = customGlyphs[text(from: cellStart[cell], to: cellEnd[cell])] + offset = cellEnd[cell] + col += span + } + } + + private func text(from start: Int, to end: Int) -> String { + String(utf16CodeUnits: Array(units[start.. Bool { + cellGlyph[cell] == nil && cellGlyph[other] == nil + && TerminalFont.face(flags: cellFlags[cell]) == TerminalFont.face(flags: cellFlags[other]) + } + + private func shapeRuns() { + var cell = 0 + while cell < cellCount { + if cellGlyph[cell] != nil { + cell += 1 + continue + } + var last = cell + while last + 1 < cellCount && shapesWith(cell, last + 1) { last += 1 } + shapeRun(first: cell, last: last) + cell = last + 1 + } + } + + private func shapeRun(first: Int, last: Int) { + let start = cellStart[first] + let end = cellEnd[last] + guard end > start else { return } + var cellOfUnit = Array(repeating: first, count: end - start) + for cell in first...last where cellEnd[cell] > cellStart[cell] { + for unit in cellStart[cell].. 0 else { continue } + let runAttributes = CTRunGetAttributes(run) as NSDictionary + let runFont = runAttributes[kCTFontAttributeName as String].map { $0 as! CTFont } + ?? font.font(flags: cellFlags[first]) + var glyphs = Array(repeating: CGGlyph(0), count: count) + var positions = Array(repeating: CGPoint.zero, count: count) + var advances = Array(repeating: CGSize.zero, count: count) + var indices = Array(repeating: CFIndex(0), count: count) + let all = CFRange(location: 0, length: 0) + CTRunGetGlyphs(run, all, &glyphs) + CTRunGetPositions(run, all, &positions) + CTRunGetAdvances(run, all, &advances) + CTRunGetStringIndices(run, all, &indices) + for index in 0.. first { + for cell in (first + 1)...last where advance[cell - first] == 0 { + startsCluster[cell - first] = false + } + } + + var groupFirst = first + for cell in first...last { + let closesGroup = cell == last || startsCluster[cell + 1 - first] + guard closesGroup else { continue } + addGroup(first: groupFirst, last: cell, glyphs: shaped.filter { $0.cell >= groupFirst && $0.cell <= cell }) + groupFirst = cell + 1 + } + } + + private func addGroup(first: Int, last: Int, glyphs: [ShapedGlyph]) { + let isBlank = (cellStart[first].. slot + overflowTolerance ? slot / advance : 1 + let offset = scale < 1 ? 0 : (slot - advance) / 2 + var runs: [GlyphRun] = [] + for glyph in glyphs { + let position = CGPoint(x: glyph.position.x - origin, y: glyph.position.y) + if let lastRun = runs.last, CFEqual(lastRun.font, glyph.font) { + runs[runs.count - 1].glyphs.append(glyph.glyph) + runs[runs.count - 1].positions.append(position) + } else { + runs.append(GlyphRun(font: glyph.font, glyphs: [glyph.glyph], positions: [position])) + } + } + groups.append(Group( + firstCell: first, lastCell: last, col: cellCol[first], offset: offset, scale: scale, runs: runs)) + extendInk(runs: runs, scale: scale) + } + + /// Grows the row's ink extent to cover the glyphs as `draw` places them, padded by a device pixel + /// for antialiasing. + private func extendInk(runs: [GlyphRun], scale: CGFloat) { + let centerY = font.cellHeight / 2 + let baselineInTextSpace = font.cellHeight / 2 - font.baseline + let pixel = 1 / font.displayScale + for run in runs { + var bounds = Array(repeating: CGRect.zero, count: run.glyphs.count) + CTFontGetBoundingRectsForGlyphs(run.font, .horizontal, run.glyphs, &bounds, run.glyphs.count) + for (glyph, position) in zip(bounds, run.positions) where !glyph.isEmpty { + let top = centerY - (position.y + baselineInTextSpace + glyph.maxY) * scale - pixel + let bottom = centerY - (position.y + baselineInTextSpace + glyph.minY) * scale + pixel + inkTop = min(inkTop, (top * font.displayScale).rounded(.down) / font.displayScale) + inkBottom = max(inkBottom, (bottom * font.displayScale).rounded(.up) / font.displayScale) + } + } + } + + private func isSelected(_ cell: Int, row: Int, selection: TerminalSelection?) -> Bool { + selection?.contains(row: row, col: cellCol[cell], span: cellSpan[cell]) ?? false + } + + private func foreground(_ cell: Int, row: Int, selection: TerminalSelection?, paint: TerminalRowPaint) -> TerminalColor { + isSelected(cell, row: row, selection: selection) ? paint.selectionForeground : cellForeground[cell] + } + + /// Draws the row with its top edge at `top`, selected cells in selection colors. + func draw(in context: CGContext, row: Int, top: CGFloat, selection: TerminalSelection?, paint: TerminalRowPaint) { + let cellWidth = font.cellWidth + let cellHeight = font.cellHeight + for cell in 0.. group.firstCell || last < group.lastCell { + let left = first > group.firstCell ? CGFloat(cellCol[first]) * cellWidth : -clipUnbounded + let right = last < group.lastCell ? CGFloat(cellCol[last] + cellSpan[last]) * cellWidth : clipUnbounded + context.clip(to: CGRect(x: left, y: -clipUnbounded, width: right - left, height: 2 * clipUnbounded)) + } + context.setFillColor(color.cgColor()) + context.translateBy(x: CGFloat(group.col) * cellWidth + group.offset, y: centerY) + context.scaleBy(x: group.scale, y: -group.scale) + context.textMatrix = .identity + for run in group.runs { + let positions = run.positions.map { CGPoint(x: $0.x, y: $0.y + baselineInTextSpace) } + CTFontDrawGlyphs(run.font, run.glyphs, positions, run.glyphs.count, context) + } + context.restoreGState() + first = last + 1 + } + } + + let baseline = top + font.baseline + for cell in 0.. { hasNotifiedMeasuredSizeRef.current = false; resetTerminal(mountInputRef.current.initialSnapshot); - callbacksRef.current.onRendererReadyChange?.({ streamKey, isReady: true }); + emitMeasuredSize({ source: "measure" }); return () => { callbacksRef.current.onRendererReadyChange?.({ streamKey, isReady: false }); clearSelectionLongPressTimeout(); @@ -604,7 +613,7 @@ function NativeTerminalEmulator({ terminalRef.current?.dispose(); terminalRef.current = null; }; - }, [clearSelectionLongPressTimeout, resetTerminal, streamKey]); + }, [clearSelectionLongPressTimeout, emitMeasuredSize, resetTerminal, streamKey]); useEffect(() => { if (focusRequestToken <= 0) return; diff --git a/packages/app/src/terminal/native-renderer/colors.ts b/packages/app/src/terminal/native-renderer/colors.ts index dfebe4eb4d..03452cd30b 100644 --- a/packages/app/src/terminal/native-renderer/colors.ts +++ b/packages/app/src/terminal/native-renderer/colors.ts @@ -1,6 +1,6 @@ import type { ITheme } from "@xterm/xterm"; -import type { TextStyle } from "react-native"; import type { TerminalCell } from "@getpaseo/protocol/messages"; +import { PASEO_TERMINAL_STYLE_FLAGS } from "../../../modules/paseo-terminal/contract"; import { darkTheme } from "@/styles/theme"; import { toXtermTheme } from "@/utils/to-xterm-theme"; @@ -29,77 +29,115 @@ const DEFAULT_COLOR_MODE = 0; const ANSI_COLOR_MODE = 1; const INDEXED_COLOR_MODE = 2; const RGB_COLOR_MODE = 3; -const FLAG_BOLD = 1 << 0; -const FLAG_ITALIC = 1 << 1; -const FLAG_UNDERLINE = 1 << 2; -const FLAG_DIM = 1 << 3; -const FLAG_INVERSE = 1 << 4; -const FLAG_STRIKETHROUGH = 1 << 5; +const OPAQUE_BLACK = 0xff000000; +const DIM_ALPHA = 0.65; -interface ResolvedCellStyle { - key: string; - style: TextStyle; - foregroundColor: string; -} - -interface ResolvedCellColors { - foreground: string; - background: string; - paintsBackground: boolean; +/** Colors are unsigned ARGB. A background of 0 is the terminal background. */ +export interface ResolvedCellStyle { + foreground: number; + background: number; + flags: number; } export interface TerminalCellStyleResolver { - readonly themeKey: string; - readonly backgroundColor: string; - readonly cursorColor: string; + readonly backgroundColor: number; + readonly cursorColor: number; resolve(cell: TerminalCell): ResolvedCellStyle; } export const DEFAULT_TERMINAL_THEME = toXtermTheme(darkTheme.colors.terminal); export const NATIVE_TERMINAL_SELECTION_COLORS = { - background: "#ffff00", - foreground: "#000000", + background: 0xffffff00, + foreground: 0xff000000, } as const; -function build256Palette(theme: ITheme): string[] { - const palette = Array.from({ length: 256 }, () => ""); +function build256Palette(theme: ITheme): number[] { + const palette = Array.from({ length: 256 }, () => OPAQUE_BLACK); for (let index = 0; index < ANSI_THEME_KEYS.length; index += 1) { const key = ANSI_THEME_KEYS[index]; - palette[index] = theme[key] ?? DEFAULT_TERMINAL_THEME[key] ?? "#000000"; + palette[index] = parseTerminalColor(theme[key] ?? DEFAULT_TERMINAL_THEME[key], OPAQUE_BLACK); } for (let red = 0; red < CUBE_LEVELS.length; red += 1) { for (let green = 0; green < CUBE_LEVELS.length; green += 1) { for (let blue = 0; blue < CUBE_LEVELS.length; blue += 1) { const index = 16 + red * 36 + green * 6 + blue; - palette[index] = `rgb(${CUBE_LEVELS[red]}, ${CUBE_LEVELS[green]}, ${CUBE_LEVELS[blue]})`; + palette[index] = rgbColor( + (CUBE_LEVELS[red] << 16) | (CUBE_LEVELS[green] << 8) | CUBE_LEVELS[blue], + ); } } } for (let index = 0; index < 24; index += 1) { const value = 8 + index * 10; - palette[232 + index] = `rgb(${value}, ${value}, ${value})`; + palette[232 + index] = rgbColor((value << 16) | (value << 8) | value); } return palette; } -function rgbColor(value: number): string { - const red = (value >> 16) & 0xff; - const green = (value >> 8) & 0xff; - const blue = value & 0xff; - return `rgb(${red}, ${green}, ${blue})`; +function rgbColor(value: number): number { + return (OPAQUE_BLACK | (value & 0xffffff)) >>> 0; +} + +function argb(input: { alpha: number; red: number; green: number; blue: number }): number { + const channel = (value: number) => Math.max(0, Math.min(255, Math.round(value))); + return ( + ((channel(input.alpha) << 24) | + (channel(input.red) << 16) | + (channel(input.green) << 8) | + channel(input.blue)) >>> + 0 + ); +} + +/** Parses the CSS color forms terminal themes use: `#rgb`, `#rrggbb`, `#rrggbbaa`, `rgb()`, `rgba()`. */ +export function parseTerminalColor(value: string | undefined, fallback: number): number { + if (!value) return fallback; + const color = value.trim().toLowerCase(); + const hex = /^#([0-9a-f]{3}|[0-9a-f]{6}|[0-9a-f]{8})$/.exec(color)?.[1]; + if (hex) { + const digits = hex.length === 3 ? [...hex].map((digit) => digit + digit).join("") : hex; + const channel = (offset: number) => Number.parseInt(digits.slice(offset, offset + 2), 16); + return argb({ + alpha: digits.length === 8 ? channel(6) : 255, + red: channel(0), + green: channel(2), + blue: channel(4), + }); + } + const functional = /^rgba?\(([^)]*)\)$/.exec(color)?.[1]; + if (functional) { + const parts = functional + .split(/[\s,/]+/) + .filter(Boolean) + .map(Number); + if (parts.length >= 3 && parts.every((part) => Number.isFinite(part))) { + return argb({ + alpha: parts.length >= 4 ? parts[3] * 255 : 255, + red: parts[0], + green: parts[1], + blue: parts[2], + }); + } + } + return fallback; +} + +function withDimAlpha(color: number): number { + const alpha = (color >>> 24) & 0xff; + return (((Math.round(alpha * DIM_ALPHA) & 0xff) << 24) | (color & 0xffffff)) >>> 0; } function resolveColor(input: { value: number | undefined; mode: number | undefined; - fallback: string; - palette: string[]; -}): string { + fallback: number; + palette: number[]; +}): number { const mode = input.mode ?? DEFAULT_COLOR_MODE; const value = input.value ?? 0; @@ -114,131 +152,50 @@ function resolveColor(input: { function cellFlags(cell: TerminalCell): number { let flags = 0; - if (cell.bold) flags |= FLAG_BOLD; - if (cell.italic) flags |= FLAG_ITALIC; - if (cell.underline) flags |= FLAG_UNDERLINE; - if (cell.dim) flags |= FLAG_DIM; - if (cell.inverse) flags |= FLAG_INVERSE; - if (cell.strikethrough) flags |= FLAG_STRIKETHROUGH; + if (cell.bold) flags |= PASEO_TERMINAL_STYLE_FLAGS.bold; + if (cell.italic) flags |= PASEO_TERMINAL_STYLE_FLAGS.italic; + if (cell.underline) flags |= PASEO_TERMINAL_STYLE_FLAGS.underline; + if (cell.strikethrough) flags |= PASEO_TERMINAL_STYLE_FLAGS.strikethrough; + if (cell.dim) flags |= PASEO_TERMINAL_STYLE_FLAGS.dim; return flags; } -function resolveCellColors(input: { - cell: TerminalCell; - palette: string[]; - foregroundColor: string; - backgroundColor: string; -}): ResolvedCellColors { - const foreground = resolveColor({ - value: input.cell.fg, - mode: input.cell.fgMode, - fallback: input.foregroundColor, - palette: input.palette, - }); - const background = resolveColor({ - value: input.cell.bg, - mode: input.cell.bgMode, - fallback: input.backgroundColor, - palette: input.palette, - }); - - const resolvedForeground = input.cell.inverse ? background : foreground; - const resolvedBackground = input.cell.inverse ? foreground : background; - return { - foreground: resolvedForeground, - background: resolvedBackground, - paintsBackground: resolvedBackground !== input.backgroundColor, - }; -} - -function textDecorationLine(flags: number): TextStyle["textDecorationLine"] | undefined { - const underline = (flags & FLAG_UNDERLINE) !== 0; - const strikethrough = (flags & FLAG_STRIKETHROUGH) !== 0; - if (underline && strikethrough) return "underline line-through"; - if (underline) return "underline"; - if (strikethrough) return "line-through"; - return undefined; -} - -function createTextStyle(input: { - foreground: string; - background: string; - paintsBackground: boolean; - flags: number; -}): TextStyle { - const style: TextStyle = { - color: input.foreground, - }; - - if (input.paintsBackground) { - style.backgroundColor = input.background; - } - if ((input.flags & FLAG_BOLD) !== 0) { - style.fontWeight = "700"; - } - if ((input.flags & FLAG_ITALIC) !== 0) { - style.fontStyle = "italic"; - } - if ((input.flags & FLAG_DIM) !== 0) { - style.opacity = 0.65; - } - - const decorationLine = textDecorationLine(input.flags); - if (decorationLine) { - style.textDecorationLine = decorationLine; - style.textDecorationColor = input.foreground; - } - - return style; -} - -function buildStyleKey(input: { - foreground: string; - background: string; - paintsBackground: boolean; - flags: number; -}): string { - const background = input.paintsBackground ? input.background : "default"; - return `${input.foreground}|${background}|${input.flags}`; -} - -function buildThemeKey(theme: ITheme): string { - return JSON.stringify(theme); -} - export function createTerminalCellStyleResolver(theme: ITheme): TerminalCellStyleResolver { const palette = build256Palette(theme); - const foregroundColor = theme.foreground ?? DEFAULT_TERMINAL_THEME.foreground ?? "#fafafa"; - const backgroundColor = theme.background ?? DEFAULT_TERMINAL_THEME.background ?? "#181B1A"; - const cursorColor = theme.cursor ?? foregroundColor; - const styleCache = new Map(); + const foregroundColor = parseTerminalColor( + theme.foreground ?? DEFAULT_TERMINAL_THEME.foreground, + 0xfffafafa, + ); + const backgroundColor = parseTerminalColor( + theme.background ?? DEFAULT_TERMINAL_THEME.background, + 0xff181b1a, + ); + const cursorColor = parseTerminalColor(theme.cursor, foregroundColor); return { - themeKey: buildThemeKey(theme), backgroundColor, cursorColor, resolve(cell: TerminalCell): ResolvedCellStyle { + const foreground = resolveColor({ + value: cell.fg, + mode: cell.fgMode, + fallback: foregroundColor, + palette, + }); + const background = resolveColor({ + value: cell.bg, + mode: cell.bgMode, + fallback: backgroundColor, + palette, + }); + const resolvedForeground = cell.inverse ? background : foreground; + const resolvedBackground = cell.inverse ? foreground : background; const flags = cellFlags(cell); - const colors = resolveCellColors({ cell, palette, foregroundColor, backgroundColor }); - const key = buildStyleKey({ - foreground: colors.foreground, - background: colors.background, - paintsBackground: colors.paintsBackground, + return { + foreground: cell.dim ? withDimAlpha(resolvedForeground) : resolvedForeground, + background: resolvedBackground === backgroundColor ? 0 : resolvedBackground, flags, - }); - const cachedStyle = styleCache.get(key); - if (cachedStyle) { - return { key, style: cachedStyle, foregroundColor: colors.foreground }; - } - - const style = createTextStyle({ - foreground: colors.foreground, - background: colors.background, - paintsBackground: colors.paintsBackground, - flags, - }); - styleCache.set(key, style); - return { key, style, foregroundColor: colors.foreground }; + }; }, }; } diff --git a/packages/app/src/terminal/native-renderer/terminal-frame.test.ts b/packages/app/src/terminal/native-renderer/terminal-frame.test.ts new file mode 100644 index 0000000000..129a7d9082 --- /dev/null +++ b/packages/app/src/terminal/native-renderer/terminal-frame.test.ts @@ -0,0 +1,199 @@ +import { describe, expect, test } from "vitest"; + +import graphemeRows from "../../../modules/paseo-terminal/android/src/androidTest/assets/grapheme-rows.json"; +import { PASEO_TERMINAL_STYLE_FLAGS } from "../../../modules/paseo-terminal/contract"; +import { encodeTerminalFrame } from "../../../modules/paseo-terminal/frame-encoding"; +import { + createTerminalCellStyleResolver, + DEFAULT_TERMINAL_THEME, + parseTerminalColor, +} from "./colors"; +import { createNativeHeadlessTerminal } from "./headless-terminal-state"; +import { buildTerminalFrame } from "./terminal-frame"; + +const resolver = createTerminalCellStyleResolver(DEFAULT_TERMINAL_THEME); +const RED = parseTerminalColor(DEFAULT_TERMINAL_THEME.red, 0); +const FOREGROUND = parseTerminalColor(DEFAULT_TERMINAL_THEME.foreground, 0); + +async function frameFor(output: string, options: { cols?: number; rows?: number } = {}) { + const terminal = createNativeHeadlessTerminal({ + cols: options.cols ?? 20, + rows: options.rows ?? 4, + }); + await terminal.write(output); + const state = terminal.getViewportState({ firstRow: 0, rowCount: options.rows ?? 4 }); + terminal.dispose(); + return { + state, + frame: buildTerminalFrame({ state, columns: state.cols, resolver, selection: null }), + }; +} + +function cellLayout(row: { text: string; cells: number[] }) { + const layout: Array<{ text: string; col: number; span: number }> = []; + let offset = 0; + let col = 0; + for (let index = 0; index < row.cells.length; index += 2) { + const [length, span] = [row.cells[index], row.cells[index + 1]]; + layout.push({ text: row.text.slice(offset, offset + length), col, span }); + offset += length; + col += span; + } + return layout; +} + +/** Indexes of the cells that start a grapheme cluster. */ +function clusterStartCells(row: { text: string; cells: number[] }, segmenter: Intl.Segmenter) { + const boundaries = new Set(Array.from(segmenter.segment(row.text), (segment) => segment.index)); + const starts: number[] = []; + let offset = 0; + for (let cell = 0; cell < row.cells.length / 2; cell += 1) { + if (boundaries.has(offset)) starts.push(cell); + offset += row.cells[cell * 2]; + } + return starts; +} + +describe("native terminal frame", () => { + test("keeps each Hangul syllable on xterm's two-column allocation", async () => { + const { frame } = await frameFor("> 안녕하세요"); + + expect(cellLayout(frame.rows[0]).slice(0, 7)).toEqual([ + { text: ">", col: 0, span: 1 }, + { text: " ", col: 1, span: 1 }, + { text: "안", col: 2, span: 2 }, + { text: "녕", col: 4, span: 2 }, + { text: "하", col: 6, span: 2 }, + { text: "세", col: 8, span: 2 }, + { text: "요", col: 10, span: 2 }, + ]); + expect(frame.cursor).toEqual({ row: 0, col: 12 }); + }); + + test("covers every column once, including trailing blanks", async () => { + const corpus = ["안녕하세요NEXT", "日本語 中文", "áé é x̸", "क्ष कि ❤️ 👩‍💻 🇰🇷"]; + const { frame } = await frameFor(corpus.join("\r\n"), { rows: 4, cols: 24 }); + + for (const row of frame.rows) { + const spans = row.cells.filter((_, index) => index % 2 === 1); + const styleColumns = row.styles.filter((_, index) => index % 4 === 0); + expect(spans.reduce((total, span) => total + span, 0)).toBe(24); + expect(styleColumns.reduce((total, span) => total + span, 0)).toBe(24); + expect(row.cells.filter((_, index) => index % 2 === 0).reduce((a, b) => a + b, 0)).toBe( + row.text.length, + ); + } + }); + + test("keeps combining marks and joiners inside the cell xterm assigned them", async () => { + const { frame } = await frameFor("éx̸ 각 क्ष 👩‍💻"); + + expect(cellLayout(frame.rows[0]).slice(0, 11)).toEqual([ + { text: "é", col: 0, span: 1 }, + { text: "x̸", col: 1, span: 1 }, + { text: " ", col: 2, span: 1 }, + { text: "각", col: 3, span: 2 }, + { text: " ", col: 5, span: 1 }, + { text: "क्", col: 6, span: 1 }, + { text: "ष", col: 7, span: 1 }, + { text: " ", col: 8, span: 1 }, + { text: "👩‍", col: 9, span: 1 }, + { text: "💻", col: 10, span: 1 }, + { text: " ", col: 11, span: 1 }, + ]); + }); + + test("splits style spans at SGR changes without moving cells", async () => { + const { frame } = await frameFor("안녕\x1b[31mNEXT\x1b[0m\x1b[1;4m!\x1b[0m"); + const row = frame.rows[0]; + + expect( + cellLayout(row) + .slice(0, 7) + .map((cell) => cell.col), + ).toEqual([0, 2, 4, 5, 6, 7, 8]); + expect(row.styles.slice(0, 12)).toEqual([ + 4, + FOREGROUND, + 0, + 0, + 4, + RED, + 0, + 0, + 1, + FOREGROUND, + 0, + PASEO_TERMINAL_STYLE_FLAGS.bold | PASEO_TERMINAL_STYLE_FLAGS.underline, + ]); + }); + + test("resolves inverse and dim into paint colors", async () => { + const { frame } = await frameFor("\x1b[7mI\x1b[0m\x1b[2mD\x1b[0m"); + const background = parseTerminalColor(DEFAULT_TERMINAL_THEME.background, 0); + + expect(frame.rows[0].styles.slice(0, 8)).toEqual([ + 1, + background, + FOREGROUND, + 0, + 1, + (((Math.round(255 * 0.65) & 0xff) << 24) | (FOREGROUND & 0xffffff)) >>> 0, + 0, + PASEO_TERMINAL_STYLE_FLAGS.dim, + ]); + }); + + test("rows follow content and style but not cursor or selection", async () => { + const plain = await frameFor("same\r\nrow"); + const moved = await frameFor("same\r\nrow\x1b[1;1H"); + const recolored = await frameFor("\x1b[31msame\x1b[0m\r\nrow"); + + expect(moved.frame.rows[0]).toEqual(plain.frame.rows[0]); + expect(moved.frame.cursor).not.toEqual(plain.frame.cursor); + expect(recolored.frame.rows[0]).not.toEqual(plain.frame.rows[0]); + + const selected = buildTerminalFrame({ + state: plain.state, + columns: plain.state.cols, + resolver, + selection: { + start: { row: 0, col: 1 }, + end: { row: 1, col: 0 }, + coordinateEpoch: 0, + }, + }); + expect(selected.rows).toEqual(plain.frame.rows); + expect(selected.selection).toEqual({ startRow: 0, startCol: 1, endRow: 1, endCol: 0 }); + }); + + test("clips the frame to the visible columns", async () => { + const { state } = await frameFor("0123456789abcdefghij"); + const frame = buildTerminalFrame({ state, columns: 8, resolver, selection: null }); + + expect(frame.columns).toBe(8); + expect(frame.rows[0].text).toBe("01234567"); + }); + + // The Android instrumented test draws these rows and compares them with each cluster drawn on its + // own. This keeps its input equal to what xterm produces, and its clusters equal to UAX #29 + // grapheme clusters. `contrastStarts` is a wrong clustering the comparison must reject. + test("Android cluster fixture matches xterm cells and grapheme clusters", async () => { + const segmenter = new Intl.Segmenter("und", { granularity: "grapheme" }); + const actual = await Promise.all( + graphemeRows.map(async ({ output, contrastStarts }) => { + const { frame } = await frameFor(output, { cols: 8, rows: 1 }); + const row = frame.rows[0]; + const [encoded] = encodeTerminalFrame({ ...frame, rows: [row] }).rows; + return { + output, + row: encoded, + clusterStarts: clusterStartCells(row, segmenter), + contrastStarts, + }; + }), + ); + + expect(actual).toEqual(graphemeRows); + }); +}); diff --git a/packages/app/src/terminal/native-renderer/terminal-frame.ts b/packages/app/src/terminal/native-renderer/terminal-frame.ts new file mode 100644 index 0000000000..584e09a274 --- /dev/null +++ b/packages/app/src/terminal/native-renderer/terminal-frame.ts @@ -0,0 +1,84 @@ +import type { + PaseoTerminalFrame, + PaseoTerminalRow, + PaseoTerminalSelection, +} from "../../../modules/paseo-terminal/contract"; +import type { TerminalCellStyleResolver } from "./colors"; +import type { NativeTerminalCell, TerminalViewportState } from "./headless-terminal-state"; +import type { TerminalSelectionRange } from "./terminal-selection"; + +/** + * Converts the xterm viewport into the native renderer's frame. Every cell keeps its own text and + * xterm's column span; continuation cells of wide characters (width 0) belong to the cell before + * them. Nothing here measures or joins text: the renderer shapes rows and places the result on + * these columns. + */ +export function buildTerminalFrame(input: { + state: TerminalViewportState; + columns: number; + resolver: TerminalCellStyleResolver; + selection: TerminalSelectionRange | null; +}): PaseoTerminalFrame { + const grid = input.state.grid as NativeTerminalCell[][]; + return { + columns: input.columns, + rows: grid.map((cells) => + buildRow({ cells, columns: input.columns, resolver: input.resolver }), + ), + cursor: input.state.cursor.hidden + ? null + : { row: input.state.cursor.row, col: input.state.cursor.col }, + selection: input.selection + ? toFrameSelection({ range: input.selection, firstRow: input.state.firstRow }) + : null, + }; +} + +function buildRow(input: { + cells: NativeTerminalCell[]; + columns: number; + resolver: TerminalCellStyleResolver; +}): PaseoTerminalRow { + let text = ""; + const cells: number[] = []; + const styles: number[] = []; + const lastColumn = Math.min(input.columns, input.cells.length); + + for (let col = 0; col < lastColumn; col += 1) { + const cell = input.cells[col]; + const span = cell.width ?? 1; + if (span <= 0) { + continue; + } + const cellText = cell.char || " "; + text += cellText; + cells.push(cellText.length, span); + + const style = input.resolver.resolve(cell); + const previous = styles.length - 4; + if ( + previous >= 0 && + styles[previous + 1] === style.foreground && + styles[previous + 2] === style.background && + styles[previous + 3] === style.flags + ) { + styles[previous] += span; + } else { + styles.push(span, style.foreground, style.background, style.flags); + } + } + + return { text, cells, styles }; +} + +function toFrameSelection(input: { + range: TerminalSelectionRange; + firstRow: number; +}): PaseoTerminalSelection { + return { + startRow: input.range.start.row - input.firstRow, + startCol: input.range.start.col, + endRow: input.range.end.row - input.firstRow, + endCol: input.range.end.col, + }; +} diff --git a/packages/app/src/terminal/native-renderer/terminal-grid-metrics.test.ts b/packages/app/src/terminal/native-renderer/terminal-grid-metrics.test.ts index 0c4d7bc25f..c0fef99104 100644 --- a/packages/app/src/terminal/native-renderer/terminal-grid-metrics.test.ts +++ b/packages/app/src/terminal/native-renderer/terminal-grid-metrics.test.ts @@ -1,68 +1,8 @@ import { describe, expect, it } from "vitest"; -import { - resolveTerminalCustomGlyphCellTransform, - resolveTerminalGridMetricsMeasurement, - resolveMeasuredTerminalCellMetrics, - resolveTerminalCursorOffset, -} from "./terminal-grid-metrics"; +import { resolveTerminalGridMetricsMeasurement } from "./terminal-grid-metrics"; describe("native terminal grid metrics", () => { - it("places each normalized custom glyph in run viewport coordinates", () => { - expect( - resolveTerminalCustomGlyphCellTransform({ - cellOffset: 57, - cellWidth: 7.4, - cellHeight: 16.2, - }), - ).toBe("matrix(7.4 0 0 16.2 421.8 0)"); - }); - - it("preserves fractional measured cell width for horizontal layout", () => { - expect( - resolveMeasuredTerminalCellMetrics({ - measuredTextWidth: 72.246, - measuredTextHeight: 16.2, - measureTextLength: 10, - roundToNearestPixel: (value) => value, - }), - ).toEqual({ - cellWidth: 7.2246, - cellHeight: 16.2, - }); - }); - - it.each([1, 2, 2.75, 3, 3.5])( - "aligns cells to the physical pixel grid at density %s", - (density) => { - const metrics = resolveMeasuredTerminalCellMetrics({ - measuredTextWidth: 72.246, - measuredTextHeight: 16.2, - measureTextLength: 10, - roundToNearestPixel: (value) => Math.round(value * density) / density, - }); - - expect(metrics.cellWidth * density).toBeCloseTo(Math.round(metrics.cellWidth * density), 10); - expect(metrics.cellHeight * density).toBeCloseTo( - Math.round(metrics.cellHeight * density), - 10, - ); - }, - ); - - it("positions the cursor with the measured fractional cell width", () => { - expect( - resolveTerminalCursorOffset({ - cursorCol: 80, - cursorRow: 3, - metrics: { cellWidth: 7.2246, cellHeight: 16 }, - }), - ).toEqual({ - x: 577.968, - y: 48, - }); - }); - it("emits measured metrics once until the measurement changes", () => { const measured = { cellWidth: 7.2, cellHeight: 16 }; diff --git a/packages/app/src/terminal/native-renderer/terminal-grid-metrics.ts b/packages/app/src/terminal/native-renderer/terminal-grid-metrics.ts index cb14111bf2..806c149c60 100644 --- a/packages/app/src/terminal/native-renderer/terminal-grid-metrics.ts +++ b/packages/app/src/terminal/native-renderer/terminal-grid-metrics.ts @@ -3,47 +3,6 @@ export interface TerminalGridCellMetrics { cellHeight: number; } -export interface TerminalGridCellMetricsInput { - measuredTextWidth: number; - measuredTextHeight: number; - measureTextLength: number; - roundToNearestPixel: (value: number) => number; -} - -export interface TerminalCursorOffsetInput { - cursorCol: number; - cursorRow: number; - metrics: TerminalGridCellMetrics; -} - -export interface TerminalCursorOffset { - x: number; - y: number; -} - -export interface TerminalCustomGlyphCellTransformInput { - cellOffset: number; - cellWidth: number; - cellHeight: number; -} - -export function resolveTerminalCustomGlyphCellTransform( - input: TerminalCustomGlyphCellTransformInput, -): string { - const translateX = input.cellOffset * input.cellWidth; - return `matrix(${input.cellWidth} 0 0 ${input.cellHeight} ${translateX} 0)`; -} - -export function resolveMeasuredTerminalCellMetrics( - input: TerminalGridCellMetricsInput, -): TerminalGridCellMetrics { - const textLength = Math.max(1, input.measureTextLength); - return { - cellWidth: snapCellMetric(input.measuredTextWidth / textLength, input.roundToNearestPixel), - cellHeight: snapCellMetric(input.measuredTextHeight, input.roundToNearestPixel), - }; -} - export function resolveTerminalGridMetricsMeasurement( previous: TerminalGridCellMetrics | null, next: TerminalGridCellMetrics, @@ -53,16 +12,3 @@ export function resolveTerminalGridMetricsMeasurement( } return next; } - -export function resolveTerminalCursorOffset( - input: TerminalCursorOffsetInput, -): TerminalCursorOffset { - return { - x: input.cursorCol * input.metrics.cellWidth, - y: input.cursorRow * input.metrics.cellHeight, - }; -} - -function snapCellMetric(value: number, roundToNearestPixel: (value: number) => number): number { - return Math.max(1, roundToNearestPixel(value)); -} diff --git a/packages/app/src/terminal/native-renderer/terminal-grid-view.native.tsx b/packages/app/src/terminal/native-renderer/terminal-grid-view.native.tsx index 04f82c1011..d88daf0679 100644 --- a/packages/app/src/terminal/native-renderer/terminal-grid-view.native.tsx +++ b/packages/app/src/terminal/native-renderer/terminal-grid-view.native.tsx @@ -1,71 +1,34 @@ -import { memo, useCallback, useMemo, useRef, useState } from "react"; +import { useCallback, useMemo, useRef, useState } from "react"; import { - PixelRatio, - Text, + useWindowDimensions, View, type LayoutChangeEvent, type StyleProp, - type TextStyle, type ViewStyle, } from "react-native"; import { StyleSheet } from "react-native-unistyles"; -import Svg, { G, Path } from "react-native-svg"; import type { ITheme } from "@xterm/xterm"; +import { + PaseoTerminalView, + type PaseoTerminalCellMetrics, + type PaseoTerminalTheme, +} from "../../../modules/paseo-terminal"; import { createTerminalCellStyleResolver, DEFAULT_TERMINAL_THEME, NATIVE_TERMINAL_SELECTION_COLORS, } from "./colors"; -import type { TerminalGlyphRect } from "./terminal-custom-glyph"; import { resolveNativeTerminalFontFamily } from "./font.native"; import type { TerminalViewportState } from "./headless-terminal-state"; import { - resolveMeasuredTerminalCellMetrics, - resolveTerminalCustomGlyphCellTransform, resolveTerminalGridMetricsMeasurement, - resolveTerminalCursorOffset, type TerminalGridCellMetrics, } from "./terminal-grid-metrics"; -import { - buildRows, - type TerminalCustomGlyphCell, - type TerminalCustomGlyphRun, - type TerminalRowModel, - type TerminalRun, -} from "./terminal-row-model"; +import { buildTerminalFrame } from "./terminal-frame"; import type { TerminalSelectionRange } from "./terminal-selection"; -const MEASURE_TEXT = "mmmmmmmmmm"; const DEFAULT_FONT_SIZE = 12; -const INITIAL_CELL_WIDTH_RATIO = 0.62; -const INITIAL_CELL_HEIGHT_RATIO = 1.35; - -interface CellMetrics { - cellWidth: number; - cellHeight: number; -} - -interface TerminalGridViewport { - width: number; - height: number; -} - -interface TerminalGridRowProps { - row: TerminalRowModel; - cellWidth: number; - cellHeight: number; - fontFamily?: string; - fontSize: number; - styleEpoch: string; -} - -interface TerminalGridRunProps { - run: TerminalRun; - cellWidth: number; - cellHeight: number; - textStyle: StyleProp; -} export interface TerminalGridViewProps { state: TerminalViewportState; @@ -77,221 +40,20 @@ export interface TerminalGridViewProps { onCellMetricsChange?: (metrics: TerminalGridCellMetrics) => void; } -function estimateCellMetrics(fontSize: number): CellMetrics { - return { - cellWidth: snapPixel(fontSize * INITIAL_CELL_WIDTH_RATIO), - cellHeight: snapPixel(fontSize * INITIAL_CELL_HEIGHT_RATIO), - }; -} - -function snapPixel(value: number): number { - return Math.max(1, PixelRatio.roundToNearestPixel(value)); -} - function resolveVisibleCols(input: { - viewportWidth: number; - cellWidth: number; + viewportWidth: number | null; + cellWidth: number | null; gridCols: number; }): number { + if (input.viewportWidth === null || input.cellWidth === null) { + return input.gridCols; + } if (input.viewportWidth <= 0 || input.cellWidth <= 0 || input.gridCols <= 0) { return 0; } return Math.min(input.gridCols, Math.max(1, Math.floor(input.viewportWidth / input.cellWidth))); } -function TerminalGridRun({ run, cellWidth, cellHeight, textStyle }: TerminalGridRunProps) { - const runStyle = useMemo>( - () => [ - styles.run, - { - backgroundColor: run.style.backgroundColor, - height: cellHeight, - width: run.cellCount * cellWidth, - }, - ], - [cellHeight, cellWidth, run.cellCount, run.style.backgroundColor], - ); - const runTextStyle = useMemo>( - () => [textStyle, run.style], - [run.style, textStyle], - ); - - return ( - - {run.renderKind === "custom-glyph" ? ( - - ) : ( - - {run.text} - - )} - - ); -} - -interface TerminalGridCustomGlyphRunProps { - run: TerminalCustomGlyphRun; - cellWidth: number; - cellHeight: number; -} - -interface TerminalGridCustomGlyphRectProps { - cell: TerminalCustomGlyphCell; - rect: TerminalGlyphRect; - cellWidth: number; - cellHeight: number; - foregroundColor: string; -} - -function TerminalGridCustomGlyphRect({ - cell, - rect, - cellWidth, - cellHeight, - foregroundColor, -}: TerminalGridCustomGlyphRectProps) { - const style = useMemo( - () => ({ - position: "absolute", - backgroundColor: foregroundColor, - left: (cell.offset + rect.x) * cellWidth, - top: rect.y * cellHeight, - width: rect.width * cellWidth, - height: rect.height * cellHeight, - }), - [cell.offset, cellHeight, cellWidth, foregroundColor, rect], - ); - - return ; -} - -function TerminalGridCustomGlyphRun({ - run, - cellWidth, - cellHeight, -}: TerminalGridCustomGlyphRunProps) { - const opacity = typeof run.style.opacity === "number" ? run.style.opacity : 1; - - return ( - - {run.glyphs.map((cell) => - cell.glyph.kind === "rects" - ? cell.glyph.rects.map((rect) => ( - - )) - : null, - )} - - - {run.glyphs.map((cell) => - cell.glyph.kind === "path" ? ( - - ) : null, - )} - - - - ); -} - -const MemoTerminalGridRun = memo(TerminalGridRun, (previous, next) => { - return ( - previous.run === next.run && - previous.cellWidth === next.cellWidth && - previous.cellHeight === next.cellHeight && - previous.textStyle === next.textStyle - ); -}); - -function TerminalGridRow({ - row, - cellWidth, - cellHeight, - fontFamily, - fontSize, -}: TerminalGridRowProps) { - const rowStyle = useMemo>( - () => [styles.row, { height: cellHeight }], - [cellHeight], - ); - const textStyle = useMemo>( - () => [ - styles.rowText, - { - height: cellHeight, - lineHeight: cellHeight, - fontFamily, - fontSize, - }, - ], - [cellHeight, fontFamily, fontSize], - ); - const accessibilityLabel = useMemo( - () => - row.runs - .map((run) => run.text) - .join("") - .trimEnd(), - [row.runs], - ); - - return ( - 0} - style={rowStyle} - testID={`terminal-row-${row.index}`} - > - {row.runs.map((run) => ( - - ))} - - ); -} - -const MemoTerminalGridRow = memo(TerminalGridRow, (previous, next) => { - return ( - previous.row.hash === next.row.hash && - previous.cellWidth === next.cellWidth && - previous.cellHeight === next.cellHeight && - previous.fontFamily === next.fontFamily && - previous.fontSize === next.fontSize && - previous.styleEpoch === next.styleEpoch - ); -}); - export function TerminalGridView({ state, xtermTheme = DEFAULT_TERMINAL_THEME, @@ -301,129 +63,61 @@ export function TerminalGridView({ selection = null, onCellMetricsChange, }: TerminalGridViewProps) { - const [metrics, setMetrics] = useState(() => estimateCellMetrics(fontSize)); + const { fontScale } = useWindowDimensions(); + const [cellWidth, setCellWidth] = useState(null); + const [viewportWidth, setViewportWidth] = useState(null); const measuredMetricsRef = useRef(null); - const [viewport, setViewport] = useState(null); const resolvedFontFamily = useMemo( () => resolveNativeTerminalFontFamily(fontFamily), [fontFamily], ); const resolver = useMemo(() => createTerminalCellStyleResolver(xtermTheme), [xtermTheme]); - const visibleCols = useMemo(() => { - const viewportWidth = viewport?.width ?? state.cols * metrics.cellWidth; - return resolveVisibleCols({ - viewportWidth, - cellWidth: metrics.cellWidth, - gridCols: state.cols, - }); - }, [metrics.cellWidth, state.cols, viewport?.width]); - const projectedGrid = useMemo( - () => state.grid.map((row) => row.slice(0, visibleCols)), - [state.grid, visibleCols], + const theme = useMemo( + () => ({ + background: resolver.backgroundColor, + cursor: resolver.cursorColor, + selectionBackground: NATIVE_TERMINAL_SELECTION_COLORS.background, + selectionForeground: NATIVE_TERMINAL_SELECTION_COLORS.foreground, + }), + [resolver], ); - const rows = useMemo( - () => - buildRows({ - grid: projectedGrid, - resolver, - selection: selection - ? { - range: selection, - firstRow: state.firstRow, - backgroundColor: NATIVE_TERMINAL_SELECTION_COLORS.background, - foregroundColor: NATIVE_TERMINAL_SELECTION_COLORS.foreground, - } - : undefined, - }), - [projectedGrid, resolver, selection, state.firstRow], + const columns = resolveVisibleCols({ viewportWidth, cellWidth, gridCols: state.cols }); + const frame = useMemo( + () => buildTerminalFrame({ state, columns, resolver, selection }), + [columns, resolver, selection, state], ); - const containerStyle = useMemo>( - () => [styles.root, { backgroundColor: resolver.backgroundColor }, style], - [resolver.backgroundColor, style], - ); - const gridStyle = useMemo>( - () => [ - styles.grid, - { - width: visibleCols * metrics.cellWidth, - height: state.grid.length * metrics.cellHeight, - }, - ], - [metrics.cellHeight, metrics.cellWidth, state.grid.length, visibleCols], - ); - const measureStyle = useMemo>( - () => [styles.measureText, { fontFamily: resolvedFontFamily, fontSize }], - [resolvedFontFamily, fontSize], - ); - const cursorStyle = useMemo>(() => { - const cursorOffset = resolveTerminalCursorOffset({ - cursorCol: state.cursor.col, - cursorRow: state.cursor.row, - metrics, - }); - return [ - styles.cursor, - { - backgroundColor: resolver.cursorColor, - width: metrics.cellWidth, - height: metrics.cellHeight, - transform: [{ translateX: cursorOffset.x }, { translateY: cursorOffset.y }], - }, - ]; - }, [metrics, state.cursor.col, state.cursor.row, resolver.cursorColor]); - - const handleMeasure = useCallback( - (event: LayoutChangeEvent) => { - const nextMetrics = resolveMeasuredTerminalCellMetrics({ - measuredTextWidth: event.nativeEvent.layout.width, - measuredTextHeight: event.nativeEvent.layout.height, - measureTextLength: MEASURE_TEXT.length, - roundToNearestPixel: (value) => PixelRatio.roundToNearestPixel(value), - }); + const handleCellMetrics = useCallback( + (metrics: PaseoTerminalCellMetrics) => { const changedMetrics = resolveTerminalGridMetricsMeasurement( measuredMetricsRef.current, - nextMetrics, + metrics, ); if (!changedMetrics) { return; } measuredMetricsRef.current = changedMetrics; - setMetrics(changedMetrics); + setCellWidth(changedMetrics.cellWidth); onCellMetricsChange?.(changedMetrics); }, [onCellMetricsChange], ); const handleContainerLayout = useCallback((event: LayoutChangeEvent) => { - const { width, height } = event.nativeEvent.layout; - setViewport((current) => { - if (current?.width === width && current.height === height) { - return current; - } - return { width, height }; - }); + const { width } = event.nativeEvent.layout; + setViewportWidth((current) => (current === width ? current : width)); }, []); return ( - - - - {MEASURE_TEXT} - - {rows.map((row) => ( - - ))} - {!state.cursor.hidden && } - + + ); } @@ -432,38 +126,7 @@ const styles = StyleSheet.create({ root: { overflow: "hidden", }, - grid: { - position: "relative", - }, - row: { - flexDirection: "row", - }, - run: { - overflow: "hidden", - }, - customGlyphRun: { + surface: { flex: 1, - position: "relative", - }, - customGlyphSvg: { - left: 0, - position: "absolute", - top: 0, - }, - rowText: { - includeFontPadding: false, - margin: 0, - padding: 0, - }, - measureText: { - includeFontPadding: false, - opacity: 0, - position: "absolute", - }, - cursor: { - left: 0, - opacity: 0.45, - position: "absolute", - top: 0, }, }); diff --git a/packages/app/src/terminal/native-renderer/terminal-row-model.test.ts b/packages/app/src/terminal/native-renderer/terminal-row-model.test.ts deleted file mode 100644 index b19922b7a9..0000000000 --- a/packages/app/src/terminal/native-renderer/terminal-row-model.test.ts +++ /dev/null @@ -1,120 +0,0 @@ -import type { TerminalCell } from "@getpaseo/protocol/messages"; -import { describe, expect, test } from "vitest"; - -import { createTerminalCellStyleResolver, DEFAULT_TERMINAL_THEME } from "./colors"; -import { buildRows } from "./terminal-row-model"; - -function cell( - char: string, - overrides: Partial & { width?: number } = {}, -): TerminalCell & { width?: number } { - return { char, ...overrides }; -} - -describe("terminal row model", () => { - test("preserves terminal cell width for styled runs and trailing spaces", () => { - const resolver = createTerminalCellStyleResolver(DEFAULT_TERMINAL_THEME); - - const rows = buildRows({ - grid: [ - [cell("n", { bg: 2, bgMode: 1 }), cell("v", { bg: 2, bgMode: 1 }), cell(" "), cell(" ")], - ], - resolver, - }); - - expect(rows[0].runs.map((run) => ({ text: run.text, cellCount: run.cellCount }))).toEqual([ - { text: "nv", cellCount: 2 }, - { text: " ", cellCount: 2 }, - ]); - }); - - test("uses xterm's authoritative width instead of guessing from the code point", () => { - const resolver = createTerminalCellStyleResolver(DEFAULT_TERMINAL_THEME); - - const rows = buildRows({ - grid: [[cell("A", { width: 2 }), cell(" ", { width: 0 }), cell("界", { width: 1 })]], - resolver, - }); - - expect(rows[0].runs.map((run) => ({ text: run.text, cellCount: run.cellCount }))).toEqual([ - { text: "A界", cellCount: 3 }, - ]); - }); - - test("redraws selected cells with the terminal selection colors", () => { - const resolver = createTerminalCellStyleResolver(DEFAULT_TERMINAL_THEME); - const grid = [[cell("a", { fg: 1, fgMode: 1 }), cell("b", { dim: true }), cell("c")]]; - - const rows = buildRows({ - grid, - resolver, - selection: { - range: { - start: { row: 7, col: 1 }, - end: { row: 7, col: 2 }, - coordinateEpoch: 1, - }, - firstRow: 7, - backgroundColor: "#ffff00", - foregroundColor: "#000000", - }, - }); - - expect( - rows[0].runs.map((run) => ({ - text: run.text, - color: run.style.color, - backgroundColor: run.style.backgroundColor, - opacity: run.style.opacity, - })), - ).toEqual([ - { - text: "a", - color: DEFAULT_TERMINAL_THEME.red, - backgroundColor: undefined, - opacity: undefined, - }, - { text: "b", color: "#000000", backgroundColor: "#ffff00", opacity: 1 }, - { text: "c", color: "#000000", backgroundColor: "#ffff00", opacity: 1 }, - ]); - expect(rows[0].hash).not.toBe(buildRows({ grid, resolver })[0].hash); - }); - - test("splits Claude box drawing from font text at exact terminal columns", () => { - const resolver = createTerminalCellStyleResolver(DEFAULT_TERMINAL_THEME); - const frame = "╭─ Claude Code ───╮"; - - const [row] = buildRows({ - grid: [frame.split("").map((char) => cell(char, { fg: 1, fgMode: 1 }))], - resolver, - }); - - expect( - row.runs.map((run) => ({ - text: run.text, - cellCount: run.cellCount, - renderKind: run.renderKind, - })), - ).toEqual([ - { text: "╭─", cellCount: 2, renderKind: "custom-glyph" }, - { text: " Claude Code ", cellCount: 13, renderKind: "text" }, - { text: "───╮", cellCount: 4, renderKind: "custom-glyph" }, - ]); - }); - - test("keeps Claude mascot glyphs custom while preserving copied row text", () => { - const resolver = createTerminalCellStyleResolver(DEFAULT_TERMINAL_THEME); - const mascot = "▐▛███▜▌ Claude Code"; - - const [row] = buildRows({ - grid: [mascot.split("").map((char) => cell(char, { fg: 1, fgMode: 1 }))], - resolver, - }); - - expect(row.runs.map((run) => [run.text, run.renderKind])).toEqual([ - ["▐▛███▜▌", "custom-glyph"], - [" Claude Code", "text"], - ]); - expect(row.runs.map((run) => run.text).join("")).toBe(mascot); - }); -}); diff --git a/packages/app/src/terminal/native-renderer/terminal-row-model.ts b/packages/app/src/terminal/native-renderer/terminal-row-model.ts deleted file mode 100644 index a65f39fd44..0000000000 --- a/packages/app/src/terminal/native-renderer/terminal-row-model.ts +++ /dev/null @@ -1,253 +0,0 @@ -import type { TextStyle } from "react-native"; -import type { TerminalCell } from "@getpaseo/protocol/messages"; - -import type { TerminalCellStyleResolver } from "./colors"; -import { resolveTerminalCustomGlyph, type TerminalCustomGlyph } from "./terminal-custom-glyph"; -import type { TerminalSelectionRange } from "./terminal-selection"; - -const WIDE_CHAR_RANGES: ReadonlyArray = [ - [0x1100, 0x115f], - [0x2329, 0x232a], - [0x2e80, 0xa4cf], - [0xac00, 0xd7a3], - [0xf900, 0xfaff], - [0xfe10, 0xfe19], - [0xfe30, 0xfe6f], - [0xff00, 0xff60], - [0xffe0, 0xffe6], - [0x1f300, 0x1faff], -]; - -interface TerminalRunBase { - key: string; - text: string; - cellCount: number; - styleKey: string; - style: TextStyle; - foregroundColor: string; -} - -export interface TerminalTextRun extends TerminalRunBase { - renderKind: "text"; -} - -export interface TerminalCustomGlyphRun extends TerminalRunBase { - renderKind: "custom-glyph"; - glyphs: TerminalCustomGlyphCell[]; -} - -export type TerminalRun = TerminalCustomGlyphRun | TerminalTextRun; - -export interface TerminalCustomGlyphCell { - key: string; - offset: number; - glyph: TerminalCustomGlyph; -} - -export interface TerminalRowModel { - index: number; - hash: string; - runs: TerminalRun[]; -} - -type TerminalRenderableCell = TerminalCell & { width?: number }; - -interface TerminalRowSelectionStyle { - range: TerminalSelectionRange; - firstRow: number; - backgroundColor: string; - foregroundColor: string; -} - -function hashStringPart(hash: number, value: string): number { - let nextHash = hash; - for (let index = 0; index < value.length; index += 1) { - nextHash = Math.imul(nextHash ^ value.charCodeAt(index), 16777619); - } - return nextHash; -} - -function finishHash(hash: number): string { - return (hash >>> 0).toString(36); -} - -function terminalCharWidth(char: string): number { - const codePoint = char.codePointAt(0); - if (codePoint === undefined) return 1; - const isWide = WIDE_CHAR_RANGES.some(([start, end]) => codePoint >= start && codePoint <= end); - return isWide ? 2 : 1; -} - -function shouldSkipSpacerCell(cells: TerminalRenderableCell[], col: number, char: string): boolean { - if (terminalCharWidth(char) < 2) { - return false; - } - return cells[col + 1]?.char === " "; -} - -function terminalCellCount(cells: TerminalRenderableCell[], col: number, char: string): number { - const authoritativeWidth = cells[col]?.width; - if (authoritativeWidth !== undefined) { - return Math.max(1, authoritativeWidth); - } - if (shouldSkipSpacerCell(cells, col, char)) { - return 2; - } - return 1; -} - -function appendRun(input: { - runs: TerminalRun[]; - text: string; - cellCount: number; - styleKey: string; - style: TextStyle; - foregroundColor: string; - customGlyph: TerminalCustomGlyph | null; - col: number; -}): void { - const renderKind = input.customGlyph ? "custom-glyph" : "text"; - const previousRun = input.runs[input.runs.length - 1]; - if ( - previousRun && - previousRun.styleKey === input.styleKey && - previousRun.renderKind === renderKind - ) { - const offset = previousRun.cellCount; - previousRun.text += input.text; - previousRun.cellCount += input.cellCount; - if (previousRun.renderKind === "custom-glyph" && input.customGlyph) { - previousRun.glyphs.push({ - key: `${input.col}:${input.text}`, - offset, - glyph: input.customGlyph, - }); - } - return; - } - - const baseRun: TerminalRunBase = { - key: `${input.col}:${input.styleKey}`, - text: input.text, - cellCount: input.cellCount, - styleKey: input.styleKey, - style: input.style, - foregroundColor: input.foregroundColor, - }; - if (input.customGlyph) { - input.runs.push({ - ...baseRun, - renderKind: "custom-glyph", - glyphs: [{ key: `${input.col}:${input.text}`, offset: 0, glyph: input.customGlyph }], - }); - return; - } - input.runs.push({ ...baseRun, renderKind: "text" }); -} - -function cellIntersectsSelection(input: { - selection: TerminalRowSelectionStyle; - row: number; - col: number; - cellCount: number; -}): boolean { - const absoluteRow = input.selection.firstRow + input.row; - const { start, end } = input.selection.range; - if (absoluteRow < start.row || absoluteRow > end.row) { - return false; - } - - const selectedStartCol = absoluteRow === start.row ? start.col : 0; - const selectedEndCol = absoluteRow === end.row ? end.col : Number.POSITIVE_INFINITY; - const cellEndCol = input.col + input.cellCount - 1; - return cellEndCol >= selectedStartCol && input.col <= selectedEndCol; -} - -function selectedCellStyle(input: { - style: TextStyle; - foregroundColor: string; - backgroundColor: string; -}): TextStyle { - return { - ...input.style, - backgroundColor: input.backgroundColor, - color: input.foregroundColor, - opacity: 1, - ...(input.style.textDecorationLine - ? { textDecorationColor: input.foregroundColor } - : undefined), - }; -} - -function buildRowModel(input: { - cells: TerminalRenderableCell[]; - index: number; - resolver: TerminalCellStyleResolver; - selection?: TerminalRowSelectionStyle; -}): TerminalRowModel { - const runs: TerminalRun[] = []; - let hash = 2166136261; - - for (let col = 0; col < input.cells.length; col += 1) { - const cell = input.cells[col]; - const text = cell.char || " "; - const resolvedStyle = input.resolver.resolve(cell); - const cellCount = terminalCellCount(input.cells, col, text); - const customGlyph = resolveTerminalCustomGlyph(text); - const selection = input.selection; - let styleKey = resolvedStyle.key; - let style = resolvedStyle.style; - let foregroundColor = resolvedStyle.foregroundColor; - if ( - selection && - cellIntersectsSelection({ - selection, - row: input.index, - col, - cellCount, - }) - ) { - styleKey = `${resolvedStyle.key}|selection:${selection.foregroundColor}:${selection.backgroundColor}`; - style = selectedCellStyle({ - style: resolvedStyle.style, - foregroundColor: selection.foregroundColor, - backgroundColor: selection.backgroundColor, - }); - foregroundColor = selection.foregroundColor; - } - appendRun({ - runs, - text, - cellCount, - styleKey, - style, - foregroundColor, - customGlyph, - col, - }); - hash = hashStringPart(hash, text); - hash = hashStringPart(hash, String(cellCount)); - hash = hashStringPart(hash, styleKey); - hash = hashStringPart(hash, customGlyph ? "custom-glyph" : "text"); - - if (cellCount > 1) { - col += 1; - } - } - - return { - index: input.index, - hash: finishHash(hash), - runs, - }; -} - -export function buildRows(input: { - grid: TerminalRenderableCell[][]; - resolver: TerminalCellStyleResolver; - selection?: TerminalRowSelectionStyle; -}): TerminalRowModel[] { - return input.grid.map((cells, index) => - buildRowModel({ cells, index, resolver: input.resolver, selection: input.selection }), - ); -} diff --git a/packages/app/vitest.config.ts b/packages/app/vitest.config.ts index b139e266c5..ba4c0fc979 100644 --- a/packages/app/vitest.config.ts +++ b/packages/app/vitest.config.ts @@ -82,6 +82,7 @@ export default defineConfig({ environment: "node", include: [ "src/**/*.{test,spec}.{ts,tsx}", + "modules/**/*.{test,spec}.{ts,tsx}", "native-release-version.test.ts", "app.config.test.ts", "vitest-environment.test.ts",