Files
paseo/packages/app/modules/paseo-hardware-keyboard/ios/PaseoHardwareKeyboardModule.swift
T
1c4400194a Support hardware keyboard shortcuts across native and desktop apps (#8625)
* feat(keyboard): support hardware keyboards on iPad

Register app shortcuts as iOS UIKeyCommands so a paired hardware keyboard
drives the same bindings as web and desktop. Overlays (menus, comboboxes,
modals, command center) get first refusal on bare keys, and a focused
terminal keeps the keys it needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(keyboard): record iOS hardware key observation spike

With a UITextView focused on iPad Pro M4/iOS 18.6, window and controller observers received Cmd+B, Cmd+P and Escape. Escape also invoked UIKeyCommand when registered, so the stream must be the sole emitter. Use observation for shortcuts and commands only for text-control ownership. Simulator intercepted Cmd+K as HID184; input capture blocked synthetic typing entirely. Keep capture off and verify the toggle via a rebind during simulator QA.

* refactor(keyboard): give overlays one key registration API

Overlay ownership now follows one layer-ordered registry on every runtime. Web retains focus trapping and restoration through its adapter; native registers the same key handler without a DOM scope. Picker navigation and dismissal no longer require a parallel native stack.

Co-authored-by: jeroenfrenken <16974549+jeroenfrenken@users.noreply.github.com>

* feat(keyboard): route native hardware keys through the shared resolver

Observe each physical press once, including native modal windows, and derive focus from declared composer and terminal surfaces. Native bindings and rebinds now use the same resolver and overlay precedence as browser and Electron input.

Co-authored-by: jeroenfrenken <16974549+jeroenfrenken@users.noreply.github.com>

* feat(keyboard): show shortcuts when hardware keyboards are present

Keep routing independent of compact layout and detect hardware capability for hints and settings. Rebind capture uses the shared overlay interface on native too. Keep the filtered sidebar model available for keyboard navigation while its panel is hidden.

Co-authored-by: Les Stroud <public@lesstroud.com>

Co-authored-by: jeroenfrenken <16974549+jeroenfrenken@users.noreply.github.com>

* test(keyboard): allow shortcut hints in compact menu assertions

Compact menus now display hardware keyboard hints. Assert action labels, order and uniqueness without treating a trailing shortcut badge as part of the action label.

* fix(keyboard): retain an overlay's opening layer in state

Capture the registry's current top when an overlay opens and retain that layer through rerenders. React Compiler removed the previous useMemo cache after the registry getter moved across modules, allowing a parent modal to rise above its child and start an infinite focus loop.

Compile this hook in Vitest as Expo does. The new hook regression failed first with layer 220 instead of 20, then passed with explicit opening state. Focused unit and browser tests pass, and the previously failing onboarding E2E completes through the real Expo compiler. Typecheck, lint and format pass.

* refactor(keyboard): let overlays own native hardware input

Remove the TextInput key handlers that duplicate declared overlay arrows and Escape in Add Project and Command Center. Remove Add Project's hidden responder now that native window sources observe keys without it. Keep soft-keyboard submit editing behavior.

* fix(keyboard): close Add Project outside its state updater

Root Escape previously called the parent close action from a React state updater, producing a cross-component render warning when navigation was still queued. Compute the previous page in the event handler and close outside the updater.

The existing Escape/Back E2E now batches navigation and dismissal through the DOM key source. It failed first with the AddProjectFlowHost render warning, then passed after the correction. Typecheck, lint and format pass; native Escape checks pass on Android and iPad.

* fix(keyboard): preserve Android dead-key composition

* fix(keyboard): recognize focused native text inputs

* test(keyboard): replace JSDOM coverage with browser tests

* refactor(menu): spell out arrow navigation branches

* fix(composer): claim fullscreen Escape on native

* refactor(composer): one fullscreen Escape module for every platform

The shared registration lived in fullscreen-exit.shared.ts with two one-line
re-exports (fullscreen-exit.ts and fullscreen-exit.native.ts). Metro needs no
native override now that the Android back button is its own adapter, so the
implementation moves into fullscreen-exit.ts and the browser test imports the
module every platform resolves.

---------

Co-authored-by: Ubuntu <jeroenfrenken@icloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: jeroenfrenken <16974549+jeroenfrenken@users.noreply.github.com>
2026-11-03 18:08:27 +07:00

238 lines
8.5 KiB
Swift

import ExpoModulesCore
import UIKit
import GameController
private let hardwareSubmitEventName = "onHardwareKeyboardSubmit"
private let hardwareShortcutEventName = "onHardwareKeyboardShortcut"
private weak var activeModule: PaseoHardwareKeyboardModule?
private var isHardwareSubmitEnabled = false
private var overlayKeys: Set<String> = []
/// UIKit spells non-character keys as sentinel input strings rather than as the
/// character they would type. JS sends the DOM `code` and the mapping lives
/// here, so the constants stay in the one place that owns UIKit vocabulary.
private let namedKeyInputs: [String: String] = [
"Escape": UIKeyCommand.inputEscape,
"ArrowUp": UIKeyCommand.inputUpArrow,
"ArrowDown": UIKeyCommand.inputDownArrow,
"Enter": "\r",
]
/// Whether the first responder is mid-IME-composition.
///
/// A committing key has to stay out of the way while a CJK candidate is being
/// chosen: Enter confirms the candidate, and consuming it as a key command
/// would submit the half-composed text instead.
private func isComposingText() -> Bool {
guard let responder = UIResponder.paseoCurrentFirstResponder as? UITextInput else {
return false
}
return responder.markedTextRange != nil
}
public class PaseoHardwareKeyboardModule: Module {
private var keyboardObservers: [NSObjectProtocol] = []
public func definition() -> ModuleDefinition {
Name("PaseoHardwareKeyboard")
Events(hardwareSubmitEventName, hardwareShortcutEventName, "onHardwareKeyboardPresenceChanged")
OnCreate {
activeModule = self
for name in [Notification.Name.GCKeyboardDidConnect, Notification.Name.GCKeyboardDidDisconnect] {
self.keyboardObservers.append(NotificationCenter.default.addObserver(forName: name, object: nil, queue: .main) { [weak self] _ in
self?.sendEvent("onHardwareKeyboardPresenceChanged", [:])
})
}
}
Function("getHardwareKeyboardPresent") { GCKeyboard.coalesced != nil }
Function("setHardwareKeyboardSubmitEnabled") { (enabled: Bool) in
DispatchQueue.main.async {
isHardwareSubmitEnabled = enabled
}
}
Function("setOverlayKeys") { (keys: [String]) in
DispatchQueue.main.async {
overlayKeys = Set(keys).intersection(["Enter", "ArrowUp", "ArrowDown"])
UIMenuSystem.main.setNeedsRebuild()
}
}
OnDestroy {
self.keyboardObservers.forEach(NotificationCenter.default.removeObserver)
self.keyboardObservers = []
if activeModule === self {
activeModule = nil
isHardwareSubmitEnabled = false
overlayKeys = []
}
}
}
fileprivate func emitHardwareKeyboardSubmit() {
sendEvent(hardwareSubmitEventName, [:])
}
fileprivate func emitHardwareKeyboardPress(_ press: UIPress, isRepeat: Bool) {
guard let key = press.key else { return }
let code = hardwareKeyCode(key)
let characters = key.characters
let named = namedKeyInputs.first { $0.value == characters }?.key
let keyName: String
if code.hasPrefix("Meta") { keyName = "Meta" }
else if code.hasPrefix("Control") { keyName = "Control" }
else if code.hasPrefix("Shift") { keyName = "Shift" }
else if code.hasPrefix("Alt") { keyName = "Alt" }
else { keyName = named ?? (characters.isEmpty ? code : characters) }
guard !code.isEmpty else { return }
sendEvent(hardwareShortcutEventName, [
"key": keyName,
"code": code,
"altKey": key.modifierFlags.contains(.alternate),
"ctrlKey": key.modifierFlags.contains(.control),
"metaKey": key.modifierFlags.contains(.command),
"shiftKey": key.modifierFlags.contains(.shift),
"repeat": isRepeat,
"isComposing": isComposingText(),
"phase": press.phase == .ended || press.phase == .cancelled ? "up" : "down",
])
}
}
public final class PaseoHardwareKeyboardWindow: UIWindow {
private var pressedCodes: Set<Int> = []
public static func observing(_ original: UIWindow) -> UIWindow {
if original is PaseoHardwareKeyboardWindow { return original }
guard let scene = original.windowScene else { return original }
let observed = PaseoHardwareKeyboardWindow(windowScene: scene)
observed.windowLevel = original.windowLevel
observed.backgroundColor = original.backgroundColor
let controller = original.rootViewController
original.rootViewController = nil
original.isHidden = true
observed.rootViewController = controller
observed.makeKeyAndVisible()
return observed
}
public override func sendEvent(_ event: UIEvent) {
if let event = event as? UIPressesEvent {
for press in event.allPresses where press.phase == .began || press.phase == .ended || press.phase == .cancelled {
guard let key = press.key else { continue }
let raw = Int(key.keyCode.rawValue)
let isRepeat = press.phase == .began && pressedCodes.contains(raw)
if press.phase == .began { pressedCodes.insert(raw) }
else { pressedCodes.remove(raw) }
activeModule?.emitHardwareKeyboardPress(press, isRepeat: isRepeat)
}
}
super.sendEvent(event)
}
public override var keyCommands: [UIKeyCommand]? {
var commands = super.keyCommands ?? []
if isHardwareSubmitEnabled && !isPlainReturnClaimed && !isComposingText()
&& UIDevice.current.userInterfaceIdiom == .pad
{
commands.append(makeSubmitCommand())
}
if !isComposingText() { commands.append(contentsOf: overlayKeys.compactMap(makeShortcutCommand)) }
return commands
}
/// Whether something registered has already asked for an unmodified Return.
///
/// The submit command is registered for as long as the composer holds focus,
/// which includes while a picker is open over it. Registering both would put
/// two commands on one press, and the composer would win a press the picker
/// asked for — Enter would create the workspace instead of choosing a project.
private var isPlainReturnClaimed: Bool {
overlayKeys.contains("Enter")
}
private func makeSubmitCommand() -> UIKeyCommand {
let command = UIKeyCommand(
input: "\r",
modifierFlags: [],
action: #selector(handleHardwareKeyboardSubmit(_:))
)
if #available(iOS 15.0, *) {
command.wantsPriorityOverSystemBehavior = true
}
return command
}
private func makeShortcutCommand(_ key: String) -> UIKeyCommand? {
guard let input = namedKeyInputs[key] else { return nil }
let command = UIKeyCommand(input: input, modifierFlags: [], action: #selector(handleHardwareKeyboardShortcut(_:)))
if #available(iOS 15.0, *) { command.wantsPriorityOverSystemBehavior = true }
return command
}
@objc
private func handleHardwareKeyboardSubmit(_ sender: UIKeyCommand) {
guard canSubmitCurrentTextInput() else {
return
}
activeModule?.emitHardwareKeyboardSubmit()
}
@objc
private func handleHardwareKeyboardShortcut(_ sender: UIKeyCommand) {}
private func canSubmitCurrentTextInput() -> Bool {
guard let responder = UIResponder.paseoCurrentFirstResponder else {
return false
}
guard let textInput = responder as? UITextInput else {
return false
}
return textInput.markedTextRange == nil
}
}
private extension UIResponder {
private static weak var currentFirstResponder: UIResponder?
static var paseoCurrentFirstResponder: UIResponder? {
currentFirstResponder = nil
UIApplication.shared.sendAction(
#selector(captureCurrentFirstResponder(_:)),
to: nil,
from: nil,
for: nil
)
return currentFirstResponder
}
@objc
private func captureCurrentFirstResponder(_ sender: Any?) {
UIResponder.currentFirstResponder = self
}
}
private func hardwareKeyCode(_ key: UIKey) -> String {
let raw = key.keyCode.rawValue
if raw >= 4 && raw <= 29 {
let letter = UnicodeScalar(Int(raw) + 61)!
return "Key" + String(letter)
}
if raw >= 30 && raw <= 38 { return "Digit" + String(raw - 29) }
if raw == 39 { return "Digit0" }
let codes: [Int: String] = [
40: "Enter", 41: "Escape", 42: "Backspace", 43: "Tab", 44: "Space",
45: "Minus", 46: "Equal", 47: "BracketLeft", 48: "BracketRight",
49: "Backslash", 51: "Semicolon", 52: "Quote", 53: "Backquote",
54: "Comma", 55: "Period", 56: "Slash", 76: "Delete", 79: "ArrowRight",
80: "ArrowLeft", 81: "ArrowDown", 82: "ArrowUp", 224: "ControlLeft",
225: "ShiftLeft", 226: "AltLeft", 227: "MetaLeft", 228: "ControlRight",
229: "ShiftRight", 230: "AltRight", 231: "MetaRight",
]
if raw >= 58 && raw <= 69 { return "F" + String(raw - 57) }
return codes[Int(raw)] ?? ""
}