Let plugins stream live events and binary data to the app (#8527)

* feat(plugins): stream events and bytes between plugins and the app

Plugins can declare a subscription with defineSubscription, handle it with
server.handle(contract, (input, { push, signal })), and listen from a surface
with useSubscription or client.subscribe. A top-level Uint8Array in an event,
an RPC output, or an RPC input crosses the wire as a binary frame (opcodes
0x20-0x2f) without JSON or base64; a nested one fails validation on the sender.

Each subscriber runs its own handler, owned by the session that subscribed,
so release, disconnect, plugin stop and reload need no plugin code. The
runtime buffers pushes until the subscribe response has gone out, so a client
never sees an event for a subscription it does not know yet.

Gated on server_info.features.pluginSubscriptions; binary outputs reach only
clients that advertise plugin_binary_payloads.

* fix(plugins): close subscription gaps found in review

- Losing daemon.manage now releases the session's plugin subscriptions;
  binary events bypass the outbound permission check, so streams would
  otherwise outlive the revocation.
- A subscribe that times out sends unsubscribe, since the plugin may
  still start the handler after the daemon gave up.
- The byte limit leaves room for the largest frame header and routing,
  and lives in assertPluginWireValue so events, outputs and inputs share it.
- A release right after the host accepted keeps `ready` resolved, so a
  handler that returns at once reports closed rather than an error.
- Session trace logging tolerates loggers without isLevelEnabled.

* fix(plugins): withhold binary plugin results after plugin access is revoked

A binary RPC output or event skipped the outbound permission check its
JSON form passes, so an RPC in flight when daemon.manage was revoked still
delivered its bytes. Each frame now checks the permission of the JSON
message it stands in for.

* test(plugins): cover subscriptions in a real browser instead of JSDOM

The app subscription tests mounted hooks in JSDOM, which docs/testing.md
rules out. The adapter and contract behavior is now a plain unit test,
and a Playwright spec drives the diff-watch and media-stream examples:
JSON events following file changes, streamed bytes, and an upload.

* test(plugins): add the plain subscription unit test and the browser spec

* fix(plugins): apply subscription schema transforms once

Both ends parsed each subscription value, and the sender sent its parse
output, so the receiver parsed a schema's output as fresh input. An event
schema like z.string().transform((s) => s.length) closed the stream as
invalid. The sender now validates and sends the value as given; the
receiver's parse produces what the handler or callback sees.

* fix(plugins): parse a subscription's input once in the plugin process

server.handle wrapped subscription handlers in a second input parse, so a
transforming input schema rejected its own parsed output before the
handler ran.

plugin-examples/diff-watch/OVERVIEW.md

@@ -0,0 +1,7 @@
+# Live changes
+
+See which files differ from `HEAD` in a workspace panel that updates as you edit. The daemon watches the workspace and pushes the changed-file list to the panel; the panel never polls.
+
+Open **Open live changes** in the Command Center, or pick **Live changes** from the workspace tab menu. Each open panel runs its own watcher, which stops when you close the panel.
+
+The watcher reads the working tree recursively, so a checkout with very large ignored directories makes it slower to start.

plugin-examples/diff-watch/README.md

@@ -0,0 +1,14 @@
+# Diff watch plugin example
+
+A workspace panel subscribes to a daemon-side watcher and lists the files that differ from `HEAD`,
+updating as files change.
+
+```bash
+paseo plugin add /absolute/path/to/paseo/plugin-examples/diff-watch
+```
+
+- `shared/diff.ts` defines the `diff.watch` subscription contract.
+- `index.server.ts` handles it: an async handler that loops until the subscriber leaves, pushing
+  the file list whenever it changes. A git failure rejects the handler and closes the subscription
+  with that error.
+- `client/diff-panel.tsx` calls `useSubscription` and renders the latest list.

plugin-examples/diff-watch/client/diff-panel.tsx

@@ -0,0 +1,65 @@
+import {
+  useSubscription,
+  useWorkspace,
+  type PluginWorkspacePanelProps,
+} from "@getpaseo/plugin/client";
+import { useMemo, useState } from "react";
+import { ScrollView, StyleSheet, Text, View } from "react-native";
+import { diffWatch, type ChangedFile } from "../shared/diff";
+
+const BASE = "HEAD";
+
+export function DiffPanel({ workspaceId, theme, layout }: PluginWorkspacePanelProps) {
+  const cwd = useWorkspace(workspaceId, (workspace) => workspace.directory);
+  const styles = useStyles(theme, layout.compact);
+  if (!cwd) return <Text style={styles.muted}>Workspace unavailable</Text>;
+  return <ChangedFiles cwd={cwd} styles={styles} />;
+}
+
+function ChangedFiles({ cwd, styles }: { cwd: string; styles: ReturnType<typeof useStyles> }) {
+  const [files, setFiles] = useState<ChangedFile[] | null>(null);
+  const subscription = useSubscription(diffWatch, { cwd, from: BASE }, (event) => {
+    setFiles(event.files);
+  });
+  return (
+    <ScrollView style={styles.screen} contentContainerStyle={styles.content}>
+      <Text style={styles.title}>Changes against {BASE}</Text>
+      <Text style={styles.muted}>{describe(subscription, files)}</Text>
+      {files?.map((file) => (
+        <View key={file.path} style={styles.row}>
+          <Text style={styles.status}>{file.status}</Text>
+          <Text style={styles.path} numberOfLines={1}>
+            {file.path}
+          </Text>
+        </View>
+      ))}
+    </ScrollView>
+  );
+}
+
+function describe(
+  subscription: ReturnType<typeof useSubscription>,
+  files: ChangedFile[] | null,
+): string {
+  if (subscription.status === "error") return subscription.error.message;
+  if (subscription.status === "closed") return "Stopped watching";
+  if (!files) return "Reading changes…";
+  if (files.length === 0) return "No changes";
+  return files.length === 1 ? "1 file changed" : `${files.length} files changed`;
+}
+
+function useStyles(theme: PluginWorkspacePanelProps["theme"], compact: boolean) {
+  return useMemo(
+    () =>
+      StyleSheet.create({
+        screen: { flex: 1, backgroundColor: theme.colors.surface0 },
+        content: { padding: compact ? 16 : 24, gap: 8 },
+        title: { color: theme.colors.foreground, fontSize: 18, fontWeight: "600" },
+        muted: { color: theme.colors.foregroundMuted },
+        row: { flexDirection: "row", gap: 12, alignItems: "center" },
+        status: { color: theme.colors.foregroundMuted, width: 72 },
+        path: { color: theme.colors.foreground, flex: 1 },
+      }),
+    [theme, compact],
+  );
+}

plugin-examples/diff-watch/index.client.tsx

@@ -0,0 +1,22 @@
+import type { PluginClientContext } from "@getpaseo/plugin/client";
+import { DiffPanel } from "./client/diff-panel";
+
+export default function contribute(client: PluginClientContext) {
+  client.addWorkspacePanel({
+    id: "changes",
+    title: "Live changes",
+    icon: "GitCompare",
+    context: "workspace",
+    Component: DiffPanel,
+  });
+  client.addCommandCenterItem({
+    id: "open-changes",
+    title: "Open live changes",
+    icon: "GitCompare",
+    context: "workspace",
+    onSelect({ openPanel }) {
+      openPanel("changes");
+    },
+  });
+  return () => {};
+}

plugin-examples/diff-watch/index.server.ts

@@ -0,0 +1,19 @@
+import type { PluginServerContext } from "@getpaseo/plugin/server";
+import { readChangedFiles, watchDirectory } from "./server/diff";
+import { diffWatch } from "./shared/diff";
+
+export default function contribute(server: PluginServerContext) {
+  // The handler runs for as long as the panel is subscribed; a git failure closes it with the error.
+  server.handle(diffWatch, async (input, { push, signal }) => {
+    const directory = watchDirectory(input.cwd, signal);
+    let previous = "";
+    while (await directory.settled()) {
+      const files = await readChangedFiles(input);
+      const current = JSON.stringify(files);
+      if (current === previous) continue;
+      previous = current;
+      await push({ type: "changes", files });
+    }
+  });
+  return () => {};
+}

plugin-examples/diff-watch/paseo-plugin.json

@@ -0,0 +1,6 @@
+{
+  "id": "diff-watch",
+  "requirements": {
+    "paseo": ">=0.15.0-beta.2"
+  }
+}

plugin-examples/diff-watch/server/diff.ts

@@ -0,0 +1,79 @@
+import { watch } from "node:fs";
+import type { SubscriptionInput } from "@getpaseo/plugin";
+import { execCommand } from "@getpaseo/plugin/server";
+import type { ChangedFile, diffWatch } from "../shared/diff";
+
+const SETTLE_MS = 150;
+
+const STATUSES: Record<string, ChangedFile["status"]> = {
+  A: "added",
+  M: "modified",
+  D: "deleted",
+  R: "renamed",
+};
+
+/** Lists changed files the way `git diff --name-status` does, plus untracked files. */
+export async function readChangedFiles({
+  cwd,
+  from,
+  to,
+}: SubscriptionInput<typeof diffWatch>): Promise<ChangedFile[]> {
+  const range = to ? [from, to] : [from];
+  const { stdout } = await execCommand("git", ["diff", "--name-status", "-z", ...range], { cwd });
+  const files = parseNameStatus(stdout);
+  if (!to) {
+    const untracked = await execCommand(
+      "git",
+      ["ls-files", "--others", "--exclude-standard", "-z"],
+      {
+        cwd,
+      },
+    );
+    for (const path of untracked.stdout.split("\0").filter(Boolean))
+      files.push({ path, status: "added" });
+  }
+  return files.sort((left, right) => left.path.localeCompare(right.path));
+}
+
+function parseNameStatus(output: string): ChangedFile[] {
+  const fields = output.split("\0").filter(Boolean);
+  const files: ChangedFile[] = [];
+  for (let index = 0; index < fields.length; index += 1) {
+    const code = fields[index][0];
+    // Renames carry the old path before the new one.
+    if (code === "R") index += 1;
+    const path = fields[index + 1];
+    index += 1;
+    if (path) files.push({ path, status: STATUSES[code] ?? "modified" });
+  }
+  return files;
+}
+
+/** Resolves each time the directory settles after a change, and with false once `signal` aborts. */
+export function watchDirectory(cwd: string, signal: AbortSignal) {
+  let changed = true;
+  let wake: (() => void) | null = null;
+  const watcher = watch(cwd, { recursive: true }, () => {
+    changed = true;
+    wake?.();
+  });
+  signal.addEventListener(
+    "abort",
+    () => {
+      watcher.close();
+      wake?.();
+    },
+    { once: true },
+  );
+  return {
+    async settled(): Promise<boolean> {
+      // The watcher and the abort both wake a wait.
+      if (!changed && !signal.aborted) await new Promise<void>((resolve) => (wake = resolve));
+      wake = null;
+      if (signal.aborted) return false;
+      await new Promise((resolve) => setTimeout(resolve, SETTLE_MS));
+      changed = false;
+      return !signal.aborted;
+    },
+  };
+}

plugin-examples/diff-watch/shared/diff.ts

@@ -0,0 +1,16 @@
+import { defineSubscription } from "@getpaseo/plugin";
+import { z } from "zod";
+
+export const ChangedFileSchema = z.object({
+  path: z.string(),
+  status: z.enum(["added", "modified", "deleted", "renamed"]),
+});
+
+export type ChangedFile = z.infer<typeof ChangedFileSchema>;
+
+/** Files that differ between `from` and `to`; without `to`, the working tree. */
+export const diffWatch = defineSubscription({
+  name: "diff.watch",
+  input: z.object({ cwd: z.string(), from: z.string(), to: z.string().optional() }),
+  event: z.object({ type: z.literal("changes"), files: z.array(ChangedFileSchema) }),
+});

plugin-examples/media-stream/OVERVIEW.md

@@ -0,0 +1,7 @@
+# Media stream
+
+Watch a moving test pattern and an audio level meter, streamed live from your machine into a workspace panel. Video and audio arrive together on one connection, and the counters show how many frames of each have played.
+
+Open **Open media stream** in the Command Center, or pick **Media stream** from the workspace tab menu. Closing the panel stops the stream.
+
+Press **Send this frame back** to upload the picture on screen to your machine. The panel shows the size and checksum your machine received. A 16 by 9 picture is 432 bytes.

plugin-examples/media-stream/README.md

@@ -0,0 +1,17 @@
+# Media stream plugin example
+
+One subscription carries two binary streams. Each pushed `Uint8Array` is a binary frame; its first
+byte says whether it is video, a keyframe, or audio.
+
+```bash
+paseo plugin add /absolute/path/to/paseo/plugin-examples/media-stream
+```
+
+- `shared/media-frames.ts` owns the byte layout: `encodeMediaFrame`, `decodeMediaFrame`, and the
+  `media.stream` contract. `decodeMediaFrame` returns a view, so the panel never copies a payload.
+- `server/test-pattern.ts` is a synthetic source; replace it with a real capture.
+- `index.server.ts` pushes each source callback as a frame and stops the source when the
+  subscription's `signal` aborts.
+- `client/media-panel.tsx` decodes each frame and routes it by opcode.
+- `shared/frame-check.ts` is an RPC whose input is a `Uint8Array`: the panel uploads the picture
+  on screen and `server/frame-check.ts` reports its size and checksum.

plugin-examples/media-stream/client/media-panel.tsx

@@ -0,0 +1,196 @@
+import { useRpc, useSubscription, type PluginWorkspacePanelProps } from "@getpaseo/plugin/client";
+import type { RpcOutput } from "@getpaseo/plugin";
+import { type UseMutationResult, useMutation } from "@tanstack/react-query";
+import { useCallback, useMemo, useState } from "react";
+import { Pressable, StyleSheet, Text, View } from "react-native";
+import { checkFrame } from "../shared/frame-check";
+import {
+  decodeMediaFrame,
+  MediaOpcode,
+  mediaStream,
+  VIDEO,
+  type MediaFrame,
+} from "../shared/media-frames";
+
+interface Playback {
+  picture: Uint8Array | null;
+  level: number;
+  videoFrames: number;
+  keyframes: number;
+  audioChunks: number;
+  bytes: number;
+}
+
+const EMPTY: Playback = {
+  picture: null,
+  level: 0,
+  videoFrames: 0,
+  keyframes: 0,
+  audioChunks: 0,
+  bytes: 0,
+};
+
+export function MediaPanel({ theme, layout }: PluginWorkspacePanelProps) {
+  const styles = useStyles(theme, layout.compact);
+  const [playback, setPlayback] = useState(EMPTY);
+  const subscription = useSubscription(mediaStream, { source: "test-pattern" }, (bytes) => {
+    const frame = decodeMediaFrame(bytes);
+    if (frame) setPlayback((current) => play(current, frame, bytes.byteLength));
+  });
+  return (
+    <View style={styles.screen}>
+      <Text style={styles.title}>Test pattern</Text>
+      <Text style={styles.muted}>
+        {subscription.status === "error" ? subscription.error.message : subscription.status}
+      </Text>
+      <Picture pixels={playback.picture} cellSize={layout.compact ? 14 : 20} />
+      <LevelMeter level={playback.level} styles={styles} />
+      <Text style={styles.muted}>
+        {playback.videoFrames} video frames ({playback.keyframes} keyframes), {playback.audioChunks}{" "}
+        audio chunks, {(playback.bytes / 1024).toFixed(0)} KiB
+      </Text>
+      <SendFrameBack picture={playback.picture} styles={styles} />
+    </View>
+  );
+}
+
+/** Uploads the picture on screen and shows what the daemon received. */
+function SendFrameBack({
+  picture,
+  styles,
+}: {
+  picture: Uint8Array | null;
+  styles: ReturnType<typeof useStyles>;
+}) {
+  const sendFrame = useRpc(checkFrame);
+  const check = useMutation({ mutationFn: sendFrame });
+  const { mutate } = check;
+  const send = useCallback(() => {
+    if (picture) mutate(picture);
+  }, [mutate, picture]);
+  return (
+    <View style={styles.upload}>
+      <Pressable
+        accessibilityRole="button"
+        disabled={!picture || check.isPending}
+        onPress={send}
+        style={styles.button}
+      >
+        <Text style={styles.buttonText}>Send this frame back</Text>
+      </Pressable>
+      <Text style={styles.muted}>{describeCheck(check)}</Text>
+    </View>
+  );
+}
+
+function describeCheck(
+  check: UseMutationResult<RpcOutput<typeof checkFrame>, Error, Uint8Array>,
+): string {
+  if (check.status === "error") return check.error.message;
+  if (check.status === "success")
+    return `The daemon received ${check.data.bytes} bytes, sha256 ${check.data.sha256.slice(0, 12)}`;
+  return check.status === "pending" ? "Sending…" : "";
+}
+
+function play(current: Playback, frame: MediaFrame, bytes: number): Playback {
+  if (frame.opcode === MediaOpcode.Audio)
+    return {
+      ...current,
+      level: audioLevel(frame.payload),
+      audioChunks: current.audioChunks + 1,
+      bytes: current.bytes + bytes,
+    };
+  return {
+    ...current,
+    picture: frame.payload,
+    videoFrames: current.videoFrames + 1,
+    keyframes: current.keyframes + (frame.opcode === MediaOpcode.Keyframe ? 1 : 0),
+    bytes: current.bytes + bytes,
+  };
+}
+
+function audioLevel(samples: Uint8Array): number {
+  const view = new DataView(samples.buffer, samples.byteOffset, samples.byteLength);
+  let sum = 0;
+  const count = samples.byteLength / 2;
+  for (let index = 0; index < count; index += 1) sum += view.getInt16(index * 2, true) ** 2;
+  return Math.min(1, Math.sqrt(sum / Math.max(1, count)) / 16384);
+}
+
+function LevelMeter({ level, styles }: { level: number; styles: ReturnType<typeof useStyles> }) {
+  const fill = useMemo(
+    () => [styles.meter, { width: `${Math.round(level * 100)}%` as const }],
+    [level, styles],
+  );
+  return (
+    <View style={styles.meterTrack}>
+      <View style={fill} />
+    </View>
+  );
+}
+
+function Picture({ pixels, cellSize }: { pixels: Uint8Array | null; cellSize: number }) {
+  const rows = useMemo(() => pictureRows(pixels, cellSize), [pixels, cellSize]);
+  return (
+    <View accessibilityLabel="Test pattern video">
+      {rows.map((row) => (
+        <View key={row.key} style={pictureStyles.row}>
+          {row.cells.map((cell) => (
+            <View key={cell.key} style={cell.style} />
+          ))}
+        </View>
+      ))}
+    </View>
+  );
+}
+
+/** One style per pixel, row by row. */
+function pictureRows(pixels: Uint8Array | null, cellSize: number) {
+  const rows = [];
+  for (let y = 0; y < VIDEO.height; y += 1) {
+    const cells = [];
+    for (let x = 0; x < VIDEO.width; x += 1) {
+      const offset = (y * VIDEO.width + x) * 3;
+      const backgroundColor = pixels
+        ? `rgb(${pixels[offset]}, ${pixels[offset + 1]}, ${pixels[offset + 2]})`
+        : "transparent";
+      cells.push({ key: `${x}`, style: { width: cellSize, height: cellSize, backgroundColor } });
+    }
+    rows.push({ key: `${y}`, cells });
+  }
+  return rows;
+}
+
+const pictureStyles = StyleSheet.create({ row: { flexDirection: "row" } });
+
+function useStyles(theme: PluginWorkspacePanelProps["theme"], compact: boolean) {
+  return useMemo(
+    () =>
+      StyleSheet.create({
+        screen: {
+          flex: 1,
+          padding: compact ? 16 : 24,
+          gap: 12,
+          backgroundColor: theme.colors.surface0,
+        },
+        title: { color: theme.colors.foreground, fontSize: 18, fontWeight: "600" },
+        muted: { color: theme.colors.foregroundMuted },
+        meterTrack: {
+          height: 8,
+          borderRadius: 4,
+          overflow: "hidden",
+          backgroundColor: theme.colors.surface2,
+        },
+        meter: { height: 8, backgroundColor: theme.colors.accent },
+        upload: { gap: 8, alignItems: "flex-start" },
+        button: {
+          paddingVertical: 8,
+          paddingHorizontal: 14,
+          borderRadius: 8,
+          backgroundColor: theme.colors.accent,
+        },
+        buttonText: { color: theme.colors.accentForeground },
+      }),
+    [theme, compact],
+  );
+}

plugin-examples/media-stream/index.client.tsx

@@ -0,0 +1,22 @@
+import type { PluginClientContext } from "@getpaseo/plugin/client";
+import { MediaPanel } from "./client/media-panel";
+
+export default function contribute(client: PluginClientContext) {
+  client.addWorkspacePanel({
+    id: "media",
+    title: "Media stream",
+    icon: "MonitorPlay",
+    context: "workspace",
+    Component: MediaPanel,
+  });
+  client.addCommandCenterItem({
+    id: "open-media",
+    title: "Open media stream",
+    icon: "MonitorPlay",
+    context: "workspace",
+    onSelect({ openPanel }) {
+      openPanel("media");
+    },
+  });
+  return () => {};
+}

plugin-examples/media-stream/index.server.ts

@@ -0,0 +1,27 @@
+import type { PluginServerContext } from "@getpaseo/plugin/server";
+import { describeFrame } from "./server/frame-check";
+import { openTestPattern } from "./server/test-pattern";
+import { checkFrame } from "./shared/frame-check";
+import { encodeMediaFrame, MediaOpcode, mediaStream } from "./shared/media-frames";
+
+export default function contribute(server: PluginServerContext) {
+  // Two streams share one subscription; each frame's opcode says which one it belongs to.
+  server.handle(mediaStream, (_input, { push, signal }) => {
+    const source = openTestPattern({
+      video: (pixels, timestampMs, keyframe) =>
+        void push(
+          encodeMediaFrame({
+            opcode: keyframe ? MediaOpcode.Keyframe : MediaOpcode.Video,
+            timestampMs,
+            payload: pixels,
+          }),
+        ),
+      audio: (samples, timestampMs) =>
+        void push(encodeMediaFrame({ opcode: MediaOpcode.Audio, timestampMs, payload: samples })),
+    });
+    signal.addEventListener("abort", () => source.close(), { once: true });
+  });
+  // A Uint8Array input arrives as the bytes the panel sent.
+  server.handle(checkFrame, describeFrame);
+  return () => {};
+}

plugin-examples/media-stream/paseo-plugin.json

@@ -0,0 +1,6 @@
+{
+  "id": "media-stream",
+  "requirements": {
+    "paseo": ">=0.15.0-beta.2"
+  }
+}

plugin-examples/media-stream/server/frame-check.ts

@@ -0,0 +1,7 @@
+import { createHash } from "node:crypto";
+import type { RpcInput, RpcOutput } from "@getpaseo/plugin";
+import type { checkFrame } from "../shared/frame-check";
+
+export function describeFrame(frame: RpcInput<typeof checkFrame>): RpcOutput<typeof checkFrame> {
+  return { bytes: frame.byteLength, sha256: createHash("sha256").update(frame).digest("hex") };
+}

plugin-examples/media-stream/server/test-pattern.ts

@@ -0,0 +1,44 @@
+import { AUDIO, VIDEO } from "../shared/media-frames";
+
+export interface MediaSourceHandlers {
+  video(pixels: Uint8Array, timestampMs: number, keyframe: boolean): void;
+  audio(samples: Uint8Array, timestampMs: number): void;
+}
+
+/** A synthetic source: a moving gradient and a tone that rises and falls. */
+export function openTestPattern(handlers: MediaSourceHandlers): { close(): void } {
+  const started = Date.now();
+  let frame = 0;
+  let sample = 0;
+  const video = setInterval(() => {
+    const pixels = new Uint8Array(VIDEO.width * VIDEO.height * 3);
+    for (let y = 0; y < VIDEO.height; y += 1)
+      for (let x = 0; x < VIDEO.width; x += 1) {
+        const offset = (y * VIDEO.width + x) * 3;
+        pixels[offset] = ((x + frame) * 16) % 256;
+        pixels[offset + 1] = (y * 28) % 256;
+        pixels[offset + 2] = ((x + y + frame) * 8) % 256;
+      }
+    handlers.video(pixels, Date.now() - started, frame % VIDEO.keyframeInterval === 0);
+    frame += 1;
+  }, 1000 / VIDEO.framesPerSecond);
+  const audio = setInterval(() => {
+    const count = (AUDIO.sampleRate * AUDIO.chunkMs) / 1000;
+    const samples = new Uint8Array(count * 2);
+    const view = new DataView(samples.buffer);
+    const amplitude = 8000 + 7000 * Math.sin((Date.now() - started) / 700);
+    for (let index = 0; index < count; index += 1, sample += 1)
+      view.setInt16(
+        index * 2,
+        amplitude * Math.sin((2 * Math.PI * 440 * sample) / AUDIO.sampleRate),
+        true,
+      );
+    handlers.audio(samples, Date.now() - started);
+  }, AUDIO.chunkMs);
+  return {
+    close() {
+      clearInterval(video);
+      clearInterval(audio);
+    },
+  };
+}

plugin-examples/media-stream/shared/frame-check.ts

@@ -0,0 +1,9 @@
+import { defineRpc } from "@getpaseo/plugin";
+import { z } from "zod";
+
+/** Sends one frame back to the daemon, which reports what arrived. */
+export const checkFrame = defineRpc({
+  name: "media.frame.check",
+  input: z.instanceof(Uint8Array),
+  output: z.object({ bytes: z.number(), sha256: z.string() }),
+});

plugin-examples/media-stream/shared/media-frames.ts

@@ -0,0 +1,47 @@
+import { defineSubscription } from "@getpaseo/plugin";
+import { z } from "zod";
+
+/** The first byte of every frame says which stream it belongs to. */
+export const MediaOpcode = { Video: 0x01, Audio: 0x02, Keyframe: 0x03 } as const;
+
+export type MediaFrame =
+  | {
+      opcode: typeof MediaOpcode.Video | typeof MediaOpcode.Keyframe;
+      timestampMs: number;
+      payload: Uint8Array;
+    }
+  | { opcode: typeof MediaOpcode.Audio; timestampMs: number; payload: Uint8Array };
+
+const HEADER_BYTES = 1 + 8;
+
+/** `[opcode u8][timestampMs f64][payload]` */
+export function encodeMediaFrame(frame: MediaFrame): Uint8Array {
+  const bytes = new Uint8Array(HEADER_BYTES + frame.payload.byteLength);
+  bytes[0] = frame.opcode;
+  new DataView(bytes.buffer).setFloat64(1, frame.timestampMs);
+  bytes.set(frame.payload, HEADER_BYTES);
+  return bytes;
+}
+
+/** Returns a view into `bytes`; the payload is not copied. */
+export function decodeMediaFrame(bytes: Uint8Array): MediaFrame | null {
+  if (bytes.byteLength < HEADER_BYTES) return null;
+  const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
+  const opcode = bytes[0];
+  const timestampMs = view.getFloat64(1);
+  const payload = bytes.subarray(HEADER_BYTES);
+  if (opcode === MediaOpcode.Audio) return { opcode, timestampMs, payload };
+  if (opcode === MediaOpcode.Video || opcode === MediaOpcode.Keyframe)
+    return { opcode, timestampMs, payload };
+  return null;
+}
+
+/** A video frame is `width * height` RGB pixels; audio is signed 16-bit mono PCM. */
+export const VIDEO = { width: 16, height: 9, framesPerSecond: 15, keyframeInterval: 30 } as const;
+export const AUDIO = { sampleRate: 8000, chunkMs: 20 } as const;
+
+export const mediaStream = defineSubscription({
+  name: "media.stream",
+  input: z.object({ source: z.enum(["test-pattern"]) }),
+  event: z.instanceof(Uint8Array),
+});