puter/paseo
1
0
Fork 0
mirror of https://github.com/getpaseo/paseo.git synced 2026-11-03 07:03:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

Let plugins stream live events and binary data to the app (#8527)

Browse Source 
* feat(plugins): stream events and bytes between plugins and the app

Plugins can declare a subscription with defineSubscription, handle it with
server.handle(contract, (input, { push, signal })), and listen from a surface
with useSubscription or client.subscribe. A top-level Uint8Array in an event,
an RPC output, or an RPC input crosses the wire as a binary frame (opcodes
0x20-0x2f) without JSON or base64; a nested one fails validation on the sender.

Each subscriber runs its own handler, owned by the session that subscribed,
so release, disconnect, plugin stop and reload need no plugin code. The
runtime buffers pushes until the subscribe response has gone out, so a client
never sees an event for a subscription it does not know yet.

Gated on server_info.features.pluginSubscriptions; binary outputs reach only
clients that advertise plugin_binary_payloads.

* fix(plugins): close subscription gaps found in review

- Losing daemon.manage now releases the session's plugin subscriptions;
  binary events bypass the outbound permission check, so streams would
  otherwise outlive the revocation.
- A subscribe that times out sends unsubscribe, since the plugin may
  still start the handler after the daemon gave up.
- The byte limit leaves room for the largest frame header and routing,
  and lives in assertPluginWireValue so events, outputs and inputs share it.
- A release right after the host accepted keeps `ready` resolved, so a
  handler that returns at once reports closed rather than an error.
- Session trace logging tolerates loggers without isLevelEnabled.

* fix(plugins): withhold binary plugin results after plugin access is revoked

A binary RPC output or event skipped the outbound permission check its
JSON form passes, so an RPC in flight when daemon.manage was revoked still
delivered its bytes. Each frame now checks the permission of the JSON
message it stands in for.

* test(plugins): cover subscriptions in a real browser instead of JSDOM

The app subscription tests mounted hooks in JSDOM, which docs/testing.md
rules out. The adapter and contract behavior is now a plain unit test,
and a Playwright spec drives the diff-watch and media-stream examples:
JSON events following file changes, streamed bytes, and an upload.

* test(plugins): add the plain subscription unit test and the browser spec

* fix(plugins): apply subscription schema transforms once

Both ends parsed each subscription value, and the sender sent its parse
output, so the receiver parsed a schema's output as fresh input. An event
schema like z.string().transform((s) => s.length) closed the stream as
invalid. The sender now validates and sends the value as given; the
receiver's parse produces what the handler or callback sees.

* fix(plugins): parse a subscription's input once in the plugin process

server.handle wrapped subscription handlers in a second input parse, so a
transforming input schema rejected its own parsed output before the
handler ran.
This commit is contained in:
Mohamed Boudra
2026-11-02 14:10:15 +07:00
committed by GitHub
parent 70cb34927a
commit d4ac64f335
80 changed files with 3237 additions and 87 deletions
Download Patch File Download Diff File Expand all files Collapse all files
public-docs/plugins/reference.md
+158
View File
@@ -2226,6 +2226,164 @@ Inputs and outputs are validated on both sides. RPC names start with a lowercase
Backend handlers receive the same `PaseoApi` as `{ paseo }`. Their connection belongs to the subprocess and closes when the plugin stops. It does not subscribe to timelines or catalog events until plugin code subscribes. Follow the [SDK event contract](/docs/sdk/events) for cleanup and timeline replacements. Backend code can use Node APIs and dependencies installed in the plugin directory.
## Stream events from the daemon {#subscriptions}
A subscription runs a daemon handler for as long as one client listens and sends that client typed
events. Use it for watchers, tails, and live media; use plugin RPC for a single answer.
Define the contract in `shared/`:
```ts
// shared/diff.ts
import { defineSubscription } from "@getpaseo/plugin";
import { z } from "zod";
export const diffWatch = defineSubscription({
name: "diff.watch",
input: z.object({ cwd: z.string(), from: z.string() }),
event: z.object({ type: z.literal("changes"), files: z.array(z.string()) }),
});
```
Handle it in `index.server.ts`. The handler receives `push` and an abort `signal` next to `paseo`:
```ts
server.handle(diffWatch, ({ cwd, from }, { push, signal }) => {
const watcher = watchChanges(cwd, from, (files) => void push({ type: "changes", files }));
signal.addEventListener("abort", () => watcher.close(), { once: true });
});
```
Subscribe from a surface with `useSubscription(contract, input, onEvent)`:
```tsx
const [files, setFiles] = useState<string[]>([]);
const subscription = useSubscription(diffWatch, { cwd, from: "HEAD" }, (event) => {
setFiles(event.files);
});
if (subscription.status === "error") return <Text>{subscription.error.message}</Text>;
```
Outside React, `client.subscribe` takes the same arguments plus options and returns a remover:
```ts
const stop = client.subscribe(diffWatch, { cwd, from: "HEAD" }, onChanges, { signal });
await stop.ready; // the daemon accepted the subscription
stop(); // or await stop.release()
```
RPCs and subscriptions share one name space; registering the same name for both fails the plugin
load.
### Handler lifetime
| The handler... | Result |
| --------------------------------------------- | ------------------------------------------------------------ |
| Throws synchronously, or the input is invalid | The subscribe call fails with the error. No events are sent. |
| Returns nothing | The subscription stays open until the client releases it. |
| Returns a promise that resolves | The subscription closes. |
| Returns a promise that rejects | The subscription closes with the error. |
`signal` aborts when the client releases the subscription, unmounts the hook, changes its input,
disconnects, or the plugin stops. Pushes after the abort are dropped. `push` validates the event
against the contract and resolves once the event has left the plugin process; await it in a loop
to pace a producer.
Each subscriber runs its own handler. Paseo does not share one handler between clients and does
not replay missed events: after a reconnect or a plugin reload, mounted surfaces subscribe again and
the handler starts from the beginning. Share an
expensive watcher inside your plugin when several subscribers need it.
`useSubscription` returns a union on `status`:
| Status | Meaning |
| ------------ | ---------------------------------------------------------------------------------------------- |
| `connecting` | Sent; the daemon has not accepted it yet. |
| `live` | The handler is running. Events arrive through `onEvent`. |
| `closed` | The handler finished. |
| `error` | The subscribe failed, the handler failed, an event failed the contract, or the plugin stopped. |
The hook subscribes again when `input` changes by shallow equality. `onEvent` is always the latest
render's callback, so an inline function does not resubscribe. `client.subscribe` takes
`{ signal, onClosed }`; `onClosed` receives `{ error? }`.
### Bytes
A top-level `Uint8Array` crosses the wire as binary, without JSON or base64. Everything else is
JSON. The rule applies to subscription events, RPC outputs, and RPC inputs. A Node `Buffer` is a
`Uint8Array`.
```ts
export const thumbnail = defineRpc({
name: "thumbnail.render",
input: z.object({ path: z.string(), width: z.number() }),
output: z.instanceof(Uint8Array),
});
// index.server.ts
server.handle(thumbnail, ({ path, width }) => renderPng(path, width));
// client
const renderThumbnail = useRpc(thumbnail);
const png = await renderThumbnail({ path, width }); // Uint8Array
```
Declare bytes with `z.instanceof(Uint8Array)`. An event can be bytes or JSON per push: declare a
union and branch with `event instanceof Uint8Array`. A `Uint8Array` nested in an object fails
validation on the sending side with an error that names the rule; send the bytes as their own
event or RPC value and the metadata as a separate JSON event.
One event or RPC value carries at most 100 MiB less 65 KiB of framing. Push large payloads in
chunks of a few hundred KiB; each `push` waits for the previous one to leave the plugin process.
Uploads from a phone have a lower limit: Android closes the connection when one send exceeds
16 MiB, so split a larger upload into several RPCs.
### Several streams on one subscription
Put a header byte in front of each payload to multiplex streams. Keep the layout in `shared/`, so
the handler and the surface call named functions:
```ts
// shared/media-frames.ts
export const MediaOpcode = { Video: 0x01, Audio: 0x02 } as const;
export function encodeMediaFrame(opcode: number, payload: Uint8Array): Uint8Array {
const bytes = new Uint8Array(1 + payload.byteLength);
bytes[0] = opcode;
bytes.set(payload, 1);
return bytes;
}
export function decodeMediaFrame(bytes: Uint8Array) {
return { opcode: bytes[0], payload: bytes.subarray(1) }; // a view, not a copy
}
export const mediaStream = defineSubscription({
name: "media.stream",
input: z.object({ source: z.string() }),
event: z.instanceof(Uint8Array),
});
```
```ts
useSubscription(mediaStream, { source }, (bytes) => {
const frame = decodeMediaFrame(bytes);
if (frame.opcode === MediaOpcode.Audio) audio.enqueue(frame.payload);
else video.draw(frame.payload);
});
```
Events from one subscription arrive in push order, JSON and binary alike.
`plugin-examples/media-stream` streams video and audio this way and uploads a frame back through
an RPC with a `Uint8Array` input; `plugin-examples/diff-watch` watches a working tree.
### Older hosts
Subscriptions and binary RPC values need a host that advertises them. Against an older host,
`useSubscription` reports `error` asking to update the host, `client.subscribe` rejects `ready`
with the same message, and an RPC with a `Uint8Array` input or output fails. Raise your plugin's
`requirements.paseo` to the release that ships subscriptions.
## Debug backend output
Backend contributions can write to stdout and stderr with normal Node logging:
public-docs/plugins/reference.zh-CN.md
+134
View File
@@ -1924,6 +1924,140 @@ export default function contribute(server: PluginServerContext) {
后端处理函数以 `{ paseo }` 的形式收到同样的 `PaseoApi`。它们的连接属于子进程，并在插件停止时关闭。在插件代码订阅之前，它不会订阅时间线或目录事件。清理和时间线替换请遵循 [SDK 事件契约](/docs/sdk/events)。后端代码可以使用 Node API 以及安装在插件目录中的依赖。
## 从 daemon 推送事件 {#subscriptions}
订阅会在一个客户端监听期间运行 daemon 处理函数，并向该客户端发送带类型的事件。监听文件、追踪日志和实时媒体使用订阅；只需要一次应答时使用插件 RPC。
在 `shared/` 中定义契约：
```ts
// shared/diff.ts
import { defineSubscription } from "@getpaseo/plugin";
import { z } from "zod";
export const diffWatch = defineSubscription({
name: "diff.watch",
input: z.object({ cwd: z.string(), from: z.string() }),
event: z.object({ type: z.literal("changes"), files: z.array(z.string()) }),
});
```
在 `index.server.ts` 中处理它。处理函数除了 `paseo`，还会收到 `push` 和一个中止 `signal`：
```ts
server.handle(diffWatch, ({ cwd, from }, { push, signal }) => {
const watcher = watchChanges(cwd, from, (files) => void push({ type: "changes", files }));
signal.addEventListener("abort", () => watcher.close(), { once: true });
});
```
在界面中用 `useSubscription(contract, input, onEvent)` 订阅：
```tsx
const [files, setFiles] = useState<string[]>([]);
const subscription = useSubscription(diffWatch, { cwd, from: "HEAD" }, (event) => {
setFiles(event.files);
});
if (subscription.status === "error") return <Text>{subscription.error.message}</Text>;
```
在 React 之外，`client.subscribe` 接受同样的参数和一个选项对象，并返回一个移除函数：
```ts
const stop = client.subscribe(diffWatch, { cwd, from: "HEAD" }, onChanges, { signal });
await stop.ready; // the daemon accepted the subscription
stop(); // or await stop.release()
```
RPC 和订阅共用一个名称空间；同一个名称同时注册为 RPC 和订阅会导致插件加载失败。
### 处理函数的生命周期 {#handler-lifetime}
| 处理函数…… | 结果 |
| ------------------------ | -------------------------------------------- |
| 同步抛出错误，或输入无效 | 订阅调用失败并返回该错误，不会发送任何事件。 |
| 不返回任何值 | 订阅保持打开，直到客户端释放它。 |
| 返回一个兑现的 Promise | 订阅关闭。 |
| 返回一个拒绝的 Promise | 订阅关闭并带上该错误。 |
当客户端释放订阅、卸载 hook、更改输入、断开连接，或者插件停止时，`signal` 会中止。中止之后的推送会被丢弃。`push` 会按契约校验事件，并在事件离开插件进程后兑现；在循环中 await 它可以控制生产者的速度。
每个订阅者运行各自的处理函数。Paseo 不会在多个客户端之间共享同一个处理函数，也不会重放错过的事件：重新连接或重新加载插件之后，已挂载的界面会重新订阅，处理函数从头开始运行。多个订阅者需要同一个昂贵的监听器时，请在插件内部共享它。
`useSubscription` 返回一个以 `status` 区分的联合类型：
| 状态 | 含义 |
| ------------ | -------------------------------------------------------- |
| `connecting` | 已发送，daemon 尚未接受。 |
| `live` | 处理函数正在运行，事件通过 `onEvent` 送达。 |
| `closed` | 处理函数已结束。 |
| `error` | 订阅失败、处理函数失败、事件不符合契约，或者插件已停止。 |
`input` 按浅比较发生变化时，hook 会重新订阅。`onEvent` 始终是最近一次渲染的回调，所以内联函数不会导致重新订阅。`client.subscribe` 接受 `{ signal, onClosed }`；`onClosed` 会收到 `{ error? }`。
### 字节 {#bytes}
顶层的 `Uint8Array` 以二进制形式传输，不经过 JSON 或 base64，其他值都是 JSON。这条规则适用于订阅事件、RPC 输出和 RPC 输入。Node 的 `Buffer` 也是 `Uint8Array`。
```ts
export const thumbnail = defineRpc({
name: "thumbnail.render",
input: z.object({ path: z.string(), width: z.number() }),
output: z.instanceof(Uint8Array),
});
// index.server.ts
server.handle(thumbnail, ({ path, width }) => renderPng(path, width));
// client
const renderThumbnail = useRpc(thumbnail);
const png = await renderThumbnail({ path, width }); // Uint8Array
```
用 `z.instanceof(Uint8Array)` 声明字节。事件可以在每次推送时选择字节或 JSON：声明一个联合类型，并用 `event instanceof Uint8Array` 分支。嵌套在对象中的 `Uint8Array` 会在发送端校验失败，错误信息会说明这条规则；把字节作为单独的事件或 RPC 值发送，元数据作为另一个 JSON 事件发送。
一个事件或 RPC 值最多 100 MiB（减去 65 KiB 的帧头）。大载荷请按几百 KiB 分块推送；每次 `push` 都会等待上一次离开插件进程。手机上传的上限更低：一次发送超过 16 MiB 时 Android 会关闭连接，所以较大的上传请拆成多个 RPC。
### 在一个订阅中传输多条流 {#several-streams-on-one-subscription}
在每个载荷前加一个头字节即可复用多条流。把布局放在 `shared/` 中，让处理函数和界面调用有名字的函数：
```ts
// shared/media-frames.ts
export const MediaOpcode = { Video: 0x01, Audio: 0x02 } as const;
export function encodeMediaFrame(opcode: number, payload: Uint8Array): Uint8Array {
const bytes = new Uint8Array(1 + payload.byteLength);
bytes[0] = opcode;
bytes.set(payload, 1);
return bytes;
}
export function decodeMediaFrame(bytes: Uint8Array) {
return { opcode: bytes[0], payload: bytes.subarray(1) }; // a view, not a copy
}
export const mediaStream = defineSubscription({
name: "media.stream",
input: z.object({ source: z.string() }),
event: z.instanceof(Uint8Array),
});
```
```ts
useSubscription(mediaStream, { source }, (bytes) => {
const frame = decodeMediaFrame(bytes);
if (frame.opcode === MediaOpcode.Audio) audio.enqueue(frame.payload);
else video.draw(frame.payload);
});
```
同一个订阅的事件按推送顺序到达，JSON 和二进制都一样。`plugin-examples/media-stream` 用这种方式传输视频和音频，并通过一个输入为 `Uint8Array` 的 RPC 把一帧上传回去；`plugin-examples/diff-watch` 监听工作树。
### 旧版主机 {#older-hosts}
订阅和二进制 RPC 值需要主机声明支持。连接旧版主机时，`useSubscription` 会报告 `error` 并提示更新主机，`client.subscribe` 的 `ready` 会以同样的信息拒绝，带有 `Uint8Array` 输入或输出的 RPC 会失败。请把插件的 `requirements.paseo` 提高到包含订阅功能的版本。
## 调试后端输出 {#debug-backend-output}
后端贡献可以用普通的 Node 日志写入 stdout 和 stderr：