puter/zopu-code
1
0
Fork 0
Code Issues 3 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

feat: consolidate project runtime deployment

Browse Source
This commit is contained in:
-Puter
2026-08-04 17:37:50 +05:30
parent 3fc80154e8
commit b4181a1271
23 changed files with 574 additions and 504 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
docs/DEPLOYMENT_PLAN.md
View File

@@ -12,7 +12,7 @@ Use a deliberately split stack:
Fast iteration (private development) Stable staging (public)
──────────────────────────────────── ────────────────────────
Mac + Tailscale Vercel + Contabo VDS
Web / Flue + in-process registry / Rivet Web SSR Agents VDS
Web / Flue + AgentOS registry (one process) Web SSR Agents VDS
│ │ │
personal Convex dev deployment staging Convex deployment
```
@@ -22,8 +22,7 @@ personal Convex dev deployment staging Convex deployment
| User-visible URL | Existing Tailscale URL on the Mac | `https://zopu-staging.vercel.app` on Vercel |
| Product backend | Personal Convex **dev** deployment | Dedicated long-lived Convex deployment |
| Web | Vite HMR on the Mac | Vercel React Router SSR |
| Agent worker | Local Flue + in-process AgentOS registry and Engine | Contabo VDS: one Flue/registry Node process + Rivet Engine |
| Delivery | Run local `pnpm dev:tailscale` | Locally built release bundle promoted on the VDS |
| Agent worker | Local Flue + AgentOS registry (one Node process) + Engine | Contabo VDS: one Flue + AgentOS registry Node process + Rivet Engine |
| State/credentials | Development-only | Independent staging secrets, OAuth app, and volumes |
**Do not create a second remote “fast iteration” stack.** It duplicates the slowest part of the loop—building and replacing container images—while the existing Mac/Tailscale stack already exercises the complete topology from a phone. Staging is the public link and integration gate; local development is the fast environment.
@@ -69,9 +68,9 @@ flowchart LR
Browser[Browser] -->|HTTPS| Vercel[Vercel: zopu-staging.vercel.app]
Browser -->|queries & mutations| Convex[Convex: joyous-cat-297]
Vercel -->|/api/auth rewrite| ConvexSite[Convex Site: joyous-cat-297.convex.site]
Convex -->|FLUE_DB_TOKEN + org header| Flue[Contabo: Flue + AgentOS registry]
Flue -->|control plane| Engine[Rivet Engine: 127.0.0.1:6420]
Engine -->|serverless callback| Flue
Convex -->|FLUE_DB_TOKEN + org header| Flue[Contabo: Flue + AgentOS registry one process]
Flue -->|control plane + outbound envoy| Engine[Rivet Engine: 127.0.0.1:6420]
Engine --> Flue
Engine --> Volumes[/srv/zopu/data/rivet]
Flue --> Workspaces[/srv/zopu/workspaces + /srv/zopu/data/flue]
CF[Cloudflare DNS] --> Vercel
@@ -93,7 +92,7 @@ The current VDS (`zopu-staging-1`, `158.220.110.74`) runs a deliberately small p
| Service | Runtime | Network exposure | Persistent data | Notes |
| --- | --- | --- | --- | --- |
| `engine` | Existing Docker Compose (`/srv/zopu/compose/docker-compose.yml`) | private loopback `127.0.0.1:6420` | `/srv/zopu/data/rivet` bind mount | Single-node RocksDB backend. Health: `http://127.0.0.1:6420/health`. |
| `agents` | systemd (`/etc/systemd/system/zopu-agents.service`) | Caddy upstream only (`127.0.0.1:3000`) | `/srv/zopu/workspaces` + `/srv/zopu/data/flue` | One Node process runs Flue and its serverless AgentOS registry. `RIVET_ENDPOINT` is the Engine control plane; `RIVET_SERVERLESS_ENDPOINT` is Engine's private callback to `http://127.0.0.1:3000/internal/rivet`. |
| `agents` | systemd (`/etc/systemd/system/zopu-agents.service`) | Caddy upstream only (`127.0.0.1:3000`) | `/srv/zopu/workspaces` + `/srv/zopu/data/flue` | One Node process runs Flue **and** hosts the AgentOS registry in native envoy Mode A (`registry.startAndWait()`). `RIVET_ENDPOINT` is the Engine control plane; the registry opens an outbound connection to the Engine, which routes actor calls back over it. No second runner process and no inbound serverless callback route. |
| `caddy` | systemd (`/etc/systemd/system/caddy.service`) | 80/443 only | Caddy certificate/config volumes | `/etc/caddy/Caddyfile` terminates TLS for `agents.zopu.puter.wtf` and forwards only the documented Flue paths to `127.0.0.1:3000`. |
Release artifacts are source controlled: `scripts/release-agents.sh` builds a curated bundle locally and atomically promotes `/srv/zopu/current`; `deploy/vds/` contains the systemd unit and secret-free environment template. The VDS materializes Linux production dependencies from the frozen lockfile, so macOS-native binaries are never uploaded.
@@ -107,7 +106,7 @@ Dokploy is intentionally excluded: it has already proved too slow and imperative
| Layer | Source of truth | Tool | Reason |
| --- | --- | --- | --- |
| VDS baseline | `infra/ansible` | **Ansible** | Idempotent OS convergence: service user, Docker, firewall, Tailscale, directories, Caddy prerequisites, and backup timer. No Pulumi SSH-command pseudo-provider. |
| VDS application topology | `deploy/vds` + `/srv/zopu/compose` | **systemd + Docker Compose** | Engine stays in its existing private Compose service; the checked-in systemd unit runs the single Node agents/registry process. |
| | VDS application topology | `deploy/vds` + `/srv/zopu/compose` | **systemd + Docker Compose** | Engine stays in its existing private Compose service; one checked-in systemd unit runs the single Flue + AgentOS registry Node process (native envoy Mode A). |
| Release execution | `scripts/release-agents.sh` now; CI later | **Local release bundle promotion** | Builds a curated cross-platform bundle, installs Linux dependencies on the VDS, atomically swaps `current`, then restarts `zopu-agents`. |
> **Current state (2026-08-03):** the Engine and Caddy retain their manually bootstrapped VDS services. The Node agents process is now represented by source-controlled `deploy/vds` artifacts and `scripts/release-agents.sh`; Pulumi/Ansible and CI are still future work.
@@ -129,12 +128,10 @@ Ansible is not redundant: it makes the Contabo host reproducible without pretend
1. Run repository validation and build the staging web artifact for the staging Convex deployment (`joyous-cat-297`, `https://joyous-cat-297.convex.cloud`), then deploy it to `zopu-staging` (`https://zopu-staging.vercel.app`).
2. From the repository root, create and promote the agents release with `scripts/release-agents.sh <vds-host> <release-id>`; it installs Linux production dependencies on the VDS, atomically swaps `/srv/zopu/current`, and leaves the running process untouched until restart.
3. Restart `zopu-agents.service` on the VDS. The unit waits for Engine health, then starts the one Node process that exposes Flue and the serverless registry.
4. Verify: `https://zopu-staging.vercel.app` returns 200, Convex auth origin `https://zopu-staging.vercel.app` is accepted, `https://agents.zopu.puter.wtf/health` returns 200, Engine health returns 200 at `http://127.0.0.1:6420/health` inside the private network, and a real signed-in staging conversation receives an agent response.
3. Restart `zopu-agents.service` on the VDS. The unit waits for Engine health, then starts the single Node process that serves Flue and starts the AgentOS registry in native envoy Mode A.
4. Verify: `https://zopu-staging.vercel.app` returns 200, Convex auth origin `https://zopu-staging.vercel.app` is accepted, `https://agents.zopu.puter.wtf/health` returns 200, Engine health returns 200 at `http://127.0.0.1:6420/health` inside the private network, `zopu-agents.service` is active, the registry envoy connected to the engine (check the process log for the startup line), and a real signed-in staging conversation receives an agent response.
5. Roll back by atomically repointing `/srv/zopu/current` at the prior release then restarting `zopu-agents.service`.
Do not couple the Vercel deployment to Gitea-native Git integration assumptions. The repository is hosted on Gitea, so start with CI invoking the Vercel CLI/API. If a Git mirror is later introduced, Vercel previews can be enabled separately.
## Required implementation backlog
The staging environment is live. Remaining work is to close the gap between the manual, on-host configuration and the declarative, CI-driven target described above:
@@ -142,7 +139,7 @@ The staging environment is live. Remaining work is to close the gap between the
1. **Convex staging deployment** — ✅ Done as `joyous-cat-297`. `SITE_URL` is set to `https://zopu-staging.vercel.app`, `FLUE_URL`/`AGENT_BACKEND_URL` to `https://agents.zopu.puter.wtf`, and `FLUE_DB_TOKEN` to the VDS agents token.
2. **Auth/webhook origin seams** — ✅ Vercel rewrites `/api/auth/*` to `https://joyous-cat-297.convex.site/api/auth/*` in `vercel.json`. Verify that Puter/Gitea webhooks are configured to post directly to the Convex Site HTTP action URL (`https://joyous-cat-297.convex.site/api/git/webhooks/...`) rather than the web origin.
3. **Vercel React Router staging project** — ✅ Done as `zopu-staging` (`https://zopu-staging.vercel.app`).
4. **VDS agent topology** — ✅ Source-controlled VDS unit, env template, and atomic release bundle now model the single Flue + in-process registry process. Apply them to the VDS and retire the former runner service from its Compose file.
4. **VDS agent topology** — ✅ Source-controlled VDS unit, env template, and atomic release bundle now model the single Flue + AgentOS registry Node process running in native envoy Mode A. Apply them to the VDS and retire the former separate runner service.
5. **Engine deployment, backups, and CI** — 🔄 Engine health and VDS backups exist. Move the existing Engine Compose service and Caddy configuration into source-controlled provisioning; add CI to run validation, promote release bundles, and deploy the web.
6. **Declarative IaC** — ⏳ Create `infra/pulumi` (Cloudflare DNS + Vercel project) and `infra/ansible` (host convergence).
7. **End-to-end staging smoke** — ⏳ Execute: sign in via GitHub OAuth, create/connect a project, send a conversation, and complete a disposable issue-to-PR job against the live `zopu-staging` + `agents.zopu.puter.wtf` stack.
@@ -153,7 +150,7 @@ The staging environment is live. Remaining work is to close the gap between the
- Use unique `FLUE_DB_TOKEN`, Rivet admin token, workspace token, model credential, and Git token per environment.
- The VDS runs code-writing agents. Use a dedicated service user; do not mount the host home directory; expose only scoped repository credentials to individual attempts; and keep staging separate from any production host.
- Back up Rivet engine state and Caddy configuration. Treat workspaces as reproducible/ephemeral unless an active job requires retention; prune completed worktrees deliberately.
- A single Flue Node instance is the correct initial staging shape. Its durable Convex adapter survives restarts, but each conversation still requires one live owner—do not add replicas until ownership routing is designed. [Flue database guide](https://flueframework.com/docs/guide/database)
- A single Flue + AgentOS registry Node instance (native envoy Mode A) is the correct initial staging shape. Its durable Convex adapter survives restarts, but each conversation still requires one live owner—do not add replicas until ownership routing is designed. [Flue database guide](https://flueframework.com/docs/guide/database)
## Sources

42
docs/LOCAL_SETUP.md
View File

@@ -7,7 +7,7 @@ This guide runs the active Zopu stack locally, including the browser chat and th
```text
Browser (React Router/Vite, :5173)
-> Convex Cloud (durable product state, auth, workflows)
-> Flue agent server + in-process AgentOS registry (:3585)
-> Flue agent server + AgentOS registry (:3585, one Node process)
-> Rivet Engine control plane (:6420)
-> AgentOS workspace actor (Git + repo clone mounted from the local checkout)
-> Gitea branch and pull request
@@ -82,15 +82,15 @@ AGENT_MODEL_MAX_TOKENS=<positive-integer>
### Rivet and AgentOS
```env
# The private Engine control plane this process connects to.
# The private Engine control plane the registry connects to.
RIVET_ENDPOINT=http://127.0.0.1:6420
# The path where the Engine calls the registry co-hosted in Flue.
RIVET_SERVERLESS_ENDPOINT=http://127.0.0.1:3585/internal/rivet
RIVET_WORKSPACE_TOKEN=<shared-random-workspace-token>
AGENT_WORKSPACE_ROOT=/absolute/path/to/zopu-agent-workspaces
# Optional: bump when the actor definition changes to drain old actors.
RIVET_ENVOY_VERSION=0.0.0
```
The registry runs in the Flue process; there is no standalone AgentOS runner. `RIVET_ENDPOINT` and `RIVET_SERVERLESS_ENDPOINT` have opposite directions and are not interchangeable. Port `6421` is an internal Engine API-peer port and must not be used as `RIVET_ENDPOINT`.
The Hono/Flue Node process hosts the AgentOS registry in **native envoy Mode A** (`registry.startAndWait()`). The registry opens one persistent outbound connection to the Rivet Engine; the Engine routes actor calls back over that connection, so no inbound HTTP callback is required. Port `6421` is an internal Engine API-peer port and must not be used as `RIVET_ENDPOINT`. Do not set `RIVET_SERVERLESS_URL` or `RIVET_SERVERLESS_TOKEN`; they are unused in this topology.
### Gitea
@@ -136,7 +136,7 @@ If work execution uses a separate agent URL, set `AGENT_BACKEND_URL`; otherwise
## Start the stack
Run each process in its own terminal. The Engine must be healthy before Flue starts because Flue serves the in-process registry that uses it.
Run each process in its own terminal. The Engine must be healthy before the agents process starts.
### 1. Start Convex development
@@ -154,7 +154,7 @@ Both commands run from `packages/backend` and explicitly load the repository-roo
Start exactly one Engine listening on `127.0.0.1:6420` using the local Engine deployment mechanism. Do not use port `6421` as the application endpoint.
### 3. Start Flue and the in-process registry
### 3. Start the agents process
```bash
# Laptop-only access
@@ -163,11 +163,11 @@ pnpm --filter @code/agents dev -- --port 3585
# Convex callbacks or access from another device
HOST=0.0.0.0 \
RIVET_ENDPOINT=http://127.0.0.1:6420 \
RIVET_SERVERLESS_ENDPOINT=http://127.0.0.1:3585/internal/rivet \
FLUE_URL=http://<tailscale-ip>:3585 \
pnpm --filter @code/agents dev -- --port 3585
```
The Flue development server serves both agent routes and the AgentOS registry at `/internal/rivet/*`; never start a separate registry runner. If shell exports override `.env`, set both endpoints explicitly as shown.
This single process serves the Hono/Flue routes **and** starts the AgentOS registry in native envoy Mode A during app module initialization (`registry.startAndWait()`). The registry opens an outbound connection to the Rivet Engine; actor calls are routed back over that connection, so there is no separate runner process and no inbound serverless callback route. If shell exports override `.env`, set `RIVET_ENDPOINT` explicitly.
### 4. Start the web application
@@ -222,12 +222,10 @@ Web sends a conversation mutation to Convex
-> the web observes Convex's reactive projection
For code execution:
-> Flue calls the local AgentOS harness
-> the harness creates an isolated Git worktree
-> a Rivet actor boots an AgentOS VM and mounts the worktree
-> Pi implements the issue and produces a candidate revision
-> the host pushes a unique branch
-> Tea creates a Gitea pull request
-> Flue calls the AgentOS actor through the Rivet Engine
-> a Rivet actor boots an AgentOS VM with the project workspace
-> the actor clones the repository and runs validated commands
-> the actor writes artifacts back through the actor RPC
```
## Common failures
@@ -245,20 +243,20 @@ Set Convex `SITE_URL` to the exact browser origin. The shared development deploy
### AgentOS fails while mounting the repository
- Confirm the Engine is healthy at `http://127.0.0.1:6420/health`.
- Confirm Flue is running with `RIVET_ENDPOINT=http://127.0.0.1:6420`.
- Confirm `RIVET_SERVERLESS_ENDPOINT` points back to the active Flue listener with `/internal/rivet` appended.
- Confirm the agents process is running and the registry started the envoy connection on startup.
- Confirm the process has `RIVET_ENDPOINT=http://127.0.0.1:6420`.
- Confirm `AGENT_WORKSPACE_ROOT` is writable.
- Restart Flue after changing registry configuration; its development server owns the in-process registry.
- Restart the agents process after changing the actor configuration; the registry re-registers actor type changes with the engine on startup.
The harness clients require CBOR encoding for actor RPC. Do not remove the `encoding: "cbor"` configuration in `packages/agents/src/adapters/agentos.ts`.
The actor client requires CBOR encoding for actor RPC. Do not remove the `encoding: "cbor"` configuration in `packages/agents/src/adapters/agentos.ts`.
### AgentOS reports an ACP completed-message resource limit
Restart Flue so the in-process registry picks up its updated actor configuration.
Restart the agents process so the registry re-registers its updated actor configuration with the engine.
### Flue connects to a remote Rivet deployment unexpectedly
### The process connects to a remote Rivet deployment unexpectedly
Environment variables exported by the shell override values loaded from `.env`. Start Flue with explicit `RIVET_ENDPOINT=http://127.0.0.1:6420` and `RIVET_SERVERLESS_ENDPOINT=http://127.0.0.1:3585/internal/rivet`.
Environment variables exported by the shell override values loaded from `.env`. Start the agents process with explicit `RIVET_ENDPOINT=http://127.0.0.1:6420`.
### Gitea issue or PR commands fail