zopu-code/deploy/zopu-runtime/README.md

Zopu Single-Node Runtime Deployment

Deployment artifacts for the Zopu execution plane on a single Debian dedicated server. Convex is already deployed as the control plane; this lane deploys only the execution plane: the daemon (Bun/Effect + AgentOS/RivetKit), the Flue agent service, Docker Engine for future Orb sandboxes, and supporting infrastructure.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Debian Dedicated Server                   │
│  ~12 CPU cores · ~40 GB RAM · single-node                    │
│                                                               │
│  ┌──────────────┐     ┌──────────────┐     ┌──────────────┐ │
│  │ zopu-daemon  │     │ zopu-agent   │     │  Docker      │ │
│  │ (systemd)    │     │ (systemd)    │     │  Engine      │ │
│  │              │     │              │     │              │ │
│  │ Effect daemon│     │ Flue Node    │     │ Orb sandboxes│ │
│  │ + RivetKit   │     │ server.mjs   │     │ (future)     │ │
│  │   in-process │     │ :3583        │     │              │ │
│  │   engine     │     │              │     │              │ │
│  │ + native     │     │              │     │              │ │
│  │   sidecar    │     │              │     │              │ │
│  │   :6420      │     │              │     │              │ │
│  └──────┬───────┘     └──────┬───────┘     └──────────────┘ │
│         │                    │                               │
│         └────────┬───────────┘                               │
│                  │                                            │
│         ┌────────▼─────────┐                                 │
│         │   Convex (cloud)  │  ← control plane (pre-deployed)│
│         └──────────────────┘                                 │
│                                                               │
│  systemd timers: health-check (60s), docker-cleanup (daily)  │
│  cron: disk-monitor (daily 06:00)                             │
│  ufw: deny-incoming, SSH + tailscale0                              │
│  Tailscale: optional private overlay                          │
└─────────────────────────────────────────────────────────────┘

Single-node RivetKit topology

The daemon calls registry.start() from rivetkit, which boots an in-process RivetKit engine (envoy mode) backed by a native Rust sidecar binary (@rivet-dev/agentos-sidecar, platform-resolved). The engine listens on RIVET_ENDPOINT (default http://localhost:6420). The daemon then calls createClient(RIVET_ENDPOINT) to connect back to its own in-process engine for actor dispatch.

Evidence: RivetKit source chunk-YDUQHING.js line 4751 — DEFAULT_ENDPOINT = "http://localhost:6420". The Registry.start() method calls #startEnvoy() → runtime.serveRegistry() for serverful mode (Mode A). The createClient() function reads RIVET_ENDPOINT env or defaults to the same http://localhost:6420.

No separate Rivet Engine process is required. The engine, actor envoy, and sidecar all run inside the daemon process. A future multi-node deployment would externalize the engine, but that is out of scope.

Docker / Orb boundary

Docker Engine is installed and the zopu service user is in the docker group. The daemon's systemd unit includes SupplementaryGroups=docker. However, no Docker-backed sandbox code is currently wired. The Orb sandbox lane contract has not landed; Docker access is provisioned now so the boundary is ready. The current agent uses the in-process AgentOS VM (Wasm/V8) sandbox, not Docker.

Files

deploy/zopu-runtime/
├── bootstrap.sh              # One-shot Debian installer
├── .env.template             # Environment template (all groups documented)
├── README.md                 # This file (runbook)
├── systemd/
│   ├── zopu-daemon.service   # Daemon systemd unit
│   ├── zopu-agent.service    # Agent systemd unit
│   ├── zopu-health.service   # Health check oneshot
│   ├── zopu-health.timer     # Health check every 60s
│   ├── zopu-docker-cleanup.service
│   └── zopu-docker-cleanup.timer
├── scripts/
│   ├── health-check.sh       # TCP/process health probes
│   ├── update.sh             # Update to branch or commit
│   ├── rollback.sh           # Roll back to previous commit
│   ├── docker-cleanup.sh     # Prune stopped containers/images/networks
│   └── disk-monitor.sh       # Disk usage alerting
└── caddy/
    └── Caddyfile             # Optional reverse proxy config (documentation)

Fresh install

# 1. SSH into the fresh Debian 12 server as root.

# 2. Set environment overrides (optional):
export ZOPU_REPO_URL="ssh://git@git.openputer.com:2222/puter/zopu-code.git"
export ZOPU_REPO_BRANCH="dogfood/v0"
# export TAILSCALE_AUTHKEY="tskey-..."
# export TAILSCALE_HOSTNAME="zopu-runtime"

# 3. Run the bootstrap script:
bash bootstrap.sh

# 4. Edit .env with real values:
nano /opt/zopu/.env

# 5. Start services:
systemctl start zopu-daemon
sleep 3
systemctl start zopu-agent

# 6. Enable timers:
systemctl enable --now zopu-health.timer zopu-docker-cleanup.timer

# 7. Verify:
/opt/zopu/deploy/zopu-runtime/scripts/health-check.sh

Start / stop / restart

# Start both services
systemctl start zopu-daemon zopu-agent

# Stop both services
systemctl stop zopu-agent zopu-daemon

# Restart (daemon first — it owns the RivetKit engine)
systemctl restart zopu-daemon && sleep 3 && systemctl restart zopu-agent

# Enable on boot
systemctl enable zopu-daemon zopu-agent

# Disable on boot
systemctl disable zopu-daemon zopu-agent

Log inspection

All service logs go to journald with SyslogIdentifier tags.

# Daemon logs (live follow)
journalctl -u zopu-daemon -f

# Agent logs (live follow)
journalctl -u zopu-agent -f

# Last 100 lines of daemon
journalctl -u zopu-daemon -n 100

# Logs since boot
journalctl -u zopu-daemon -b

# Health check timer logs
journalctl -u zopu-health.service -n 50

# Docker cleanup logs
journalctl -u zopu-docker-cleanup.service -n 50

# Disk monitor logs (cron → file)
tail -100 /var/log/zopu/disk-monitor.log

# All Zopu syslog identifiers
journalctl -t zopu-daemon -t zopu-agent --since "1 hour ago"

Health checks

# Manual health check (prints all probes)
/opt/zopu/deploy/zopu-runtime/scripts/health-check.sh

# Quiet mode (exit code only)
/opt/zopu/deploy/zopu-runtime/scripts/health-check.sh --quiet

# Check systemd timer is running
systemctl status zopu-health.timer
systemctl list-timers zopu-health.timer

The health check probes:

1. zopu-daemon systemd unit is active
2. zopu-agent systemd unit is active
3. RivetKit engine port (default 6420) accepts TCP connections
4. Flue agent port (default 3583) accepts TCP connections
5. Docker daemon responds to docker info

No HTTP health endpoints are assumed. Flue does not expose one by design (per Flue docs: "Flue does not add a health endpoint"). RivetKit's health route is internal to the registry runtime and not documented as publicly addressable on the engine endpoint.

Update to commit

# Update to latest of dogfood/v0 (default)
/opt/zopu/deploy/zopu-runtime/scripts/update.sh

# Update to a specific branch
/opt/zopu/deploy/zopu-runtime/scripts/update.sh dogfood/runtime-deploy

# Update to a specific commit
/opt/zopu/deploy/zopu-runtime/scripts/update.sh abc123def456

The update script:

1. Records current HEAD to .last-deployed-sha
2. Fetches, resolves branch-or-commit, checks out
3. bun install, builds daemon and agent
4. Restarts daemon, waits, restarts agent
5. Runs health check; reports failure and rollback instructions

Rollback

# Roll back to the previously deployed commit
/opt/zopu/deploy/zopu-runtime/scripts/rollback.sh

# Roll back to a specific commit
/opt/zopu/deploy/zopu-runtime/scripts/rollback.sh abc123def456

Rollback reads .last-deployed-sha (written by update.sh), checks out that commit, rebuilds, and restarts services. The pre-rollback SHA is saved to .pre-rollback-sha for re-rollback if needed.

Docker cleanup

# Manual cleanup
/opt/zopu/deploy/zopu-runtime/scripts/docker-cleanup.sh

# Check Docker disk usage
docker system df

# Timer runs daily; check its schedule
systemctl list-timers zopu-docker-cleanup.timer

Cleanup prunes:

• Stopped containers older than 24 hours
• Dangling (untagged) images
• Unused networks

Named volumes and running containers are never removed.

Disk-space monitoring

# Manual check
/opt/zopu/deploy/zopu-runtime/scripts/disk-monitor.sh

# Custom threshold (90%)
/opt/zopu/deploy/zopu-runtime/scripts/disk-monitor.sh --warn-percent 90

A cron job runs at 06:00 daily and writes to /var/log/zopu/disk-monitor.log. Default alert threshold is 80%.

Firewall and private networking

The firewall (ufw) is deny-by-default:

• SSH (port 22) is allowed on all interfaces
• All traffic on tailscale0 is allowed (Tailscale private overlay)
• All other incoming traffic is denied

The RivetKit engine (:6420) and Flue agent (:3583) ports are not exposed on public interfaces. Reachability options:

1. Tailscale (recommended): bootstrap runs ufw allow in on tailscale0 so all ports are reachable over the private overlay. Set TAILSCALE_AUTHKEY before running bootstrap to configure automatically. Other Tailscale-connected machines can reach the agent at http://zopu-runtime:3583 and the engine at http://zopu-runtime:6420.
2. Custom private interface: if you have a non-Tailscale private network (e.g. a VLAN or wireguard interface), add an explicit rule:

   ufw allow in on eth1  # or your private interface name
   

   Do NOT assume direct private IP access works by default — the deny-incoming policy blocks it until an interface-specific rule is added.
3. Caddy (optional): install Caddy and use the annotated Caddyfile in caddy/ if you need TLS termination or a public ingress point.

Environment groups

See .env.template for the full annotated template. The eight required groups:

Group	Variables
Convex	CONVEX_URL, CONVEX_SITE_URL, SITE_URL
Gitea	GITEA_URL, GITEA_TOKEN
Model gateway	AGENT_MODEL_*
AgentOS/RivetKit	RIVET_ENDPOINT (optional)
Zopu agent	FLUE_DB_TOKEN, PORT
Daemon	DAEMON_ID, DAEMON_NAME, DAEMON_VERSION, DAEMON_HEARTBEAT_MS, DAEMON_COMMAND_LEASE_MS
Docker sandbox	group membership (no env vars)
Service auth	AUTH_SECRET (if needed)

What is NOT deployed

• Web app: the web frontend is not deployed in this lane.
• Kubernetes: no container orchestration.
• PostgreSQL for Rivet: the in-process RivetKit engine uses its own storage; no external PostgreSQL is required.
• Multi-node coordination: single-node only.
• Public administration endpoints: no admin HTTP surface.
• Secrets in source: .env is never committed; .env.template contains only placeholder values.
• Production generated-app hosting: only the execution plane runs here.
• Docker-backed Orb sandboxes: Docker is installed and access is provisioned, but no Orb sandbox code is wired. This is a boundary prepared for the Orb lane, not a working feature.

Reproducibility

The deployment does not require the developer's MacBook to remain online. Once bootstrap completes and .env is filled in:

1. Services run under systemd with Restart=always.
2. Logs persist in journald.
3. Health checks run every 60 seconds via systemd timer.
4. Docker cleanup runs daily.
5. Disk usage is monitored daily.
6. Unattended-upgrades handles Debian security patches.