feat(gateway): deliverable mode — ship artifacts as native uploads from any agent surface (#27813)

The agent can now produce a chart, PDF, spreadsheet, or any other supported
file type and have it land in Slack / Discord / Telegram / WhatsApp / etc.
as a native attachment, just by mentioning the absolute path in its
response. Same primitive works for kanban-worker completions: workers
attach artifacts via kanban_complete(artifacts=[...]) and the gateway
notifier uploads them alongside the completion message.

Changes:

- gateway/platforms/base.py: extract_local_files now covers PDFs, docx,
  spreadsheets (xlsx/csv/json/yaml), presentations (pptx), archives
  (zip/tar/gz), audio (mp3/wav/...), and html — not just images and video.
  Image/video extensions still embed inline; everything else routes to
  send_document via the existing dispatch partition in gateway/run.py.

- tools/kanban_tools.py + hermes_cli/kanban_db.py: kanban_complete gains
  an explicit ``artifacts`` parameter. The handler stashes it in
  metadata.artifacts (for downstream workers) and the kernel promotes
  it onto the completed-event payload so the notifier can find it
  without a second SQL round-trip.

- gateway/run.py: _kanban_notifier_watcher now calls a new helper
  _deliver_kanban_artifacts after sending the completion text. The
  helper reads payload.artifacts (preferred), falls back to scanning
  the payload summary and task.result with extract_local_files, then
  partitions images / videos / documents and uploads each via
  send_multiple_images / send_video / send_document.

- website/docs/user-guide/features/deliverable-mode.md + sidebars.ts:
  user-facing docs page covering the extension list, the kanban
  artifacts pattern, and the MCP-for-connector-breadth recommendation.

Tests:

- tests/gateway/test_extract_local_files.py: 7 new test cases
  (documents, spreadsheets, presentations, audio, archives, html,
  chart-pdf canonical case). 44 passing, 0 regressions.
- tests/tools/test_kanban_tools.py: 4 new cases covering the artifacts
  arg shape (list / string / merge with existing metadata / type
  rejection). 17 passing.
- tests/hermes_cli/test_kanban_notify.py: 2 new cases covering full
  notifier → artifact-upload path and missing-file silent-skip. 12
  passing.
- E2E (real files, real kanban kernel, real BasePlatformAdapter):
  worker calls kanban_complete(artifacts=[png,pdf,csv]) → metadata +
  event payload land → notifier helper partitions correctly →
  send_multiple_images called once with the PNG, send_document called
  twice with PDF + CSV.

What's NOT in this PR (deferred to follow-ups):

- Ad-hoc "research this for two hours, ping the thread when done"
  slash command — covered today by kanban subscriptions; a dedicated
  slash command can ride a follow-up PR if needed.
- Setup-wizard prompt for recommended MCP servers (Notion, GitHub,
  Linear, etc.) — docs page lists them; UI is a separate change.

Plan and rationale captured in ~/.hermes/docs/perplexity-computer-parity.pdf
(local doc, not shipped).
This commit is contained in:
Teknium
2026-05-18 02:14:43 -07:00
committed by GitHub
parent dadc8aa255
commit f2fdb9a178
9 changed files with 671 additions and 8 deletions

View File

@@ -2157,12 +2157,20 @@ class BasePlatformAdapter(ABC):
@staticmethod
def extract_local_files(content: str) -> Tuple[List[str], str]:
"""
Detect bare local file paths in response text for native media delivery.
Detect bare local file paths in response text for native delivery.
Matches absolute paths (/...) and tilde paths (~/) ending in common
image or video extensions. Validates each candidate with
``os.path.isfile()`` to avoid false positives from URLs or
non-existent paths.
image, video, audio, or document extensions. Validates each
candidate with ``os.path.isfile()`` to avoid false positives from
URLs or non-existent paths.
The extension list is broader than just images/video so the agent
can produce arbitrary artifacts (charts, PDFs, spreadsheets, code
archives, CSVs) and have them ship to the user as native uploads
without needing an explicit ``MEDIA:`` tag. Image / video
extensions still embed inline where the platform supports it;
document extensions route through ``send_document``. The dispatch
partition lives in ``gateway/run.py``.
Paths inside fenced code blocks (``` ... ```) and inline code
(`...`) are ignored so that code samples are never mutilated.
@@ -2172,8 +2180,22 @@ class BasePlatformAdapter(ABC):
raw path strings removed).
"""
_LOCAL_MEDIA_EXTS = (
'.png', '.jpg', '.jpeg', '.gif', '.webp',
# Images (embed inline)
'.png', '.jpg', '.jpeg', '.gif', '.webp', '.bmp', '.tiff', '.svg',
# Video (embed inline where supported)
'.mp4', '.mov', '.avi', '.mkv', '.webm',
# Audio (delivered as voice/audio where supported)
'.mp3', '.wav', '.ogg', '.m4a', '.flac',
# Documents (uploaded as file attachments)
'.pdf', '.docx', '.doc', '.odt', '.rtf', '.txt', '.md',
# Spreadsheets / data
'.xlsx', '.xls', '.ods', '.csv', '.tsv', '.json', '.xml', '.yaml', '.yml',
# Presentations
'.pptx', '.ppt', '.odp', '.key',
# Archives
'.zip', '.tar', '.gz', '.tgz', '.bz2', '.xz', '.7z', '.rar',
# Web / rendered output
'.html', '.htm',
)
ext_part = '|'.join(e.lstrip('.') for e in _LOCAL_MEDIA_EXTS)

View File

@@ -4474,6 +4474,29 @@ class GatewayRunner:
"kanban notifier: delivered %s event for %s to %s/%s on board %s",
kind, sub["task_id"], platform_str, sub["chat_id"], board_slug,
)
# After delivering the text notification, surface
# any artifact paths the worker referenced in
# ``kanban_complete(summary=..., artifacts=[...])``
# (or the legacy ``result`` field) as native
# uploads. ``extract_local_files`` finds bare
# absolute paths in the summary;
# ``send_document`` / ``send_image_file`` uploads
# them. Only fires on the ``completed`` event so
# we never spam attachments on retries.
if kind == "completed":
try:
await self._deliver_kanban_artifacts(
adapter=adapter,
chat_id=sub["chat_id"],
metadata=metadata,
event_payload=getattr(ev, "payload", None),
task=task,
)
except Exception as art_exc:
logger.debug(
"kanban notifier: artifact delivery for %s failed: %s",
sub["task_id"], art_exc,
)
# Reset the failure counter on success.
sub_fail_counts.pop(sub_key, None)
except Exception as exc:
@@ -4591,6 +4614,110 @@ class GatewayRunner:
finally:
conn.close()
async def _deliver_kanban_artifacts(
self,
*,
adapter,
chat_id: str,
metadata: dict,
event_payload: Optional[dict],
task,
) -> None:
"""Upload artifact files referenced by a completed kanban task.
Workers passing ``kanban_complete(artifacts=[...])`` ship absolute
file paths through the completion event so downstream humans get
the deliverable as a native upload instead of a path printed in
chat.
Sources scanned, in priority order:
1. ``event_payload['artifacts']`` (explicit list — preferred)
2. ``event_payload['summary']`` (truncated first line)
3. ``task.result`` (legacy fallback)
Files are deduplicated, missing files are silently skipped (the
path may have been mentioned for reference only), and delivery
errors are logged but do not break the notifier loop.
"""
from pathlib import Path as _Path
candidates: list[str] = []
seen: set[str] = set()
def _add(path: str) -> None:
if not path:
return
expanded = os.path.expanduser(path)
if expanded in seen:
return
if not os.path.isfile(expanded):
return
seen.add(expanded)
candidates.append(expanded)
# 1. Explicit artifacts list in payload.
if isinstance(event_payload, dict):
raw = event_payload.get("artifacts")
if isinstance(raw, (list, tuple)):
for item in raw:
if isinstance(item, str):
_add(item)
# 2. Paths embedded in the payload summary.
summary = event_payload.get("summary")
if isinstance(summary, str) and summary:
paths, _ = adapter.extract_local_files(summary)
for p in paths:
_add(p)
# 3. Legacy: paths embedded in task.result.
if task is not None and getattr(task, "result", None):
result_text = str(task.result)
paths, _ = adapter.extract_local_files(result_text)
for p in paths:
_add(p)
if not candidates:
return
_IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".gif", ".webp", ".bmp", ".tiff", ".svg"}
_VIDEO_EXTS = {".mp4", ".mov", ".avi", ".mkv", ".webm", ".3gp"}
from urllib.parse import quote as _quote
# Partition images so they ride a single send_multiple_images call
# on platforms that support batch image uploads (Signal/Slack RPCs).
image_paths = [p for p in candidates if _Path(p).suffix.lower() in _IMAGE_EXTS]
other_paths = [p for p in candidates if _Path(p).suffix.lower() not in _IMAGE_EXTS]
if image_paths:
try:
batch = [(f"file://{_quote(p)}", "") for p in image_paths]
await adapter.send_multiple_images(
chat_id=chat_id, images=batch, metadata=metadata,
)
except Exception as exc:
logger.warning(
"kanban notifier: image batch upload failed: %s", exc,
)
for path in other_paths:
ext = _Path(path).suffix.lower()
try:
if ext in _VIDEO_EXTS:
await adapter.send_video(
chat_id=chat_id, video_path=path, metadata=metadata,
)
else:
await adapter.send_document(
chat_id=chat_id, file_path=path, metadata=metadata,
)
except Exception as exc:
logger.warning(
"kanban notifier: artifact upload (%s) failed: %s",
path, exc,
)
async def _kanban_dispatcher_watcher(self) -> None:
"""Embedded kanban dispatcher — one tick every `dispatch_interval_seconds`.