common/hermes
2
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fix: prevent streaming cursor from appearing as standalone messages (#9538)

Browse Source 
During rapid tool-calling, the model often emits 1-2 tokens before
switching to tool calls. The stream consumer would create a new message
with 'X ▉' (short text + cursor), and if the follow-up edit to strip
the cursor was rate-limited by the platform, the cursor remained as
a permanent standalone message — reported on Telegram as 'white box'
artifacts.

Add a minimum-content guard in _send_or_edit: when creating a new
standalone message (no existing message_id), require at least 4
visible characters alongside the cursor before sending. Shorter text
accumulates into the next streaming segment instead.

This prevents cursor-only 'tofu' messages across all platforms without
affecting normal streaming (edits to existing messages, final sends
without cursor, and messages with substantial text are all unaffected).

Reported by @michalkomar on X.
This commit is contained in:
Teknium
2026-04-14 01:52:42 -07:00
committed by GitHub
parent 2558d28a9b
commit b4fcec6412
2 changed files with 102 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
gateway/stream_consumer.py
View File

@@ -632,10 +632,26 @@ class GatewayStreamConsumer:
visible_without_cursor = text
if self.cfg.cursor:
visible_without_cursor = visible_without_cursor.replace(self.cfg.cursor, "")
if not visible_without_cursor.strip():
_visible_stripped = visible_without_cursor.strip()
if not _visible_stripped:
return True # cursor-only / whitespace-only update
if not text.strip():
return True # nothing to send is "success"
# Guard: do not create a brand-new standalone message when the only
# visible content is a handful of characters alongside the streaming
# cursor. During rapid tool-calling the model often emits 1-2 tokens
# before switching to tool calls; the resulting "X ▉" message risks
# leaving the cursor permanently visible if the follow-up edit (to
# strip the cursor on segment break) is rate-limited by the platform.
# This was reported on Telegram, Matrix, and other clients where the
# ▉ block character renders as a visible white box ("tofu").
# Existing messages (edits) are unaffected — only first sends gated.
_MIN_NEW_MSG_CHARS = 4
if (self._message_id is None
and self.cfg.cursor
and self.cfg.cursor in text
and len(_visible_stripped) < _MIN_NEW_MSG_CHARS):
return True # too short for a standalone message — accumulate more
try:
if self._message_id is not None:
if self._edit_supported: