Teknium
9e992df8ae
fix(telegram): use UTF-16 code units for message length splitting ( #8725 )
...
Port from nearai/ironclaw#2304 : Telegram's 4096 character limit is
measured in UTF-16 code units, not Unicode codepoints. Characters
outside the Basic Multilingual Plane (emoji like 😀 , CJK Extension B,
musical symbols) are surrogate pairs: 1 Python char but 2 UTF-16 units.
Previously, truncate_message() used Python's len() which counts
codepoints. This could produce chunks exceeding Telegram's actual limit
when messages contain many astral-plane characters.
Changes:
- Add utf16_len() helper and _prefix_within_utf16_limit() for
UTF-16-aware string measurement and truncation
- Add _custom_unit_to_cp() binary-search helper that maps a custom-unit
budget to the largest safe codepoint slice position
- Update truncate_message() to accept optional len_fn parameter
- Telegram adapter now passes len_fn=utf16_len when splitting messages
- Fix fallback truncation in Telegram error handler to use
_prefix_within_utf16_limit instead of codepoint slicing
- Update send_message_tool.py to use utf16_len for Telegram platform
- Add comprehensive tests: utf16_len, _prefix_within_utf16_limit,
truncate_message with len_fn (emoji splitting, content preservation,
code block handling)
- Update mock lambdas in reply_mode tests to accept **kw for len_fn
2026-04-12 19:06:20 -07:00
..
2026-02-26 03:20:08 +03:00
2026-04-10 21:18:34 -07:00
2026-03-26 13:19:43 -07:00
2026-03-29 21:29:13 -07:00
2026-04-10 16:51:44 -07:00
2026-04-05 12:31:10 -07:00
2026-04-12 18:03:16 -07:00
2026-04-07 03:28:44 -07:00
2026-04-11 13:47:25 -07:00
2026-04-11 02:03:20 -07:00
2026-04-01 12:05:02 -07:00
2026-04-10 03:52:00 -07:00
2026-03-17 03:52:15 -07:00
2026-04-10 03:52:00 -07:00
2026-04-10 03:21:45 -07:00
2026-04-06 13:20:01 -07:00
2026-04-12 03:03:07 -07:00
2026-04-10 03:52:00 -07:00
2026-04-10 21:16:53 -07:00
2026-04-11 19:23:29 -07:00
2026-03-13 23:59:18 -07:00
2026-04-05 19:46:58 -07:00
2026-04-10 03:44:43 -07:00
2026-04-11 13:59:52 -07:00
2026-03-09 02:20:57 -07:00
2026-04-09 23:25:27 -07:00
2026-04-10 05:19:26 -07:00
2026-04-11 19:22:14 -07:00
2026-04-09 02:26:33 -07:00
2026-04-11 13:59:52 -07:00
2026-03-14 09:32:05 -07:00
2026-03-13 08:52:54 -07:00
2026-03-14 14:27:21 +03:00
2026-04-10 03:52:00 -07:00
2026-04-12 19:06:20 -07:00
2026-04-03 10:24:07 -07:00
2026-04-09 23:25:27 -07:00
2026-03-23 06:50:09 -07:00
2026-04-11 13:59:52 -07:00
2026-04-11 17:20:34 -07:00
2026-04-03 18:20:50 -07:00
2026-04-03 13:16:26 -07:00
2026-03-29 15:38:32 -07:00
2026-03-17 01:47:34 -07:00
2026-04-10 21:16:56 -07:00
2026-04-10 05:54:56 -07:00
2026-04-07 22:45:14 -07:00
2026-04-12 13:05:56 -07:00
2026-04-06 16:54:16 -07:00
2026-04-03 13:10:11 -07:00
2026-04-08 20:01:06 -07:00
2026-04-10 21:18:34 -07:00
2026-03-13 07:40:38 -07:00
2026-03-29 20:05:59 -07:00
2026-04-12 00:36:22 -07:00
2026-03-15 03:50:45 -07:00
2026-04-12 18:05:41 -07:00
2026-04-10 21:15:59 -07:00
2026-04-12 07:24:46 +05:30
2026-04-11 13:59:52 -07:00
2026-04-10 05:04:28 -07:00
2026-02-28 03:38:27 -05:00
2026-03-11 09:15:34 +01:00
2026-04-10 03:07:00 -07:00
2026-04-10 02:58:42 -07:00
2026-03-05 18:39:37 -08:00
2026-04-10 03:44:43 -07:00
2026-03-14 21:28:51 -07:00
2026-04-12 19:06:20 -07:00
2026-04-07 09:58:45 -07:00
2026-04-11 14:43:53 -07:00
2026-04-08 03:36:44 -07:00
2026-04-10 21:18:34 -07:00
2026-04-11 03:11:34 -07:00
2026-03-14 21:19:22 -07:00
2026-03-10 23:34:52 -07:00
2026-04-11 16:21:39 -07:00
2026-03-28 14:25:12 -07:00
2026-04-11 19:38:05 -07:00
2026-03-27 04:03:13 -07:00
2026-04-04 19:05:34 -07:00
2026-04-10 21:18:34 -07:00
2026-04-10 03:01:59 -07:00
2026-04-11 15:35:04 -07:00
2026-03-30 19:17:07 -07:00
2026-03-26 19:27:58 -07:00
2026-04-11 03:11:34 -07:00
2026-04-06 13:20:01 -07:00
2026-04-10 21:18:34 -07:00
2026-03-22 09:33:39 -07:00
2026-04-10 03:44:43 -07:00
2026-04-12 13:05:56 -07:00
2026-04-11 13:59:52 -07:00
2026-04-07 11:12:08 -07:00
2026-04-09 14:07:32 -07:00
2026-04-10 05:04:28 -07:00
2026-04-11 14:05:38 -07:00
2026-03-27 11:33:19 -07:00
2026-03-15 23:04:34 -07:00
2026-04-07 17:27:09 -07:00
2026-04-11 19:22:14 -07:00
2026-04-02 20:54:27 -07:00
2026-03-01 05:28:12 -08:00
2026-04-11 16:21:39 -07:00
2026-04-11 14:43:53 -07:00
2026-04-07 17:19:07 -07:00
2026-04-07 14:08:59 -07:00
2026-04-11 13:59:52 -07:00
2026-04-03 13:16:26 -07:00
2026-03-27 04:03:13 -07:00
2026-03-29 21:53:59 -07:00
2026-03-27 04:03:13 -07:00
2026-03-28 14:23:27 -07:00
2026-03-15 03:50:45 -07:00
2026-04-10 05:34:33 -07:00
2026-04-12 19:06:20 -07:00
2026-03-17 02:49:57 -07:00
2026-04-05 11:59:28 -07:00
2026-04-09 23:25:27 -07:00
2026-03-14 14:27:20 +03:00
2026-03-04 21:34:40 +03:00
2026-03-31 10:42:03 -07:00
2026-04-05 11:59:28 -07:00
2026-04-05 00:28:58 -07:00
2026-04-12 04:52:59 -07:00
2026-04-10 02:33:01 -07:00
2026-04-11 17:20:34 -07:00
2026-04-03 10:24:07 -07:00
2026-04-12 18:05:41 -07:00
2026-04-07 17:27:09 -07:00
2026-03-28 14:33:35 -07:00
2026-04-07 17:27:09 -07:00
2026-04-11 15:22:49 -07:00
2026-04-09 23:25:27 -07:00
2026-04-12 16:43:25 -07:00
2026-03-29 16:25:20 -07:00
2026-04-03 01:16:39 -07:00
2026-03-17 10:44:37 -07:00
2026-04-05 11:07:47 -07:00
2026-04-10 03:38:44 -07:00