tito: support agent-layer-inserted assistant messages in append segmentation#9
Open
DavidBellamy wants to merge 44 commits intomainfrom
Open
tito: support agent-layer-inserted assistant messages in append segmentation#9DavidBellamy wants to merge 44 commits intomainfrom
DavidBellamy wants to merge 44 commits intomainfrom
Conversation
…pdating (radixark#890) Co-authored-by: Yueming Yuan <yym022502@gmail.com>
Co-authored-by: Yueming Yuan <yueming@Mac.attlocal.net>
…adixark#654) Co-authored-by: GuanxingLu <gxlu02@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… CLI args in swe-agent-v2 (radixark#954) Co-authored-by: Shi Dong <shi.dong@radixark.ai>
…#952) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Yueming Yuan <yym022502@gmail.com>
…adixark#926) Co-authored-by: guapisolo <guapisolo@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rmers >=5.0 (radixark#927) Co-authored-by: guapisolo <guapisolo@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#948) Co-authored-by: guapisolo <guapisolo@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: maocheng23 <35615230+maocheng23@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
…log dtype (radixark#975) Co-authored-by: yueming-yuan <yym022502@gmail.com>
…adixark#974) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on (radixark#949) Co-authored-by: Yanbin Jiang <jybsuper@gmail.com>
Co-authored-by: Ethan (Yusheng) Su <yushengsu.thu@gmail.com>
Co-authored-by: Zhichen Zeng <zczeng@uw.edu>
…t between sgl & miles router (radixark#1015)
…ntation
TITO is designed around the invariant that assistant turns are produced by
the inference engine and arrive as pretokenized IDs, so `_split_appended_
segments` and `tokenize_additional_non_assistant` only handled tool/user/
system roles. That invariant does not hold for agent frameworks that
insert their own non-generated assistant messages between tool calls
(e.g., terminus-2 self-reflection / planning turns).
When such a turn is appended to a session, the session server currently
500s with:
ValueError: unsupported appended role for TITO segmentation: assistant
Add assistant as a supported append role alongside user/system. Inserted
assistant turns are tokenized with [_DUMMY_SYSTEM, _DUMMY_USER] as the
synthetic base context so chat-template renders produce correct
assistant-turn boundary tokens (most templates require an assistant turn
to follow a user/tool turn, not a system turn directly).
The function name `tokenize_additional_non_assistant` is kept for backward
compatibility; its docstring is updated to reflect the broadened semantics
("tokenize appended messages that were not generated by the inference
engine," which includes agent-layer assistant insertions).
Callers still need to pass `--tito-allowed-append-roles` with `assistant`
included for the outer `assert_messages_append_only_with_allowed_role`
check to accept these turns. That CLI choice is already configurable per
the existing `allowed_append_roles` argument on TITOTokenizer.__init__.
DavidBellamy
force-pushed
the
fix/tito-assistant-append-role
branch
from
0b02324 to
1f58c11
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
deployfor RL360's next image rebuild without waiting for upstream review.Error this fixes
Reproduced 100% on RL360 job 1571608 (pd-hicache-l3-ep4, tblite + terminus-2).
Change
_split_appended_segmentsnow acceptsassistantalongsideuser/system._tokenize_assistant_segmentuses[_DUMMY_SYSTEM, _DUMMY_USER]as the synthetic base context so chat templates render assistant-turn boundary tokens correctly.tokenize_additional_non_assistantgets theelif role == "assistant"branch. Function name retained for backward compatibility; docstring broadened.--tito-allowed-append-rolesto includeassistant(already configured in RL360's agent cookbook).Why this goes here, not only upstream
Production RL training on M2 is blocked on this. Upstream review cycle is unpredictable; LLM360/miles:deploy lets us unblock this week. Once radixark#1024 merges, octopus naturally reconciles.
Deploy path
deploybranch gets rebuilt tonight (or sooner if manually dispatched)agentic-rl-nightly-<YYYYMMDD>with new MILES_SHARL360_IMAGE_TAG+MILES_SHAindocker/versions.env