fix: remove off-by-one in inference prompt truncation

[Chessing234]

Summary

When the prompt must be truncated to fit context_len, max_src_len subtracted an extra token (- 1), so input_ids[-max_src_len:] dropped one more prompt token than needed.

Root cause

With max_new_tokens == context_len, max_src_len became -1, and input_ids[-(-1):] is input_ids[1:], dropping the first prompt token (e.g. <s>). Slicing the generated output with input_echo_len then left part of the prompt in the decoded response.

Fix

Use max_src_len = context_len - max_new_tokens. When the budget is exactly full, max_src_len = 0 and the full truncated prompt is kept; when max_new_tokens exceeds context_len, only the minimum required tokens are dropped.

Fixes #1912

Made with Cursor