At the moment when pruning we trim older messages to stay within the context ratio. However, the first few messages often see the overall reasoning and context for that session in the first place. When pruning we should maintain the first few messages, insert a "truncated history" type message drop remaining old messages like we do now.
At the moment when pruning we trim older messages to stay within the context ratio. However, the first few messages often see the overall reasoning and context for that session in the first place. When pruning we should maintain the first few messages, insert a "truncated history" type message drop remaining old messages like we do now.