Skip to content

Feature: cap message history in prompt at 30 recent messages #43

@Rustam-Z

Description

@Rustam-Z

Summary

The harness currently injects recent Telegram message history into every CC prompt. Cap that window at 30 most recent messages per chat to keep prompts short and predictable.

Why

  • Token cost. Long histories balloon the prompt — every turn pays for context the model rarely needs past the last ~30 messages.
  • Latency. Shorter prompts → faster first-token, especially on -p mode (relevant until Feature: switch CC invocation from claude -p to claude --resume #42 lands).
  • Cache hit rate. A bounded history is easier to keep stable across turns; an unbounded one rotates entries in/out and busts the cache.
  • Quality. Long, irrelevant tails dilute attention. The recent 30 are almost always what matters; older context is better surfaced via memory / query_db on demand when actually relevant.

Behavior

  • Per chat (DM or group), include only the last 30 messages in the inbound prompt.
  • Reply chains (<reply_chain> blocks already injected for direct replies) still resolve outside the 30-message window — they're explicit lookups, not the rolling history.
  • Older messages remain in the DB; nothing is deleted. The model can still query_db for them when needed.
  • Configurable knob (e.g. MESSAGE_HISTORY_LIMIT=30 env var) so the number can be tuned without code changes.

Edge cases

  • Group chats with high traffic — 30 messages may cover only a few minutes. Acceptable: model has memory + query_db for digging deeper when asked.
  • DMs — 30 messages is plenty for context continuity.
  • Multi-chat interleaving — limit is per chat, not global, so a busy group doesn't crowd out a DM's history.

Suggested implementation

  • In the prompt-assembly path: change the message-history query from LIMIT 100 (or whatever current) to LIMIT 30, ordered by created_at DESC, then re-sort ASC for prompt insertion.
  • Surface the active limit in /health for ops visibility.

Severity

Feature, low risk. Easy to ship, immediate prompt-size win.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions