feat: auto-compact context when token usage exceeds ENV threshold#17
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces an auto-compaction feature configured via the ZCODE_ACP_AUTO_COMPACT_THRESHOLD environment variable. When context usage exceeds this threshold, the server automatically triggers session/compact after a successful turn. The feedback highlights two main improvement opportunities: first, running maybeAutoCompact asynchronously without await to prevent blocking the prompt response and introducing user-visible latency; second, passing the already-fetched context usage to maybeAutoCompact to avoid a redundant session/read roundtrip to the backend.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| if (result.stopReason === "end_turn") { | ||
| const { maybeAutoCompact } = await import("../config/auto-compact.js"); | ||
| await maybeAutoCompact(server, cx, params.sessionId, zcodeSid); | ||
| } |
There was a problem hiding this comment.
Awaiting maybeAutoCompact here blocks the prompt response from returning to the client. Since compaction can take a significant amount of time (up to 5 minutes as configured in compact's waitForTurnIdle), this introduces severe user-visible latency after the turn has already successfully ended.
Since maybeAutoCompact is fully wrapped in try/catch blocks and is guaranteed not to throw or reject, you can safely run it in the background by calling it without await (using void). This allows the prompt response to return immediately, vastly improving the user experience.
if (result.stopReason === "end_turn") {
const { maybeAutoCompact } = await import("../config/auto-compact.js");
// Run auto-compaction in the background so we don't block the prompt response.
// Compaction can take up to several minutes, and awaiting it here would introduce
// severe user-visible latency after the turn has already successfully ended.
void maybeAutoCompact(server, cx, params.sessionId, zcodeSid);
}| // Read current context usage via session/read. | ||
| let used = 0; | ||
| try { | ||
| const backend = server.ensureBackend(); | ||
| const resp = await backend.request( | ||
| server.nextId(), | ||
| "session/read", | ||
| { sessionId: zcodeSid }, | ||
| 5000, | ||
| ); | ||
| if (resp.error) return; | ||
| const result = (resp.result ?? {}) as { projection?: { contextUsed?: number } }; | ||
| used = result.projection?.contextUsed ?? 0; | ||
| } catch (e) { | ||
| warn(`auto-compact: session/read failed (${e instanceof Error ? e.message : String(e)})`); | ||
| return; | ||
| } |
There was a problem hiding this comment.
This session/read call is redundant. Right before maybeAutoCompact is triggered in prompt(), runEventTurn has already fetched the latest session state via buildSnapshot (which internally calls session/read to get the projection and context usage).
To avoid an extra roundtrip to the backend subprocess, consider passing the known context usage (or the snapshot) down from prompt() to maybeAutoCompact. For example, you could retrieve the last usage from the differ or return it from runEventTurn and pass it as an optional parameter to maybeAutoCompact.
1 participant
No description provided.