bug: 10s Latency on Gemini CLI Initialization (Cold Start)

[Dev-020]

🛠 Environment & System Info

• Orion Core Version: v1.0.0
• Frontend/Client: Orion Core (Gemini CLI Wrapper)
• LLM Backend: [x] Gemini CLI
• OS: [x] Windows

🐛 Bug Description

Current implementation of OrionCoreGeminiCLI initializes a new gemini subprocess for every prompt. This "Cold Start" triggers a full Java/Node startup and authentication handshake, resulting in a consistent ~10-second latency before prompt processing begins.

✅ Expected Behavior

Sub-second response initiation (excluding AI generation time).

❌ Actual Behavior

The CLI process exits after every turn, requiring a complete re-initialization of the runtime for the next turn.

---

Important

AI-Generated Report: This issue was drafted by an AI Agent (Gemini CLI) and reviewed/approved by the repository owner before submission.

[Dev-020]

💡 Proposed Solution: The "Warm Start" Strategy

Based on code analysis of backends/orion_core_geminicli.py (Lines 216-233), the primary bottleneck is the subprocess.Popen one-shot execution.

Strategy

1. Persistent Daemon Mode: Modify OrionCoreGeminiCLI.__init__ to start the Gemini CLI in interactive mode during core startup.
2. Standard I/O Pipes: Keep stdin=subprocess.PIPE and stdout=subprocess.PIPE open for the duration of the Orion session.
3. Async Communication: Feed user prompts directly into the running CLI's stdin and parse the JSONL output from stdout in real-time.
4. Latency Gain: This bypasses the JVM/Node startup and Auth handshake for every prompt after the initial "Cold Start," targeting sub-second response times.

---

Important

AI-Generated Report: This comment was drafted by an AI Agent (Gemini CLI) and reviewed/approved by the repository owner before submission.

[Dev-020]

Update: Latency Analysis & Implementation Findings

After conducting isolated baseline tests and attempting a persistent process implementation, we have identified the following:

1. Cold Start Overhead: Current baseline tests confirm an average turn latency of 30 - 33 seconds. This is primarily driven by the Node.js/Java runtime initialization and authentication handshake required for every one-shot execution.
2. Headless Mode Limitation: We found that the Gemini CLI's headless mode (--prompt or -p) is inherently designed for one-off tasks. Even when wrapped in a persistent script, it appears to trigger significant internal overhead that prevents achieving sub-second response times.
3. Warm Start Implementation Challenges: Initial attempts to maintain a persistent subprocess via stdin/stdout pipes resulted in I/O deadlocks and hangs. The CLI does not currently offer a dedicated "daemon" mode for JSON-based programmatic interaction.
4. Hypothesis on Interactive Mode: Observation suggests that the CLI's Interactive Mode (REPL) responds significantly faster than headless mode. However, programmatically extracting structured responses from the REPL remains a challenge as it is optimized for TUI display rather than JSON parsing.

Next Steps:

• Investigate if there is a way to force the Interactive Mode to output raw API logs or JSON objects that can be parsed programmatically without the headless mode overhead.
• Explore the possibility of a "proxy" script that keeps the authentication session alive at a lower level.