[DEPLOYMENT] Language Toggle, RAG Ingestion Pipeline & Rate Limiting Issues

[mjunaidca]

Deployment Issues

Issue 1: Language Toggle - Cannot Switch Back to English from Urdu

Description: After switching to Urdu locale, users cannot switch back to English. The language toggle appears to be stuck in Urdu mode.

Steps to Reproduce:

1. Navigate to any page
2. Click language toggle (globe icon)
3. Select "Urdu" (UR) → Works ✅
4. Click language toggle again
5. Select "English" (EN) → Fails ❌

Expected: Should switch back to /en/ route
Actual: Does not switch back to English

Affected Component: robolearn-interface/src/components/LanguageToggle/index.tsx
Likely Issue: Locale path detection/removal logic (lines 76-95)

Impact: High - Users stuck in Urdu mode cannot access English content

---

Issue 2: RAG Ingestion Pipeline - Manual Sync Required

Description: RAG ingestion pipeline requires manual execution. Should automatically sync when content changes.

Current Behavior:

• Must run manually: uv run python scripts/ingest.py ingest
• No automatic sync on content changes
• No CI/CD integration

Expected Behavior:

• Auto-sync on git push (when docs/ changes)
• GitHub Actions workflow triggers ingestion
• Incremental updates only (changed files)

Proposed Solution:

1. Create .github/workflows/sync-rag.yml
2. Trigger on push to main when docs/** or robolearn-interface/docs/** changes
3. Run rag-agent/scripts/ingest.py ingest (incremental mode)

Impact: Medium - Content updates require manual intervention

---

Issue 3: Chat Agent - Rate Limiting Missing (CRITICAL for Public Launch)

Description: ChatKit endpoint (/chatkit) has no rate limiting. This is a critical security and cost risk for public launch.

Current Behavior:

• No rate limiting on /chatkit endpoint
• Unlimited requests per user/IP
• No protection against abuse or cost overruns
• Vulnerable to DDoS or excessive API usage

Risk Assessment:

• Security: High - Vulnerable to abuse, potential DDoS
• Cost: High - Unlimited OpenAI API calls could cause cost overruns
• Availability: Medium - No protection against resource exhaustion

Expected Behavior:

• Rate limiting per user (e.g., 20 requests/minute per authenticated user)
• Rate limiting per IP (e.g., 10 requests/minute per IP for unauthenticated)
• Graceful error responses (429 Too Many Requests)
• Rate limit headers in responses (X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset)

Proposed Solution:

1. Add slowapi or fastapi-limiter library
2. Configure per-user limits (20 req/min authenticated)
3. Configure per-IP limits (10 req/min unauthenticated)
4. Add 429 responses with Retry-After header
5. Add rate limit headers to responses

Affected File: rag-agent/app.py - POST /chatkit endpoint

Impact: CRITICAL - Must be implemented before public launch

---

Acceptance Criteria

Language Toggle Fix

• Can switch Urdu → English
• Language preference persists
• Works with root and subpath baseUrl
• No console errors

Auto-Sync Ingestion

• GitHub Actions triggers on content changes
• Incremental ingestion runs automatically
• Only changed files processed
• Workflow logs show progress

Rate Limiting (CRITICAL)

• Rate limiting middleware installed
• Per-user limits (20 req/min authenticated)
• Per-IP limits (10 req/min unauthenticated)
• 429 responses with Retry-After header
• Rate limit headers in responses
• Tested with load

---

Related Files:

• robolearn-interface/src/components/LanguageToggle/index.tsx
• rag-agent/scripts/ingest.py
• rag-agent/app.py (rate limiting)
• .github/workflows/deploy.yml