feat(#291): CodeRabbit rate limit tracker and quota management#303
feat(#291): CodeRabbit rate limit tracker and quota management#303xsovad06 wants to merge 3 commits into
Conversation
|
Warning Review limit reached
Next review available in: 28 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (10)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
SonarCloud flagged json.JSONDecodeError as redundant when caught alongside ValueError (its parent). Remove the redundant class. Add 16 tests covering all previously uncovered code paths in coderabbit_quota.py: unlimited quota, empty sync, default timestamps, GitHub API fetch (success/failure/bad JSON/exceptions), and PR review parsing (non-CR users, missing fields, PENDING state, bad dates, null login). Coverage: 56% -> 100%.
Cover all paths in sova/dashboard/routers/quota.py: - GET /api/quota/coderabbit with enabled config (success + error) - POST /api/quota/coderabbit/sync (success, no-repo, error) - Fix line-length violations in test file Raises new-code coverage from 61.4% to meet SonarCloud 80% gate.
|
xsovad06
left a comment
xsovad06
left a comment
There was a problem hiding this comment.
PR Review: CodeRabbit Rate Limit Tracker (#291)
Summary: Well-structured feature adding CodeRabbit quota tracking across config, service, DB model, dashboard API, and UI widget. 1229 lines total, 682 of which are tests (55%). All CI checks pass, SonarCloud reports 100% coverage on new code.
Findings
[HIGH] Plan defaults may not match CodeRabbit actual limits
sova/config/models.py:368 -- The plan defaults {"free": 4, "pro": 5, "pro_plus": 10} are unverified for paid tiers. CodeRabbit Pro uses adaptive rate limiting, not a fixed per-hour cap. Consider defaulting paid plans to 0 (unlimited) since rate limiting is primarily a free-tier concern:
_plan_defaults = {"free": 4, "pro": 0, "pro_plus": 0}[HIGH] No Alembic migration for CodeRabbitEvent table
sova/db/models.py:347-363 -- New coderabbit_events table has no migration. Acknowledged in the PR body as "future PR", and mitigated by enabled=False default. Existing databases will crash with "no such table" if the feature is enabled before migration lands. Ensure the migration PR is tracked.
[MEDIUM] 20 sequential API calls on sync
sova/supervisor/coderabbit_quota.py:249-258 -- sync_from_github fetches reviews for up to 20 PRs individually (semaphore=5). Consider filtering by updated_at within the rolling window, or batching via GraphQL to reduce API calls.
[MEDIUM] Two coderabbit-prefixed config sections
sova/config/models.py:351 vs :286 -- [external_reviews.coderabbit] and [coderabbit_quota] coexist with similar env prefixes (SOVA_CODERABBIT_ vs SOVA_CODERABBIT_QUOTA_). The separation is justified (supervisor vs review-tool concerns) but may confuse users. Consider a note in settings descriptions referencing the relationship.
Verdict: Comment (no blocking issues)
The feature defaults to disabled, limiting blast radius. The two HIGH findings are worth addressing but neither requires blocking the PR.
What is done well
- Triple-registration discipline -- config model, loader, and settings metadata all present and correct per architecture rules.
- Thorough test coverage -- 28 test cases covering config defaults, service logic edge cases, API failures, bad JSON, deduplication, project isolation, and all router endpoints.
- Clean module placement -- new
sova/supervisor/subsystem with proper__init__.pydocstring sets up the namespace cleanly for future additions.
1 participant
Summary
Changes
Config System
[coderabbit]section withenabled,plan,reviews_per_hour,min_pr_spacing_minutesfieldsCore Service (
sova/supervisor/coderabbit_quota.py)get_review_history()- queries GitHub API for coderabbitai[bot] PR reviews with DB cachingcan_create_pr()- checks quota availability against rolling 1-hour windownext_available_slot()- calculates when next review slot opensrequest_review()- posts@coderabbitai reviewcomment on PRsget_review_status()- distinguishes reviewed/rate_limited/pending/summary_only statesDatabase
CodeRabbitEventmodel with repo/pr_number/event_type/recorded_at fieldsDashboard
/api/quota/coderabbitendpoint returning quota statuscoderabbit.enabledconfigReview guidance
Key design decisions:
CodeRabbitQuotaConfigfrom existingCodeRabbitConfig(external_reviews) - quota tracking is supervisor-level concern, not review-tool integration/pulls/{pr}/reviewsnot/issues/{pr}/commentsrecorded_at > now - timedelta(hours=1))Verification focus:
repofieldTest plan
Closes #291