Skip to content

fix(orb): stop double-counting PRs already tracked by the own-ledger#5257

Merged
JSONbored merged 1 commit into
mainfrom
fix/orb-double-count
Jul 12, 2026
Merged

fix(orb): stop double-counting PRs already tracked by the own-ledger#5257
JSONbored merged 1 commit into
mainfrom
fix/orb-double-count

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • The public homepage's "PRs reviewed" / "Decision accuracy" counter double-counts PRs. getOrbGlobalStats (src/orb/outcomes.ts) folds in every registered Orb installation's orb_pr_outcomes unconditionally, re-counting PRs the frozen own-ledger (pull_requests + audit_events, frozen since the self-host cutover — the source App has since been fully deleted, so it can never grow again) already counted.
  • Quantified directly against production data (2026-07-12): 243 PRs (173 merged + 70 closed, 96% concentrated in one repo) counted twice, inflating merged/closed/accuracyPct on the live public endpoint.
  • Fix: a NOT EXISTS anti-join in getOrbGlobalStats's SQL, skipping any orb_pr_outcomes row whose (repo, pr_number) already has a github_app.pr_public_surface_published audit event (the same event the own-ledger's disposition query itself reads). The two sums become strictly additive over disjoint PR sets — no new env var, no bind-parameter list, no schema change.
  • Updated the file-header comment in public-stats.ts that previously (incorrectly) characterized this as "a small, bounded double-count" — measured, it wasn't.
  • Confirmed a full retirement of the own-ledger (using only orb_pr_outcomes) would have been the wrong fix: self-host's own Postgres and orb_pr_outcomes only cover PRs since the ~06-27 cutover, while 3,244 of the 3,487 own-ledger PRs predate that window and exist nowhere else now that the source App is deleted — dropping the own-ledger would have under-counted by ~39%.

Validation

  • npm run typecheck
  • New test: test/integration/orb-outcomes.test.ts — excludes a PR already counted by the own-ledger, still counts an unrelated one
  • Fixed a stale test-double SQL-routing classifier in test/unit/public-stats.test.ts (isDispositions) that would have wrongly matched the new anti-join's SQL text
  • npm run test:ci (full local gate, unsharded) — green
  • Coverage on both changed files: 100% statements/lines/functions; the two files' overall branch % shown by a narrow run reflects pre-existing, unrelated lines outside this diff (verified via git diff)

Safety

  • No secrets, no schema/migration change.
  • Own-ledger data untouched — this only changes which orb_pr_outcomes rows get summed.

getOrbGlobalStats folded in every registered Orb installation's
outcomes unconditionally, re-counting PRs the frozen own-ledger
(pull_requests + audit_events, frozen since the self-host cutover)
already counted. Quantified: 243 PRs (173 merged + 70 closed, 96%
in one repo), inflating the public homepage's "PRs reviewed" and
"Decision accuracy" tiles.

Adds a NOT EXISTS anti-join so orb_pr_outcomes rows whose (repo,
pr_number) already has a github_app.pr_public_surface_published
audit event are excluded — the two sums are now over disjoint PR
sets. No data loss: the own-ledger side is untouched, and no
backfill is needed since the two sources' actual PR-number ranges
barely overlap outside the double-counted set.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.27%. Comparing base (a6d93c6) to head (b92a62b).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5257   +/-   ##
=======================================
  Coverage   94.27%   94.27%           
=======================================
  Files         474      474           
  Lines       40078    40078           
  Branches    14612    14612           
=======================================
  Hits        37782    37782           
  Misses       1609     1609           
  Partials      687      687           
Flag Coverage Δ
shard-1 46.24% <ø> (-0.15%) ⬇️
shard-2 34.70% <ø> (+0.11%) ⬆️
shard-3 32.09% <ø> (+0.05%) ⬆️
shard-4 31.72% <ø> (-0.40%) ⬇️
shard-5 33.69% <ø> (-0.03%) ⬇️
shard-6 44.99% <ø> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/orb/outcomes.ts 87.50% <ø> (ø)
src/review/public-stats.ts 96.66% <ø> (ø)
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@JSONbored JSONbored self-assigned this Jul 12, 2026
@gittensory-orb gittensory-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 12, 2026
@gittensory-orb

gittensory-orb Bot commented Jul 12, 2026

Copy link
Copy Markdown

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

Review updated: 2026-07-12 12:12:11 UTC

4 files · 1 AI reviewer · 2 blockers · readiness 93/100 · CI green · unstable

⏸️ Suggested Action - Manual Review

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.

Review summary
This PR adds a NOT EXISTS anti-join in getOrbGlobalStats to skip orb_pr_outcomes rows whose (repo, pr_number) already has a github_app.pr_public_surface_published audit event, fixing a real double-count of 243 PRs that was inflating the public homepage counter. The fix targets the correct layer (the aggregation query itself, not a downstream display clamp) and both sides of the previous overlap reasoning are corrected — the file-header comment in public-stats.ts is honestly updated to reflect the measured 243-PR figure instead of the prior unverified 'small, bounded' claim. The new integration test (orb-outcomes.test.ts) exercises the real anti-join by inserting a genuine audit_events row and a colliding orb_pr_outcomes row, which is a legitimate reachable-path test, and the isOrbGlobal routing guard in public-stats.test.ts correctly prevents the new SQL from being misrouted through the existing isDispositions test stub.

Blockers

Nits — 5 non-blocking
  • src/orb/outcomes.ts:63-67 — the NOT EXISTS correlated subquery filters on audit_events by target_key string match; confirm there's an index covering (event_type, target_key) or this anti-join does a full scan per orb_pr_outcomes row as that table grows.
  • The external brief's 'magic numbers' flags on src/orb/outcomes.ts:48 are noise — those are measured figures (243 PRs, 2026-07-12 date) in a documentation comment, not logic constants that need naming.
  • Consider adding a covering index on audit_events(event_type, target_key) if one doesn't already exist, since both the disposition query and this new anti-join filter on that pair.
  • The comment in outcomes.ts:48 could be trimmed post-merge once the 243-PR figure is stale, to avoid a comment that reads like a permanent invariant when it's really a point-in-time measurement.
  • nit: `src/orb/outcomes.ts:getOrbGlobalStats` puts `LOWER(ae.target_key)` on the anti-join comparison, so confirm the existing `audit_events` indexes still make this acceptable on production-sized ledgers or switch to comparing against the canonical target key shape if casing is already normalized.

Concerns raised — review before merging

  • No linked issue detected — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue — Link the relevant issue (for example Closes #123) before opening the PR.
Signal Result Evidence
Code review ❌ 2 blockers 1 reviewer
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 44 registered-repo PR(s), 36 merged, 470 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 44 PR(s), 470 issue(s).
Gate result ❌ Blocking Repo-configured hard blocker found.
Improvement ✅ Minor risk: clean · value: minor — Code changes are accompanied by test evidence.
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 44 PR(s), 470 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Explain no-issue PR.
  • Link the issue being solved, or explicitly explain why this is a no-issue PR.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
[BETA] Chat with Gittensory

Ask Gittensory a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @gittensory ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @gittensory chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @gittensory mention with a real question is routed to the closest matching read-only command automatically -- no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/gittensory-commands

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

  • Re-run Gittensory review

@gittensory-orb gittensory-orb Bot added the manual-review Gittensor contributor context label Jul 12, 2026
@JSONbored JSONbored merged commit 5b9e9e1 into main Jul 12, 2026
18 checks passed
@JSONbored JSONbored deleted the fix/orb-double-count branch July 12, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Development

Successfully merging this pull request may close these issues.

1 participant