docs: add PR approval waiting-time analysis#10506
Draft
basvandijk wants to merge 2 commits into
Draft
Conversation
Adds a reproducible analysis of how long PRs in dfinity/ic wait for code-review approval, to assess review as a productivity bottleneck. - Fetches all PRs from the last 12 months (reviews + review-request timeline) via the GitHub GraphQL API. - Filters out drafts and abandoned (stale) open PRs; reports bots separately. - Classifies PRs as same-team vs cross-team using CODEOWNERS review requests, canonicalized to stable umbrella teams to account for CODEOWNERS reorganizations over the window. - Reports time-to-first-approval and time-to-full-approval (wall-clock and weekday-only) for 12- and 6-month windows. See pr-approval-analysis/REPORT.md for findings and pr-approval-analysis/README.md to reproduce.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new pr-approval-analysis/ directory containing a reproducible (scripts + checked-in outputs) analysis of PR approval waiting times in dfinity/ic, split into same-team vs cross-team approval dynamics.
Changes:
- Adds Python scripts to fetch PR/review-request data from GitHub, snapshot CODEOWNERS umbrella-team membership, derive a stable team mapping across CODEOWNERS reorganizations, and compute summary statistics.
- Checks in the generated report (
REPORT.md) plus the derived team mapping (team_mapping.json) and supporting reproduction docs (README.md,.gitignore).
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| pr-approval-analysis/README.md | Reproduction instructions and artifact inventory for the analysis. |
| pr-approval-analysis/REPORT.md | Generated findings (12-month and 6-month windows) and methodology/limitations. |
| pr-approval-analysis/analyze.py | Core analysis and report rendering from fetched PR/review data. |
| pr-approval-analysis/fetch_prs.py | GraphQL-based PR data fetcher (reviews + timeline review-request events), resumable. |
| pr-approval-analysis/fetch_teams.py | Fetches current membership of CODEOWNERS umbrella teams via gh api. |
| pr-approval-analysis/build_team_mapping.py | Derives fine-team → umbrella-team mapping from historical vs current CODEOWNERS resolution. |
| pr-approval-analysis/team_mapping.json | Checked-in derived mapping used to canonicalize teams across time. |
| pr-approval-analysis/.gitignore | Ignores fetched datasets and intermediate artifacts produced by scripts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+655
to
+658
| report = render(results, home, prs) | ||
| with open("report.md", "w") as f: | ||
| f.write(report) | ||
| print("\nwrote report.md") |
Comment on lines
+9
to
+11
|
|
||
| Writes report.md and prints a summary. | ||
| """ |
Comment on lines
+59
to
+63
| reviewDecision | ||
| reviews(first: 50) { | ||
| totalCount | ||
| nodes { state submittedAt author { login __typename } } | ||
| } |
Comment on lines
+159
to
+160
| "research": "core-protocol", | ||
| "p2p-systems-research-team": "core-protocol", |
Comment on lines
+163
to
+165
| "canister-os": "node", | ||
| "platform-operations": "dre", | ||
| "governance": "governance-team", |
Comment on lines
+167
to
+170
| "cross-chain-team": "defi", | ||
| "languages": "sdk", | ||
| "motoko": "sdk", | ||
| "utopia": "sdk", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a reproducible analysis of how long PRs in
dfinity/icwait for code-review approval, to assess whether review is a productivity bottleneck as AI makes writing code cheaper.Everything lives under
pr-approval-analysis/— see REPORT.md for the full findings and README.md to reproduce.Headline findings (last 12 months, 3,953 human PRs after filtering)
Method notes
consensus/execution/team-dsm→core-protocol).Notes
@dfinity/ic-owners-ownersas a code owner.