Skip to content

fix: paginate GitHub GraphQL queries (#675)#955

Open
maxachis wants to merge 1 commit intodevfrom
fix/675-graphql-pagination
Open

fix: paginate GitHub GraphQL queries (#675)#955
maxachis wants to merge 1 commit intodevfrom
fix/675-graphql-pagination

Conversation

@maxachis
Copy link
Copy Markdown
Contributor

@maxachis maxachis commented May 2, 2026

Summary

GitHub's GraphQL API caps the first argument at 100, so any query that may return more than 100 nodes needs cursor-based pagination via pageInfo { endCursor hasNextPage } and an after: cursor on subsequent calls.

This PR adds that pagination to the two queries where the 100-node cap is reachable in practice:

  • get_github_issue_project_statuses in middleware/third_party_interaction_logic/github/helpers.py (project items grow with every data request issue).
  • GithubLabelManager.get_labels in middleware/third_party_interaction_logic/github/label_manager.py (repository labels).

Other first: N usages have small bounded N (labels(first: 10) per issue, fields(first: 20) for project fields) and are intentionally left as-is.

convert_graph_ql_result_to_issue_info now accepts either a single response dict or a list of page dicts so callers can hand in all paginated responses at once.

Closes #675

Test plan

  • New unit tests in tests/middleware/test_github_graphql_pagination.py cover:
    • Multi-page project-status fetch threads endCursor into the next after: argument and merges nodes.
    • Single-page response stops after one call.
    • convert_graph_ql_result_to_issue_info correctly merges multiple page payloads.
    • GithubLabelManager paginates label results across pages.
  • uv run ruff check . passes.
  • uv run basedpyright --level error passes (0 errors).
  • Full uv run pytest tests should be run in CI (local DB unavailable in this environment, so only the four new unit tests were executed locally via --noconftest).

Notes for reviewers

  • No callers outside of these two modules use the changed helpers, so the dict | list[dict] union on convert_graph_ql_result_to_issue_info is purely forward-compatible.
  • The after: cursor is interpolated as a string literal into the query rather than being passed as a GraphQL variable, matching the existing string-templated style in this module. Could be migrated to $variables later if desired.

GitHub's GraphQL API caps the `first` argument at 100, so queries that
can return more than 100 nodes must follow `pageInfo.endCursor` /
`hasNextPage` to retrieve every result. This adds cursor-based
pagination to the two queries where the cap is reachable in practice:

- `get_github_issue_project_statuses` (project items, can grow with
  every data request issue).
- `GithubLabelManager.get_labels` (repository labels).

Other `first: N` usages (per-issue label and project field subqueries)
have small bounded N and are left as-is.

Closes #675

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant