-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
docs: Clarify GitHub status checks for AI Code Review #15679
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rohan-at-sentry
wants to merge
4
commits into
getsentry:master
Choose a base branch
from
rohan-at-sentry:docs/ai-code-review-status-checks
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+63
−4
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
c40b63d
docs: Clarify GitHub status checks for AI Code Review
a03fbfd
Merge branch 'master' into docs/ai-code-review-status-checks
sfanahata 4b50c50
Merge branch 'master' into docs/ai-code-review-status-checks
jaffrepaul 6e564be
Address PR review comments
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -26,6 +26,20 @@ To enable AI Code Review in your GitHub organization or on specific repositories | |||||
| - `Show Generative AI Features` | ||||||
| - `Enable AI Code Review` | ||||||
|
|
||||||
| ### GitHub Permissions | ||||||
|
|
||||||
| AI Code Review requires specific GitHub permissions to function properly. When you install or update the Sentry GitHub integration, you'll need to accept the following permissions: | ||||||
|
|
||||||
| - **Pull Requests (Read & Write)**: To read PR content and write code review comments | ||||||
| - **Checks (Read & Write)**: To create status checks that show the AI Code Review results on your PRs | ||||||
| - **Commit Statuses (Read & Write)**: To post status checks to commits and integrate with branch protection rules | ||||||
|
|
||||||
| If you have previously installed the GitHub integration, you may need to update your permissions to include these. GitHub will prompt you to accept updated permissions when Sentry requests them. | ||||||
|
|
||||||
| <Alert level="info"> | ||||||
| Denying or ignoring permission update requests won't affect your current Sentry usage, but will prevent AI Code Review and other features from working. | ||||||
| </Alert> | ||||||
|
|
||||||
| ## AI Code Review Commands | ||||||
|
|
||||||
| Ways AI Code Review can help you: | ||||||
|
|
@@ -38,6 +52,41 @@ Once you have added a comment, the assistant will reply, acknowledging the reque | |||||
|
|
||||||
|  | ||||||
|
|
||||||
| ## GitHub Status Checks | ||||||
|
|
||||||
| AI Code Review creates GitHub status checks (also called "checks") on your pull requests to indicate whether the code review passed or found potential issues. These status checks appear in the PR's checks section and can be integrated with GitHub's branch protection rules. | ||||||
|
|
||||||
| ### Status Check Behavior | ||||||
|
|
||||||
| - **Success**: If AI Code Review completes successfully, the status check will show as successful (green checkmark). This happens regardless of whether issues were found - the check indicates the review completed, not that the code is error-free. | ||||||
| - **Neutral**: If AI Code Review runs but has no specific pass/fail outcome. | ||||||
| - **Error**: If there's an error running the AI Code Review itself, such as a service issue. | ||||||
| - **Cancelled**: If the review was cancelled, typically because a new commit was pushed and the previous review was superseded. | ||||||
| - **Timed out**: If the review takes too long to complete. | ||||||
|
|
||||||
rohan-at-sentry marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| ### Visibility | ||||||
|
|
||||||
| Status checks from AI Code Review are visible to: | ||||||
| - All users with read access to the repository | ||||||
| - Anyone viewing the pull request on GitHub | ||||||
| - GitHub Actions and other CI/CD tools that check PR status | ||||||
|
|
||||||
| The detailed code review comments are also visible to all users with repository access, appearing as review comments on the pull request. | ||||||
|
|
||||||
| ### Using with Branch Protection | ||||||
|
|
||||||
| You can optionally configure GitHub branch protection rules to require the AI Code Review status check before merging. To do this: | ||||||
|
|
||||||
| 1. Go to your GitHub repository Settings > Branches | ||||||
| 2. Add or edit a branch protection rule for your target branch (e.g., `main`) | ||||||
| 3. Enable "Require status checks to pass before merging" | ||||||
| 4. Search for and select the "Seer Code Review" status check | ||||||
| 5. Save your changes | ||||||
|
|
||||||
| <Alert level="warning"> | ||||||
| We recommend not making the AI Code Review status check a required check. Requiring it will block PR merges if the check fails due to service disruptions, and may conflict with future personal configuration options that allow users to opt out of code review. | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| </Alert> | ||||||
|
|
||||||
| ## Frequently Asked Questions | ||||||
|
|
||||||
| - **What data does AI Code Review need access to for the AI system to function, and what information is sent to third-party AI providers?** | ||||||
|
|
@@ -58,6 +107,16 @@ Once you have added a comment, the assistant will reply, acknowledging the reque | |||||
|
|
||||||
| Error Prediction is automatically triggered by the following GitHub pull_request webhook events: | ||||||
| - `opened`: when you open a new pull request (we skip those opened in `draft` state) | ||||||
| - `ready_for_review`: when a draft pull request is marked as “Ready for review” | ||||||
| - `ready_for_review`: when a draft pull request is marked as "Ready for review" | ||||||
|
|
||||||
| To manually run error prediction and get a general review, comment `@sentry review` in the PR. | ||||||
|
|
||||||
| - **Why is the AI Code Review status check showing an error?** | ||||||
|
|
||||||
| The status check may show an error if there's a temporary service issue or the review timed out. The status check does not fail based on code quality issues found during review - those appear as review comments on your PR. | ||||||
|
|
||||||
| If you don't see a status check at all, ensure that the `Show Generative AI Features` and `Enable AI Code Review` settings are enabled in your [organization settings](https://sentry.io/orgredirect/settings/:orgslug/), and that your GitHub integration has the required Checks permission. | ||||||
|
|
||||||
| - **Can I disable status checks while keeping AI Code Review comments?** | ||||||
|
|
||||||
| Status checks are automatically created whenever AI Code Review runs. If you don't want them to appear or affect your workflow, simply don't add them as a required check in your branch protection rules. The checks will still appear on PRs but won't block merging. | ||||||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we don't need this one. We use this endpoint and it lists "Checks" repository permissions (write)
Let's check other endpoints used:
Looks like they all are for the
checks:write