Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions .github/actions/code-review/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: 'protoLabs Code Review'
description: 'Async-parallel code review via the protoLabs LLM gateway. Maps the repo into bounded feature records, runs a model-backed review, posts findings as a sticky PR comment, and appends them to the GitHub Actions step summary. Non-blocking by design.'
author: 'protoLabs'
branding:
icon: 'eye'
color: 'blue'

inputs:
gateway-api-key:
description: 'Bearer token for the LLM gateway (passed as OPENAI_API_KEY to review-code).'
required: true
gateway-base-url:
description: 'LLM gateway base URL without `/v1`. `/v1` is appended automatically.'
required: false
default: 'https://api.proto-labs.ai'
model:
description: 'Gateway model alias. Defaults to review-code default (protolabs/smart).'
required: false
default: ''
timeout-ms:
description: 'Per-request timeout for the LLM gateway in milliseconds.'
required: false
default: '300000'
github-token:
description: 'Token used to post the sticky PR comment. Defaults to the workflow GITHUB_TOKEN.'
required: false
default: ${{ github.token }}
pr-number:
description: 'Pull request number to comment on. Defaults to github.event.pull_request.number.'
required: false
default: ${{ github.event.pull_request.number }}
release-tools-ref:
description: 'release-tools ref (branch/tag/commit) to install. Defaults to main.'
required: false
default: 'main'

outputs:
findings:
description: 'Number of review findings produced. Empty string if the review step failed.'
value: ${{ steps.review.outputs.findings }}
report-path:
description: 'Filesystem path of the generated markdown report. Empty string if no report was generated.'
value: ${{ steps.report.outputs.report_path }}

runs:
using: composite
steps:
- name: Initialize review state
shell: bash
run: |
npx --yes -p "github:protoLabsAI/release-tools#${{ inputs.release-tools-ref }}" review-code init

- name: Map features
shell: bash
run: |
npx --yes -p "github:protoLabsAI/release-tools#${{ inputs.release-tools-ref }}" review-code map

- name: Run review against PR diff
id: review
shell: bash
continue-on-error: true
env:
OPENAI_API_KEY: ${{ inputs.gateway-api-key }}
OPENAI_BASE_URL: ${{ inputs.gateway-base-url }}/v1
CODE_REVIEW_TIMEOUT_MS: ${{ inputs.timeout-ms }}
CODE_REVIEW_MODEL: ${{ inputs.model }}
run: |
set -o pipefail
ARGS="--all"
if [ -n "${{ inputs.model }}" ]; then ARGS="$ARGS --model ${{ inputs.model }}"; fi
npx --yes -p "github:protoLabsAI/release-tools#${{ inputs.release-tools-ref }}" review-code run $ARGS 2>&1 | tee /tmp/review-run.log
findings=$(grep -oE 'findings=[0-9]+' /tmp/review-run.log | tail -1 | cut -d= -f2)
echo "findings=${findings:-0}" >> "$GITHUB_OUTPUT"

- name: Build markdown report
id: report
if: always()
shell: bash
run: |
npx --yes -p "github:protoLabsAI/release-tools#${{ inputs.release-tools-ref }}" review-code report --output /tmp/code-review.md 2>&1 | tee /tmp/report.log || true
if [ -f /tmp/code-review.md ]; then
echo "report_path=/tmp/code-review.md" >> "$GITHUB_OUTPUT"
fi
Comment on lines +80 to +83
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Avoid stale report reuse by using a unique temp file per run.

Line 80 writes to a fixed /tmp/code-review.md and Line 81 only checks existence. On persistent runners, a previous file can be reused if report generation fails, leading to stale findings being posted.

Proposed fix
-        npx --yes -p "github:protoLabsAI/release-tools#${{ inputs.release-tools-ref }}" review-code report --output /tmp/code-review.md 2>&1 | tee /tmp/report.log || true
-        if [ -f /tmp/code-review.md ]; then
-          echo "report_path=/tmp/code-review.md" >> "$GITHUB_OUTPUT"
+        report_file="${RUNNER_TEMP}/code-review-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}.md"
+        rm -f "$report_file"
+        npx --yes -p "github:protoLabsAI/release-tools#${{ inputs.release-tools-ref }}" review-code report --output "$report_file" 2>&1 | tee /tmp/report.log || true
+        if [ -s "$report_file" ]; then
+          echo "report_path=$report_file" >> "$GITHUB_OUTPUT"
         fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
npx --yes -p "github:protoLabsAI/release-tools#${{ inputs.release-tools-ref }}" review-code report --output /tmp/code-review.md 2>&1 | tee /tmp/report.log || true
if [ -f /tmp/code-review.md ]; then
echo "report_path=/tmp/code-review.md" >> "$GITHUB_OUTPUT"
fi
report_file="${RUNNER_TEMP}/code-review-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}.md"
rm -f "$report_file"
npx --yes -p "github:protoLabsAI/release-tools#${{ inputs.release-tools-ref }}" review-code report --output "$report_file" 2>&1 | tee /tmp/report.log || true
if [ -s "$report_file" ]; then
echo "report_path=$report_file" >> "$GITHUB_OUTPUT"
fi
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/actions/code-review/action.yml around lines 80 - 83, Replace the
fixed /tmp/code-review.md with a unique per-run temp file and update the
subsequent existence check and GITHUB_OUTPUT write to use that variable;
specifically change the npx review-code command invocation (the line calling
"npx ... review-code report --output /tmp/code-review.md") to write to a temp
file created at runtime (e.g., via mktemp or using RUNNER_TEMP/GITHUB_RUN_ID)
and then change the if check and the echo that sets report_path to reference
that same temp variable instead of the hard-coded path, optionally also ensure
the script verifies the file is non-empty before exporting report_path to avoid
using stale/empty files.


- name: Post findings as sticky PR comment
if: always() && steps.report.outputs.report_path != '' && inputs.pr-number != ''
shell: bash
env:
GH_TOKEN: ${{ inputs.github-token }}
PR_NUMBER: ${{ inputs.pr-number }}
REPORT_PATH: ${{ steps.report.outputs.report_path }}
FINDINGS_COUNT: ${{ steps.review.outputs.findings }}
run: node "${{ github.action_path }}/post-findings.mjs"

- name: Append findings to step summary
if: always() && steps.report.outputs.report_path != ''
shell: bash
run: |
{
echo "## Code Review — ${{ steps.review.outputs.findings }} finding(s)"
echo
cat "${{ steps.report.outputs.report_path }}"
} >> "$GITHUB_STEP_SUMMARY"
88 changes: 88 additions & 0 deletions .github/actions/code-review/post-findings.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
#!/usr/bin/env node
/**
* Post a code-review findings report as a sticky PR comment.
*
* Reads REPORT_PATH, PR_NUMBER, FINDINGS_COUNT, and GH_TOKEN from the
* environment. Uses the `gh` CLI to PATCH a previously-posted comment
* tagged with the `code-review:findings` marker, so re-runs replace
* prior output instead of stacking new comments on the PR.
*
* Called from the composite action at
* `.github/actions/code-review/action.yml`. Designed to be non-blocking:
* any failure is logged and exit code is 0.
*/
import { readFileSync, writeFileSync, mkdtempSync } from 'node:fs';
import { execFileSync } from 'node:child_process';
import { join } from 'node:path';
import { tmpdir } from 'node:os';

const MARKER = '<!-- code-review:findings -->';

const reportPath = process.env.REPORT_PATH;
const prNumber = process.env.PR_NUMBER;
const findingsCount = process.env.FINDINGS_COUNT || '?';

if (!reportPath || !prNumber) {
console.error('REPORT_PATH and PR_NUMBER must be set');
process.exit(0); // non-blocking
}

let report;
try {
report = readFileSync(reportPath, 'utf8');
} catch (err) {
console.error(`Cannot read report at ${reportPath}: ${err.message}`);
process.exit(0);
}

const body = `${MARKER}
## Code Review — ${findingsCount} finding(s)

> Async review running parallel to CodeRabbit. Findings are advisory; not all are merge blockers.

${report}
`;

function gh(args, opts = {}) {
return execFileSync('gh', args, { encoding: 'utf8', ...opts });
}

const workdir = mkdtempSync(join(tmpdir(), 'code-review-'));
const bodyFile = join(workdir, 'body.md');
writeFileSync(bodyFile, body);

let existingId = null;
try {
const comments = JSON.parse(gh(['pr', 'view', prNumber, '--json', 'comments']));
const match = (comments.comments ?? []).find((c) => c.body?.includes(MARKER));
if (match) existingId = match.id;
Comment on lines +56 to +58
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

❓ Verification inconclusive

🏁 Script executed:

#!/bin/bash
# Verify whether comment IDs from `gh pr view --json comments` are REST-patchable IDs.
set -euo pipefail

PR_NUMBER="${PR_NUMBER:?set PR_NUMBER}"
# Shows the shape of IDs returned by pr view
gh pr view "$PR_NUMBER" --json comments --jq '.comments[] | {id, url}' | head -n 5

# Shows numeric IDs from REST issue comments for the same PR
gh api "repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" --jq '.[] | {id, html_url}' | head -n 5

Repository: protoLabsAI/release-tools

Length of output: 113


Confirm PATCH /issues/comments/{comment_id} uses REST numeric IDs; existingId currently comes from gh pr view --json comments.

post-findings.mjs sets existingId = match.id from gh pr view <prNumber> --json comments, then PATCHes /repos/${owner}/${repo}/issues/comments/${existingId}. The {comment_id} path for that REST endpoint must be the REST comment ID; ensure the id returned by gh pr view --json comments is numeric/REST-compatible, or the script may not update the existing comment.

Proposed fix (lookup via REST comments so IDs are patchable)
-let existingId = null;
+let existingId = null;
 try {
-  const comments = JSON.parse(gh(['pr', 'view', prNumber, '--json', 'comments']));
-  const match = (comments.comments ?? []).find((c) => c.body?.includes(MARKER));
+  const comments = JSON.parse(
+    gh([
+      'api',
+      `repos/${process.env.GITHUB_REPOSITORY}/issues/${prNumber}/comments`,
+      '--paginate',
+    ]),
+  );
+  const match = (comments ?? []).find((c) => c.body?.includes(MARKER));
   if (match) existingId = match.id;
 } catch (err) {
   console.error(`Failed to list comments (continuing as new): ${err.message}`);
 }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/actions/code-review/post-findings.mjs around lines 56 - 58, The
current code assigns existingId = match.id from gh(['pr','view', prNumber,
'--json', 'comments']) but then PATCHes
/repos/${owner}/${repo}/issues/comments/${existingId}, which requires the REST
numeric comment ID; update the lookup so existingId is the REST comment id:
replace the gh pr view comments fetch with a call to the REST API (e.g., gh api
repos/:owner/:repo/issues/:issue_number/comments) and find the comment whose
body includes MARKER, then set existingId to that comment's id before calling
the PATCH endpoint; ensure the variables referenced are existingId, match (or
the found comment), and the PATCH URL
/repos/${owner}/${repo}/issues/comments/${existingId}.

} catch (err) {
console.error(`Failed to list comments (continuing as new): ${err.message}`);
}

if (existingId) {
try {
const payloadFile = join(workdir, 'patch.json');
writeFileSync(payloadFile, JSON.stringify({ body }));
gh([
'api',
'--method',
'PATCH',
`/repos/{owner}/{repo}/issues/comments/${existingId}`,
'--input',
payloadFile,
]);
console.log(`Updated existing code-review comment ${existingId}`);
process.exit(0);
} catch (err) {
console.error(`Patch failed, posting new comment instead: ${err.message}`);
}
}

try {
gh(['pr', 'comment', prNumber, '--body-file', bodyFile]);
console.log('Posted new code-review comment');
} catch (err) {
console.error(`Failed to post comment: ${err.message}`);
process.exit(0);
}
54 changes: 54 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,60 @@ The harness keeps prompts bounded, skips external symlinks, rejects findings
outside the reviewed file allowlist, and preserves existing triage fields when
a finding is regenerated.

### Use it as a GitHub Action

For PR-level async-parallel review (runs alongside CodeRabbit, posts findings
as a sticky PR comment), use the bundled composite action:

```yaml
# .github/workflows/code-review.yml in your repo
name: Code Review

on:
pull_request:
branches: [main]

permissions:
contents: read
pull-requests: write

jobs:
review:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- uses: actions/setup-node@v4
with: { node-version: "22" }
- uses: protoLabsAI/release-tools/.github/actions/code-review@main
with:
gateway-api-key: ${{ secrets.GATEWAY_API_KEY }}
gateway-base-url: ${{ secrets.GATEWAY_BASE_URL }}
```

#### Action inputs

| Input | Required | Default | Description |
| ------------------- | -------- | ----------------------------------------- | ---------------------------------------------------------- |
| `gateway-api-key` | yes | — | Bearer token for the gateway. Passed as `OPENAI_API_KEY`. |
| `gateway-base-url` | no | `https://api.proto-labs.ai` | Gateway base URL without `/v1` — appended automatically. |
| `model` | no | `protolabs/smart` (review-code default) | Override the gateway model alias. |
| `timeout-ms` | no | `300000` | Per-request timeout for the LLM gateway in milliseconds. |
| `pr-number` | no | `${{ github.event.pull_request.number }}` | PR number to comment on; skip the sticky comment if empty. |
| `github-token` | no | `${{ github.token }}` | Token for posting the sticky PR comment. |
| `release-tools-ref` | no | `main` | release-tools ref to install (branch, tag, or commit). |

#### Behavior

- Non-blocking — review failures or timeouts do not gate the PR (`continue-on-error: true` on the review step).
- Posts a **single sticky PR comment** tagged with the `code-review:findings` marker; re-runs `PATCH` the existing comment rather than stacking new ones.
- Appends the markdown report to `$GITHUB_STEP_SUMMARY` so findings are visible in the Actions UI even when no PR comment is posted (e.g., manual `workflow_dispatch`).

#### Required secrets

- `GATEWAY_API_KEY` — same secret used by the release-notes action.
- `GATEWAY_BASE_URL` — only required if your gateway isn't at the default `https://api.proto-labs.ai`.

## Development

```bash
Expand Down
Loading