-
-
Notifications
You must be signed in to change notification settings - Fork 638
Description
Fix CI Failures for Dependabot PRs
Summary
Dependabot PRs consistently fail CI because they don't have access to the REACT_ON_RAILS_PRO_LICENSE secret. This blocks automated dependency updates from being merged.
Example PR
PR #2170 - Bump webpack and webpack-dev-server
Root Cause
GitHub restricts Dependabot from accessing repository secrets for security reasons. When Dependabot creates a PR, the CI workflows that require the REACT_ON_RAILS_PRO_LICENSE secret fail with:
```
ReactOnRailsPro::Error: [React on Rails Pro] License validation error: [React on Rails Pro] No license found.
Please set REACT_ON_RAILS_PRO_LICENSE environment variable or create
/home/runner/work/react_on_rails/react_on_rails/react_on_rails_pro/spec/dummy/config/react_on_rails_pro_license.key file.
```
Affected Workflows
- React on Rails Pro - Package Tests - Fails at "Generate file-system based entrypoints"
- React on Rails Pro - Lint - Fails due to missing license
- React on Rails Pro - Integration Tests - Fails due to missing license
- JS unit tests for Renderer package - May fail if it requires Pro license
Why This Happens
- Dependabot PRs run in a restricted context with no access to secrets
- Pro package workflows require the
REACT_ON_RAILS_PRO_LICENSEenv var - The Pro package's license validation happens during Rails initialization
- Without a valid license, the Pro package raises an error and exits
Proposed Solutions
Option 1: Skip Pro Workflows for Dependabot PRs (Recommended)
Add a condition to skip Pro-specific workflows when triggered by Dependabot:
```yaml
In .github/workflows/pro-integration-tests.yml, pro-lint.yml, etc.
jobs:
build:
# Skip for Dependabot PRs since they can't access secrets
if: github.actor != 'dependabot[bot]'
runs-on: ubuntu-22.04
# ...
```
Pros:
- Simple to implement
- Dependabot PRs will pass open-source tests
- Maintainers can re-run Pro tests after merge or by checking out the branch
Cons:
- Pro tests won't run on dependency updates automatically
- Need to manually verify Pro compatibility before merging
Option 2: Use Dependabot Secrets
GitHub allows Dependabot-specific secrets. Add the Pro license as a Dependabot secret:
- Go to Settings → Secrets and variables → Dependabot
- Add
REACT_ON_RAILS_PRO_LICENSEsecret
Pros:
- Full CI runs for Dependabot PRs
- No workflow changes needed
Cons:
- Need to maintain separate secret
- May have security implications
Option 3: Allow Evaluation License for CI
Modify the Pro package to accept a special "CI_EVALUATION" mode that skips license validation for Dependabot PRs:
```ruby
In react_on_rails_pro initializer
if ENV['CI'] == 'true' && ENV['GITHUB_ACTOR'] == 'dependabot[bot]'
Skip license validation for Dependabot CI
ReactOnRailsPro.configure { |c| c.skip_license_validation = true }
end
```
Pros:
- Full CI runs for Dependabot PRs
- No secret management needed
Cons:
- Requires code changes to Pro package
- Could be bypassed (security consideration)
Option 4: Hybrid Approach
- Skip Pro workflows for Dependabot PRs (Option 1)
- Add a manual workflow trigger that maintainers can run after reviewing the PR
- Require the manual workflow to pass before merging
```yaml
.github/workflows/pro-manual-test.yml
name: Pro Tests (Manual)
on:
workflow_dispatch:
inputs:
pr_number:
description: 'PR number to test'
required: true
```
Recommended Implementation
Use Option 1 (Skip Pro workflows) as the primary solution with documentation explaining the limitation.
Implementation Steps
-
Update Pro workflows to skip Dependabot PRs:
```yaml
.github/workflows/pro-integration-tests.yml
jobs:
detect-changes:
if: github.actor != 'dependabot[bot]'
``` -
Update Dependabot auto-merge settings (if any) to not auto-merge without Pro tests
-
Document the limitation in CONTRIBUTING.md:
Dependabot PRs skip React on Rails Pro tests due to secret restrictions.
Maintainers should manually verify Pro compatibility before merging dependency updates. -
Add a comment bot (optional) that posts a reminder on Dependabot PRs about missing Pro tests
Affected Files
.github/workflows/pro-integration-tests.yml.github/workflows/pro-lint.yml.github/workflows/pro-test-package-and-gem.yml.github/workflows/package-js-tests.yml(if it tests Pro packages)
Related
- GitHub Docs: Dependabot Secrets
- PR #2170 - Example failing Dependabot PR
Questions
- Should we add the Pro license as a Dependabot secret for full CI coverage?
- Is there a security concern with Dependabot having access to the Pro license?
- Should we create a separate "verify Pro compatibility" workflow that maintainers trigger manually?