diff --git a/Changelog.md b/Changelog.md index 325243a352..9c7c2d61c0 100644 --- a/Changelog.md +++ b/Changelog.md @@ -25,6 +25,7 @@ - Fixed filter Canvas Test Student from roster sync (#7926) ### 🔧 Internal changes +- Add release automation scripts (#7914) - Added seed task to assign TAs to A1 groupings and criteria (#7867) - Updated autotest seed files to ensure settings follow tester JSON schema (#7775) - Refactored grade entry form helper logic into `GradeEntryFormsController` and removed the newly-unused helper file. (#7789) diff --git a/release/RELEASING.md b/release/RELEASING.md new file mode 100644 index 0000000000..e8a125d4d5 --- /dev/null +++ b/release/RELEASING.md @@ -0,0 +1,210 @@ +# Releasing MarkUs + +Step-by-step guide for cutting a MarkUs minor release. Helper scripts in this directory automate the tedious parts — each step shows the manual command and the script alternative. + +## Prerequisites + +- `gh` CLI authenticated (`gh auth status`) +- Docker running (`docker compose up`) +- A GitHub milestone exists for the target version with all relevant PRs merged and tagged +- Clean working tree + +## Phase 1: Setup + +```bash +git fetch origin +git checkout release && git pull origin release +git checkout -b v2.X.Y # branch from release, not master +``` + +**Verify:** `git log --oneline -1` matches the latest release branch commit. + +## Phase 2: Recon — discover what to cherry-pick + +```bash +RECON=$(ruby release/recon.rb v2.X.Y) +echo "$RECON" | ruby release/recon-format.rb --summary +echo "$RECON" | ruby release/recon-format.rb --plan +``` + +This queries the milestone, checks which PRs are already on the release branch, resolves file-overlap dependencies, and outputs a JSON plan. The `$RECON` variable is reused in later phases (release notes, PR body). + +Review the plan. Note any non-PR commits (direct pushes, fork merges) — decide whether to include or skip. + +## Phase 3: Cherry-pick + +**Automated (recommended):** + +```bash +release/cherry-pick.sh v2.X.Y +``` + +This cherry-picks all milestone PRs in dependency order, auto-resolves Changelog conflicts, skips empty commits, and verifies each pick for contamination. It stops on code conflicts or contamination and tells you exactly what to do. + +After fixing a problem, resume from where it stopped: +```bash +release/cherry-pick.sh v2.X.Y --resume +``` + +At the end it prints the PR list and the `changelog.rb` command to run next. + +
+Manual alternative + +For each PR in the order from recon: + +```bash +git cherry-pick -m1 +ruby release/verify.rb +``` + +Conflict handling: +- **Changelog.md only:** `git checkout --ours Changelog.md && git add Changelog.md && GIT_EDITOR=true git cherry-pick --continue` +- **Code files:** Stop. Resolve by comparing against `gh pr diff `. +- **Empty commit:** Already on release. `git cherry-pick --skip`. +
+ +## Phase 4: Rebuild the Changelog + +The Changelog is always corrupted after cherry-picks. Rebuild it: + +```bash +ruby release/changelog.rb --mode=release --version=v2.X.Y --prs=7783,7851,7858 +``` + +Pass the comma-separated list of cherry-picked PR numbers. The script reads `origin/release` and `origin/master`, filters master's unreleased entries to only the included PRs, and outputs a clean Changelog. + +```bash +ruby release/changelog.rb --mode=release --version=v2.X.Y --prs= > Changelog.md +``` + +**Validate:** +```bash +bash release/validate_changelog.sh v2.X.Y +``` + +All 6 checks should pass: no conflict markers, empty unreleased, version section exists with entries, correct ordering, no duplicate PRs, older sections unchanged. + +## Phase 5: Version bump and commit + +```bash +echo "VERSION=v2.X.Y,PATCH_LEVEL=DEV" > app/MARKUS_VERSION +git add Changelog.md app/MARKUS_VERSION +git commit -m "v2.X.Y" +``` + +`PATCH_LEVEL=DEV` is a legacy field — always keep it as-is. + +## Phase 6: Test + +```bash +docker compose exec rails bundle exec rspec +docker compose exec rails npx jest --no-coverage +``` + +Pre-existing failures on the release branch are expected. Verify no NEW failures were introduced by the cherry-picks. + +## Phase 7: Dependency and settings check + +```bash +git diff origin/release -- Gemfile Gemfile.lock package.json package-lock.json +git diff origin/release -- markus.control config/settings.yml +git diff origin/release --name-only -- db/migrate/ +``` + +If any of these show changes, notify sysadmins before deployment. They may need to `bundle install`, `npm install`, apply new settings to `settings.local.yml`, or run migrations. + +## Phase 8: Push and PR + +```bash +git push -u origin v2.X.Y +gh pr create --base release --title "v2.X.Y" --body "Release v2.X.Y" +``` + +Wait for CI. Get reviewer approval. **Merge with "Create a merge commit"** (never squash into release). + +## Phase 9: GitHub Release + +After the PR is merged: + +```bash +# Re-run if your shell session expired since Phase 2 +RECON=$(ruby release/recon.rb v2.X.Y) +gh release create v2.X.Y --repo MarkUsProject/Markus --target release --title "v2.X.Y" --notes "$(echo "$RECON" | ruby release/recon-format.rb --release-notes)" +``` + +Or create manually via GitHub UI: Releases > Create > tag `v2.X.Y`, target `release`. + +## Phase 10: Milestone management + +```bash +# Close released milestone +MILESTONE_ID=$(gh api repos/MarkUsProject/Markus/milestones --jq ".[] | select(.title==\"v2.X.Y\") | .number") +gh api -X PATCH "repos/MarkUsProject/Markus/milestones/$MILESTONE_ID" -f state=closed + +# Create next milestone +gh api repos/MarkUsProject/Markus/milestones -f title="v2.X.Z" +``` + +## Phase 11: Sync Changelog to master + +Move released entries from `[unreleased]` into a new version section on master: + +```bash +git checkout master && git pull origin master +git checkout -b v2.X.Y-changelog + +ruby release/changelog.rb --mode=master-sync --version=v2.X.Y --prs= > Changelog.md +bash release/validate_changelog_master.sh v2.X.Y + +git add Changelog.md +git commit -m "Update changelog with new release v2.X.Y [ci skip]" +git push -u origin v2.X.Y-changelog +gh pr create --base master --title "Update changelog for v2.X.Y" --body "Sync released entries." +``` + +Squash-merge is fine here (same branch lineage, `[ci skip]` skips CI). + +## Phase 12: Satellite repos (Wiki, Autotester) + +Check each repo's milestone for PRs. If any exist, follow the same cherry-pick + PR + release flow. If none, still create a GitHub release with the version tag. + +## Phase 13: Cleanup + +Delete version branches: +```bash +git push origin --delete v2.X.Y v2.X.Y-changelog +git branch -d v2.X.Y v2.X.Y-changelog +``` + +--- + +## Helper Scripts Reference + +| Script | What it does | +|--------|-------------| +| `cherry-pick.sh ` | Automated cherry-pick loop with verification. `--resume` to continue after fixing a conflict | +| `recon.rb ` | Queries milestone, resolves cherry-pick order, outputs JSON | +| `recon-format.rb --flag` | Formats recon JSON (pipe from stdin). Flags: `--summary`, `--plan`, `--order`, `--pr-list`, `--pr-body`, `--release-notes`, `--skipped` | +| `verify.rb ` | Compares last cherry-pick diff against original PR. Exit 0 = clean, 1 = contaminated | +| `changelog.rb --mode=MODE --version=V --prs=N,N` | Rebuilds Changelog. Modes: `release` (for release branch), `master-sync` (for master) | +| `validate_changelog.sh ` | 6-check validation for release branch changelog | +| `validate_changelog_master.sh ` | 5-check validation for master changelog sync | + +All scripts accept `--help` for usage details. + +--- + +## Pitfalls + +| Problem | Solution | +|---------|----------| +| Changelog corrupted after cherry-picks | Always rebuild with `changelog.rb`. Never trust the auto-merged result. | +| Cherry-pick pulls in extra code | Git 3-way merge can import dependency PR code. Always run `verify.rb` after each pick. | +| Empty cherry-pick commit | PR was in a prior release. Check changelog, `git cherry-pick --skip`. | +| `PATCH_LEVEL=RELEASE` | Wrong. Always `PATCH_LEVEL=DEV`. Legacy field, unused at runtime. | +| API returns 404 | Include `/csc108` prefix. Use `MarkUsAuth` not `Bearer`. | +| Jest flag | `--testPathPatterns` (plural), not singular. | +| Rails runner `!` escaping | Pipe via stdin: `echo '...' | docker compose exec -T rails bundle exec rails runner -` | +| Squash-merge into release | Never. Use "Create a merge commit" to preserve commit history. | +| Copying release Changelog to master | Never overwrite. Use `--mode=master-sync` to move entries from unreleased. | diff --git a/release/changelog.rb b/release/changelog.rb new file mode 100755 index 0000000000..f5cc9463ce --- /dev/null +++ b/release/changelog.rb @@ -0,0 +1,157 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# Changelog Rebuild — Produces a clean Changelog.md after cherry-picks. +# +# Usage: +# ruby release/changelog.rb --mode=release --version=v2.9.6 --prs=7783,7851,7858 +# ruby release/changelog.rb --mode=master-sync --version=v2.9.6 --prs=7783,7851,7858 +# ruby release/changelog.rb --help +# +# Modes: +# release — Empty [unreleased] + new version section + old sections from origin/release +# master-sync — Move cherry-picked entries from [unreleased] to new version section on master + +require_relative 'common' + +CATEGORIES = [ + "### \u{1F6E1}\u{FE0F} Security", + "### \u{1F6A8} Breaking changes", + "### \u{2728} New features and improvements", + "### \u{1F41B} Bug fixes", + "### \u{1F527} Internal changes" +].freeze + +def parse_version_header(line) + return unless line =~ /^## \[(unreleased|v[\d.]+)\]/i + + Regexp.last_match(1).downcase == 'unreleased' ? 'unreleased' : Regexp.last_match(1) +end + +def process_changelog_line(line, result, current) + ver = parse_version_header(line) + return init_version_section(result, current, ver) if ver + return init_category(result, current, line) if line.start_with?('### ') && current[:ver] + + append_entry(result, current, line) if line.start_with?('- ') && current[:ver] && current[:cat] +end + +def init_version_section(result, current, ver) + current[:ver] = ver + result[:order] << ver + result[:sections][ver] = {} + current[:cat] = nil +end + +def init_category(result, current, line) + current[:cat] = line + result[:sections][current[:ver]][line] ||= [] +end + +def append_entry(result, current, line) + result[:sections][current[:ver]][current[:cat]] << line +end + +def warn_unknown_categories(sections) + unknown = sections.values.flat_map(&:keys).uniq - CATEGORIES + unknown.each { |c| warn "Warning: unknown category '#{c}' — entries may be dropped" } if unknown.any? +end + +# Parses Changelog.md into { "sections" => { version => { category => [entries] } }, "version_order" => [...] } +def parse_changelog(text) + result = { sections: {}, order: [] } + current = { ver: nil, cat: nil } + text.each_line { |line| process_changelog_line(line.rstrip, result, current) } + warn_unknown_categories(result[:sections]) + { 'sections' => result[:sections], 'version_order' => result[:order] } +end + +def emit_categories(out, entries_by_cat, skip_empty: false) + CATEGORIES.each do |cat| + entries = entries_by_cat[cat] || [] + next if skip_empty && entries.empty? + + out << cat + entries.each { |e| out << e } + out << '' + end +end + +def emit_old_sections_verbatim(out, raw_text, skip:) + emitting = false + raw_text.each_line do |line| + line = line.chomp + if line =~ /^## \[(unreleased|v[\d.]+)\]/i + ver = Regexp.last_match(1).downcase == 'unreleased' ? 'unreleased' : Regexp.last_match(1) + emitting = skip.exclude?(ver) + end + out << line if emitting + end +end + +def partition_entries(unreleased, pr_list) + matched = {} + unmatched = {} + unreleased.each do |cat, entries| + matched[cat], unmatched[cat] = entries.partition { |e| pr_list.any? { |n| e.match?(/\##{n}(?!\d)/) } } + end + [matched, unmatched] +end + +def build_changelog_sections(mode, version, matched, unmatched) + out = ['# Changelog', '', '## [unreleased]', ''] + emit_categories(out, mode == 'release' ? {} : unmatched, skip_empty: false) + out << "## [#{version}]" + out << '' + emit_categories(out, matched, skip_empty: true) + out +end + +def build_changelog(mode, version, pr_list) + release_raw = ReleaseHelpers.run('git', 'show', 'origin/release:Changelog.md') + master_raw = ReleaseHelpers.run('git', 'show', 'origin/master:Changelog.md') + unreleased = parse_changelog(master_raw)['sections']['unreleased'] || {} + matched, unmatched = partition_entries(unreleased, pr_list) + + out = build_changelog_sections(mode, version, matched, unmatched) + source_raw = mode == 'release' ? release_raw : master_raw + emit_old_sections_verbatim(out, source_raw, skip: ['unreleased', version]) + "#{out.join("\n")}\n" +end + +# --- Main --- + +if ARGV.empty? || ARGV.intersect?(['-h', '--help']) + warn <<~HELP + Usage: ruby release/changelog.rb --mode=MODE --version=VERSION --prs=N,N,N + + Modes: + release Build changelog for release branch (empty unreleased + new version) + master-sync Build changelog for master (move entries from unreleased to version) + + Options: + --mode=MODE release or master-sync (required) + --version=VERSION Target version, e.g. v2.9.6 (required) + --prs=N,N,N Comma-separated PR numbers (required) + HELP + exit(ARGV.empty? ? 1 : 0) +end + +args = {} +ARGV.each { |a| args[Regexp.last_match(1)] = Regexp.last_match(2) if a =~ /^--(\w[\w-]*)=(.+)$/ } + +mode = args['mode'] +version = args['version'] +pr_list = (args['prs'] || '').split(',').map(&:strip).reject(&:empty?) + +unless mode && version && pr_list.any? + warn 'Error: --mode, --version, and --prs are all required. Run with --help.' + exit 1 +end +unless %w[release master-sync].include?(mode) + warn "Error: --mode must be 'release' or 'master-sync', got '#{mode}'" + exit 1 +end +ReleaseHelpers.validate_version!(version) + +puts build_changelog(mode, version, pr_list) diff --git a/release/cherry-pick.sh b/release/cherry-pick.sh new file mode 100755 index 0000000000..f0c4f06443 --- /dev/null +++ b/release/cherry-pick.sh @@ -0,0 +1,209 @@ +#!/bin/bash +# Cherry-pick automation for MarkUs releases. +# +# Usage: +# release/cherry-pick.sh v2.9.6 Run recon and cherry-pick all PRs +# release/cherry-pick.sh v2.9.6 --resume Skip already-picked PRs, continue +# release/cherry-pick.sh --help +# +# Stops on code conflicts or contamination. Re-run with --resume after fixing. +# Outputs the comma-separated PR list at the end for use with changelog.rb. + +set -euo pipefail + +HELPERS="$(cd "$(dirname "$0")" && pwd)" + +VERSION="" +RESUME=false + +for arg in "$@"; do + case "$arg" in + --resume) RESUME=true ;; + --help|-h) + echo "Usage: release/cherry-pick.sh [--resume]" + echo "" + echo "Cherry-picks all milestone PRs onto the current branch." + echo "Stops on code conflicts or contamination." + echo "Re-run with --resume after fixing to continue." + exit 0 + ;; + v[0-9]*) VERSION="$arg" ;; + *) echo "Unknown argument: $arg"; exit 1 ;; + esac +done + +if [[ -z "$VERSION" ]]; then + echo "Usage: release/cherry-pick.sh [--resume]" + exit 1 +fi + +GREEN='\033[0;32m'; YELLOW='\033[0;33m'; RED='\033[0;31m' +CYAN='\033[0;36m'; BOLD='\033[1m'; RESET='\033[0m' +info() { echo -e "${CYAN}>>>${RESET} $1"; } +success() { echo -e "${GREEN} OK${RESET} $1"; } +warn() { echo -e "${YELLOW}WARN${RESET} $1"; } +fail() { echo -e "${RED}FAIL${RESET} $1"; } + +stop() { + [[ -n "$PICKED" ]] && echo " PRs picked so far: $PICKED" + exit 1 +} + +# Show hook diff and stop for user review +stop_for_hook_review() { + local hook_diff="$1" + local diff_lines + diff_lines=$(echo "$hook_diff" | wc -l | tr -d ' ') + echo "" + warn "Pre-commit hook modified files in #$PR ($diff_lines lines):" + echo "$hook_diff" | head -40 | sed 's/^/ /' + if [[ "$diff_lines" -gt 40 ]]; then + echo " ... ($((diff_lines - 40)) more lines, run: git diff)" + fi + echo "" + echo " If the above is only formatting, accept and continue." + echo "" + echo " To accept: git add -u && git cherry-pick --continue" + echo " release/cherry-pick.sh $VERSION --resume" + echo " To reject: git checkout -- . && git cherry-pick --abort" + echo " release/cherry-pick.sh $VERSION --resume" + echo "" + stop +} + +# --- Recon --- + +info "Running recon for $VERSION..." +RECON_JSON=$(ruby "$HELPERS/recon.rb" "$VERSION") +ORDER=$(echo "$RECON_JSON" | ruby "$HELPERS/recon-format.rb" --order) +TOTAL=$(echo "$ORDER" | wc -l | tr -d ' ') + +if [[ "$TOTAL" -eq 0 ]]; then + warn "No PRs to cherry-pick." + exit 0 +fi + +echo -e "\n${BOLD}Cherry-pick plan ($TOTAL PRs):${RESET}" +echo "$RECON_JSON" | ruby "$HELPERS/recon-format.rb" --plan +echo "" + +# --- Cherry-pick loop --- + +PICKED="" +SKIPPED="" +NUM=0 +BRANCH_LOG="$(git log --oneline HEAD --not origin/release)" + +while IFS= read -r entry; do + PR="${entry%%:*}" + SHA="${entry##*:}" + NUM=$((NUM + 1)) + + # Resume: skip if a cherry-picked commit for this PR is already on the branch + if $RESUME && [[ "$BRANCH_LOG" == *"(#$PR)"* ]]; then + success "[$NUM/$TOTAL] #$PR — already picked, skipping" + PICKED="${PICKED:+$PICKED,}$PR" + continue + fi + + info "[$NUM/$TOTAL] Cherry-picking #$PR (${SHA:0:10})..." + + CHERRY_OUT=$(git cherry-pick -m1 "$SHA" 2>&1) || { + # Empty commit — PR already on release via different path + if [[ "$CHERRY_OUT" =~ (empty|nothing.*to\ commit) ]]; then + git cherry-pick --skip 2>/dev/null + success "#$PR — already on release, skipped" + SKIPPED="${SKIPPED:+$SKIPPED,}$PR" + continue + fi + + CONFLICTED_FILES=$(git diff --name-only --diff-filter=U 2>/dev/null) + + # No merge conflicts — likely a pre-commit hook failure on a clean pick + if [[ -z "$CONFLICTED_FILES" ]]; then + HOOK_DIFF=$(git diff 2>/dev/null) + [[ -n "$HOOK_DIFF" ]] && stop_for_hook_review "$HOOK_DIFF" + # No conflicts, no hook changes — truly empty commit + git cherry-pick --skip 2>/dev/null + success "#$PR — already on release, skipped" + SKIPPED="${SKIPPED:+$SKIPPED,}$PR" + continue + fi + + # Check if all conflicts are auto-resolvable (Changelog.md, Gemfile.lock) + HAS_CODE_CONFLICT=false + while IFS= read -r f; do + case "$f" in + Changelog.md|Gemfile.lock) ;; + *) HAS_CODE_CONFLICT=true; break ;; + esac + done <<< "$CONFLICTED_FILES" + + if $HAS_CODE_CONFLICT; then + echo "" + fail "Code conflict in #$PR" + echo "$CONFLICTED_FILES" | sed 's/^/ /' + echo "" + echo " To fix:" + echo " 1. Compare: gh pr diff $PR" + echo " 2. Resolve conflicts, then: git add && git cherry-pick --continue" + echo " 3. Re-run: release/cherry-pick.sh $VERSION --resume" + echo "" + echo " To skip this PR:" + echo " git cherry-pick --abort" + echo " release/cherry-pick.sh $VERSION --resume" + echo "" + stop + fi + + # Auto-resolve: Changelog with ours (rebuilt later), Gemfile.lock by accepting incoming version + while IFS= read -r f; do + case "$f" in + Changelog.md) git checkout --ours "$f" && git add "$f" ;; + Gemfile.lock) + # Resolve conflict markers: drop ours, keep theirs (incoming version) + awk '/^<<<<<<>>>>>>/{next} !skip{print}' "$f" > "$f.tmp" \ + && mv "$f.tmp" "$f" && git add "$f" ;; + esac + done <<< "$CONFLICTED_FILES" + if ! GIT_EDITOR=true git cherry-pick --continue 2>/dev/null; then + # Commit failed — check if a pre-commit hook modified files + HOOK_DIFF=$(git diff 2>/dev/null) + [[ -n "$HOOK_DIFF" ]] && stop_for_hook_review "$HOOK_DIFF" + # No hook changes — commit became empty after resolution (PR already applied) + git checkout -- . 2>/dev/null + git cherry-pick --skip 2>/dev/null + success "#$PR — already on release, skipped" + SKIPPED="${SKIPPED:+$SKIPPED,}$PR" + continue + fi + success "#$PR — auto-resolved ($(echo "$CONFLICTED_FILES" | paste -sd, -))" + BRANCH_LOG="$(git log --oneline HEAD --not origin/release)" + } + + if ! VERIFY_OUT=$(ruby "$HELPERS/verify.rb" "$PR" 2>/dev/null); then + echo "$VERIFY_OUT" + echo "" + warn "Contamination detected in #$PR" + echo "" + echo " To undo: git reset --hard HEAD~1" + echo " To accept: release/cherry-pick.sh $VERSION --resume" + echo "" + stop + fi + + success "#$PR — verified clean" + PICKED="${PICKED:+$PICKED,}$PR" + BRANCH_LOG="$(git log --oneline HEAD --not origin/release)" +done <<< "$ORDER" + +# --- Summary --- + +echo "" +echo -e "${BOLD}Cherry-pick complete${RESET}" +echo " Picked: $PICKED" +[[ -n "$SKIPPED" ]] && echo " Skipped: $SKIPPED" +echo "" +echo "Next steps:" +echo " ruby release/changelog.rb --mode=release --version=$VERSION --prs=$PICKED > Changelog.md" +echo " bash release/validate_changelog.sh $VERSION" diff --git a/release/common.rb b/release/common.rb new file mode 100644 index 0000000000..8410398f74 --- /dev/null +++ b/release/common.rb @@ -0,0 +1,33 @@ +# frozen_string_literal: true + +require 'open3' + +# Shared helper methods for release scripts. +module ReleaseHelpers + REPO = 'MarkUsProject/Markus' + + class CommandError < RuntimeError; end + + def self.run(*cmd) + stdout, stderr, status = Open3.capture3(*cmd) + return stdout if status.success? + + raise CommandError, "Command failed: #{cmd.inspect}\n#{stderr}".strip + end + + def self.run_stripped(*cmd) + run(*cmd).strip + end + + def self.command_succeeds?(*cmd) + _, _, status = Open3.capture3(*cmd) + status.success? + end + + def self.validate_version!(version) + return if version.match?(/\Av\d+\.\d+\.\d+\z/) + + warn "Error: version must match vX.Y.Z format, got '#{version}'" + exit 1 + end +end diff --git a/release/recon-format.rb b/release/recon-format.rb new file mode 100755 index 0000000000..5dd7ef49b4 --- /dev/null +++ b/release/recon-format.rb @@ -0,0 +1,64 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# Recon Format — Extracts fields from recon JSON for bash consumption. +# +# Usage (reads JSON from stdin): +# ruby recon-format.rb --summary # PR counts and dependency info +# ruby recon-format.rb --plan # Cherry-pick plan table +# ruby recon-format.rb --order # number:ref pairs (one per line) +# ruby recon-format.rb --pr-list # Comma-separated PR numbers +# ruby recon-format.rb --pr-body # Markdown PR body for gh pr create +# ruby recon-format.rb --release-notes # Bulleted PR list for gh release create + +require 'json' + +data = JSON.parse($stdin.read) +prs = data['milestone_prs'].index_by { |p| p['number'] } +order = data['proposed_cherry_pick_order'] +skipped = data['skipped'] + +case ARGV[0] +when '--summary' + puts "Milestone PRs: #{data['milestone_prs'].length}" + puts "To cherry-pick: #{order.length}" + puts "Already on release: #{skipped.length}" + puts '' + puts data['dependency_changes']['summary'] + puts data['dependency_changes']['settings'] + +when '--plan' + order.each do |item| + pr = prs[item['number']] + deps = (pr['dependencies'] || []).empty? ? '--' : pr['dependencies'].map { |d| "##{d}" }.join(', ') + printf " %2d. #%-5d %-50s deps: %<deps>s\n", + order: item['order'], number: item['number'], title: pr['title'][0..49], deps: deps + end + +when '--skipped' + skipped.each { |s| puts " ##{s['number']} — #{s['reason']}" } + +when '--order' + order.each { |item| puts "#{item['number']}:#{item['ref']}" } + +when '--pr-list' + puts order.pluck('number').join(',') + +when '--pr-body' + puts "## Release #{data['version']}" + puts '' + puts '### Cherry-picked PRs' + order.each { |item| puts "- ##{item['number']} — #{prs[item['number']]['title']}" } + puts '' + puts '### Notes' + puts "- Dependencies: #{data['dependency_changes']['summary']}" + puts "- Settings: #{data['dependency_changes']['settings']}" + +when '--release-notes' + order.each { |item| puts "- ##{item['number']} — #{prs[item['number']]['title']}" } + +else + warn 'Usage: echo JSON | ruby recon-format.rb COMMAND' + warn 'Commands: --summary --plan --skipped --order --pr-list --pr-body --release-notes' + exit 1 +end diff --git a/release/recon.rb b/release/recon.rb new file mode 100755 index 0000000000..c618872c0b --- /dev/null +++ b/release/recon.rb @@ -0,0 +1,161 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# Release Recon — Discovers milestone PRs and builds a cherry-pick plan. +# +# Usage: +# ruby release/recon.rb <version> +# ruby release/recon.rb --help +# +# Output: JSON to stdout with milestone PRs, cherry-pick order, and skipped items. + +require 'json' +require 'time' +require_relative 'common' + +def fetch_pr_files(number) + ReleaseHelpers.run_stripped( + 'gh', 'pr', 'diff', number.to_s, + '--repo', ReleaseHelpers::REPO, '--name-only' + ).split("\n") +end + +def ancestor_of_release?(sha) + ReleaseHelpers.command_succeeds?( + 'git', 'merge-base', '--is-ancestor', sha, 'origin/release' + ) +end + +def enrich_pr(pull) + pull['files_changed'] = fetch_pr_files(pull['number']) + sha = pull.dig('mergeCommit', 'oid') + pull['merge_commit'] = sha + pull['already_in_release'] = sha.present? && ancestor_of_release?(sha) +end + +def fetch_milestone_prs(version) + raw = ReleaseHelpers.run_stripped( + 'gh', 'pr', 'list', '--repo', ReleaseHelpers::REPO, + '--state', 'merged', '--search', "milestone:#{version}", + '--json', 'number,title,mergedAt,mergeCommit,author', + '--jq', 'sort_by(.mergedAt)', '--limit', '100' + ) + prs = JSON.parse(raw) + warn "Warning: No merged PRs found in milestone #{version}" if prs.empty? + prs.each { |pr| enrich_pr(pr) } +end + +# Heuristic: lines containing "(#" are PR merge commits; others are direct pushes +def find_non_pr_commits + ReleaseHelpers.run_stripped('git', 'log', 'origin/release..origin/master', '--oneline') + .split("\n") + .reject { |l| l.include?('(#') || l.strip.empty? } + .map do |line| + hash, *msg = line.split + { 'hash' => hash, 'message' => msg.join(' ') } + end +end + +def earlier_prs(prs, current, timestamps) + prs.select do |o| + o['number'] != current['number'] && + timestamps[o['number']] < timestamps[current['number']] + end +end + +def detect_dependencies(pending_prs) + timestamps = pending_prs.to_h { |pr| [pr['number'], Time.zone.parse(pr['mergedAt'])] } + pending_prs.each do |pr| + earlier = earlier_prs(pending_prs, pr, timestamps) + overlapping = earlier.select { |o| pr['files_changed'].intersect?(o['files_changed']) } + pr['dependencies'] = overlapping.map { |o| o['number'] } + end +end + +def find_ready(remaining, by_num, placed) + ready = remaining.select { |n| (by_num[n]['dependencies'] - placed).empty? } + ready = [remaining.min_by { |n| by_num[n]['mergedAt'] }] if ready.empty? + ready.sort_by { |n| by_num[n]['mergedAt'] } +end + +def build_order_entry(position, number, pull) + { 'order' => position, 'ref' => pull['merge_commit'], + 'type' => 'milestone_pr', 'number' => number } +end + +def build_order(remaining, by_num) + placed = [] + order = [] + while remaining.any? + ready = find_ready(remaining, by_num, placed) + ready.each { |n| order << build_order_entry(order.length + 1, n, by_num[n]) } + placed.concat(ready) + remaining -= ready + end + order +end + +def topo_sort(pending_prs) + by_num = pending_prs.index_by { |pr| pr['number'] } + remaining = pending_prs.pluck('number') + build_order(remaining, by_num) +end + +# Resolves file-overlap dependencies and returns topologically sorted cherry-pick order. +def build_cherry_pick_order(pending_prs) + detect_dependencies(pending_prs) + topo_sort(pending_prs) +end + +# --- Main --- + +version = ARGV[0] + +if version.nil? || ['-h', '--help'].include?(version) + warn <<~HELP + Usage: ruby release/recon.rb <version> + e.g. ruby release/recon.rb v2.9.6 + + Queries the GitHub milestone for merged PRs, checks ancestry, + resolves cherry-pick order, and outputs a JSON plan to stdout. + HELP + exit(version.nil? ? 1 : 0) +end + +prs = fetch_milestone_prs(version) +pending = prs.reject { |pr| pr['already_in_release'] } +order = build_cherry_pick_order(pending) + +dep_diff = ReleaseHelpers.run_stripped( + 'git', 'diff', 'origin/release..origin/master', '--', + 'Gemfile', 'Gemfile.lock', 'package.json', 'package-lock.json' +) +settings_diff = ReleaseHelpers.run_stripped( + 'git', 'diff', 'origin/release..origin/master', '--', + 'markus.control', 'config/settings.yml' +) + +dep_line_count = dep_diff.lines.count { |l| l.start_with?('+', '-') } + +result = { + 'version' => version, + 'timestamp' => Time.now.utc.iso8601, + 'release_branch_tip' => ReleaseHelpers.run_stripped('git', 'log', 'origin/release', '--oneline', '-1'), + 'milestone_prs' => prs.map do |pr| + { 'number' => pr['number'], 'title' => pr['title'], 'author' => pr.dig('author', 'login'), + 'merged_at' => pr['mergedAt'], 'merge_commit' => pr['merge_commit'], + 'files_changed' => pr['files_changed'], 'already_in_release' => pr['already_in_release'], + 'dependencies' => pr['dependencies'] || [] } + end, + 'non_pr_commits' => find_non_pr_commits, + 'proposed_cherry_pick_order' => order, + 'skipped' => prs.select { |pr| pr['already_in_release'] }.map do |pr| + { 'ref' => pr['merge_commit'], 'number' => pr['number'], 'reason' => 'Already ancestor of release branch' } + end, + 'dependency_changes' => { + 'summary' => dep_diff.empty? ? 'No dependency changes' : "Dependency files changed (#{dep_line_count} lines)", + 'settings' => settings_diff.empty? ? 'No settings changes' : 'Settings files changed — notify sysadmin' + } +} + +puts JSON.pretty_generate(result) diff --git a/release/test/test_helpers.rb b/release/test/test_helpers.rb new file mode 100755 index 0000000000..f4fcb26b19 --- /dev/null +++ b/release/test/test_helpers.rb @@ -0,0 +1,361 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# Unit tests for release helper pure functions. +# Run: ruby release/test/test_helpers.rb + +require 'json' +require 'set' +require 'open3' + +$LOAD_PATH.unshift File.expand_path('..', __dir__) +require 'common' + +# Simple test harness state. +module TestState + @pass = 0 + @fail_count = 0 + @errors = [] + + class << self + attr_accessor :pass, :fail_count, :errors + end +end + +def assert(description, condition) + if condition + $stdout.puts " \e[32mPASS\e[0m #{description}" + TestState.pass += 1 + else + $stdout.puts " \e[31mFAIL\e[0m #{description}" + TestState.errors << description + TestState.fail_count += 1 + end +end + +def assert_eq(description, actual, expected) + if actual == expected + $stdout.puts " \e[32mPASS\e[0m #{description}" + TestState.pass += 1 + else + $stdout.puts " \e[31mFAIL\e[0m #{description}" + $stdout.puts " expected: #{expected.inspect}" + $stdout.puts " actual: #{actual.inspect}" + TestState.errors << description + TestState.fail_count += 1 + end +end + +# ============================================================================ +# Pure functions copied from scripts (avoids loading scripts with side effects) +# ============================================================================ + +def parse_version_header(line) + return unless line =~ /^## \[(unreleased|v[\d.]+)\]/i + + Regexp.last_match(1).downcase == 'unreleased' ? 'unreleased' : Regexp.last_match(1) +end + +def process_line(line, result, current) + ver = parse_version_header(line) + return init_ver(result, current, ver) if ver + return init_cat(result, current, line) if line.start_with?('### ') && current[:ver] + + add_entry(result, current, line) if line.start_with?('- ') && current[:ver] && current[:cat] +end + +def init_ver(result, current, ver) + current[:ver] = ver + result[:order] << ver + result[:sections][ver] = {} + current[:cat] = nil +end + +def init_cat(result, current, line) + current[:cat] = line + result[:sections][current[:ver]][line] ||= [] +end + +def add_entry(result, current, line) + result[:sections][current[:ver]][current[:cat]] << line +end + +def parse_changelog(text) + result = { sections: {}, order: [] } + current = { ver: nil, cat: nil } + text.each_line { |line| process_line(line.rstrip, result, current) } + { 'sections' => result[:sections], 'version_order' => result[:order] } +end + +def entry_matches_prs?(entry, pr_numbers) + pr_numbers.any? { |num| entry.match?(/\##{num}(?!\d)/) } +end + +def find_ready(remaining, by_num, placed) + ready = remaining.select { |n| (by_num[n]['dependencies'] - placed).empty? } + ready = [remaining.min_by { |n| by_num[n]['mergedAt'] }] if ready.empty? + ready.sort_by { |n| by_num[n]['mergedAt'] } +end + +def build_order(remaining, by_num) + placed = [] + order = [] + while remaining.any? + ready = find_ready(remaining, by_num, placed) + ready.each { |n| order << { 'order' => order.length + 1, 'number' => n, 'ref' => by_num[n]['merge_commit'] } } + placed.concat(ready) + remaining -= ready + end + order +end + +def topological_sort(pending_prs) + by_num = pending_prs.index_by { |pr| pr['number'] } + remaining = pending_prs.pluck('number') + build_order(remaining, by_num) +end + +def extract_file_diff(full_diff, filename) + full_diff.lines + .slice_before { |l| l.start_with?('diff --git') } + .find { |chunk| chunk.first.include?("b/#{filename}") } + &.join || '' +end + +def change_lines(diff_text) + diff_text.lines + .select { |l| l.start_with?('+', '-') } + .reject { |l| l.start_with?('+++', '---') } +end + +def format_cmd(json, flag) + script = File.expand_path('../recon-format.rb', __dir__) + stdout, _, status = Open3.capture3('ruby', script, flag, stdin_data: json) + [stdout.strip, status.success?] +end + +# ============================================================================ +# Tests +# ============================================================================ + +puts "\n\e[1m--- parse_changelog ---\e[0m" + +changelog = <<~MD + # Changelog + + ## [unreleased] + + ### ✨ New features and improvements + - Feature A (#100) + - Feature B (#101) + + ### 🐛 Bug fixes + - Fix C (#102) + + ## [v2.9.5] + + ### 🛡️ Security + - Security fix (#90) + + ### ✨ New features and improvements + - Old feature (#80) + + ## [v2.9.4] + + ### 🐛 Bug fixes + - Old bug fix (#70) +MD + +parsed = parse_changelog(changelog) +assert_eq 'version_order count', parsed['version_order'].length, 3 +assert_eq 'version_order values', parsed['version_order'], %w[unreleased v2.9.5 v2.9.4] +assert_eq 'unreleased features', parsed['sections']['unreleased']['### ✨ New features and improvements'].length, 2 +assert_eq 'unreleased bug fixes', parsed['sections']['unreleased']['### 🐛 Bug fixes'].length, 1 +assert_eq 'v2.9.5 security', parsed['sections']['v2.9.5']['### 🛡️ Security'].length, 1 +assert_eq 'v2.9.4 bug fixes', parsed['sections']['v2.9.4']['### 🐛 Bug fixes'].length, 1 + +empty = parse_changelog("# Changelog\n") +assert_eq 'empty changelog', empty['version_order'], [] + +# --- entry_matches_prs? --- + +puts "\n\e[1m--- entry_matches_prs? ---\e[0m" + +assert 'matches single PR', entry_matches_prs?('- Feature A (#100)', ['100']) +assert 'matches in list', entry_matches_prs?('- Feature A (#100)', %w[99 100 101]) +assert 'no match for missing PR', !entry_matches_prs?('- Feature A (#100)', ['200']) +assert 'no false-match on substring', !entry_matches_prs?('- Feature (#1001)', ['100']) +assert 'matches multi-PR entry', entry_matches_prs?('- Fix (#100, #200)', ['200']) +assert 'matches PR at end of line', entry_matches_prs?('- Fix #100', ['100']) + +# --- partition --- + +puts "\n\e[1m--- partition_entries ---\e[0m" + +entries = { + 'features' => ['- A (#100)', '- B (#101)', '- C (#200)'], + 'fixes' => ['- D (#100)'] +} +pr_list = %w[100 200] + +matched = {} +unmatched = {} +entries.each do |cat, ents| + matched[cat], unmatched[cat] = ents.partition { |e| entry_matches_prs?(e, pr_list) } +end + +assert_eq 'matched features', matched['features'].length, 2 +assert_eq 'unmatched features', unmatched['features'], ['- B (#101)'] +assert_eq 'matched fixes', matched['fixes'].length, 1 + +# --- topological_sort --- + +puts "\n\e[1m--- topological_sort ---\e[0m" + +linear = [ + { 'number' => 1, 'mergedAt' => '2026-01-01', 'merge_commit' => 'a', 'dependencies' => [] }, + { 'number' => 2, 'mergedAt' => '2026-01-02', 'merge_commit' => 'b', 'dependencies' => [1] }, + { 'number' => 3, 'mergedAt' => '2026-01-03', 'merge_commit' => 'c', 'dependencies' => [2] } +] +assert_eq 'linear: 1,2,3', topological_sort(linear).pluck('number'), [1, 2, 3] + +diamond = [ + { 'number' => 1, 'mergedAt' => '2026-01-01', 'merge_commit' => 'a', 'dependencies' => [] }, + { 'number' => 2, 'mergedAt' => '2026-01-02', 'merge_commit' => 'b', 'dependencies' => [1] }, + { 'number' => 3, 'mergedAt' => '2026-01-03', 'merge_commit' => 'c', 'dependencies' => [1] }, + { 'number' => 4, 'mergedAt' => '2026-01-04', 'merge_commit' => 'd', 'dependencies' => [2, 3] } +] +nums = topological_sort(diamond).pluck('number') +assert_eq 'diamond: first is 1', nums[0], 1 +assert_eq 'diamond: last is 4', nums[3], 4 +assert 'diamond: 2 before 4', nums.index(2) < nums.index(4) +assert 'diamond: 3 before 4', nums.index(3) < nums.index(4) + +independent = [ + { 'number' => 3, 'mergedAt' => '2026-01-03', 'merge_commit' => 'c', 'dependencies' => [] }, + { 'number' => 1, 'mergedAt' => '2026-01-01', 'merge_commit' => 'a', 'dependencies' => [] }, + { 'number' => 2, 'mergedAt' => '2026-01-02', 'merge_commit' => 'b', 'dependencies' => [] } +] +assert_eq 'independent: date order', topological_sort(independent).pluck('number'), [1, 2, 3] + +cycle = [ + { 'number' => 1, 'mergedAt' => '2026-01-01', 'merge_commit' => 'a', 'dependencies' => [2] }, + { 'number' => 2, 'mergedAt' => '2026-01-02', 'merge_commit' => 'b', 'dependencies' => [1] } +] +result = topological_sort(cycle) +assert_eq 'cycle: 2 items', result.length, 2 +assert_eq 'cycle: chronological fallback', result.pluck('number'), [1, 2] + +single = [{ 'number' => 1, 'mergedAt' => '2026-01-01', 'merge_commit' => 'a', 'dependencies' => [] }] +assert_eq 'single PR', topological_sort(single).pluck('number'), [1] + +assert_eq 'empty input', topological_sort([]), [] + +# --- extract_file_diff --- + +puts "\n\e[1m--- extract_file_diff ---\e[0m" + +diff = <<~DIFF + diff --git a/file1.rb b/file1.rb + --- a/file1.rb + +++ b/file1.rb + @@ -1,3 +1,4 @@ + line1 + +added + line2 + diff --git a/file2.rb b/file2.rb + --- a/file2.rb + +++ b/file2.rb + @@ -1,2 +1,2 @@ + -old + +new +DIFF + +assert 'file1 extraction', extract_file_diff(diff, 'file1.rb').include?('+added') +assert 'file1 excludes file2', extract_file_diff(diff, 'file1.rb').exclude?('+new') +assert 'file2 extraction', extract_file_diff(diff, 'file2.rb').include?('+new') +assert_eq 'missing file', extract_file_diff(diff, 'nope.rb'), '' + +# --- change_lines --- + +puts "\n\e[1m--- change_lines ---\e[0m" + +cl = change_lines("--- a/f.rb\n+++ b/f.rb\n context\n-removed\n+added\n context\n") +assert_eq 'change_lines count', cl.length, 2 +assert('includes -removed', cl.any? { |l| l.strip == '-removed' }) +assert('includes +added', cl.any? { |l| l.strip == '+added' }) +assert('excludes ---', cl.none? { |l| l.start_with?('---') }) + +# --- recon-format.rb --- + +puts "\n\e[1m--- recon-format.rb ---\e[0m" + +sample = JSON.generate({ + 'version' => 'v2.9.6', + 'milestone_prs' => [ + { 'number' => 100, 'title' => 'Feature A', 'dependencies' => [] }, + { 'number' => 200, 'title' => 'Feature B', 'dependencies' => [100] } + ], + 'proposed_cherry_pick_order' => [ + { 'order' => 1, 'ref' => 'aaa', 'type' => 'milestone_pr', 'number' => 100 }, + { 'order' => 2, 'ref' => 'bbb', 'type' => 'milestone_pr', 'number' => 200 } + ], + 'skipped' => [{ 'number' => 50, 'reason' => 'Already ancestor' }], + 'dependency_changes' => { 'summary' => 'No dependency changes', + 'settings' => 'No settings changes' } +}) + +out, ok = format_cmd(sample, '--pr-list') +assert('pr-list ok', ok) +assert_eq('pr-list output', out, '100,200') + +out, ok = format_cmd(sample, '--order') +assert('order ok', ok) +assert_eq('order lines', out.split("\n"), ['100:aaa', '200:bbb']) + +out, ok = format_cmd(sample, '--summary') +assert('summary ok', ok) +assert('summary has count', out.include?('2')) + +out, ok = format_cmd(sample, '--release-notes') +assert('release-notes ok', ok) +assert('release-notes has #100', out.include?('#100')) + +out, ok = format_cmd(sample, '--skipped') +assert('skipped ok', ok) +assert('skipped has #50', out.include?('#50')) + +out, ok = format_cmd(sample, '--pr-body') +assert('pr-body ok', ok) +assert('pr-body has header', out.include?('Release v2.9.6')) + +_, ok = format_cmd(sample, '--bogus') +assert('invalid flag rejects', !ok) + +# --- validate_version! --- + +puts "\n\e[1m--- validate_version! ---\e[0m" + +v = /\Av\d+\.\d+\.\d+\z/ +assert 'v2.9.6 valid', 'v2.9.6'.match?(v) +assert 'v0.0.1 valid', 'v0.0.1'.match?(v) +assert 'no v prefix invalid', !'2.9.6'.match?(v) +assert 'two parts invalid', !'v2.9'.match?(v) +assert 'verbose invalid', !'verbose'.match?(v) +assert 'rc suffix invalid', !'v2.9.6-rc1'.match?(v) + +# ============================================================================ +# Summary +# ============================================================================ + +total = TestState.pass + TestState.fail_count +puts "\n#{'=' * 60}" +if TestState.fail_count.zero? + puts "\e[32m#{total} tests passed\e[0m" +else + puts "\e[31m#{TestState.fail_count} of #{total} tests failed:\e[0m" + TestState.errors.each { |e| puts " - #{e}" } +end +puts '=' * 60 + +exit(TestState.fail_count.zero? ? 0 : 1) diff --git a/release/validate_changelog.sh b/release/validate_changelog.sh new file mode 100755 index 0000000000..e4c3811af6 --- /dev/null +++ b/release/validate_changelog.sh @@ -0,0 +1,229 @@ +#!/bin/bash +# ============================================================================= +# Changelog Validator for MarkUs Releases +# ============================================================================= +# +# Validates Changelog.md on a release/version branch to catch common issues +# introduced by cherry-pick conflict resolution. +# +# Usage: +# ./validate_changelog.sh <version> [changelog_path] +# +# Examples: +# ./validate_changelog.sh v2.9.3 +# ./validate_changelog.sh v2.9.3 /path/to/Changelog.md +# +# What it checks: +# 1. No git conflict markers left in file +# 2. [unreleased] section exists and is empty +# 3. Target version section exists and has entries +# 4. Version sections appear in correct descending order +# 5. No PR numbers duplicated across different version sections +# 6. Sections below the release version match the original release branch +# +# ============================================================================= + +set -uo pipefail + +VERSION="${1:-}" +CHANGELOG="${2:-Changelog.md}" +ERRORS=0 +WARNINGS=0 + +pass() { echo " PASS $1"; } +fail() { echo " FAIL $1"; ERRORS=$((ERRORS + 1)); } +warn() { echo " WARN $1"; WARNINGS=$((WARNINGS + 1)); } +info() { echo " INFO $1"; } +divider() { echo "------------------------------------------------------------------------"; } + +# Extract bullet entries from a section. $1 = section name (e.g., "unreleased", "$VERSION") +section_entries() { + awk -v section="$1" ' + BEGIN { IGNORECASE = 1 } + /^## \[/ { + if (found) exit + header = $0; gsub(/^## \[/, "", header); gsub(/\].*$/, "", header) + if (tolower(header) == tolower(section)) found = 1 + next + } + found && /^- / { print } + ' "$CHANGELOG" +} + +# Extract version strings from ## [vX.Y.Z] headers +version_headers() { + grep '^## \[v' "$CHANGELOG" | sed 's/## \[\(.*\)\]/\1/' +} + +if [ -z "$VERSION" ]; then + echo "Usage: $0 <version> [changelog_path]" + echo " e.g. $0 v2.9.3" + exit 2 +fi + +if [ ! -f "$CHANGELOG" ]; then + echo "Error: $CHANGELOG not found. Run from the repo root or pass the path." + exit 2 +fi + +echo "" +echo "Changelog Validation: $VERSION" +echo "File: $CHANGELOG" +divider + +# ============================================================================= +# CHECK 1: No conflict markers +# ============================================================================= + +echo "" +echo "[1/6] Conflict markers" + +MARKERS=$(grep -n '^\(<<<<<<<\|=======\|>>>>>>>\)' "$CHANGELOG" 2>/dev/null || true) +if [ -z "$MARKERS" ]; then + pass "No conflict markers" +else + fail "Conflict markers found in file:" + echo "$MARKERS" | head -10 | sed 's/^/ /' +fi + +# ============================================================================= +# CHECK 2: [unreleased] section exists and is empty +# ============================================================================= + +echo "" +echo "[2/6] Unreleased section" + +if ! head -5 "$CHANGELOG" | grep -qi '## \[unreleased\]'; then + fail "[unreleased] section missing or not at top of file" +else + pass "[unreleased] section present at top of file" +fi + +UNRELEASED_ENTRIES=$(section_entries "unreleased" | wc -l | tr -d ' ') + +if [ "$UNRELEASED_ENTRIES" -eq 0 ]; then + pass "[unreleased] section is empty" +else + warn "[unreleased] section has $UNRELEASED_ENTRIES entries (should be empty on release branch)" + section_entries "unreleased" | head -5 | sed 's/^/ /' +fi + +# ============================================================================= +# CHECK 3: Target version section exists and has entries +# ============================================================================= + +echo "" +echo "[3/6] Version section [$VERSION]" + +if ! grep -q "^## \[$VERSION\]" "$CHANGELOG"; then + fail "[$VERSION] section missing" +else + pass "[$VERSION] section found" +fi + +VERSION_ENTRIES=$(section_entries "$VERSION" | wc -l | tr -d ' ') + +if [ "$VERSION_ENTRIES" -eq 0 ]; then + warn "[$VERSION] section has no entries" +else + pass "[$VERSION] section has $VERSION_ENTRIES entries" +fi + +# ============================================================================= +# CHECK 4: Version sections are in descending order +# ============================================================================= + +echo "" +echo "[4/6] Section ordering" + +VERSIONS_IN_FILE=$(version_headers | head -10) +SORTED_VERSIONS=$(echo "$VERSIONS_IN_FILE" | sort -t. -k1,1r -k2,2rn -k3,3rn) + +if [ "$VERSIONS_IN_FILE" != "$SORTED_VERSIONS" ]; then + warn "Version sections may be out of order" + info "Found order:" + echo "$VERSIONS_IN_FILE" | head -5 | sed 's/^/ /' +else + pass "Version sections in correct descending order" +fi + +FIRST_VERSION=$(version_headers | head -1) +if [ "$FIRST_VERSION" != "$VERSION" ]; then + warn "Expected [$VERSION] as first version, found [$FIRST_VERSION]" +else + pass "[$VERSION] is the first version after [unreleased]" +fi + +# ============================================================================= +# CHECK 5: No PR numbers duplicated across version sections +# ============================================================================= + +echo "" +echo "[5/6] Duplicate PR references" + +TARGET_PRS=$(section_entries "$VERSION" | grep -o '#[0-9]\+' | sort -u) +OTHER_PRS=$(awk "found && /^## \[v/{p=1} p{print} /^## \[$VERSION\]/{found=1}" "$CHANGELOG" \ + | grep -o '#[0-9]\+' | sort -u) + +DUPLICATES="" +for pr in $TARGET_PRS; do + if echo "$OTHER_PRS" | grep -q "^${pr}$"; then + DUPLICATES="$DUPLICATES $pr" + fi +done + +if [ -n "$DUPLICATES" ]; then + warn "PRs appearing in both [$VERSION] and older sections:$DUPLICATES" + info "This may indicate cherry-pick entries leaked into wrong sections" + for pr in $DUPLICATES; do + info " $pr appears on lines: $(grep -n "$pr" "$CHANGELOG" | cut -d: -f1 | tr '\n' ' ')" + done +else + pass "No PR numbers duplicated between [$VERSION] and older sections" +fi + +# ============================================================================= +# CHECK 6: Older sections unchanged from release branch +# ============================================================================= + +echo "" +echo "[6/6] Integrity of older sections" + +PREV_VERSION=$(version_headers | awk "found{print; exit} /^$VERSION\$/{found=1}") + +if [ -n "$PREV_VERSION" ] && git show origin/release:Changelog.md &>/dev/null; then + # Extract everything from the previous version onwards in both files + CURRENT_TAIL=$(awk "/^## \[$PREV_VERSION\]/{found=1} found{print}" "$CHANGELOG") + ORIGINAL_TAIL=$(git show origin/release:Changelog.md | awk "/^## \[$PREV_VERSION\]/{found=1} found{print}") + + if [ "$CURRENT_TAIL" != "$ORIGINAL_TAIL" ]; then + fail "Sections from [$PREV_VERSION] onwards differ from origin/release" + info "This suggests cherry-pick entries leaked into older sections" + info "Compare with: git show origin/release:Changelog.md" + else + pass "Sections from [$PREV_VERSION] onwards match origin/release" + fi +else + info "Skipped (could not determine previous version or origin/release not available)" +fi + +# ============================================================================= +# Summary +# ============================================================================= + +echo "" +divider +if [ $ERRORS -eq 0 ] && [ $WARNINGS -eq 0 ]; then + echo "RESULT: All checks passed" +elif [ $ERRORS -eq 0 ]; then + echo "RESULT: Passed with $WARNINGS warning(s)" +else + echo "RESULT: FAILED with $ERRORS error(s) and $WARNINGS warning(s)" + echo "" + echo "To debug:" + echo " View master reference: git show master:Changelog.md | head -60" + echo " View release reference: git show origin/release:Changelog.md | head -30" + echo " Search for markers: grep -n '<<<<<<' Changelog.md" +fi +echo "" +exit $ERRORS diff --git a/release/validate_changelog_master.sh b/release/validate_changelog_master.sh new file mode 100755 index 0000000000..bd71579477 --- /dev/null +++ b/release/validate_changelog_master.sh @@ -0,0 +1,281 @@ +#!/bin/bash +# ============================================================================= +# Changelog Validator — Master Branch Sync +# ============================================================================= +# +# Validates that the Changelog.md edit on master correctly moves released +# entries from [unreleased] into a new version section, without touching +# anything else. +# +# This script compares the working copy of Changelog.md against origin/master +# (the pre-edit state). It must be run from the repo root on the changelog +# branch (v2.X.Y-changelog), BEFORE pushing. +# +# Usage: +# ./validate_changelog_master.sh <version> [changelog_path] +# +# Examples: +# ./validate_changelog_master.sh v2.9.4 +# ./validate_changelog_master.sh v2.9.4 /path/to/Changelog.md +# +# What it checks: +# 1. [unreleased] section still has entries (not wiped out) +# 2. New [v2.X.Y] section exists between [unreleased] and previous version +# 3. Every entry in [v2.X.Y] was present in origin/master's [unreleased] +# 4. No entries disappeared — every entry removed from [unreleased] is in [v2.X.Y] +# 5. Everything from the previous version downward is identical to origin/master +# +# ============================================================================= + +set -uo pipefail + +VERSION="${1:-}" +CHANGELOG="${2:-Changelog.md}" +REFERENCE="origin/master" +ERRORS=0 +WARNINGS=0 + +pass() { echo " PASS $1"; } +fail() { echo " FAIL $1"; ERRORS=$((ERRORS + 1)); } +warn() { echo " WARN $1"; WARNINGS=$((WARNINGS + 1)); } +info() { echo " INFO $1"; } +divider() { echo "------------------------------------------------------------------------"; } + +# Extract bullet entries (lines starting with "- ") from a given section. +# Reads from a file. Outputs sorted entries for stable comparison. +# $1 = file path +# $2 = section name (e.g., "unreleased" or "v2.9.4") +extract_entries() { + local file="$1" + local section="$2" + + awk -v section="$section" ' + BEGIN { found = 0; IGNORECASE = 1 } + /^## \[/ { + if (found) exit + header = $0 + gsub(/^## \[/, "", header) + gsub(/\].*$/, "", header) + if (tolower(header) == tolower(section)) found = 1 + next + } + found && /^- / { print } + ' "$file" | sort +} + +# Extract everything from a given version section header to EOF. +# $1 = file path +# $2 = version string (e.g., "v2.9.3") +extract_from_version() { + local file="$1" + local version="$2" + + awk -v version="$version" ' + BEGIN { found = 0 } + $0 ~ "^## \\[" version "\\]" { found = 1 } + found { print } + ' "$file" +} + +if [ -z "$VERSION" ]; then + echo "Usage: $0 <version> [changelog_path]" + echo " e.g. $0 v2.9.4" + echo "" + echo "Validates changelog edits when syncing released entries to master." + echo "Run from the repo root on the v2.X.Y-changelog branch." + exit 2 +fi + +if [ ! -f "$CHANGELOG" ]; then + echo "Error: $CHANGELOG not found. Run from the repo root or pass the path." + exit 2 +fi + +if ! git show "${REFERENCE}:Changelog.md" &>/dev/null; then + echo "Error: Cannot read ${REFERENCE}:Changelog.md" + echo "Make sure you've fetched origin and are in the git repo root." + exit 2 +fi + +TMPDIR_VALIDATE=$(mktemp -d) +trap 'rm -rf "$TMPDIR_VALIDATE"' EXIT + +REF_CHANGELOG="$TMPDIR_VALIDATE/ref_changelog.md" +git show "${REFERENCE}:Changelog.md" > "$REF_CHANGELOG" + +echo "" +echo "Master Changelog Validation: $VERSION" +echo "Working file: $CHANGELOG" +echo "Reference: ${REFERENCE}:Changelog.md" +divider + +# ============================================================================= +# CHECK 1: [unreleased] section still has entries +# ============================================================================= + +echo "" +echo "[1/5] Unreleased section has entries" + +if ! head -5 "$CHANGELOG" | grep -qi '## \[unreleased\]'; then + fail "[unreleased] section missing or not at top of file" +else + NEW_UNRELEASED_COUNT=$(extract_entries "$CHANGELOG" "unreleased" | wc -l | tr -d ' ' || true) + + if [ "$NEW_UNRELEASED_COUNT" -gt 0 ]; then + pass "[unreleased] section has $NEW_UNRELEASED_COUNT entries" + else + fail "[unreleased] section is empty — entries were wiped instead of moved" + info "The [unreleased] section should retain entries not part of $VERSION" + fi +fi + +# ============================================================================= +# CHECK 2: New version section exists in correct position +# ============================================================================= + +echo "" +echo "[2/5] New [$VERSION] section exists in correct position" + +if ! grep -q "^## \[$VERSION\]" "$CHANGELOG"; then + fail "[$VERSION] section not found" +else + pass "[$VERSION] section found" + + if grep -q "^## \[$VERSION\]" "$REF_CHANGELOG"; then + warn "[$VERSION] section already existed in $REFERENCE — is this a re-run?" + fi + + FIRST_VERSIONED=$(grep '^## \[v' "$CHANGELOG" | head -1 | sed 's/## \[\(.*\)\]/\1/') + if [ "$FIRST_VERSIONED" != "$VERSION" ]; then + fail "[$VERSION] is not the first version section (found [$FIRST_VERSIONED] first)" + else + pass "[$VERSION] is the first version section after [unreleased]" + fi + + VERSION_COUNT=$(extract_entries "$CHANGELOG" "$VERSION" | wc -l | tr -d ' ' || true) + if [ "$VERSION_COUNT" -eq 0 ]; then + fail "[$VERSION] section has no entries" + else + pass "[$VERSION] section has $VERSION_COUNT entries" + fi +fi + +# ============================================================================= +# CHECK 3: Every entry in [VERSION] was in origin/master's [unreleased] +# ============================================================================= + +echo "" +echo "[3/5] All [$VERSION] entries came from [unreleased]" + +OLD_UNRELEASED="$TMPDIR_VALIDATE/old_unreleased.txt" +extract_entries "$REF_CHANGELOG" "unreleased" > "$OLD_UNRELEASED" + +VERSION_ENTRIES_FILE="$TMPDIR_VALIDATE/version_entries.txt" +extract_entries "$CHANGELOG" "$VERSION" > "$VERSION_ENTRIES_FILE" + +VERSION_COUNT=$(wc -l < "$VERSION_ENTRIES_FILE" | tr -d ' ') + +FROM_UNRELEASED=0 +NEW_ENTRIES=0 +while IFS= read -r entry; do + [ -z "$entry" ] && continue + if grep -qFx -- "$entry" "$OLD_UNRELEASED"; then + FROM_UNRELEASED=$((FROM_UNRELEASED + 1)) + else + if [ "$NEW_ENTRIES" -eq 0 ]; then + info "Some [$VERSION] entries are NEW (not in ${REFERENCE}'s [unreleased]):" + fi + NEW_ENTRIES=$((NEW_ENTRIES + 1)) + echo " $entry" + fi +done < "$VERSION_ENTRIES_FILE" + +if [ "$NEW_ENTRIES" -gt 0 ]; then + warn "$NEW_ENTRIES entries in [$VERSION] were not in ${REFERENCE}'s [unreleased] (e.g., dep bumps added during release)" + info "Verify these entries belong in this release" +fi +if [ "$FROM_UNRELEASED" -gt 0 ]; then + pass "$FROM_UNRELEASED of $VERSION_COUNT entries in [$VERSION] came from ${REFERENCE}'s [unreleased]" +fi + +# ============================================================================= +# CHECK 4: No entries vanished — removed entries accounted for in [VERSION] +# ============================================================================= + +echo "" +echo "[4/5] No entries lost — every removal from [unreleased] is in [$VERSION]" + +NEW_UNRELEASED_FILE="$TMPDIR_VALIDATE/new_unreleased.txt" +extract_entries "$CHANGELOG" "unreleased" > "$NEW_UNRELEASED_FILE" + +MISSING_ENTRIES=0 +MOVED_COUNT=0 +while IFS= read -r entry; do + [ -z "$entry" ] && continue + if ! grep -qFx -- "$entry" "$NEW_UNRELEASED_FILE"; then + # This entry was removed from unreleased — it must be in [VERSION] + MOVED_COUNT=$((MOVED_COUNT + 1)) + if ! grep -qFx -- "$entry" "$VERSION_ENTRIES_FILE"; then + if [ "$MISSING_ENTRIES" -eq 0 ]; then + fail "Entries removed from [unreleased] but NOT in [$VERSION]:" + fi + MISSING_ENTRIES=$((MISSING_ENTRIES + 1)) + echo " $entry" + fi + fi +done < "$OLD_UNRELEASED" + +if [ "$MISSING_ENTRIES" -gt 0 ]; then + info "$MISSING_ENTRIES entries were deleted instead of moved" +else + pass "All $MOVED_COUNT entries removed from [unreleased] are present in [$VERSION]" +fi + +# ============================================================================= +# CHECK 5: Everything from previous version downward is identical +# ============================================================================= + +echo "" +echo "[5/5] Older sections unchanged" + +PREV_VERSION=$(grep '^## \[v' "$REF_CHANGELOG" | head -1 | sed 's/## \[\(.*\)\]/\1/') + +if [ -z "$PREV_VERSION" ]; then + warn "Could not determine previous version section in $REFERENCE" +else + CURRENT_TAIL=$(extract_from_version "$CHANGELOG" "$PREV_VERSION") + ORIGINAL_TAIL=$(extract_from_version "$REF_CHANGELOG" "$PREV_VERSION") + + if [ "$CURRENT_TAIL" != "$ORIGINAL_TAIL" ]; then + fail "Content from [$PREV_VERSION] onwards differs from $REFERENCE" + info "Older sections must not be modified" + DIFF_OUTPUT=$(diff <(echo "$CURRENT_TAIL") <(echo "$ORIGINAL_TAIL") | head -20) + if [ -n "$DIFF_OUTPUT" ]; then + info "First differences:" + echo "$DIFF_OUTPUT" | sed 's/^/ /' + fi + else + pass "Everything from [$PREV_VERSION] onwards is identical to $REFERENCE" + fi +fi + +# ============================================================================= +# Summary +# ============================================================================= + +echo "" +divider +if [ $ERRORS -eq 0 ] && [ $WARNINGS -eq 0 ]; then + echo "RESULT: All checks passed ✅" +elif [ $ERRORS -eq 0 ]; then + echo "RESULT: Passed with $WARNINGS warning(s)" +else + echo "RESULT: FAILED with $ERRORS error(s) and $WARNINGS warning(s)" + echo "" + echo "To debug:" + echo " View current: head -60 $CHANGELOG" + echo " View reference: git show $REFERENCE:Changelog.md | head -60" + echo " Full diff: git diff $REFERENCE -- Changelog.md" +fi +echo "" +exit $ERRORS diff --git a/release/verify.rb b/release/verify.rb new file mode 100755 index 0000000000..32fee386dd --- /dev/null +++ b/release/verify.rb @@ -0,0 +1,82 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +# Cherry-pick Verification — Detects contamination after a cherry-pick. +# +# Usage: +# ruby release/verify.rb <pr_number> +# ruby release/verify.rb --help +# +# Exit codes: 0 = PASS, 1 = FAIL (contamination), 2 = usage error + +require 'set' +require_relative 'common' + +EXCLUDE_FILES = Set['Changelog.md'].freeze +# Files where only additions (+lines) are compared, not deletions. +# Gemfile.lock base versions differ between release and master, so removed lines always mismatch. +ADDITIONS_ONLY_FILES = Set['Gemfile.lock'].freeze + +def file_set(*cmd) + ReleaseHelpers.run(*cmd).strip.split("\n").to_set - EXCLUDE_FILES +end + +def extract_file_diff(full_diff, filename) + full_diff.lines + .slice_before { |l| l.start_with?('diff --git') } + .find { |chunk| chunk.first.include?("b/#{filename}") } + &.join || '' +end + +def change_lines(diff_text) + diff_text.lines + .select { |l| l.start_with?('+', '-') } + .reject { |l| l.start_with?('+++', '---') } +end + +# --- Main --- + +pr_number = ARGV[0] + +if pr_number.nil? || ['-h', '--help'].include?(pr_number) + warn <<~HELP + Usage: ruby release/verify.rb <pr_number> + e.g. ruby release/verify.rb 7851 + + Compares the last commit's diff against the original PR diff. + Detects contamination from Git's 3-way merge during cherry-pick. + Excludes Changelog.md by default. + HELP + exit(pr_number.nil? ? 2 : 0) +end + +cherry_files = file_set('git', 'diff', 'HEAD~1..HEAD', '--name-only') +pr_files = file_set('gh', 'pr', 'diff', pr_number, '--repo', ReleaseHelpers::REPO, '--name-only') + +extra = cherry_files - pr_files +missing = pr_files - cherry_files +shared = cherry_files & pr_files + +pr_full_diff = ReleaseHelpers.run('gh', 'pr', 'diff', pr_number, '--repo', ReleaseHelpers::REPO) +comparable_lines = ->(lines, file) do + return lines.select { |l| l.start_with?('+') } if ADDITIONS_ONLY_FILES.include?(file) + + lines +end + +mismatched = shared.reject do |file| + cherry = change_lines(ReleaseHelpers.run('git', 'diff', 'HEAD~1..HEAD', '--', file)) + original = change_lines(extract_file_diff(pr_full_diff, file)) + comparable_lines.call(cherry, file).sort == comparable_lines.call(original, file).sort +end + +if [extra, missing, mismatched].any? { |s| !s.empty? } + puts "FAIL PR ##{pr_number} — contamination detected" + extra.each { |f| puts " + #{f} (extra)" } + missing.each { |f| puts " - #{f} (missing)" } + mismatched.each { |f| puts " ~ #{f} (line mismatch)" } + exit 1 +end + +puts "PASS PR ##{pr_number} — cherry-pick matches original diff" +puts " Files: #{shared.size} checked, #{EXCLUDE_FILES.to_a.join(', ')} excluded"