diff --git a/.github/workflows/CLEANUP_DOCUMENTATION.md b/.github/workflows/CLEANUP_DOCUMENTATION.md new file mode 100644 index 000000000..c97507a77 --- /dev/null +++ b/.github/workflows/CLEANUP_DOCUMENTATION.md @@ -0,0 +1,102 @@ +# PR Documentation Folder Cleanup + +## Overview + +This document describes the automated cleanup mechanism for PR documentation folders in `/www/mie-docs/public/`. + +## Problem + +When pull requests are created, the CI workflow builds documentation and deploys it to `/www/mie-docs/public/{branch-name}/`. Previously, these folders were never cleaned up when PRs were closed or merged, leading to accumulation of stale folders. + +## Solution + +Two cleanup mechanisms have been implemented: + +### 1. Automatic Cleanup on PR Close/Merge + +**Workflow:** `.github/workflows/pull_request.yml` + +When a PR is closed or merged, the cleanup job automatically: +- Extracts the branch name from the PR +- Removes the corresponding folder from `/www/mie-docs/public/` +- Sends a notification to Rocket.Chat +- Handles cases where the folder doesn't exist gracefully + +**Trigger:** Runs automatically when a PR is closed (including merged PRs) + +### 2. Scheduled Cleanup of Stale Folders + +**Workflow:** `.github/workflows/cleanup_stale_pr_folders.yml` + +Weekly scheduled job that: +- Fetches list of all open PRs via GitHub API +- Compares with folders in `/www/mie-docs/public/` +- Removes folders that don't correspond to any active PR +- Keeps the `master` branch folder +- Provides a summary of removed and kept folders + +**Schedule:** Runs every Sunday at 2:00 AM UTC + +**Manual Trigger:** Can be manually triggered via GitHub Actions UI using the "workflow_dispatch" event + +## Technical Details + +### Branch Name Sanitization + +Branch names are sanitized to match filesystem naming conventions: +- Forward slashes (`/`) are replaced with hyphens (`-`) +- Example: `feature/my-feature` becomes `feature-my-feature` + +### Error Handling + +- If a folder doesn't exist during cleanup, a warning is logged but the job continues +- If the scheduled cleanup can't access the folder, it exits with an error +- Both workflows include notifications to Rocket.Chat regardless of success/failure + +### Notifications + +All cleanup activities are reported to the `#miedocs` Rocket.Chat channel: +- PR close cleanup: Notifies when a specific PR folder is removed +- Scheduled cleanup: Provides summary of how many folders were removed vs kept + +## Maintenance + +### Monitoring + +Check the following for cleanup status: +1. GitHub Actions workflow runs in the repository +2. Rocket.Chat `#miedocs` channel for notifications +3. `/www/mie-docs/public/` directory for any unexpected folders + +### Manual Cleanup + +If needed, you can manually trigger the scheduled cleanup: +1. Go to GitHub Actions tab +2. Select "Cleanup Stale PR Folders" workflow +3. Click "Run workflow" +4. Select the branch (usually `master`) +5. Click "Run workflow" button + +### Troubleshooting + +**Folder not being cleaned up after PR close:** +- Check the workflow run logs in GitHub Actions +- Verify the branch name matches the folder name (with `/` replaced by `-`) +- Check if the self-hosted runner has permissions to delete the folder + +**Scheduled cleanup not running:** +- Verify the cron schedule is correct +- Check if the self-hosted runner is online +- Review workflow run history for any errors + +**Too many folders being kept:** +- Verify PRs are actually closed, not just merged +- Check if there are branches with the same name as folders +- Review the scheduled cleanup logs to see which folders were identified as active + +## Future Improvements + +Potential enhancements that could be considered: +- Add retention period before deleting folders (e.g., keep for 7 days after PR close) +- Implement archiving instead of deletion for important PRs +- Add metrics/dashboard for folder lifecycle diff --git a/.github/workflows/cleanup_stale_pr_folders.yml b/.github/workflows/cleanup_stale_pr_folders.yml new file mode 100644 index 000000000..9e6c14bd6 --- /dev/null +++ b/.github/workflows/cleanup_stale_pr_folders.yml @@ -0,0 +1,104 @@ +# Scheduled cleanup workflow for stale PR documentation folders +# This runs weekly to clean up any folders that don't correspond to active PRs + +name: Cleanup Stale PR Folders + +on: + schedule: + # Run every Sunday at 2 AM UTC + - cron: '0 2 * * 0' + workflow_dispatch: # Allow manual triggering + +jobs: + cleanup-stale-folders: + runs-on: [ self-hosted, docsqa ] + permissions: + contents: read + pull-requests: read + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Get list of active PR branches + id: active_branches + uses: actions/github-script@v7.0.1 + with: + script: | + const { data: pullRequests } = await github.rest.pulls.list({ + owner: context.repo.owner, + repo: context.repo.repo, + state: 'open' + }); + + // Sanitize branch names to match filesystem naming (replace / with -) + const branches = pullRequests.map(pr => { + return pr.head.ref.replace(/\//g, '-'); + }); + + // Also include master branch as it's deployed + branches.push('master'); + + console.log('Active branches:', branches); + return branches; + + - name: Clean up stale folders + env: + ACTIVE_BRANCHES: ${{ steps.active_branches.outputs.result }} + run: | + echo "Active branches: $ACTIVE_BRANCHES" + + # Convert JSON array to bash array using readarray + readarray -t active_branches < <(echo "$ACTIVE_BRANCHES" | jq -r '.[]') + + # Navigate to the public docs folder + cd /www/mie-docs/public/ || exit 1 + + removed_count=0 + skipped_count=0 + + # Iterate through all folders + for folder in */; do + # Remove trailing slash + folder_name="${folder%/}" + + # Check if this folder corresponds to an active branch + is_active=false + for branch in "${active_branches[@]}"; do + if [ "$folder_name" = "$branch" ]; then + is_active=true + break + fi + done + + if [ "$is_active" = false ]; then + echo "🗑️ Removing stale folder: $folder_name" + rm -rf "$folder_name" + removed_count=$((removed_count + 1)) + else + echo "✅ Keeping active folder: $folder_name" + skipped_count=$((skipped_count + 1)) + fi + done + + echo "" + echo "=== Cleanup Summary ===" + echo "Folders removed: $removed_count" + echo "Active folders kept: $skipped_count" + echo "=======================" + + # Set output for notification + echo "removed_count=$removed_count" >> $GITHUB_OUTPUT + echo "skipped_count=$skipped_count" >> $GITHUB_OUTPUT + id: cleanup_summary + + - name: Rocket.Chat Cleanup Notification + uses: wreiske/Rocket.Chat.GitHub.Action.Notification@1.5.1 + if: always() + with: + type: ${{ job.status }} + job_name: '*Scheduled Cleanup* Removed ${{ steps.cleanup_summary.outputs.removed_count }} stale PR folder(s), kept ${{ steps.cleanup_summary.outputs.skipped_count }} active' + channel: '#miedocs' + url: ${{ secrets.ROCKETCHAT_WEBHOOK }} + commit: false + token: ${{ github.token }} diff --git a/.github/workflows/pull_request.yml b/.github/workflows/pull_request.yml index 3c55cdb2b..d95b6bd66 100644 --- a/.github/workflows/pull_request.yml +++ b/.github/workflows/pull_request.yml @@ -6,6 +6,7 @@ name: Pull Request pushed # events but only for the master branch on: pull_request: + types: [opened, synchronize, reopened, closed] push: branches: [ master ] @@ -15,6 +16,8 @@ jobs: build: # The type of runner that the job will run on runs-on: [ self-hosted, docsqa ] + # Only run build if PR is not being closed + if: github.event.action != 'closed' # Steps represent a sequence of tasks that will be executed as part of the job steps: @@ -68,3 +71,44 @@ jobs: body: '👋 @${{github.actor}}, Your documentation has been pushed to https://docs-qa.med-web.com/${{ steps.extract_branch.outputs.branch }}/ for commit ${{ github.sha }}' }) if: github.ref != 'refs/heads/master' + + # Cleanup job to remove PR folders when PR is closed or merged + cleanup: + runs-on: [ self-hosted, docsqa ] + # Only run cleanup when PR is closed (includes merged PRs) + if: github.event.action == 'closed' + permissions: + contents: read + + steps: + - name: Extract branch name from PR + shell: bash + run: | + # Use the head ref from the PR event, sanitize it for filesystem + branch_name="${{ github.head_ref }}" + branch_name="${branch_name//\//-}" + echo "branch=${branch_name}" >> $GITHUB_OUTPUT + id: extract_branch + + - name: Remove PR documentation folder + run: | + FOLDER="/www/mie-docs/public/${{ steps.extract_branch.outputs.branch }}" + if [ -d "$FOLDER" ]; then + echo "Removing folder: $FOLDER" + rm -rf "$FOLDER" + echo "✅ Successfully removed PR documentation folder" + else + echo "⚠️ Folder not found: $FOLDER (may have been already cleaned up)" + fi + + - name: Rocket.Chat Cleanup Notification + uses: wreiske/Rocket.Chat.GitHub.Action.Notification@1.5.1 + if: always() + with: + type: ${{ job.status }} + job_name: '*Cleanup* PR #${{ github.event.pull_request.number }} documentation removed' + channel: '#miedocs' + url: ${{ secrets.ROCKETCHAT_WEBHOOK }} + commit: false + token: ${{ github.token }} + diff --git a/README.md b/README.md index af1b49550..ede1f3d19 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,8 @@ themes.gohugo.io) for WC and EH. - [SHORTCODES.md](SHORTCODES.md) documentation for all available Hugo shortcodes 4. Automation to automate the process as well as a set of scripts to update a qa-server in realtime watching for changes in Google Drive and near instant update. - [Actions](.github/workflows) - github scripts that automate changes out to production and test Pull Requests to see if they break the build process. + - **pull_request.yml** - Builds and deploys PR documentation to `/www/mie-docs/public/{branch-name}/`, and automatically cleans up folders when PRs are closed or merged + - **cleanup_stale_pr_folders.yml** - Scheduled cleanup job (runs weekly on Sundays) that removes folders for PRs that are no longer active - [build.sh](build.sh) a script for testing and building the static page generation locally on your own machine, GitHub or CloudFlare. ## Setup