Skip to content

rethinsilvester/gitlab-upgrade-assistant

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GitLab Upgrade Assistant

An interactive bash script to streamline GitLab self-managed instance upgrades with safety checks, runner management, and comprehensive health monitoring.

License GitLab Bash

Features

  • 🚀 Interactive Menu System - Step-by-step guided upgrade process
  • 🔄 Runner Management - Pause/resume instance and project runners automatically
  • 📢 Broadcast Messages - Set warning and maintenance notifications for users
  • 🛡️ Safety Checks - Background migration verification, health endpoints
  • 📦 RPM/Package Management - Download and verify GitLab packages with checksum
  • 🔧 Maintenance Mode - Enable/disable read-only mode during upgrades
  • 📊 Health Monitoring - Comprehensive post-upgrade health checks
  • 📝 Detailed Logging - All operations logged for audit and troubleshooting

Requirements

  • GitLab Self-Managed version 16.0 or higher
  • Operating System: RHEL/CentOS 8+, Ubuntu 20.04+, or Debian 11+
  • Bash version 4.0 or higher
  • curl for API interactions
  • wget for package downloads
  • Root/sudo access on the GitLab server
  • GitLab Personal Access Token with admin privileges

Required Token Scopes

Your GitLab Personal Access Token needs the following scopes:

  • api - Full API access
  • read_repository - Read repository (for Evaluate pipeline, if enabled)

Installation

Quick Install

# Clone the repository
git clone https://github.com/yourusername/gitlab-upgrade-assistant.git
cd gitlab-upgrade-assistant

# Copy and customize configuration
cp upgrade_gitlab.conf.example upgrade_gitlab.conf
nano upgrade_gitlab.conf  # Edit with your settings

# Make executable
chmod +x upgrade_gitlab.sh

# Run the script
sudo ./upgrade_gitlab.sh

Manual Installation

  1. Download the script:

    wget https://raw.githubusercontent.com/yourusername/gitlab-upgrade-assistant/main/upgrade_gitlab.sh
    wget https://raw.githubusercontent.com/yourusername/gitlab-upgrade-assistant/main/upgrade_gitlab.conf.example
  2. Configure:

    cp upgrade_gitlab.conf.example upgrade_gitlab.conf
    # Edit upgrade_gitlab.conf with your environment details
  3. Run:

    chmod +x upgrade_gitlab.sh
    sudo ./upgrade_gitlab.sh

Configuration

Copy upgrade_gitlab.conf.example to upgrade_gitlab.conf and customize:

# Required: Your GitLab URLs
PROD_URL="https://gitlab.yourcompany.com"
DEV_URL="https://gitlab-dev.yourcompany.com"  # Optional

# Package source configuration
GITLAB_DISTRIBUTION="el/8"  # or ubuntu/jammy, debian/bullseye, etc.
GITLAB_EDITION="gitlab-ee"   # or gitlab-ce for Community Edition

# Optional features
ENABLE_EVALUATE_PIPELINE="false"
ENABLE_VM_SNAPSHOT_AUTOMATION="false"

⚠️ Security Note: Never commit upgrade_gitlab.conf with tokens to version control. The file is included in .gitignore by default.

Usage

Starting the Script

sudo ./upgrade_gitlab.sh

You'll be prompted for:

  1. Environment (dev/prod)
  2. GitLab Personal Access Token
  3. Target GitLab version (e.g., 18.2.4)

Menu Options

The interactive menu provides options for each upgrade phase:

Pre-Upgrade Steps

Option Description
1 Set warning broadcast message (advance notice to users)
2 Run pre-upgrade evaluation pipeline (if configured)

Upgrade Steps

Option Description
3 Set critical broadcast message (maintenance notice)
4 Download GitLab RPM/package
5 Verify package checksum (MD5)
6 Pause GitLab runners
7 Enable maintenance mode (read-only)
8 Create GitLab backup
9 Check background migrations
10 Take external backup (Cohesity/other)
11 Take VM snapshot
12 Perform upgrade
13 Restart GitLab services

Post-Upgrade Steps

Option Description
14 Confirm GitLab version
15 Run health checks (background)
16 Check health check status
17 Check GitLab health endpoint
18 Disable maintenance mode
19 Resume GitLab runners
20 Run post-upgrade evaluation
21-23 Delete broadcast messages

Recommended Upgrade Workflow

1. Set warning broadcast (1-7 days before)
2. On upgrade day:
   a. Set critical broadcast
   b. Download and verify RPM
   c. Pause runners
   d. Enable maintenance mode
   e. Create backup
   f. Verify background migrations complete
   g. Take VM snapshot (recommended)
   h. Perform upgrade
   i. Restart GitLab
   j. Verify version
   k. Run health checks
   l. Disable maintenance mode
   m. Resume runners
   n. Delete broadcast messages

Supported Platforms

GitLab Editions

  • GitLab Enterprise Edition (EE)
  • GitLab Community Edition (CE)

Operating Systems

OS Package Format Status
RHEL/CentOS 8+ RPM ✅ Fully Tested
RHEL/CentOS 9 RPM ✅ Supported
Ubuntu 20.04+ DEB 🔄 In Progress
Debian 11+ DEB 🔄 In Progress

Logs and Output

All operations are logged to a timestamped directory:

/tmp/gitlab_upgrade_<from_version>_to_<to_version>_<timestamp>/
├── main_upgrade.log          # Main operation log
├── gitlab_upgrade_output.log # RPM install output
├── gitlab_restart_output.log # Service restart output
└── evaluate/                 # Evaluate pipeline info (if enabled)

Troubleshooting

Common Issues

Token validation failed

Error: Invalid GitLab token or unable to connect to GitLab API
  • Verify your token has api scope
  • Check the GitLab URL is correct
  • Ensure network connectivity to GitLab

Background migrations pending

Warning: Background migrations still pending (remaining: X)
  • Wait for migrations to complete before upgrading
  • Check GitLab Admin > Monitoring > Background Migrations

Runner pause failed

Error: Failed to pause runner (HTTP: 403)
  • Verify token has admin privileges
  • Check if runner is a shared/group runner (may need different permissions)

Debug Mode

Set DEBUG=true before running for verbose output:

DEBUG=true sudo ./upgrade_gitlab.sh

Contributing

Contributions are welcome! Please read CONTRIBUTING.md for guidelines.

Development Setup

# Fork and clone
git clone https://github.com/yourusername/gitlab-upgrade-assistant.git
cd gitlab-upgrade-assistant

# Create feature branch
git checkout -b feature/your-feature

# Make changes and test
# Submit pull request

Running Tests

# Shellcheck for linting
shellcheck upgrade_gitlab.sh

# Dry-run mode (coming soon)
./upgrade_gitlab.sh --dry-run

Roadmap

  • DEB package support for Ubuntu/Debian
  • Dry-run mode for testing
  • Slack/Teams notifications
  • Automated rollback support
  • Multi-node/HA GitLab support
  • Omnibus to Helm migration helper

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

  • GitLab Inc. for their excellent upgrade documentation
  • The GitLab self-managed community for feedback and testing

Disclaimer

This tool is provided as-is. Always:

  • Test upgrades in a non-production environment first
  • Have verified backups before upgrading
  • Follow GitLab's official upgrade paths
  • Review GitLab release notes for breaking changes

Found this helpful? Give it a ⭐ on GitHub!

Questions? Open an issue

About

Interactive bash script for GitLab self-managed instance upgrades

Topics

Resources

License

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages