Skip to content

Commit 2dd2df2

Browse files
tsweckardtsweckard
authored
Merge pull request #24 from Trihydro/us/17033-azure-gh-pipeline
Automate GH JAR Upload to DevOps
2 parents 9874a2d + 003e60d commit 2dd2df2

2 files changed

Lines changed: 274 additions & 8 deletions

File tree

‎.github/BUILD-PIPELINE.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
# GraphHopper Build Pipeline Documentation
2+
3+
## Overview
4+
5+
The GraphHopper build pipeline (`build.yml`) is a GitHub Actions workflow that automates building, testing, and deploying the GraphHopper routing engine to Azure DevOps. The pipeline runs on every push to the repository.
6+
7+
## Pipeline Structure
8+
9+
### Job 1: Build and Test
10+
**Trigger:** Every push to any branch
11+
12+
This job builds and tests the GraphHopper project across multiple Java versions to ensure compatibility.
13+
14+
**Steps:**
15+
1. **Checkout code** - Retrieves the repository code
16+
2. **Setup Java** - Configures Java using the Temurin distribution
17+
- Tested versions: Java 25 and Java 26-ea (early access)
18+
- Uses a matrix strategy to test both versions in parallel
19+
3. **Cache Maven artifacts** - Caches `~/.m2/repository` to speed up builds
20+
4. **Cache node** - Caches `web-bundle/node` directory
21+
5. **Cache node_modules** - Caches `web-bundle/node_modules` directory
22+
6. **Build and Test** - Runs `mvn -B clean test`
23+
24+
### Job 2: Push to Azure DevOps
25+
**Trigger:** Only runs on successful build when pushing to the `master` branch
26+
27+
This job packages the GraphHopper JAR file and pushes it to the Azure DevOps repository for deployment.
28+
29+
**Configuration Variables:**
30+
- `AZURE_ORG`: trihydro
31+
- `AZURE_PROJECT`: SDX
32+
- `AZURE_REPO`: graphhopper
33+
- `JAVA_VERSION`: 25
34+
35+
**Steps:**
36+
1. **Checkout code** - Retrieves the repository code
37+
2. **Package JAR** - Runs `mvn -B package -DskipTests` to create the JAR file
38+
3. **Get JAR info** - Extracts JAR filename and creates a timestamp
39+
4. **Clone Azure DevOps Repository** - Clones the Azure repo using PAT authentication
40+
5. **Commit and push JAR** - Creates a new branch, backs up old JAR, commits new JAR
41+
6. **Create Pull Request** - Creates PR in Azure DevOps with designated reviewers
42+
43+
## Required Secrets
44+
45+
The following GitHub repository secrets must be configured for the pipeline to work:
46+
47+
### `AZURE_DEVOPS_PAT`
48+
**Type:** Personal Access Token
49+
**Purpose:** Authenticates GitHub Actions with Azure DevOps
50+
**Permissions Required:**
51+
- Code: Read & Write
52+
53+
**How to Update:**
54+
1. Go to Azure DevOps → User Settings → Personal Access Tokens
55+
2. Create a new token or regenerate existing one with the required permissions
56+
3. Copy the token value
57+
4. Go to GitHub repository → Settings → Secrets and variables → Actions
58+
5. Update or create `AZURE_DEVOPS_PAT` secret with the new token value
59+
60+
### `AZURE_GH_REPO_ID`
61+
**Type:** Repository GUID
62+
**Purpose:** Identifies the Azure DevOps repository for API calls
63+
64+
**How to Find:**
65+
1. Run command: `az repos list --organization "https://dev.azure.com/trihydro" --project "SDX" --query "[?name == 'graphhopper'].id" --output tsv`
66+
67+
### `TEAGHEN_DEVOPS_USER_ID` & `CHAN_DEVOPS_USER_ID`
68+
**Type:** User GUID
69+
**Purpose:** Adds Teaghen and Chan as a PR reviewer automatically
70+
71+
**How to Find/Update:**
72+
1. Make API call to: https://vssps.dev.azure.com/trihydro/_apis/graph/users?api-version=7.1-preview.1
73+
1. Authorization header: Basic `<AZURE_DEVOPS_PAT>`
74+
2. Content-Type header: application/json
75+
2. Update the GitHub secret with the originId value.
76+
77+
## Maintenance Guide
78+
79+
### Updating Java Versions
80+
To test against different Java versions:
81+
1. Edit `.github/workflows/build.yml`
82+
2. Modify the `matrix.java-version` array (currently `[25, 26-ea]`)
83+
3. Update the `JAVA_VERSION` environment variable in the `push-to-azure` job if changing the deployment version
84+
85+
### Updating Azure DevOps Configuration
86+
To change the target Azure DevOps organization, project, or repository:
87+
1. Edit `.github/workflows/build.yml`
88+
2. Update the environment variables in the `push-to-azure` job:
89+
- `AZURE_ORG`
90+
- `AZURE_PROJECT`
91+
- `AZURE_REPO`
92+
93+
### Adding/Removing Reviewers
94+
To modify automatic PR reviewers:
95+
1. Add the new reviewer user ID secrets in GitHub repository settings
96+
2. Edit the `Create Pull Request in Azure DevOps` step in `build.yml`
97+
3. Add/remove reviewer objects in the `reviewers` array of the PR payload
98+
99+
### Monitoring Secrets Expiration
100+
**Recommended Schedule:**
101+
- Azure DevOps PATs typically expire after 90 days depending on configuration
102+
- Set up calendar reminders 2 weeks before known expiration dates
103+
104+
### Troubleshooting Common Issues
105+
106+
**Build fails but tests pass locally:**
107+
- Check Java version compatibility (pipeline uses Java 25/26)
108+
- Clear Maven cache by removing the cache action temporarily
109+
110+
**Azure push fails with an authentication error:**
111+
- `AZURE_DEVOPS_PAT` has likely expired - regenerate it
112+
- Verify the PAT has Code and Pull Request permissions
113+
114+
**Pull Request creation fails:**
115+
- Verify `AZURE_GH_REPO_ID` is correct
116+
- Check that reviewer IDs are valid and users have access to the repository
117+
- Ensure the target branch (master) exists in Azure DevOps repo
118+
119+
## Workflow Diagram
120+
121+
```
122+
┌─────────────────┐
123+
│ Push Event │
124+
└────────┬────────┘
125+
│
126+
▼
127+
┌─────────────────────────┐
128+
│ Build & Test Job │
129+
│ - Java 25 │
130+
│ - Java 26-ea │
131+
│ - Maven clean test │
132+
└────────┬────────────────┘
133+
│
134+
▼
135+
[master branch?]
136+
│ Yes
137+
▼
138+
┌─────────────────────────┐
139+
│ Push to Azure Job │
140+
│ 1. Package JAR │
141+
│ 2. Clone Azure repo │
142+
│ 3. Create branch │
143+
│ 4. Commit JAR │
144+
│ 5. Push branch │
145+
│ 6. Create PR │
146+
└─────────────────────────┘
147+
```
148+

‎.github/workflows/build.yml‎

Lines changed: 126 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
name: Build and Test
22
on: push
3+
34
jobs:
45
build:
56
runs-on: ubuntu-latest
@@ -36,14 +37,131 @@ jobs:
3637
${{ runner.os}}-node_modules-
3738
- name: Build ${{ matrix.java-version }}
3839
run: mvn -B clean test
40+
41+
push-to-azure:
42+
runs-on: ubuntu-latest
43+
needs: build
44+
if: github.ref == 'refs/heads/master'
45+
env:
46+
AZURE_ORG: trihydro
47+
AZURE_PROJECT: SDX
48+
AZURE_REPO: graphhopper
49+
JAVA_VERSION: 25
50+
steps:
51+
- uses: actions/checkout@v4
52+
- uses: actions/setup-java@v4
53+
with:
54+
java-version: ${{ env.JAVA_VERSION }}
55+
distribution: temurin
56+
cache: maven
57+
3958
- name: Package JAR
40-
if: github.ref == 'refs/heads/master'
4159
run: mvn -B package -DskipTests
42-
- name: Upload JAR Artifact
43-
if: github.ref == 'refs/heads/master'
44-
uses: actions/upload-artifact@v4
45-
with:
46-
name: graphhopper-jar-${{ matrix.java-version }}
47-
path: web/target/*.jar
48-
retention-days: 90
4960

61+
- name: Get JAR info and set date
62+
id: jar-info
63+
run: |
64+
JAR_FILE=$(ls web/target/*.jar | head -n 1)
65+
if [ -z "$JAR_FILE" ] || [ ! -f "$JAR_FILE" ]; then
66+
echo "Error: No JAR file found in web/target/. Maven package step may have failed to produce a JAR." >&2
67+
exit 1
68+
fi
69+
JAR_NAME=$(basename "$JAR_FILE" | sed 's/-SNAPSHOT//')
70+
DATE=$(date +%Y%m%d-%H%M%S)
71+
72+
{
73+
echo "jar_file=$JAR_FILE"
74+
echo "jar_name=$JAR_NAME"
75+
echo "date=$DATE"
76+
} >> "$GITHUB_OUTPUT"
77+
78+
- name: Clone Azure DevOps Repository
79+
env:
80+
AZURE_DEVOPS_PAT: ${{ secrets.AZURE_DEVOPS_PAT }}
81+
run: |
82+
git clone https://$AZURE_DEVOPS_PAT@dev.azure.com/$AZURE_ORG/$AZURE_PROJECT/_git/$AZURE_REPO azure-repo
83+
84+
- name: Commit and push JAR to new branch
85+
working-directory: azure-repo
86+
run: |
87+
# Set variables
88+
JAR_FILE="${{ steps.jar-info.outputs.jar_file }}"
89+
JAR_NAME="${{ steps.jar-info.outputs.jar_name }}"
90+
BUILD_DATE="${{ steps.jar-info.outputs.date }}"
91+
COMMIT_SHA="${{ github.sha }}"
92+
93+
# Configure git
94+
git config user.name "GitHub Actions Bot"
95+
git config user.email "github-actions@users.noreply.github.com"
96+
97+
# Create a new branch
98+
BRANCH_NAME="update-graphhopper-jar-$BUILD_DATE"
99+
git checkout -b $BRANCH_NAME
100+
101+
# Create resources directory if it doesn't exist
102+
mkdir -p resources
103+
104+
# Rename existing JAR file if it exists
105+
if [ -f "resources/$JAR_NAME" ]; then
106+
OLD_NAME="resources/$(basename $JAR_NAME .jar)-old-$BUILD_DATE.jar"
107+
mv "resources/$JAR_NAME" "$OLD_NAME"
108+
fi
109+
110+
# Copy new JAR file
111+
cp ../$JAR_FILE resources/$JAR_NAME
112+
113+
# Commit changes
114+
git add resources/
115+
git commit -m "Update GraphHopper JAR from GitHub build - $BUILD_DATE" \
116+
-m "Built from commit: $COMMIT_SHA" \
117+
-m "Java version: $JAVA_VERSION"
118+
119+
# Push the branch
120+
git push origin $BRANCH_NAME
121+
122+
# Save branch name for PR creation
123+
echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV
124+
125+
- name: Create Pull Request in Azure DevOps
126+
env:
127+
AZURE_DEVOPS_PAT: ${{ secrets.AZURE_DEVOPS_PAT }}
128+
AZURE_GH_REPO_ID: ${{ secrets.AZURE_GH_REPO_ID }}
129+
REVIEWER_ID_TEAGHEN: ${{ secrets.TEAGHEN_DEVOPS_USER_ID }}
130+
REVIEWER_ID_CHAN: ${{ secrets.CHAN_DEVOPS_USER_ID }}
131+
run: |
132+
# Set variables
133+
BUILD_DATE="${{ steps.jar-info.outputs.date }}"
134+
COMMIT_SHA="${{ github.sha }}"
135+
AUTH_HEADER=$(echo -n ":${AZURE_DEVOPS_PAT}" | base64 | tr -d '\n')
136+
137+
PR_PAYLOAD=$(cat <<EOF
138+
{
139+
"sourceRefName": "refs/heads/${{ env.BRANCH_NAME }}",
140+
"targetRefName": "refs/heads/master",
141+
"title": "Update GraphHopper JAR - $BUILD_DATE",
142+
"description": "Automated PR to update GraphHopper JAR file.\n\n**Source:** GitHub repository\n**Commit:** $COMMIT_SHA\n**Java Version:** $JAVA_VERSION\n**Build Date:** $BUILD_DATE\n\nThis JAR was automatically built and uploaded by GitHub Actions.",
143+
"reviewers": [
144+
{
145+
"id": "${REVIEWER_ID_TEAGHEN}"
146+
},
147+
{
148+
"id": "${REVIEWER_ID_CHAN}"
149+
}
150+
]
151+
}
152+
EOF
153+
)
154+
155+
# Make API request
156+
HTTP_CODE=$(curl -sSL -w "%{http_code}" -o /tmp/pr_response.json \
157+
-X POST \
158+
-H "Content-Type: application/json" \
159+
-H "Authorization: Basic $AUTH_HEADER" \
160+
-d "$PR_PAYLOAD" \
161+
"https://dev.azure.com/${AZURE_ORG}/${AZURE_PROJECT}/_apis/git/repositories/${AZURE_GH_REPO_ID}/pullrequests?api-version=7.1")
162+
163+
if [ "$HTTP_CODE" -ne 201 ]; then
164+
echo "Failed to create Pull Request (HTTP $HTTP_CODE)"
165+
cat /tmp/pr_response.json
166+
exit 1
167+
fi

0 commit comments

Comments
 (0)