Skip to content

Commit 31305f2

Browse files
per1234per1234
committed
Add infrastructure for checking sync of category order metadata
In order to allow contributors to include the specific order of the category listings in proposals, the repository contains a metadata file that defines the category order. This file is automatically generated using an API call. A task and GitHub Actions workflow are added for the maintenance of this data. On any change to a relevant file, and periodically, the workflow will run and compare the data in the repository against the actual category structure of the forum to ensure the data remains in sync. If the category order metadata file is found to be out of sync, the updated file is uploaded to a workflow artifact and a message notifying the contributor of its availability printed to the workflow run logs. This will assist contributors in easily bringing the file into sync.
1 parent c610df5 commit 31305f2

File tree

8 files changed

+297
-2
lines changed

8 files changed

+297
-2
lines changed

‎.github/workflows/check-order-metadata-task.yml‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
name: Check Category Order Metadata
2+
3+
# See: https://docs.github.com/actions/using-workflows/events-that-trigger-workflows
4+
on:
5+
create:
6+
push:
7+
paths:
8+
- ".github/workflows/check-order-metadata-task.ya?ml"
9+
- "poetry.lock"
10+
- "pyproject.toml"
11+
- "Taskfile.ya?ml"
12+
- "assets/generate-category-order-metadata/*"
13+
- "content/categories/order.md"
14+
pull_request:
15+
paths:
16+
- ".github/workflows/check-order-metadata-task.ya?ml"
17+
- "poetry.lock"
18+
- "pyproject.toml"
19+
- "Taskfile.ya?ml"
20+
- "assets/generate-category-order-metadata/*"
21+
- "content/categories/order.md"
22+
schedule:
23+
# Run periodically to catch breakage caused by external changes.
24+
- cron: "0 3 * * FRI"
25+
workflow_dispatch:
26+
repository_dispatch:
27+
28+
jobs:
29+
run-determination:
30+
runs-on: ubuntu-latest
31+
outputs:
32+
result: ${{ steps.determination.outputs.result }}
33+
steps:
34+
- name: Determine if the rest of the workflow should run
35+
id: determination
36+
run: |
37+
if [[ "${{ secrets.ARDUINO_FORUM_API_KEY }}" == "" ]]; then
38+
# Required API key secret is not defined.
39+
RESULT="false"
40+
else
41+
RESULT="true"
42+
fi
43+
44+
echo "result=$RESULT" >> $GITHUB_OUTPUT
45+
46+
check-order-metadata:
47+
needs: run-determination
48+
if: needs.run-determination.outputs.result == 'true'
49+
runs-on: ubuntu-latest
50+
env:
51+
ARTIFACT_NAME: category-order-metadata
52+
CATEGORY_ORDER_METADATA_FILE_PATH: content/categories/order.md
53+
steps:
54+
- name: Checkout repository
55+
uses: actions/checkout@v3
56+
57+
- name: Install Poetry
58+
run: |
59+
pipx install \
60+
--python "$(which python)" \
61+
poetry
62+
63+
- name: Install Task
64+
uses: arduino/setup-task@v1
65+
with:
66+
repo-token: ${{ secrets.GITHUB_TOKEN }}
67+
version: 3.x
68+
69+
- name: Generate category order metadata
70+
env:
71+
ARDUINO_FORUM_API_KEY: ${{ secrets.ARDUINO_FORUM_API_KEY }}
72+
run: |
73+
task \
74+
--silent \
75+
forum:generate-category-order-metadata
76+
77+
- name: Check category order metadata
78+
id: check-category-order-metadata
79+
run: git diff --color --exit-code
80+
81+
- name: Upload corrected file to workflow artifact
82+
if: failure() && steps.check-category-order-metadata.outcome == 'failure'
83+
uses: actions/upload-artifact@v3
84+
with:
85+
if-no-files-found: error
86+
name: ${{ env.ARTIFACT_NAME }}
87+
path: ${{ env.CATEGORY_ORDER_METADATA_FILE_PATH }}
88+
89+
- name: Add artifact availability notice
90+
if: failure() && steps.check-category-order-metadata.outcome == 'failure'
91+
run: |
92+
echo "::notice file=${{ env.CATEGORY_ORDER_METADATA_FILE_PATH }}::Corrected file was saved to the ${{ env.ARTIFACT_NAME }} workflow artifact"

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# Arduino Forum Assets
22

3+
[![Check Category Order Metadata](https://github.com/arduino/forum-assets/actions/workflows/check-order-metadata-task.yml/badge.svg)](https://github.com/arduino/forum-assets/actions/workflows/check-order-metadata-task.yml)
34
[![Check General Formatting status](https://github.com/arduino/forum-assets/actions/workflows/check-general-formatting-task.yml/badge.svg)](https://github.com/arduino/forum-assets/actions/workflows/check-general-formatting-task.yml)
45
[![Check Files status](https://github.com/arduino/forum-assets/actions/workflows/check-files-task.yml/badge.svg)](https://github.com/arduino/forum-assets/actions/workflows/check-files-task.yml)
56
[![Check License status](https://github.com/arduino/forum-assets/actions/workflows/check-license.yml/badge.svg)](https://github.com/arduino/forum-assets/actions/workflows/check-license.yml)

‎Taskfile.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,21 @@ tasks:
5454
# This is an "umbrella" task used to call any documentation generation processes the project has.
5555
# It can be left empty if there are none.
5656

57+
forum:generate-category-order-metadata:
58+
desc: Generate the category order metadata file
59+
deps:
60+
- task: poetry:install-deps
61+
cmds:
62+
- |
63+
if [[ "$ARDUINO_FORUM_API_KEY" == "" ]]; then
64+
echo "ARDUINO_FORUM_API_KEY environment variable not set."
65+
echo "This task requires a forum.arduino.cc Discourse API key with \"categories > list\" scope."
66+
exit 1
67+
fi
68+
69+
poetry run \
70+
python assets/generate-category-order-metadata/generate-category-order-metadata.py
71+
5772
# Source: https://github.com/arduino/tooling-project-assets/blob/main/workflow-templates/assets/check-files-task/Taskfile.yml
5873
general:check-filenames:
5974
desc: Check for non-portable filenames

‎assets/generate-category-order-metadata/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Category Order Metadata Generator
2+
3+
This Python script uses the Arduino Forum's [Discourse API](https://docs.discourse.org/#tag/Categories/operation/listCategories) to generate the forum category order metadata file (`order.md`).

‎assets/generate-category-order-metadata/category-skip-list.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
[186]

‎assets/generate-category-order-metadata/generate-category-order-metadata.py‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
import os
2+
import requests
3+
import json
4+
import pathlib
5+
6+
7+
def get_category_list(categories_data, level):
8+
category_list = ""
9+
for category_data in categories_data:
10+
if category_data["id"] in skip_list:
11+
continue
12+
13+
category_list += "{indent}- {name}\n".format(indent=" " * level, name=category_data["name"])
14+
15+
if "subcategory_list" in category_data:
16+
category_list += get_category_list(categories_data=category_data["subcategory_list"], level=level + 1)
17+
18+
return category_list
19+
20+
21+
response = requests.get(
22+
url="https://forum.arduino.cc/categories.json",
23+
params={"include_subcategories": "true"},
24+
headers={"Api-Key": os.environ["ARDUINO_FORUM_API_KEY"], "Api-Username": "system"},
25+
)
26+
27+
script_location = pathlib.PurePath(os.path.dirname(os.path.realpath(__file__)))
28+
with open(file=script_location.joinpath("category-skip-list.json"), encoding="utf-8") as skip_list_file:
29+
skip_list = json.load(fp=skip_list_file)
30+
31+
category_list = get_category_list(categories_data=response.json()["category_list"]["categories"], level=0)
32+
33+
with open(file="content/categories/order.md", mode="w", encoding="utf-8") as order_file:
34+
order_file.write(category_list)

0 commit comments

Comments
 (0)