ci: add automated skill review for SKILL.md pull requests#56
Conversation
Adds a lightweight GitHub Action that reviews any SKILL.md files changed in a pull request using tessl skill review. Posts results as a PR comment with minimal permissions (pull-requests: write, contents: read). Signed-off-by: Alan Pope <alan@popey.com>
|
Apologies for the slow review — this should not have sat for over a month. Direction is good — automated linting on 1. Pin the third-party action to a SHA, not ```yaml
2. Brief docs entry in A line under "What to expect when you submit a SKILL.md PR" explaining: this workflow runs, here's what it does, here's how to disable for in-progress PRs ( Once those two are in, this is a clean merge. Apologies again for the wait. |
|
Final ping on this one @popey — apologies for the persistent wait. The two asks from the Apr-4 review still stand:
The CI workflow itself is fine and the direction is right. If these two are easy on your end, push them whenever — happy to merge same-day after. If you're out of bandwidth, comment here and I'll pick up the changes myself and ship them with your authorship credit. Either is fine. If I haven't heard back by end of next week (2026-05-22), I'll close this with a note pointing at a fresh smaller version I'll open myself, and we can revisit if you want to re-engage. Genuine appreciation for the original work — automated SKILL.md review is a real win for the project. |
shreyas-lyzr
left a comment
There was a problem hiding this comment.
The intent is good — automated quality signal on SKILL.md PRs before manual review. One concern that needs addressing before merge:
The workflow pins tesslio/skill-review@main, which means any push to that third-party repo’s main branch runs with pull-requests: write permissions in your CI. That is a supply-chain risk: a compromised or malicious update to tesslio/skill-review could post arbitrary PR comments or read repository contents on your behalf.
Suggestion: pin to a specific commit SHA instead of @main:
- uses: tesslio/skill-review@<full-sha-here>
Check the latest commit hash on https://github.com/tesslio/skill-review and pin there. Add a comment with the version tag for human readability.
Also worth confirming: does tesslio/skill-review require any secrets or tokens passed in? The workflow doesn’t set GITHUB_TOKEN explicitly — if the action needs it to post comments, this may silently no-op on some GitHub configurations. Check the action’s README for required inputs.
Once the pin is in place and token wiring is confirmed, happy to approve.
shreyas-lyzr
left a comment
There was a problem hiding this comment.
Lightweight and well-scoped CI addition. A few things worth checking before merge:
-
tesslio/skill-review@main pins to the default branch tip, not a specific SHA or tag. Any upstream change can silently alter review behavior or break the workflow. Consider pinning to a specific SHA or release tag for reproducibility.
-
The workflow triggers on pull_request events, but the pull-requests: write permission will not be available on fork PRs due to GitHubs security model. The comment posting will silently fail on fork-originated PRs. Worth noting this or switching to pull_request_target if fork coverage is desired.
-
The checkout step runs before skill-review, which is correct. Verify the action does not require additional setup beyond checkout.
If tesslio/skill-review handles its own requirements and fork PRs are out of scope, this looks good.
shreyas-lyzr
left a comment
There was a problem hiding this comment.
Lightweight and well-scoped CI addition. Three things to address:
- Pin tesslio/skill-review to a SHA or tag rather than main to avoid unexpected behavior changes from upstream.
- pull-requests: write permission does not apply to fork PRs under GitHubs security model, so comment posting will silently fail on forks. Document this limitation or consider pull_request_target if needed.
- Confirm that the tesslio/skill-review action does not require additional setup beyond the checkout step already present.
Otherwise this is a good quality gate to add.
Hullo! Thanks for merging the skill improvements earlier. This is a follow-up that adds a lightweight GitHub Action to automatically review any
SKILL.mdfiles when they're changed in a PR, using tessl skill review.**/SKILL.mdpull-requests: writeandcontents: readThis way you and your contributors get an instant quality signal on skill changes before manual review — no signup or tokens needed.