Record and expose the PR number for GitHub jobs

[emilyalbini]

Right now buildomat tracks some metadata through tags in GitHub jobs (for example gong.run.github_id or gong.head.branch).

I'm working on the Omicron integration with Trunk, which requires running their CLI with a secret token to upload JUnit XML reports to their platform. As Buildomat doesn't support secrets, I created a GitHub Actions workflow with the secret that calls a script responsible for fetching all the metadata Trunk requires, the JUnit XML report, and uploading it to their platform.

This works for the most part, except with PRs coming from forks of omicron outside our organization (which some of our colleagues use). This is because the GitHub API (and the API only, not the web UI) doesn't attach PRs coming from forks to a commit (example check run):

$ gh api repos/oxidecomputer/omicron/check-runs/49281946346 | jq '.head_sha, .pull_requests'
"29f079bdce96a0c4ba328c7e5155c7cd2123bee9"
[]

$ gh api repos/oxidecomputer/omicron/commits/29f079bdce96a0c4ba328c7e5155c7cd2123bee9/pulls
[]

GitHub does return the information if I look directly in the fork, but there is also no way to reliably know which fork the commit comes from:

$ gh api repos/karencfv/omicron/commits/cd38f25aafc93560877f7ec4aa8f21ec81488bd6/pulls | jq '.[].number'
8846

On the other hand, Buildomat does receive the PR number attached to a workflow run in the pull_request event. Exposing the PR number in a machine-readable information would make the script more reliable. A possible implementation would be to add a gong.pr.number tag when the pull_request event is received, and returning a JSON representation of a subset of the details page when Accept is not text/html:

$ curl https://buildomat.eng.oxide.computer/wg/0/details/01K5DJN34XQB9MKG2DPV4X4MAS/GolsAfx7CS23Y9l5J9miocea0lK1p0dQdYoQii1LPXrdxXSR/01K5DJNDHFNE5HV56PE56FBJS0
{
    "tags": {
        "gong.run.id": 12345,
        "gong.pr.number": 234,
        # ...
    },
    "artefacts": [
        {
            "name": "path/to/junit.xml",
            "downlaod": "https://..."
        }
    ]
}

[jclulow]

The URLs under /wg are definitely not an API, so much as just a web view of things.

For the chunk of this that runs inside GHA, I assume GitHub generates some kind of short-lived limited token to inject into that environment. Is there some way to use that token to authenticate to a GitHub Application like buildomat?

[emilyalbini]

The URLs under /wg are definitely not an API, so much as just a web view of things.

I guess I just see this as an easier way to parse the information that is already there publicly available. Anyone wanting to access the data today can parse the HTML, it's just more painful. If the proper way we add to retrieve this data turns out to be even more painful to access (for example requiring authentication, even if you already know the URL with the secret in it) I'll probably end up just scraping the HTML.

For the chunk of this that runs inside GHA, I assume GitHub generates some kind of short-lived limited token to inject into that environment. Is there some way to use that token to authenticate to a GitHub Application like buildomat?

There are two tokens GitHub injects into build environments:

• An installation token of the github-actions app, with configurable permissions.
• OIDC token that can be used to authenticate with third party services.

I'm not aware of any way to use the installation token of an app to authenticate as another app, except for very bad ways of doing so (sending that token to buildomat and buildomat probing what that token can do, let's not do this).

We could implement OIDC authentication in buildomat, but I'd prefer we don't. Right now the script to upload XML reports to Trunk can be run locally with just a GitHub token (which can be a personal one), and if we were to require OIDC we wouldn't be able to run it locally anymore. I really don't want to lose the ability to debug the script locally.

[jclulow]

The URLs under /wg are definitely not an API, so much as just a web view of things.

I guess I just see this as an easier way to parse the information that is already there publicly available. Anyone wanting to access the data today can parse the HTML, it's just more painful. If the proper way we add to retrieve this data turns out to be even more painful to access (for example requiring authentication, even if you already know the URL with the secret in it) I'll probably end up just scraping the HTML.

I would not recommend it. The structure of the URLs and the HTML in the pages is, as I said, not an API. It will change in the future. Presumably that will break the program if it relies on it.

buildomat has an actual stable API, it's mostly just a question of crafting an appropriate authentication and authorisation situation for new use cases. We have, to date, side-stepped a lot of those issues by relying on the authorisation implied by, say, webhook deliveries and by poking at the GitHub API itself. But that's definitely not the only way to interact with the system.

even if you already know the URL with the secret in it

How do you know what the URL is already? Presumably this is also via some unstable scraped interface?

We could implement OIDC authentication in buildomat, but I'd prefer we don't. Right now the script to upload XML reports to Trunk can be run locally with just a GitHub token (which can be a personal one), and if we were to require OIDC we wouldn't be able to run it locally anymore.

I had a quick squiz and I believe OIDC is the way to go, alas. In theory one ought to be able to parlay the context of an appropriately privileged GitHub Actions environment into a similar level of buildomat API access.

I really don't want to lose the ability to debug the script locally.

I don't believe the program needs to be undebuggable, FWIW.

I'll get some more details together on the OIDC situation soon.

[emilyalbini]

How do you know what the URL is already? Presumably this is also via some unstable scraped interface?

Buildomat configures it as the "See more details" URL, making it available on the GitHub API when looking up the check run (without requiring any authentication with GitHub either, since Omicron is a public repository):

$ curl -s https://api.github.com/repos/oxidecomputer/omicron/check-runs/49281946346 | jq .details_url -r
https://buildomat.eng.oxide.computer/wg/0/details/01K40SBYH10EC5CYZ4YGA6TYFE/npqSZwRbYLQZf1XfB7TLEX0Foih9TvOmPpxJo1nikyPq2KJL/01K40SC85BVH6KYVRNH6ACAQWR

I had a quick squiz and I believe OIDC is the way to go, alas. In theory one ought to be able to parlay the context of an appropriately privileged GitHub Actions environment into a similar level of buildomat API access.

Sure, long term that's probably the better option! But right now that information is available without the need of any buildomat credential if you know the secret URL, and rendering a subset of that same endpoint in JSON would avoid the implementation complexity of OIDC in the short term (when everyone is busy with other projects).

I don't believe the program needs to be undebuggable, FWIW.

Buildomat credentials could be configured for me to test the script, but as far as I know most of the company doesn't have them right now. Ideally I'd like for anyone to be able to run the script locally and debug it (without having to ask for a set of credentials). The current status is that anyone can create their own GitHub token in the UI and test the script end to end.