Skip to content

docs(skills): add E2E CI architecture, queue dashboard, and skill INDEX #394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
castrojo wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs(skills): add E2E CI architecture, queue dashboard, and skill INDEX #394

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/skills/INDEX.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# docs/skills — Index

Agent skill docs for the `projectbluefin/common` repo.

| File | What it covers |
|---|---|
| [hive-review.md](hive-review.md) | `~/src/hive-status` — session start, P0/P1 triage, hive label taxonomy |
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify whether the linked docs file exists in the repository.
fd -i '^hive-review\.md$' docs

Repository: projectbluefin/common

Length of output: 47

Fix dead docs link: hive-review.md is missing under docs/

The docs/skills/INDEX.md entry links to hive-review.md, but no hive-review.md file is present anywhere in docs/ (the link target appears to be a dead reference).

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/skills/INDEX.md` at line 7, The INDEX entry in docs/skills/INDEX.md
points to hive-review.md which doesn't exist; fix by either adding the missing
docs/skills/hive-review.md file with the intended content or updating the link
in docs/skills/INDEX.md to the correct existing document path/title (ensure the
target filename `hive-review.md` or its replacement is present in docs/ so the
link is not dead and docs build/links pass).

| [queue-dashboard.md](queue-dashboard.md) | queue.projectbluefin.io — PR tiers, merge ruleset (2 approvals), refresh cadence |
| [e2e-ci.md](e2e-ci.md) | Post-merge E2E CI architecture — common suite, brew tools masked in CI, MOTD fix, known quarantined scenarios |
94 changes: 94 additions & 0 deletions docs/skills/e2e-ci.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# E2E CI Architecture — projectbluefin/common

Reference for agents working on test failures in the post-merge E2E suite.

## How the common suite runs

The common post-merge E2E (`common/.github/workflows/e2e.yml`) calls the reusable
workflow at `projectbluefin/testsuite/.github/workflows/e2e.yml@main` with `suites: common`.

**The common suite is special** — it runs behave directly on the GHA runner (not inside
a pre-built container). The runner SSHes to the VM at `127.0.0.1:2222` as `bluefin-test`.

```yaml
# testsuite e2e.yml (simplified)
if: env.SUITE == 'common'
run: |
python3 -m pip install behave
behave tests/common/features/
else
# Load pre-built runner container, then run behave inside it
```

The pre-built runner container does **not** affect the common suite because of this conditional.

## VM kernel args

```
systemd.mask=brew-setup.service
```
Comment on lines +27 to +29
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add a language hint to the fenced code block.

This block is missing a fence language (MD040). Use text (or another appropriate lexer) to satisfy markdown linting.

Suggested fix 
-```
+```text
 systemd.mask=brew-setup.service
</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 27-27: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/skills/e2e-ci.md` around lines 27 - 29, The fenced code block containing
"systemd.mask=brew-setup.service" is missing a language hint (MD040); update
that markdown block in docs/skills/e2e-ci.md by adding a fence language (e.g.,
use "text") so the block becomes ```text ... ``` and satisfies the markdown
linter while preserving the exact inner content.


`brew-setup.service` is masked for every test VM. This means:

- Homebrew itself is present at `/home/linuxbrew/.linuxbrew/bin/brew`
- **Brew packages are NOT installed** — `eza`, `fd`, `rg`, `bat`, `fzf`, `starship`, etc.
are all from `cli.Brewfile` and require brew-setup to run first
- Any test that checks for these tools will fail in CI

## Known CI failures and their status

### brew CLI tools (eza, fd, ripgrep, bat, fzf, starship) — QUARANTINED

**Root cause:** These are `cli.Brewfile` packages. `brew-setup.service` is masked.
The `ublue-os/brew` image only ships a bare Homebrew tarball at `/usr/share/homebrew.tar.zst`.

**Status:** Quarantined in `tests/common/features/common_shell.feature` via `@quarantine`.
Tracking issue: projectbluefin/testsuite#210

**Options to unblock:**
1. Un-mask `brew-setup.service` in CI (adds ~60s)
2. Install tools as RPMs in the Containerfile
3. Add a CI step to `brew bundle install` before the suite runs

### zsh / fish — QUARANTINED

**Root cause:** zsh and fish ARE installed as RPMs (`/usr/bin/zsh`, `/usr/bin/fish`).
However, under the `bash -lc '...brew_shellenv...; zsh --version'` SSH command that
`environment.py` uses, they return `command not found`. PATH does not include
`/usr/bin` in this non-interactive login context for the fresh `bluefin-test` user.

**Status:** Quarantined. Tracking issue: projectbluefin/testsuite#210

### Dakota MOTD — FIXED (testsuite PR #208)

**Root cause:** `run_ssh()` in `ssh_steps.py` wraps commands in `bash -lc` (login shell)
when `ssh_command_prefix` is set. This triggers `/etc/profile.d/ublue-motd.sh`, which
calls `ublue-motd` and prints the MOTD to stdout. The `vm_reachable_over_ssh` step
checked `stdout == "ok"` (exact match) but got MOTD prepended.

Only Dakota prints a MOTD; Bluefin stable/LTS pass unaffected.

**Fix (testsuite PR #208):**
- Creates `~/.config/no-show-user-motd` for the CI user in VM setup
- Changes assertion to `stdout.strip().split('\n')[-1] == "ok"`

## SSH command execution model

```python
# testsuite/tests/shared/ssh_steps.py
# When context.ssh_command_prefix is set (always true for common suite):
cmd_wrapped = f"bash -lc {shlex.quote(f'{prefix}; {cmd}')}"
```

`environment.py` sets `ssh_command_prefix` to include `eval $(brew shellenv)` so brew
tools are on PATH inside the VM. Side effect: login shell triggers profile.d scripts.

## Images tested

| Job | Image |
|---|---|
| E2E — Bluefin LTS | `ghcr.io/ublue-os/bluefin:lts` |
| E2E — Bluefin Stable | `ghcr.io/ublue-os/bluefin:latest` |
| E2E — Dakota | `ghcr.io/projectbluefin/dakota:latest` |

All three run `suites: common` only. Post-merge only — not triggered on PRs.
54 changes: 54 additions & 0 deletions docs/skills/queue-dashboard.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Queue Dashboard — queue.projectbluefin.io

Reference for agents reviewing PRs and checking merge readiness.

## What it shows

https://queue.projectbluefin.io/ is a static site regenerated hourly by GitHub Actions
from `projectbluefin/queue`. It shows:

- **P0 / P1 issues** — hive-tracked blockers and this-cycle items across the org
- **PR tiers** — open PRs bucketed by review state
- **Victories** — merged PRs and closed issues (7-day and 30-day windows)

## PR tiers (how your PR appears)

The generator queries GitHub search with these filters:

```
is:pr is:open -is:draft status:success review:approved → ✅ Approved (ready to merge)
is:pr is:open -is:draft status:success review:required → ⚠️ Needs reviews
is:pr is:open -is:draft status:success review:none → 🔵 No reviews yet
```
Comment on lines +18 to +22
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Specify a language for the query filter code fence.

This fence triggers MD040 because it has no language. text is sufficient here.

Suggested fix 
-```
+```text
 is:pr is:open -is:draft status:success review:approved   → ✅ Approved (ready to merge)
 is:pr is:open -is:draft status:success review:required   → ⚠️  Needs reviews
 is:pr is:open -is:draft status:success review:none       → 🔵  No reviews yet
</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 18-18: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/skills/queue-dashboard.md` around lines 18 - 22, The code fence in
docs/skills/queue-dashboard.md is missing a language tag which triggers MD040;
update the triple-backtick fence that contains the PR query examples so it
starts with ```text (i.e., change the opening ``` to ```text) to explicitly mark
the block as plain text and satisfy the linter while keeping the content
unchanged.


**Key:** `review:approved` only turns green when ALL required approvals are satisfied.
`review:required` means at least one review is still needed.

## Merge ruleset for projectbluefin/common

Ruleset: `main-review-required-with-renovate-bypass`

- **Required approvals: 2** — one review leaves a PR in `review:required`, not `review:approved`
- **Required status check: `Build and push image`** only
- Smoke tests (`E2E — */GNOME 50 — smoke`) are **not** required checks — stale failures
from prior runs don't block merge
- Renovate PRs bypass the review requirement (auto-merge eligible)

## When a PR has one review

`review:required` → appears in the yellow "Needs reviews" tier on the dashboard.
A second human approval moves it to the green "Approved" tier.

## Refresh cadence

- Regenerated hourly via GitHub Actions in `projectbluefin/queue`
- `hive-progress-sync.yml` in this repo posts Common-specific stats to the org project board
at :20 past the hour (staggered from bluefin :15, dakota :00, knuckle :30, lts :45)

## Repos tracked

`bluefin`, `bluefin-lts`, `common`, `dakota`, `actions`, `renovate-config`, `bonedigger`, `knuckle`

## Source

Generator: https://github.com/projectbluefin/queue/blob/main/generate.js
Loading