Skip to content

Commit 2b77dcc

Browse files
ewegaewega
authored
Release 0.3.7: supported plugins + env cleanup (#53)
* refactor: make CLI tool-agnostic, fix Copilot test endpoint Major refactor to remove GitHub/Copilot-specific assumptions across the CLI: Bug fixes: - Add Name field to ConnectionTestRequest (fixes Copilot test 500 error) - Write early state checkpoint in deploy_azure after RG creation - Allow cleanup --resource-group without state file ConnectionDef as single source of truth: - Add RateLimitPerHour, EnableGraphql, TokenPrompt, OrgPrompt, EnterprisePrompt fields to ConnectionDef - BuildCreateRequest/BuildTestRequest read fields from def (no plugin branches) - pluginDisplayName sources from ConnectionDef registry Token resolution data-driven: - Rewrite token.Resolve to accept ResolveOpts struct - Remove pluginEnvFileKeys/pluginEnvVarKeys/pluginDisplayName switch funcs - Callers pass def.EnvFileKeys, def.EnvVarNames, def.DisplayName directly Per-plugin token + org resolution: - runConnectionsInternal resolves token, org, enterprise per-plugin - Remove upfront GitHub org prompt from init wizard - Org/enterprise derived from connection results for downstream phases Tool-agnostic strings (~25 replacements): - Flag descriptions: Organization slug, Enterprise slug, PAT - Plugin lists generated dynamically from availablePluginSlugs() - Error messages use registry data, not hardcoded plugin names Generic project descriptions: - Replace HasGitHub/HasCopilot booleans with PluginNames []string - Project description built from active plugin display names Scope handling cleanup: - Remove SkipGitHub/SkipCopilot from ScopeOpts - runConfigureScopes uses single-plugin flow with generic validation - ensureScopeConfig accepts plugin parameter (no hardcoded github) - discoverConnections iterates AvailableConnections() dynamically * refactor: replace github/copilot-connection-id with generic --connection-id * docs: update AGENTS.md design principles, add plugin-registry skill * docs: clarify supported plugins and token/state files * cli: cleanup env files and align token resolution
1 parent e4a304e commit 2b77dcc

34 files changed

Lines changed: 3137 additions & 2107 deletions

‎.github/skills/plugin-registry/SKILL.md‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
---
2+
name: plugin-registry
3+
description: How the ConnectionDef plugin registry works — adding plugins, token resolution, scope handling, and the tool-agnostic design. Use when adding a new DevOps tool plugin, modifying ConnectionDef fields, changing how connections/scopes/tokens are resolved, or debugging plugin-specific behavior.
4+
---
5+
6+
# Plugin Registry — ConnectionDef as Single Source of Truth
7+
8+
## ConnectionDef Fields
9+
10+
Each plugin is a `ConnectionDef` struct in `cmd/connection_types.go`:
11+
12+
```go
13+
type ConnectionDef struct {
14+
Plugin string // DevLake plugin slug (e.g. "github", "gh-copilot")
15+
DisplayName string // User-facing name (e.g. "GitHub Copilot")
16+
Available bool // false = "coming soon" in menus
17+
Endpoint string // Default API endpoint
18+
NeedsOrg bool // Prompt for org during connection creation
19+
NeedsEnterprise bool // Prompt for enterprise during connection creation
20+
SupportsTest bool // Test connection before creating
21+
RateLimitPerHour int // API rate limit (0 = default 4500)
22+
EnableGraphql bool // Send enableGraphql=true in payloads
23+
RequiredScopes []string // PAT scopes for documentation
24+
ScopeHint string // Displayed in token prompt
25+
TokenPrompt string // Label for masked token prompt
26+
OrgPrompt string // Label for org prompt (empty = not prompted)
27+
EnterprisePrompt string // Label for enterprise prompt (empty = not prompted)
28+
EnvVarNames []string // Environment variables for token resolution
29+
EnvFileKeys []string // .devlake.env keys for token resolution
30+
}
31+
```
32+
33+
## Adding a New Plugin
34+
35+
1. Add a `ConnectionDef` entry to `connectionRegistry` in `cmd/connection_types.go`
36+
2. Set `Available: true` when ready (false = "coming soon")
37+
3. Add a scope function (e.g. `scopeGitLab`) in `cmd/configure_scopes.go`
38+
4. Add a `case` in the scope dispatch switch in `init.go`, `configure_full.go`, and `configure_scopes.go`
39+
40+
No other registration needed — token resolution, connection creation, help text, and menu labels all derive from the `ConnectionDef` fields.
41+
42+
## Token Resolution Chain
43+
44+
Token resolution is fully data-driven via `token.ResolveOpts`:
45+
46+
```go
47+
token.Resolve(token.ResolveOpts{
48+
FlagValue: flagToken,
49+
EnvFilePath: envFile,
50+
EnvFileKeys: def.EnvFileKeys, // e.g. ["GITHUB_PAT", "GITHUB_TOKEN"]
51+
EnvVarNames: def.EnvVarNames, // e.g. ["GITHUB_TOKEN", "GH_TOKEN"]
52+
DisplayName: def.DisplayName, // e.g. "GitHub Copilot"
53+
ScopeHint: def.ScopeHint, // e.g. "manage_billing:copilot, read:org"
54+
})
55+
```
56+
57+
Priority: `--token` flag → `.devlake.env` file → environment variable → interactive masked prompt.
58+
59+
There are NO switch/case statements on plugin names in the token package. All lookup keys come from the caller (ConnectionDef fields).
60+
61+
## Build/Test Request Construction
62+
63+
`BuildCreateRequest` and `BuildTestRequest` on `ConnectionDef` read fields directly:
64+
65+
- `RateLimitPerHour` → `req.RateLimitPerHour` (default 4500 if 0)
66+
- `EnableGraphql` → `req.EnableGraphql`
67+
- `NeedsOrg` → conditionally includes `req.Organization`
68+
- `NeedsEnterprise` → conditionally includes `req.Enterprise`
69+
70+
No `if d.Plugin == "github"` branches — behavior is declarative.
71+
72+
The `ConnectionTestRequest` includes a `Name` field (required by some plugins like gh-copilot for validation).
73+
74+
## Scope Handling
75+
76+
Each plugin has its own scope function:
77+
78+
| Plugin | Scope Function | Data Scope |
79+
|--------|---------------|------------|
80+
| `github` | `scopeGitHub()` | Repos (via `gh repo list` or `--repos` flag) |
81+
| `gh-copilot` | `scopeCopilot()` | Org/enterprise (computed as scope ID) |
82+
83+
### Copilot Scope ID Convention
84+
85+
```go
86+
func copilotScopeID(org, enterprise string) string
87+
// enterprise + org → "enterprise/org"
88+
// enterprise only → "enterprise"
89+
// org only → "org"
90+
```
91+
92+
This must match `listGhCopilotRemoteScopes` in the DevLake backend exactly.
93+
94+
### Scope Config
95+
96+
Each plugin has different scope config shapes:
97+
- **GitHub**: `deploymentPattern`, `productionPattern`, `issueTypeIncident` (DORA patterns)
98+
- **Copilot**: `baselinePeriodDays`, `implementationDate`
99+
100+
`ensureScopeConfig(client, plugin, connID, opts)` accepts the plugin string as a parameter.
101+
102+
## Per-Plugin Resolution in Orchestrators
103+
104+
`runConnectionsInternal` (used by `configure full` and `init`) resolves per-plugin:
105+
106+
```
107+
for each selected ConnectionDef:
108+
1. Resolve token using def.EnvVarNames, def.TokenPrompt
109+
2. Prompt for org if def.NeedsOrg (using def.OrgPrompt)
110+
3. Prompt for enterprise if def.NeedsEnterprise (using def.EnterprisePrompt)
111+
4. Test & create connection
112+
```
113+
114+
This supports mixed-plugin flows where different plugins need different tokens (e.g. GitHub + GitLab).
115+
116+
## Scope ID Resolution in Projects
117+
118+
`listConnectionScopes()` resolves scope IDs differently per plugin:
119+
- GitHub: uses `githubId` (int) from scope response
120+
- Copilot: uses `id` (string) from scope response
121+
122+
This is pragmatic — DevLake's scope API returns plugin-specific ID fields.
123+
124+
## State Files
125+
126+
Connection results are saved to state files (`.devlake-azure.json` or `.devlake-local.json`):
127+
128+
```go
129+
StateConnection{
130+
Plugin: "gh-copilot",
131+
ConnectionID: 1,
132+
Name: "GitHub Copilot - my-org",
133+
Organization: "my-org",
134+
Enterprise: "my-enterprise",
135+
}
136+
```
137+
138+
State enables command chaining — `configure scope` and `configure project` read connection IDs from state.
139+
140+
## Common Pitfalls
141+
142+
- Forgetting to set `RateLimitPerHour` → defaults to 4500, which may be wrong for some plugins
143+
- Not setting `TokenPrompt` → generic "PAT" label in prompts
144+
- Copilot scope ID must match backend's `listGhCopilotRemoteScopes` logic exactly
145+
- On Windows, always build with `-o gh-devlake.exe` — PowerShell resolves `.exe` preferentially

‎AGENTS.md‎

Lines changed: 19 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,15 @@ internal/
3030
- **Generic API helpers**: `doPost[T]`, `doGet[T]`, `doPut[T]`, `doPatch[T]` in `internal/devlake/client.go`
3131

3232
### Plugin System
33-
Plugins are defined via `ConnectionDef` structs in `cmd/connection_types.go`. Each entry declares the plugin slug, endpoint, required fields (`NeedsOrg`, `NeedsEnterprise`), and PAT scopes. To add a new DevOps tool, add a `ConnectionDef` to `connectionRegistry` — no other registration needed.
33+
Plugins are defined via `ConnectionDef` structs in `cmd/connection_types.go`. Each entry declares the plugin slug, endpoint, rate limits, prompt labels, and PAT resolution keys. To add a new DevOps tool, add a `ConnectionDef` to `connectionRegistry` — token resolution, org prompts, and connection creation all derive from these fields automatically. See the `plugin-registry` skill for full details.
3434

35-
**One plugin per invocation.** Flag-based commands target a single `--plugin`. Interactive mode walks through plugins sequentially. This keeps plugin-specific fields (org, enterprise, repos, tokens) self-contained.
35+
**One plugin per invocation.** Flag-based commands target a single `--plugin`. Interactive mode walks through plugins sequentially.
36+
37+
### Design Principles
38+
- **Tool-agnostic**: No hardcoded plugin names outside `connectionRegistry` and plugin-specific scope functions
39+
- **Per-plugin resolution**: Orchestrators resolve token, org, and enterprise independently for each plugin
40+
- **Declarative over imperative**: Plugin behavior comes from `ConnectionDef` fields, not switch/case branches
41+
- **Interactive orchestrators**: `init` and `configure full` are interactive-only; flag-driven automation uses individual commands
3642

3743
### Copilot Scope ID Convention
3844
The `gh-copilot` plugin computes scope IDs as: enterprise + org → `"enterprise/org"`, enterprise only → `"enterprise"`, org only → `"org"`. See `copilotScopeID()` in `cmd/configure_scopes.go`. The scope ID must match the plugin's `listGhCopilotRemoteScopes` logic exactly or blueprint references will break.
@@ -105,6 +111,10 @@ import (
105111
- Command-specific flags as package-level vars with `cmd.Flags().StringVar()`
106112
- Required flags validated in `RunE` (not via `MarkFlagRequired`)
107113

114+
## Documentation
115+
116+
Each command group has a reference file in `docs/`. When adding a new command, create or update the matching `docs/<command>.md` and add a row to the Command Reference table in `README.md`.
117+
108118
## Testing
109119

110120
- Unit tests: `*_test.go` alongside source
@@ -114,10 +124,12 @@ import (
114124
## Build & Run
115125

116126
```bash
117-
go build -o gh-devlake . # Build the extension
127+
go build -o gh-devlake.exe . # Windows (always use .exe suffix)
128+
go build -o gh-devlake . # Linux/macOS
118129
gh extension install . # Install locally for testing
119-
gh devlake status # Run via gh CLI
120-
gh devlake configure connection # Create a plugin connection
121-
gh devlake configure scope # Configure collection scopes
122-
gh devlake configure project # Create a project and start data collection
130+
gh devlake init # Guided wizard (deploy + configure)
131+
gh devlake configure full # Configure connections + scopes + project
132+
gh devlake status # Health check and connection summary
123133
```
134+
135+
> **Windows**: Always build with `-o gh-devlake.exe`. PowerShell resolves `.exe` preferentially — a stale `.exe` will shadow a freshly built binary without the extension.

0 commit comments

Comments
 (0)