Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ jobs:
sudo mv wfctl /usr/local/bin/wfctl
- name: Test
run: GOWORK=off go test ./...
- name: Checkout workflow-compute for provider alignment
uses: actions/checkout@v4
with:
repository: GoCodeAlone/workflow-compute
token: ${{ secrets.RELEASES_TOKEN }}
path: workflow-compute
- name: Check workflow-compute provider alignment
run: scripts/check-workflow-compute-alignment.sh "$GITHUB_WORKSPACE/workflow-compute"
- name: Validate workflow config
run: wfctl validate workflow.yaml
- name: Build with wfctl
Expand Down
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Workflow external plugin for dispatching work to
[`workflow-compute`](https://github.com/GoCodeAlone/workflow-compute).

The plugin is the Workflow-facing adapter. It should provide modules and steps
for compute providers, pools, dispatch, waiting, and fanout while delegating
orchestration, leasing, proof verification, accounting, and dashboard state to
the core compute service.
for compute providers, pools, provider contract catalogs, dispatch, waiting, and
fanout while delegating orchestration, leasing, proof verification, accounting,
and dashboard state to the core compute service.

## Intended Use

Expand All @@ -33,6 +33,10 @@ wfcompute control plane." It is not a wfcompute worker/provider node. Provider
nodes, supervisors, package updates, proof verification, rewards, and dashboard
state belong to `workflow-compute`.

`compute.provider_catalog` consumes `workflow-compute/pkg/protocol.ProviderContract`
records. It intentionally does not define a separate plugin-local executor,
dependency, verification, reward, or network provider shape.

If the wfcompute control plane exposes a public client surface, it should expose
only the scoped APIs needed by external Workflow clients, such as task submit,
task status, proof reads, credential lifecycle, and readiness. Provider
Expand Down
5 changes: 5 additions & 0 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,15 @@ C7: Standalone repo verification uses `GOWORK=off` unless parent `go.work` inclu
C8: `wfctl compute` CLI adapter owns operator UX only; scheduler/ledger/proof semantics stay core.
C9: External Workflow apps may use plugin outside wfcompute deployment/network if they can reach scoped control-plane client APIs.
C10: Public client control-plane access ≠ provider/admin mutation ingress.
C11: Plugin provider catalog details track `workflow-compute`'s typed `ProviderContract`, not a parallel plugin-local provider shape.

§I

repo: `workflow-plugin-compute` → Workflow external plugin adapter
core: `workflow-compute` → scheduler, worker, ledger, proof, reward, dashboard
module: `compute.provider` → control-plane connection + auth refs
module: `compute.pool` → org/pool/policy defaults
module: `compute.provider_catalog` → core `protocol.ProviderContract` declarations
step: `step.compute_dispatch` → submit task
step: `step.compute_wait` → wait/read proof
step: `step.compute_map` → fanout deterministic task set
Expand All @@ -51,6 +53,7 @@ V15: `wfctl compute submit` supports command/container-build without leaking wor
V16: `wfctl compute accounting export` includes raw contribution units and policy reward outputs without leaking token
V17: docs/examples distinguish `compute.provider` Workflow connection from wfcompute provider/worker node
V18: plugin guidance for public control-plane use excludes bootstrap token, provider mutation, package/campaign/trust-root mutation, and raw agent/supervisor control APIs
V19: PR CI checks plugin provider catalog tests against current `GoCodeAlone/workflow-compute` main with a local module replace

§T

Expand All @@ -65,6 +68,7 @@ T7|x|add `wfctl compute github-runner` adapter commands for runner register/job
T8|x|add `wfctl compute submit command|container-build` for ad hoc workload demos|I.cmd,I.wfctl,C8,V10,V12,V15
T9|x|include rewards in `wfctl compute accounting export`|I.cmd,I.wfctl,C8,V10,V16
T10|x|document external Workflow client use cases and public client-surface boundary|C9,C10,V17,V18
T11|x|align provider catalog details with workflow-compute `ProviderContract` and gate drift in PR CI|C11,I.module,V19

§B

Expand All @@ -77,3 +81,4 @@ B5|2026-05-09|CI exposed `compute_map` timeout test receiving raw context-deadli
B6|2026-05-10|live `wfctl compute run` task used CLI-specific signature key rejected by core v0 verifier|V11
B7|2026-05-10|review found token-bearing CLI could post to cleartext non-loopback `http://` server|V12
B8|2026-05-10|review found `wfctl compute run` stdout included full workload + signature envelope|V13
B9|2026-05-20|plugin provider catalog draft used grouped executor/dependency/proof/reward/network details while workflow-compute had moved to typed `ProviderContract`; plugin manifest also omitted the new module|C11,V19
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ toolchain go1.26.2

require (
github.com/GoCodeAlone/workflow v0.27.0
github.com/GoCodeAlone/workflow-compute v0.0.0-20260509213854-e62dca3c1662
github.com/GoCodeAlone/workflow-compute v0.0.0-20260520232556-8e112579a8c0
)

require (
Expand Down Expand Up @@ -115,7 +115,7 @@ require (
github.com/itchyny/timefmt-go v0.1.7 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgx/v5 v5.9.1 // indirect
github.com/jackc/pgx/v5 v5.9.2 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jcmturner/aescts/v2 v2.0.0 // indirect
github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect
Expand Down
4 changes: 4 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ github.com/GoCodeAlone/workflow v0.27.0 h1:oufPjwWTuwbZw5PckBEDrIah+w7JJcj51nKQz
github.com/GoCodeAlone/workflow v0.27.0/go.mod h1:Ue+5YDScTZgtA36q6r/kDaIRxGJFkyxXbeyJVNVJ0Cc=
github.com/GoCodeAlone/workflow-compute v0.0.0-20260509213854-e62dca3c1662 h1:nC/EvC0w5KQpCVbzxN+d2iBEbmAunp/0qchf3cUENto=
github.com/GoCodeAlone/workflow-compute v0.0.0-20260509213854-e62dca3c1662/go.mod h1:Loml3Kueb3XI4Nh+CIPR55dUlLXkd/bnfsE3rf/WVlw=
github.com/GoCodeAlone/workflow-compute v0.0.0-20260520232556-8e112579a8c0 h1:gQC+CIChZZFvUwQ4omKHz+fZ+GscRhZcUyoV641U9C8=
github.com/GoCodeAlone/workflow-compute v0.0.0-20260520232556-8e112579a8c0/go.mod h1:m1GFY/28DcdOp2ok+tTlvqnJqNYhWk5cwJs/8zUFMh4=
github.com/GoCodeAlone/yaegi v0.17.2 h1:WK6Y6e0t1a6U7r+S2dN3CGWW1PizYD3zO0zneToZPxM=
github.com/GoCodeAlone/yaegi v0.17.2/go.mod h1:z5Pr6Wse6QJcQvpgxTxzMAevFarH0N37TG88Y9dprx0=
github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.32.0 h1:rIkQfkCOVKc1OiRCNcSDD8ml5RJlZbH/Xsq7lbpynwc=
Expand Down Expand Up @@ -285,6 +287,8 @@ github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7Ulw
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.9.1 h1:uwrxJXBnx76nyISkhr33kQLlUqjv7et7b9FjCen/tdc=
github.com/jackc/pgx/v5 v5.9.1/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/pgx/v5 v5.9.2 h1:3ZhOzMWnR4yJ+RW1XImIPsD1aNSz4T4fyP7zlQb56hw=
github.com/jackc/pgx/v5 v5.9.2/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jcmturner/aescts/v2 v2.0.0 h1:9YKLH6ey7H4eDBXW8khjYslgyqG2xZikXP0EQFKrle8=
Expand Down
79 changes: 72 additions & 7 deletions internal/module.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"strings"
"time"

"github.com/GoCodeAlone/workflow-compute/pkg/protocol"
sdk "github.com/GoCodeAlone/workflow/plugin/external/sdk"
)

Expand Down Expand Up @@ -73,12 +74,13 @@ func (m *providerModule) Stop(context.Context) error {
}

type poolConfig struct {
ProviderRef string `json:"provider_ref"`
OrgID string `json:"org_id"`
PoolID string `json:"pool_id"`
PolicyID string `json:"policy_id"`
Mode string `json:"mode"`
Labels map[string]string `json:"labels,omitempty"`
ProviderRef string `json:"provider_ref"`
ProviderCatalogRef string `json:"provider_catalog_ref,omitempty"`
OrgID string `json:"org_id"`
PoolID string `json:"pool_id"`
PolicyID string `json:"policy_id"`
Mode string `json:"mode"`
Labels map[string]string `json:"labels,omitempty"`
}

type poolModule struct {
Expand Down Expand Up @@ -157,11 +159,74 @@ func poolModuleSchema() sdk.ModuleSchemaData {
Category: "Compute",
Description: "Default org, pool, and policy routing for compute tasks.",
ConfigFields: []sdk.ConfigField{
{Name: "provider_ref", Type: "string", Description: "Name of the compute.provider module.", Required: true},
{Name: "provider_ref", Type: "string", Description: "Name of the compute.provider control-plane module.", Required: true},
{Name: "provider_catalog_ref", Type: "string", Description: "Optional name of the compute.provider_catalog module that constrains provider policy."},
{Name: "org_id", Type: "string", Description: "Organization id for submitted work.", Required: true},
{Name: "pool_id", Type: "string", Description: "Pool id for submitted work.", Required: true},
{Name: "policy_id", Type: "string", Description: "Policy id for submitted work.", Required: true},
{Name: "mode", Type: "string", Description: "Pool mode.", Required: true, Options: []string{"private", "priority", "public"}},
},
}
}

type providerCatalogConfig struct {
Contracts []protocol.ProviderContract `json:"contracts"`
}

type providerCatalogModule struct {
name string
config providerCatalogConfig
}

func newProviderCatalogModule(name string, raw map[string]any) (*providerCatalogModule, error) {
var cfg providerCatalogConfig
if err := decodeStrictMap(raw, &cfg); err != nil {
return nil, fmt.Errorf("compute.provider_catalog %q: %w", name, err)
}
if err := cfg.validate(); err != nil {
return nil, fmt.Errorf("compute.provider_catalog %q: %w", name, err)
}
return &providerCatalogModule{name: name, config: cfg}, nil
}

func (c providerCatalogConfig) validate() error {
var errs []error
if len(c.Contracts) == 0 {
errs = append(errs, errors.New("contracts is required"))
}
seen := make(map[string]int, len(c.Contracts))
for i, contract := range c.Contracts {
if err := contract.Validate(); err != nil {
errs = append(errs, fmt.Errorf("contracts[%d]: %w", i, err))
}
if prior, ok := seen[contract.ID]; ok && contract.ID != "" {
errs = append(errs, fmt.Errorf("contracts[%d].id duplicates contracts[%d]", i, prior))
}
seen[contract.ID] = i
}
return errors.Join(errs...)
}

func (m *providerCatalogModule) Init() error {
return nil
}

func (m *providerCatalogModule) Start(context.Context) error {
return nil
}

func (m *providerCatalogModule) Stop(context.Context) error {
return nil
}

func providerCatalogModuleSchema() sdk.ModuleSchemaData {
return sdk.ModuleSchemaData{
Type: "compute.provider_catalog",
Label: "Compute Provider Catalog",
Category: "Compute",
Description: "Declarative workflow-compute provider contracts validated against the core provider catalog protocol.",
ConfigFields: []sdk.ConfigField{
{Name: "contracts", Type: "array", Description: "workflow-compute ProviderContract records.", Required: true},
},
}
}
119 changes: 116 additions & 3 deletions internal/module_test.go
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
package internal

import (
"encoding/json"
"os"
"strings"
"testing"

"github.com/GoCodeAlone/workflow-compute/pkg/protocol"
sdk "github.com/GoCodeAlone/workflow/plugin/external/sdk"
)

Expand All @@ -13,11 +16,30 @@ func TestModuleTypes(t *testing.T) {
t.Fatal("plugin must implement ModuleProvider")
}
got := modules.ModuleTypes()
if len(got) != 2 || got[0] != "compute.provider" || got[1] != "compute.pool" {
if len(got) != 3 || got[0] != "compute.provider" || got[1] != "compute.pool" || got[2] != "compute.provider_catalog" {
t.Fatalf("module types: got %#v", got)
}
}

func TestPluginManifestModuleTypesMatchRuntime(t *testing.T) {
modules := NewPlugin().(sdk.ModuleProvider)
data, err := os.ReadFile("../plugin.json")
if err != nil {
t.Fatalf("read plugin.json: %v", err)
}
var manifest struct {
Capabilities struct {
ModuleTypes []string `json:"moduleTypes"`
} `json:"capabilities"`
}
if err := json.Unmarshal(data, &manifest); err != nil {
t.Fatalf("decode plugin.json: %v", err)
}
if strings.Join(manifest.Capabilities.ModuleTypes, ",") != strings.Join(modules.ModuleTypes(), ",") {
t.Fatalf("manifest module types %v do not match runtime %v", manifest.Capabilities.ModuleTypes, modules.ModuleTypes())
}
}

func TestProviderModuleStrictConfig(t *testing.T) {
_, err := newProviderModule("bad", map[string]any{
"server_url": "https://compute.example.test",
Expand Down Expand Up @@ -86,10 +108,101 @@ func TestModuleSchemas(t *testing.T) {
t.Fatal("plugin must implement SchemaProvider")
}
got := schemas.ModuleSchemas()
if len(got) != 2 {
if len(got) != 3 {
t.Fatalf("schema count: got %d", len(got))
}
if got[0].Type != "compute.provider" || got[1].Type != "compute.pool" {
if got[0].Type != "compute.provider" || got[1].Type != "compute.pool" || got[2].Type != "compute.provider_catalog" {
t.Fatalf("schemas: got %#v", got)
}
}

func TestProviderCatalogUsesWorkflowComputeProviderContracts(t *testing.T) {
contract := validProviderContract()
raw := map[string]any{
"contracts": []any{toMap(t, contract)},
}
module, err := newProviderCatalogModule("catalog", raw)
if err != nil {
t.Fatalf("newProviderCatalogModule: %v", err)
}
if got := module.config.Contracts[0]; got.PluginID != "workflow-plugin-compute" || got.ProviderID != "workflow-compute-control-plane" {
t.Fatalf("contract tuple: got plugin=%q provider=%q", got.PluginID, got.ProviderID)
}
}

func TestProviderCatalogRejectsLegacyGroupedProviderDetails(t *testing.T) {
_, err := newProviderCatalogModule("catalog", map[string]any{
"executor_providers": []any{map[string]any{
"name": "command",
"type": "command",
"workload_kinds": []any{"command"},
}},
})
if err == nil || !strings.Contains(err.Error(), "executor_providers") {
t.Fatalf("expected strict rejection for legacy grouped provider details, got %v", err)
}
}

func TestProviderCatalogRejectsMalformedWorkflowComputeContract(t *testing.T) {
contract := validProviderContract()
contract.ConfigSchemaDigest = "sha256:not-hex"
_, err := newProviderCatalogModule("catalog", map[string]any{
"contracts": []any{toMap(t, contract)},
})
if err == nil || !strings.Contains(err.Error(), "config_schema_digest") {
t.Fatalf("expected ProviderContract validation error, got %v", err)
}
}

func validProviderContract() protocol.ProviderContract {
return protocol.ProviderContract{
ProtocolVersion: protocol.Version,
ID: "workflow-compute-control-plane-v1",
PluginID: "workflow-plugin-compute",
ProviderID: "workflow-compute-control-plane",
ContractID: "workflow-compute.control-plane.v1",
Version: "v1.0.0",
DisplayName: "workflow-compute Control Plane",
ConfigSchemaRef: "schema://providers/workflow-plugin-compute/workflow-compute-control-plane/v1",
ConfigSchemaDigest: "sha256:" + strings.Repeat("b", 64),
OperatingModes: []protocol.NetworkOperatingMode{protocol.NetworkModeBatch},
WorkloadKinds: []string{string(protocol.WorkloadCommand), string(protocol.WorkloadContainerBuild)},
ExecutorProviders: []string{"sandboxed-command"},
ExecutionSecurityTiers: []protocol.ExecutionSecurityTier{protocol.ExecutionSandboxedContainer},
ProofTiers: []protocol.ProofTier{protocol.ProofArtifactHash},
NetworkModes: []protocol.NetworkMode{protocol.NetworkModeRelay},
RuntimeContract: protocol.ProviderRuntimeContract{
Profiles: []protocol.ProviderRuntimeProfile{{
ID: "sandboxed-command-oci",
RuntimeProfile: protocol.RuntimeProfileSandboxedOCI,
ExecutorProvider: "sandboxed-command",
ExecutionSecurityTier: protocol.ExecutionSandboxedContainer,
ProofTier: protocol.ProofArtifactHash,
AllowedRuntimeTools: []protocol.ContainerRuntimeTool{protocol.ContainerRuntimePodman, protocol.ContainerRuntimeDocker, protocol.ContainerRuntimeNerdctl},
ImageDigestRequired: true,
RootFSDigestRequired: true,
AllowedMountRefs: []string{"workspace"},
WritableRootFS: protocol.RuntimePermissionForbidden,
Privileged: protocol.RuntimePermissionForbidden,
HostNamespaces: protocol.RuntimePermissionForbidden,
HostSocket: protocol.RuntimePermissionForbidden,
SeccompDisable: protocol.RuntimePermissionForbidden,
NoNewPrivilegesDisable: protocol.RuntimePermissionForbidden,
ConformanceProfiles: []string{"sandboxed-oci-v1"},
}},
},
}
}

func toMap(t *testing.T, value any) map[string]any {
t.Helper()
data, err := json.Marshal(value)
if err != nil {
t.Fatalf("marshal value: %v", err)
}
var out map[string]any
if err := json.Unmarshal(data, &out); err != nil {
t.Fatalf("decode value: %v", err)
}
return out
}
4 changes: 4 additions & 0 deletions internal/plugin.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ func (p *computePlugin) ModuleTypes() []string {
return []string{
"compute.provider",
"compute.pool",
"compute.provider_catalog",
}
}

Expand All @@ -36,6 +37,8 @@ func (p *computePlugin) CreateModule(typeName, name string, config map[string]an
return newProviderModule(name, config)
case "compute.pool":
return newPoolModule(name, config)
case "compute.provider_catalog":
return newProviderCatalogModule(name, config)
default:
return nil, fmt.Errorf("compute plugin: unknown module type %q", typeName)
}
Expand Down Expand Up @@ -66,5 +69,6 @@ func (p *computePlugin) ModuleSchemas() []sdk.ModuleSchemaData {
return []sdk.ModuleSchemaData{
providerModuleSchema(),
poolModuleSchema(),
providerCatalogModuleSchema(),
}
}
Loading
Loading