Skip to content

Accept engine pseudo-versions in plugin evidence#631

Merged
intel352 merged 1 commit into
mainfrom
codex/plugin-conformance-pseudo-version
May 11, 2026
Merged

Accept engine pseudo-versions in plugin evidence#631
intel352 merged 1 commit into
mainfrom
codex/plugin-conformance-pseudo-version

Conversation

@intel352
Copy link
Copy Markdown
Contributor

Summary

  • allow CanonicalEngineVersion to accept Go pseudo-versions such as v0.51.3-0.20260511085732-8246535de8bd
  • keep plugin release versions strict, so plugin version metadata is not widened
  • cover release, pseudo, invalid, and plugin-version rejection cases

Why

Plugin conformance CI may need to pin a post-release workflow commit before the next tagged release. Evidence should record the exact engine build under test instead of pretending it was the prior release.

Verification

  • GOWORK=off go test ./cmd/wfctl -run 'TestPluginCompatVersion|TestPluginConformanceLocalJSONPass' -count=1
  • GOWORK=off go test ./cmd/wfctl -run 'TestPluginCompat|TestPluginConformance' -count=1
  • git diff --check

Review

Manual antagonistic pass: checked that only engine versions accept pseudo-version shape; CanonicalPluginVersion still rejects pseudo-versions.

Copilot AI review requested due to automatic review settings May 11, 2026 09:08
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates wfctl’s plugin compatibility evidence model to allow engine versions (only) to be recorded as Go pseudo-versions, enabling conformance CI to pin an exact post-release engine commit while keeping plugin release versions strictly MAJOR.MINOR.PATCH.

Changes:

  • Extend CanonicalEngineVersion to accept either strict semver or a pseudo-version-like semver form.
  • Keep CanonicalPluginVersion strict (release-only) and add a regression test rejecting pseudo-versions for plugins.
  • Add test coverage for engine pseudo-version canonicalization.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
cmd/wfctl/plugin_compat_model.go Adds pseudo-version acceptance logic for engine version canonicalization.
cmd/wfctl/plugin_compat_model_test.go Adds tests covering engine pseudo-versions and ensures plugin versions still reject pseudo-versions.

Comment thread cmd/wfctl/plugin_compat_model.go Outdated
@intel352 intel352 force-pushed the codex/plugin-conformance-pseudo-version branch from a9010b7 to 4ce86b7 Compare May 11, 2026 09:15
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 11, 2026

⏱ Benchmark Results

No significant performance regressions detected.

benchstat comparison (baseline → PR)
## benchstat: baseline → PR
baseline-bench.txt:262: parsing iteration count: invalid syntax
baseline-bench.txt:435176: parsing iteration count: invalid syntax
baseline-bench.txt:855903: parsing iteration count: invalid syntax
baseline-bench.txt:1256035: parsing iteration count: invalid syntax
baseline-bench.txt:1688123: parsing iteration count: invalid syntax
baseline-bench.txt:2105421: parsing iteration count: invalid syntax
benchmark-results.txt:262: parsing iteration count: invalid syntax
benchmark-results.txt:307322: parsing iteration count: invalid syntax
benchmark-results.txt:560094: parsing iteration count: invalid syntax
benchmark-results.txt:873451: parsing iteration count: invalid syntax
benchmark-results.txt:1215071: parsing iteration count: invalid syntax
benchmark-results.txt:1543815: parsing iteration count: invalid syntax
goos: linux
goarch: amd64
pkg: github.com/GoCodeAlone/workflow/dynamic
cpu: AMD EPYC 7763 64-Core Processor                
                            │ benchmark-results.txt │
                            │        sec/op         │
InterpreterCreation-4                 3.438m ± 236%
ComponentLoad-4                       3.610m ±   0%
ComponentExecute-4                    2.000µ ±   1%
PoolContention/workers-1-4            1.099µ ±   2%
PoolContention/workers-2-4            1.097µ ±   2%
PoolContention/workers-4-4            1.101µ ±   0%
PoolContention/workers-8-4            1.105µ ±   1%
PoolContention/workers-16-4           1.107µ ±   2%
ComponentLifecycle-4                  3.627m ±   1%
SourceValidation-4                    2.311µ ±   1%
RegistryConcurrent-4                  792.4n ±   4%
LoaderLoadFromString-4                3.632m ±   1%
geomean                               17.74µ

                            │ benchmark-results.txt │
                            │         B/op          │
InterpreterCreation-4                  2.027Mi ± 0%
ComponentLoad-4                        2.180Mi ± 0%
ComponentExecute-4                     1.203Ki ± 0%
PoolContention/workers-1-4             1.203Ki ± 0%
PoolContention/workers-2-4             1.203Ki ± 0%
PoolContention/workers-4-4             1.203Ki ± 0%
PoolContention/workers-8-4             1.203Ki ± 0%
PoolContention/workers-16-4            1.203Ki ± 0%
ComponentLifecycle-4                   2.183Mi ± 0%
SourceValidation-4                     1.984Ki ± 0%
RegistryConcurrent-4                   1.133Ki ± 0%
LoaderLoadFromString-4                 2.182Mi ± 0%
geomean                                15.25Ki

                            │ benchmark-results.txt │
                            │       allocs/op       │
InterpreterCreation-4                   15.68k ± 0%
ComponentLoad-4                         18.02k ± 0%
ComponentExecute-4                       25.00 ± 0%
PoolContention/workers-1-4               25.00 ± 0%
PoolContention/workers-2-4               25.00 ± 0%
PoolContention/workers-4-4               25.00 ± 0%
PoolContention/workers-8-4               25.00 ± 0%
PoolContention/workers-16-4              25.00 ± 0%
ComponentLifecycle-4                    18.07k ± 0%
SourceValidation-4                       32.00 ± 0%
RegistryConcurrent-4                     2.000 ± 0%
LoaderLoadFromString-4                  18.06k ± 0%
geomean                                  183.3

cpu: AMD EPYC 9V74 80-Core Processor                
                            │ baseline-bench.txt │
                            │       sec/op       │
InterpreterCreation-4               3.954m ± 89%
ComponentLoad-4                     2.720m ± 12%
ComponentExecute-4                  1.429µ ±  1%
PoolContention/workers-1-4          794.6n ±  2%
PoolContention/workers-2-4          786.8n ±  1%
PoolContention/workers-4-4          788.2n ±  1%
PoolContention/workers-8-4          787.2n ±  1%
PoolContention/workers-16-4         788.2n ±  1%
ComponentLifecycle-4                2.713m ±  1%
SourceValidation-4                  1.623µ ±  1%
RegistryConcurrent-4                610.9n ±  6%
LoaderLoadFromString-4              2.732m ±  0%
geomean                             13.43µ

                            │ baseline-bench.txt │
                            │        B/op        │
InterpreterCreation-4               2.027Mi ± 0%
ComponentLoad-4                     2.180Mi ± 0%
ComponentExecute-4                  1.203Ki ± 0%
PoolContention/workers-1-4          1.203Ki ± 0%
PoolContention/workers-2-4          1.203Ki ± 0%
PoolContention/workers-4-4          1.203Ki ± 0%
PoolContention/workers-8-4          1.203Ki ± 0%
PoolContention/workers-16-4         1.203Ki ± 0%
ComponentLifecycle-4                2.183Mi ± 0%
SourceValidation-4                  1.984Ki ± 0%
RegistryConcurrent-4                1.133Ki ± 0%
LoaderLoadFromString-4              2.182Mi ± 0%
geomean                             15.25Ki

                            │ baseline-bench.txt │
                            │     allocs/op      │
InterpreterCreation-4                15.68k ± 0%
ComponentLoad-4                      18.02k ± 0%
ComponentExecute-4                    25.00 ± 0%
PoolContention/workers-1-4            25.00 ± 0%
PoolContention/workers-2-4            25.00 ± 0%
PoolContention/workers-4-4            25.00 ± 0%
PoolContention/workers-8-4            25.00 ± 0%
PoolContention/workers-16-4           25.00 ± 0%
ComponentLifecycle-4                 18.07k ± 0%
SourceValidation-4                    32.00 ± 0%
RegistryConcurrent-4                  2.000 ± 0%
LoaderLoadFromString-4               18.06k ± 0%
geomean                               183.3

pkg: github.com/GoCodeAlone/workflow/middleware
cpu: AMD EPYC 7763 64-Core Processor                
                                  │ benchmark-results.txt │
                                  │        sec/op         │
CircuitBreakerDetection-4                     306.5n ± 6%
CircuitBreakerExecution_Success-4             21.54n ± 1%
CircuitBreakerExecution_Failure-4             66.48n ± 0%
geomean                                       76.00n

                                  │ benchmark-results.txt │
                                  │         B/op          │
CircuitBreakerDetection-4                    144.0 ± 0%
CircuitBreakerExecution_Success-4            0.000 ± 0%
CircuitBreakerExecution_Failure-4            0.000 ± 0%
geomean                                                 ¹
¹ summaries must be >0 to compute geomean

                                  │ benchmark-results.txt │
                                  │       allocs/op       │
CircuitBreakerDetection-4                    1.000 ± 0%
CircuitBreakerExecution_Success-4            0.000 ± 0%
CircuitBreakerExecution_Failure-4            0.000 ± 0%
geomean                                                 ¹
¹ summaries must be >0 to compute geomean

cpu: AMD EPYC 9V74 80-Core Processor                
                                  │ baseline-bench.txt │
                                  │       sec/op       │
CircuitBreakerDetection-4                  230.3n ± 3%
CircuitBreakerExecution_Success-4          17.58n ± 0%
CircuitBreakerExecution_Failure-4          55.14n ± 0%
geomean                                    60.66n

                                  │ baseline-bench.txt │
                                  │        B/op        │
CircuitBreakerDetection-4                 144.0 ± 0%
CircuitBreakerExecution_Success-4         0.000 ± 0%
CircuitBreakerExecution_Failure-4         0.000 ± 0%
geomean                                              ¹
¹ summaries must be >0 to compute geomean

                                  │ baseline-bench.txt │
                                  │     allocs/op      │
CircuitBreakerDetection-4                 1.000 ± 0%
CircuitBreakerExecution_Success-4         0.000 ± 0%
CircuitBreakerExecution_Failure-4         0.000 ± 0%
geomean                                              ¹
¹ summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/module
cpu: AMD EPYC 7763 64-Core Processor                
                                 │ benchmark-results.txt │
                                 │        sec/op         │
JQTransform_Simple-4                        888.5n ± 29%
JQTransform_ObjectConstruction-4            1.488µ ± 12%
JQTransform_ArraySelect-4                   3.353µ ±  1%
JQTransform_Complex-4                       38.35µ ±  1%
JQTransform_Throughput-4                    1.812µ ±  1%
SSEPublishDelivery-4                        66.75n ±  1%
geomean                                     1.655µ

                                 │ benchmark-results.txt │
                                 │         B/op          │
JQTransform_Simple-4                      1.273Ki ± 0%
JQTransform_ObjectConstruction-4          1.773Ki ± 0%
JQTransform_ArraySelect-4                 2.625Ki ± 0%
JQTransform_Complex-4                     16.22Ki ± 0%
JQTransform_Throughput-4                  1.984Ki ± 0%
SSEPublishDelivery-4                        0.000 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

                                 │ benchmark-results.txt │
                                 │       allocs/op       │
JQTransform_Simple-4                        10.00 ± 0%
JQTransform_ObjectConstruction-4            15.00 ± 0%
JQTransform_ArraySelect-4                   30.00 ± 0%
JQTransform_Complex-4                       324.0 ± 0%
JQTransform_Throughput-4                    17.00 ± 0%
SSEPublishDelivery-4                        0.000 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

cpu: AMD EPYC 9V74 80-Core Processor                
                                 │ baseline-bench.txt │
                                 │       sec/op       │
JQTransform_Simple-4                     627.8n ± 33%
JQTransform_ObjectConstruction-4         1.065µ ±  1%
JQTransform_ArraySelect-4                2.622µ ±  1%
JQTransform_Complex-4                    32.00µ ±  1%
JQTransform_Throughput-4                 1.321µ ±  1%
SSEPublishDelivery-4                     50.28n ±  2%
geomean                                  1.245µ

                                 │ baseline-bench.txt │
                                 │        B/op        │
JQTransform_Simple-4                   1.273Ki ± 0%
JQTransform_ObjectConstruction-4       1.773Ki ± 0%
JQTransform_ArraySelect-4              2.625Ki ± 0%
JQTransform_Complex-4                  16.22Ki ± 0%
JQTransform_Throughput-4               1.984Ki ± 0%
SSEPublishDelivery-4                     0.000 ± 0%
geomean                                             ¹
¹ summaries must be >0 to compute geomean

                                 │ baseline-bench.txt │
                                 │     allocs/op      │
JQTransform_Simple-4                     10.00 ± 0%
JQTransform_ObjectConstruction-4         15.00 ± 0%
JQTransform_ArraySelect-4                30.00 ± 0%
JQTransform_Complex-4                    324.0 ± 0%
JQTransform_Throughput-4                 17.00 ± 0%
SSEPublishDelivery-4                     0.000 ± 0%
geomean                                             ¹
¹ summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/schema
cpu: AMD EPYC 7763 64-Core Processor                
                                    │ benchmark-results.txt │
                                    │        sec/op         │
SchemaValidation_Simple-4                       1.106µ ± 4%
SchemaValidation_AllFields-4                    1.685µ ± 5%
SchemaValidation_FormatValidation-4             1.586µ ± 1%
SchemaValidation_ManySchemas-4                  1.814µ ± 4%
geomean                                         1.522µ

                                    │ benchmark-results.txt │
                                    │         B/op          │
SchemaValidation_Simple-4                      0.000 ± 0%
SchemaValidation_AllFields-4                   0.000 ± 0%
SchemaValidation_FormatValidation-4            0.000 ± 0%
SchemaValidation_ManySchemas-4                 0.000 ± 0%
geomean                                                   ¹
¹ summaries must be >0 to compute geomean

                                    │ benchmark-results.txt │
                                    │       allocs/op       │
SchemaValidation_Simple-4                      0.000 ± 0%
SchemaValidation_AllFields-4                   0.000 ± 0%
SchemaValidation_FormatValidation-4            0.000 ± 0%
SchemaValidation_ManySchemas-4                 0.000 ± 0%
geomean                                                   ¹
¹ summaries must be >0 to compute geomean

cpu: AMD EPYC 9V74 80-Core Processor                
                                    │ baseline-bench.txt │
                                    │       sec/op       │
SchemaValidation_Simple-4                    871.2n ± 4%
SchemaValidation_AllFields-4                 1.265µ ± 1%
SchemaValidation_FormatValidation-4          1.218µ ± 2%
SchemaValidation_ManySchemas-4               1.227µ ± 3%
geomean                                      1.133µ

                                    │ baseline-bench.txt │
                                    │        B/op        │
SchemaValidation_Simple-4                   0.000 ± 0%
SchemaValidation_AllFields-4                0.000 ± 0%
SchemaValidation_FormatValidation-4         0.000 ± 0%
SchemaValidation_ManySchemas-4              0.000 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

                                    │ baseline-bench.txt │
                                    │     allocs/op      │
SchemaValidation_Simple-4                   0.000 ± 0%
SchemaValidation_AllFields-4                0.000 ± 0%
SchemaValidation_FormatValidation-4         0.000 ± 0%
SchemaValidation_ManySchemas-4              0.000 ± 0%
geomean                                                ¹
¹ summaries must be >0 to compute geomean

pkg: github.com/GoCodeAlone/workflow/store
cpu: AMD EPYC 7763 64-Core Processor                
                                   │ benchmark-results.txt │
                                   │        sec/op         │
EventStoreAppend_InMemory-4                   1.189µ ± 28%
EventStoreAppend_SQLite-4                     1.364m ±  2%
GetTimeline_InMemory/events-10-4              13.81µ ±  1%
GetTimeline_InMemory/events-50-4              77.35µ ±  1%
GetTimeline_InMemory/events-100-4             124.1µ ±  2%
GetTimeline_InMemory/events-500-4             639.9µ ±  1%
GetTimeline_InMemory/events-1000-4            1.302m ±  1%
GetTimeline_SQLite/events-10-4                107.6µ ±  1%
GetTimeline_SQLite/events-50-4                251.0µ ±  2%
GetTimeline_SQLite/events-100-4               430.3µ ±  1%
GetTimeline_SQLite/events-500-4               1.832m ±  1%
GetTimeline_SQLite/events-1000-4              3.557m ±  2%
geomean                                       221.0µ

                                   │ benchmark-results.txt │
                                   │         B/op          │
EventStoreAppend_InMemory-4                     801.5 ± 7%
EventStoreAppend_SQLite-4                     1.984Ki ± 3%
GetTimeline_InMemory/events-10-4              7.953Ki ± 0%
GetTimeline_InMemory/events-50-4              46.62Ki ± 0%
GetTimeline_InMemory/events-100-4             94.48Ki ± 0%
GetTimeline_InMemory/events-500-4             472.8Ki ± 0%
GetTimeline_InMemory/events-1000-4            944.3Ki ± 0%
GetTimeline_SQLite/events-10-4                16.74Ki ± 0%
GetTimeline_SQLite/events-50-4                87.14Ki ± 0%
GetTimeline_SQLite/events-100-4               175.4Ki ± 0%
GetTimeline_SQLite/events-500-4               846.1Ki ± 0%
GetTimeline_SQLite/events-1000-4              1.639Mi ± 0%
geomean                                       67.43Ki

                                   │ benchmark-results.txt │
                                   │       allocs/op       │
EventStoreAppend_InMemory-4                     7.000 ± 0%
EventStoreAppend_SQLite-4                       53.00 ± 0%
GetTimeline_InMemory/events-10-4                125.0 ± 0%
GetTimeline_InMemory/events-50-4                653.0 ± 0%
GetTimeline_InMemory/events-100-4              1.306k ± 0%
GetTimeline_InMemory/events-500-4              6.514k ± 0%
GetTimeline_InMemory/events-1000-4             13.02k ± 0%
GetTimeline_SQLite/events-10-4                  382.0 ± 0%
GetTimeline_SQLite/events-50-4                 1.852k ± 0%
GetTimeline_SQLite/events-100-4                3.681k ± 0%
GetTimeline_SQLite/events-500-4                18.54k ± 0%
GetTimeline_SQLite/events-1000-4               37.29k ± 0%
geomean                                        1.162k

cpu: AMD EPYC 9V74 80-Core Processor                
                                   │ baseline-bench.txt │
                                   │       sec/op       │
EventStoreAppend_InMemory-4                741.6n ± 11%
EventStoreAppend_SQLite-4                  2.058m ± 77%
GetTimeline_InMemory/events-10-4           10.62µ ±  2%
GetTimeline_InMemory/events-50-4           45.43µ ± 32%
GetTimeline_InMemory/events-100-4          89.65µ ±  1%
GetTimeline_InMemory/events-500-4          462.6µ ±  1%
GetTimeline_InMemory/events-1000-4         941.0µ ±  1%
GetTimeline_SQLite/events-10-4             66.40µ ±  0%
GetTimeline_SQLite/events-50-4             175.9µ ±  1%
GetTimeline_SQLite/events-100-4            309.0µ ±  0%
GetTimeline_SQLite/events-500-4            1.339m ±  1%
GetTimeline_SQLite/events-1000-4           2.605m ±  0%
geomean                                    163.4µ

                                   │ baseline-bench.txt │
                                   │        B/op        │
EventStoreAppend_InMemory-4                 846.5 ± 12%
EventStoreAppend_SQLite-4                 1.981Ki ±  2%
GetTimeline_InMemory/events-10-4          7.953Ki ±  0%
GetTimeline_InMemory/events-50-4          46.62Ki ±  0%
GetTimeline_InMemory/events-100-4         94.48Ki ±  0%
GetTimeline_InMemory/events-500-4         472.8Ki ±  0%
GetTimeline_InMemory/events-1000-4        944.3Ki ±  0%
GetTimeline_SQLite/events-10-4            16.74Ki ±  0%
GetTimeline_SQLite/events-50-4            87.14Ki ±  0%
GetTimeline_SQLite/events-100-4           175.4Ki ±  0%
GetTimeline_SQLite/events-500-4           846.1Ki ±  0%
GetTimeline_SQLite/events-1000-4          1.639Mi ±  0%
geomean                                   67.72Ki

                                   │ baseline-bench.txt │
                                   │     allocs/op      │
EventStoreAppend_InMemory-4                  7.000 ± 0%
EventStoreAppend_SQLite-4                    53.00 ± 0%
GetTimeline_InMemory/events-10-4             125.0 ± 0%
GetTimeline_InMemory/events-50-4             653.0 ± 0%
GetTimeline_InMemory/events-100-4           1.306k ± 0%
GetTimeline_InMemory/events-500-4           6.514k ± 0%
GetTimeline_InMemory/events-1000-4          13.02k ± 0%
GetTimeline_SQLite/events-10-4               382.0 ± 0%
GetTimeline_SQLite/events-50-4              1.852k ± 0%
GetTimeline_SQLite/events-100-4             3.681k ± 0%
GetTimeline_SQLite/events-500-4             18.54k ± 0%
GetTimeline_SQLite/events-1000-4            37.29k ± 0%
geomean                                     1.162k

Benchmarks run with go test -bench=. -benchmem -count=6.
Regressions ≥ 20% are flagged. Results compared via benchstat.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 11, 2026

Codecov Report

❌ Patch coverage is 72.72727% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
cmd/wfctl/plugin_compat_model.go 72.72% 1 Missing and 2 partials ⚠️

📢 Thoughts on this report? Let us know!

@intel352 intel352 merged commit 8e42478 into main May 11, 2026
19 checks passed
@intel352 intel352 deleted the codex/plugin-conformance-pseudo-version branch May 11, 2026 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants