fix: add failureReason to conformance evidence and strip +dirty from wfctlVersion#649
Merged
Merged
Conversation
⏱ Benchmark Results✅ No significant performance regressions detected. benchstat comparison (baseline → PR)
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…buildVersion Agent-Logs-Url: https://github.com/GoCodeAlone/workflow/sessions/539e0b4b-78fb-4780-8176-5d1712ae7d55 Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Update wfctl plugin for conformance and failure diagnostics
fix: add failureReason to conformance evidence and strip +dirty from wfctlVersion
May 13, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves wfctl plugin conformance evidence by making failed typed-IaC checks easier to diagnose and ensuring dirty build metadata does not invalidate evidence version normalization.
Changes:
- Adds
failureReasonto compatibility evidence and populates it on conformance failures. - Ensures passing conformance evidence omits
failureReason. - Strips
+dirtyfromwfctlbuild versions before evidence stamping.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
cmd/wfctl/plugin_conformance.go |
Captures the conformance error message in failed evidence. |
cmd/wfctl/plugin_conformance_test.go |
Adds assertions for failure reasons and dirty-version suffix absence. |
cmd/wfctl/plugin_compat_model.go |
Extends compatibility evidence JSON with optional failureReason. |
cmd/wfctl/main.go |
Normalizes build versions by removing a trailing +dirty marker. |
cmd/wfctl/main_test.go |
Adds coverage for dirty-marker stripping behavior. |
Contributor
|
@copilot need improved coverage: #649 (comment) |
…ipping logic Agent-Logs-Url: https://github.com/GoCodeAlone/workflow/sessions/b77c3555-47a4-45f7-aa6e-57b196c51d2d Co-authored-by: intel352 <77607+intel352@users.noreply.github.com>
Contributor
Author
Addressed in 5ea6ce8. Extracted a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Conformance evidence for failed typed-IaC checks omitted the actual error message, forcing maintainers to reproduce failures locally. Additionally,
wfctlVersionin evidence could be stamped with+dirtypseudo-versions that are not valid Go module pseudo-versions, causingCanonicalEvidenceEngineVersionto silently fall back tov0.0.0.Changes
PluginCompatibilityEvidence(plugin_compat_model.go): AddfailureReason string(json:"failureReason,omitempty"). Included inevidenceDigestcomputation when non-empty; absent from passing evidence.runPluginConformanceCheck(plugin_conformance.go): Setev.FailureReason = err.Error()before normalizing failed evidence. The error message (e.g. "typed IaC service not registered", "legacy InvokeService dispatch removed") is now captured in the evidence JSON alongsidestderrTail.buildVersion(main.go): Strip+dirtysuffix from build info versions. The marker reflects build-environment VCS state, not a meaningful version difference. Strings likev0.22.8-0.20260510180701-a851625d3bf0+dirtyare not valid Go pseudo-versions and would silently downgradewfctlVersionin evidence tov0.0.0; stripping the suffix lets the real commit-bound pseudo-version propagate.Failed evidence now looks like:
{ "status": "fail", "failureReason": "plugin uses legacy InvokeService dispatch removed in workflow v1.0.0", "stderrTail": "...", "wfctlVersion": "v0.22.8-0.20260510180701-a851625d3bf0", ... }