CUE v0.14.1: import statements in outputs[].valueFrom cause panic and "reference not found" in downstream steps

[agentali]

Describe the bug

Workflows that use CUE import statements in step outputs[].valueFrom scripts fail after the CUE library upgrade (v0.6 → v0.14.1) in kubevela/workflow. Two distinct failures occur:

Bug 1 — Panic in SetValueByScript:

invalid cue task for evaluation: interface conversion: *ast.File is not ast.Expr: missing method declNode

Bug 2 — Reference error in downstream steps:

reference "json" not found

To Reproduce

1. Create a ConfigMap prerequisite:

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
  namespace: default
data:
  config.json: |
    {
      "region": "us-east-1",
      "security_groups": [{"id": "sg-001"}, {"id": "sg-002"}],
      "subnets": [{"id": "subnet-001"}, {"id": "subnet-002"}]
    }

2. Apply a workflow that uses import "encoding/json" in outputs[].valueFrom:

apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: test-cue-import-bug
  namespace: default
spec:
  components:
    - name: test-component
      type: helm
      properties:
        chart: some-chart
        repoType: oci
        url: oci://example.com/charts
        values:
          region: ""
          vpcOptions: {}
  workflow:
    steps:
      - name: read-config
        type: read-object
        properties:
          apiVersion: v1
          kind: ConfigMap
          name: my-config
          namespace: default
        outputs:
          - name: region
            valueFrom: |
              import "encoding/json"
              json.Unmarshal(output.value.data["config.json"]).region

          - name: vpcOptions
            valueFrom: |
              import "encoding/json"
              #d: json.Unmarshal(output.value.data["config.json"])
              {
                  securityGroupIds: [ for sg in #d.security_groups { sg.id } ]
                  subnetIds: [ for subnet in #d.subnets { subnet.id } ]
              }

      - name: apply
        type: apply-component
        dependsOn: [read-config]
        inputs:
          - from: region
            parameterKey: properties.values.region
          - from: vpcOptions
            parameterKey: properties.values.vpcOptions
        properties:
          component: test-component

3. Observe the error:

• Bug 1: The read-config step (or the subsequent apply step during input processing) fails with the *ast.File is not ast.Expr panic.
• Bug 2: If Bug 1 doesn't manifest (depending on the exact code path), the apply step fails with reference "json" not found.

4. Unit test reproduction — add to pkg/cue/model/value/value_test.go:

func TestCueImportInOutputScript(t *testing.T) {
	r := require.New(t)

	taskValue := cuecontext.New().CompileString(`
let OUTPUT = {
	#do:       "read"
	#provider: "kube"
	$params: {
		cluster: ""
		value: {
			apiVersion: "v1"
			kind:       "ConfigMap"
			metadata: { name: "test", namespace: "default" }
		}
	}
	$returns: {
		value: {
			apiVersion: "v1"
			kind:       "ConfigMap"
			data: {
				"config.json": "{\"region\":\"us-east-1\",\"security_groups\":[{\"id\":\"sg-1\"},{\"id\":\"sg-2\"}]}"
			}
		}
	}
}
output: OUTPUT
`)
	r.NoError(taskValue.Err())

	script := `import "encoding/json"
#d: json.Unmarshal(output.value.data["config.json"])
{
	securityGroupIds: [ for sg in #d.security_groups { sg.id } ]
}
`

	v, err := LookupValueByScript(taskValue, script)
	if err != nil {
		t.Fatalf("LookupValueByScript failed: %v", err)
	}

	jsonBytes, jsonErr := v.MarshalJSON()
	r.NoError(jsonErr, "value should be concrete and JSON-serializable")
	t.Logf("JSON output: %s", string(jsonBytes))

	str, _ := sets.ToString(v)
	r.NotContains(str, "import", "serialized output must not contain import statements")
}

Expected behavior

Both workflow steps succeed:

• read-config extracts values from the ConfigMap using json.Unmarshal
• apply receives clean scalar/struct values and applies the component

This works correctly in KubeVela v1.9.9 (which uses CUE v0.6).

Screenshots

N/A — see error messages and data flow diagram in Additional context below.

Workflow Version

• Failing: KubeVela v1.10.6, v1.10.7 (using kubevela/workflow with cuelang.org/go v0.14.1)
• Working: KubeVela v1.9.9 (using kubevela/workflow v0.6.0 with older CUE)

Cluster information

• Kubernetes 1.28+
• Not cluster-version-specific — the bug is in the workflow controller's CUE evaluation logic

Additional context

Root Cause Analysis

The CUE library upgrade from v0.6 to v0.14.1 introduced two behavioral changes:

Bug 1: cue.Value.Syntax() returns *ast.File instead of ast.Expr

Location: pkg/cue/model/value/value.go → SetValueByScript()

In CUE v0.6, cue.Value.Syntax() returned ast.Expr types (*ast.StructLit, *ast.BasicLit, etc.). In v0.14.1, when the value retains parent context (imports, let bindings), Syntax() returns *ast.File instead. Since *ast.File does not implement ast.Expr (missing declNode()), the type assertion panics.

Call chain:

data_passing.go Input()
  → SetValueByScript(base, v, path...)
    → v.Syntax(cue.ResolveReferences(true))   // returns *ast.File
    → setValue(node, expr, selectors)          // PANIC: *ast.File is not ast.Expr

Bug 2: cue.Final() preserves $returns wrapper — output.value no longer resolves

Location: pkg/cue/model/value/value.go → LookupValueByScript()

In CUE v0.6, util.ToString(taskValue) with cue.Final() flattened the provider's internal structure so that output.value pointed directly to the returned data. In v0.14.1, cue.Final() preserves the $returns wrapper, so the data lives at output.$returns.value instead of output.value.

When a script references output.value, #d becomes json.Unmarshal(<undefined>.data[...]) which is non-concrete. MarshalJSON() fails, the JSON round-trip cleanup is skipped, and the contaminated value (carrying import "encoding/json" and unresolved #d) leaks into the workflow context ConfigMap.

When the next step loads this ConfigMap, CUE encounters json.Unmarshal without import "encoding/json" in scope → reference "json" not found.

The existing $returns fallback in data_passing.go (Output hook, lines ~72-84) handles simple single-expression scripts by retrying with output.$returns.value. But it does NOT handle multi-line scripts with hidden defs (like the #d: pattern) because:

1. strings.Split(output.ValueFrom, "output.") doesn't work for multi-line scripts with multiple output. references
2. The fallback checks v.Err() != nil, but hidden def resolution errors don't bubble up to the parent value's Err()

Data Flow

read-object step completes
    ↓
taskValue serialized via util.ToString(taskValue) [cue.Final()]
    CUE v0.14.1 preserves: output: { $returns: { value: { ... } } }
    (CUE v0.6 flattened to: output: { value: { ... } })
    ↓
Script references "output.value" — DOES NOT EXIST in serialized form
    #d: json.Unmarshal(output.value.data["config.json"])  → UNRESOLVED
    ↓
MarshalJSON() fails: "undefined field: value"
    ↓
JSON round-trip cleanup skipped → contaminated value returned
    Value carries: import "encoding/json", #d: json.Unmarshal(...)
    ↓
ctx.SetVar(v, name) → sets.ToString(v) → FillRaw into vars
    Workflow vars now contain import "encoding/json" at top level
    ↓
Vars written to workflow context ConfigMap
    ↓
Next step loads ConfigMap → CompileString(varsStr)
    FAILS: reference "json" not found

Proposed Fix

Four changes (branch: fix/cue-v0.14.1-workflow-output-imports):

Fix 1: nodeToExpr() — Safe *ast.File → ast.Expr conversion

File: pkg/cue/model/value/value.go

Add a helper to safely convert the ast.Node returned by Syntax() into an ast.Expr, handling the *ast.File case by extracting the embedded expression or wrapping declarations in a *ast.StructLit.

Fix 2: Rewrite output.value → output.$returns.value (Root Cause)

File: pkg/cue/model/value/value.go, in LookupValueByScript()

After serializing the task value, detect when $returns is present and rewrite script references for backward compatibility.

Fix 3: JSON round-trip in LookupValueByScript() (Defense in Depth)

Round-trip concrete values through JSON to strip parent CUE context (let bindings, hidden defs, imports).

Fix 4: concretizeValue() in the Output hook (Defense in Depth)

File: pkg/hooks/data_passing.go, called before ctx.SetVar()

A second safety net to ensure values stored in workflow context are clean.

Fix	Purpose	Without it
#1 nodeToExpr	Prevents panic on *ast.File → ast.Expr type assertion	Step panics immediately during input processing
#2 output.value rewrite	Restores backward compatibility with pre-v0.14.1 script references	Scripts reference output.value which doesn't exist; #d becomes unresolvable
#3 JSON round-trip	Strips CUE artifacts from concrete output values	Values carry parent context (imports, let bindings) that corrupt downstream serialization
#4 concretizeValue	Second safety net for values entering SetVar	Edge cases where Fix #3 is skipped still contaminate the ConfigMap

Fix #2 is the root cause fix. Fixes #3 and #4 are defense-in-depth.

The bug affects any workflow step type that produces outputs with import statements in valueFrom scripts, not just read-object. It will affect any provider-based step where the result is wrapped in $returns.