Skip to content

[BUG] ec should fail if provided config references a non-existent rule or collection #1753

@ralphbean

Description

@ralphbean

Describe the Bug

I typo'd something in an EnterpriseContractPolicy but ec didn't fail, which incorrectly led me to believe that I had successfully created the policy I wanted.

Steps to Reproduce

Construct a EnterpriseContractResource with

apiVersion: appstudio.redhat.com/v1alpha1
kind: EnterpriseContractPolicy
spec:
  description: 'An example policy for this issue'
  publicKey: 'k8s://openshift-pipelines/public-key'
  sources:
    - name: Release Policies
      policy:
        - oci::quay.io/enterprise-contract/ec-release-policy:git-0cdcba3@sha256:febe9ada08701fad1cec392dfde8e66cc2408989d1396adb4073af6f646855d1
      config:
        include:
          - "@slsa3"
          - LOL not a rule

Use that policy to evaluate ~any artifact, and it will pass - even though "LOL not a rule" is not a rule

Expected Behavior

I expect ec to fail with "LOL not a rule" not found in any policy source among ... (and then, list the policy sources provided like oci::quay.io/enterprise-contract/ec-release-policy:git-0cdcba3@sha256:febe9ada08701fad1cec392dfde8e66cc2408989d1396adb4073af6f646855d1

Actual Behavior

ec exits successfully without even a warning. (IMO, a hard failure would be best in this situation, not just a warning.)

Screenshots or Terminal Output

If applicable, add screenshots or copy-paste the terminal output to help explain your problem.

Environment Details

❯ ec version                                                  
Version            v0.5.13
Source ID          0527fad71b065e9a0694a30ca70dbf04625ac811
Change date        2024-06-14 20:29:28 +0000 UTC (3 weeks ago)
ECC                v0.1.47
OPA                v0.65.0
Conftest           v0.53.0
Cosign             v2.2.4
Sigstore           v1.8.4
Rekor              v1.3.6
Tekton Pipeline    v0.54.0
Kubernetes Client  v0.29.5

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions