Config follow-up: Implement component override functionality for Config resource

[andreaskaris]

This commit adds the ability to mark components as unmanaged in the
bpfman-operator, preventing the operator from creating or updating
specific objects. The implementation includes:

• Added ComponentOverride struct to Config API with fields for
  kind, group, namespace, name, and unmanaged flag
• Modified assureResource function to check for overrides and skip
  management of unmanaged components
• Implemented isOverridden helper function to match objects against
  override specifications
• Added tests to verify override functionality works correctly
  across all component types

Documentation

Component Overrides

The overrides field allows you to mark specific components as unmanaged, preventing the operator from creating or updating them. This is intended as a debug/escape hatch for advanced users who need to customize specific resources without operator interference.

YAML Configuration

Add overrides directly to the Config resource:

apiVersion: bpfman.io/v1alpha1
kind: Config
metadata:
  name: bpfman-config
spec:
  # Existing configuration...
  namespace: bpfman
  daemon:
    image: quay.io/bpfman/bpfman:latest
    logLevel: info
  agent:
    image: quay.io/bpfman/bpfman-agent:latest
    logLevel: info
  configuration: |
    # bpfman.toml content here

  # Component override for the bpfman daemon DaemonSet
  overrides:
  - apiVersion: apps/v1
    kind: DaemonSet
    namespace: bpfman
    name: bpfman-daemon
    unmanaged: true

JSON Patch for Precise Control

Using kubectl patch to add overrides:

kubectl patch config bpfman-config --type='json' -p='[
  {
    "op": "add",
    "path": "/spec/overrides",
    "value": [
      {
        "apiVersion": "apps/v1",
        "kind": "DaemonSet",
        "namespace": "bpfman",
        "name": "bpfman-daemon",
        "unmanaged": true
      }
    ]
  }
]'

Adding Multiple Overrides

To append to existing overrides without replacing them:

kubectl patch config bpfman-config --type='json' -p='[
  {
    "op": "add",
    "path": "/spec/overrides/-",
    "value": {
      "apiVersion": "apps/v1",
      "kind": "DaemonSet",
      "namespace": "bpfman",
      "name": "bpfman-daemon",
      "unmanaged": true
    }
  }
]'

The /- notation appends to the existing overrides array rather than replacing it.

Examples

Override the metrics proxy DaemonSet:

overrides:
- apiVersion: apps/v1
  kind: DaemonSet
  namespace: bpfman
  name: bpfman-metrics-proxy
  unmanaged: true

Override a ConfigMap:

overrides:
- apiVersion: v1
  kind: ConfigMap
  namespace: bpfman
  name: bpfman-config
  unmanaged: true

Override multiple resources:

overrides:
- apiVersion: apps/v1
  kind: DaemonSet
  namespace: bpfman
  name: bpfman-daemon
  unmanaged: true
- apiVersion: apps/v1
  kind: DaemonSet
  namespace: bpfman
  name: bpfman-metrics-proxy
  unmanaged: true
- apiVersion: storage.k8s.io/v1
  kind: CSIDriver
  namespace: ""  # cluster-scoped resource
  name: csi.bpfman.io
  unmanaged: true

Note: The apiVersion field matches what you see in kubectl output (e.g., apps/v1 for DaemonSets, v1 for core resources like ConfigMaps), making it easy to copy/paste from existing manifests.

[mergify]

@andreaskaris, this pull request is now in conflict and requires a rebase.

[frobware]

ConfigMap should not be overrideable given that it's now an internal implementation detail of the Config CRD (post #461). Should isOverrideable narrow the list down to the DaemonSet, CSIDriver, SCC -- everthing else is excluded.

[frobware]

I played around with the functionality and have some suggestions for UX in #489.

[frobware]

Replying to my own question here after experimenting with this.

For a debug/escape hatch, I think blocking creation is the right behaviour. The semantics become simple and predictable: unmanaged = the operator pretends this resource doesn't exist.

If you mark something as unmanaged and then delete it, you presumably meant to delete it. Recreating something marked "unmanaged" would be semantically incoherent - you're telling the operator not to manage it, but then expecting it to manage recreation.

If deletion was accidental, the recovery is straightforward: remove the override, and the operator recreates the resource on the next reconcile. I tested this flow and it works as expected.

The alternative (only blocking updates, allowing recreation) adds complexity and creates confusing edge cases. For a feature explicitly framed as a debug escape hatch for advanced users, I'd keep it simple and trust the user knows what they're doing.

[andreaskaris]

• Re the override suggestion in Config overrides: use apiVersion instead of group for easier UX #489

The reason for the format that I chose is that OpenShift does it like that:
https://github.com/search?q=org%3Aopenshift%20ComponentOverride&type=code

It doesn't have to follow OCP's example, though, your version feels cleaner to me

[andreaskaris]

@frobware Ready for another review, I added your changes

[frobware]

Testing Component Overrides

Helper script: https://github.com/frobware/nix-bpfman/blob/main/scripts/bpfman-operator-component-override

Initial state - no overrides

$ kubectl get config bpfman-config -o jsonpath="{.spec.overrides}"
(empty)

Add daemon override

$ bpfman-operator-component-override add daemon
config.bpfman.io/bpfman-config patched
Added override: daemon

$ kubectl get config bpfman-config -o jsonpath="{.spec.overrides}" | jq .
[
  {
    "apiVersion": "apps/v1",
    "kind": "DaemonSet",
    "name": "bpfman-daemon",
    "namespace": "bpfman",
    "unmanaged": true
  }
]

Operator logs confirm skipping

$ kubectl logs -n bpfman deployment/bpfman-operator --tail=50 | grep -i skip
{"level":"info","ts":"2026-01-19T11:39:46Z","logger":"Config","msg":"Skipping unmanaged resource (override in place)","kind":"DaemonSet","namespace":"bpfman","name":"bpfman-daemon"}

Add all components

$ bpfman-operator-component-override add ALL
Already overridden: daemon
config.bpfman.io/bpfman-config patched
Added override: csi
config.bpfman.io/bpfman-config patched
Added override: metrics-proxy
config.bpfman.io/bpfman-config patched
Added override: config

$ kubectl get config bpfman-config -o jsonpath="{.spec.overrides}" | jq .
[
  {
    "apiVersion": "apps/v1",
    "kind": "DaemonSet",
    "name": "bpfman-daemon",
    "namespace": "bpfman",
    "unmanaged": true
  },
  {
    "apiVersion": "storage.k8s.io/v1",
    "kind": "CSIDriver",
    "name": "csi.bpfman.io",
    "namespace": "",
    "unmanaged": true
  },
  {
    "apiVersion": "apps/v1",
    "kind": "DaemonSet",
    "name": "bpfman-metrics-proxy",
    "namespace": "bpfman",
    "unmanaged": true
  },
  {
    "apiVersion": "v1",
    "kind": "ConfigMap",
    "name": "bpfman-config",
    "namespace": "bpfman",
    "unmanaged": true
  }
]

Reset all overrides

$ bpfman-operator-component-override reset
config.bpfman.io/bpfman-config patched
All overrides removed

$ kubectl get config bpfman-config -o jsonpath="{.spec.overrides}"
(empty)

[frobware]

Follow-up: Verify unmanaged resource is not recreated

$ bpfman-operator-component-override add daemon
config.bpfman.io/bpfman-config patched
Added override: daemon

$ kubectl get daemonset -n bpfman bpfman-daemon
NAME            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
bpfman-daemon   1         1         1       1            1           <none>          76m

$ kubectl delete daemonset -n bpfman bpfman-daemon
daemonset.apps "bpfman-daemon" deleted

$ sleep 15; kubectl get daemonset -n bpfman bpfman-daemon
Error from server (NotFound): daemonsets.apps "bpfman-daemon" not found

$ bpfman-operator-component-override delete daemon
config.bpfman.io/bpfman-config patched
Removed override: daemon

$ sleep 5; kubectl get daemonset -n bpfman bpfman-daemon
NAME            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
bpfman-daemon   1         1         1       1            1           <none>          8s

The operator correctly:

1. Does not recreate deleted resources when marked as unmanaged
2. Recreates resources once the override is removed

[frobware]

All components tested

Each component was marked as unmanaged, deleted, verified not recreated, then the override was removed and verified the operator recreated it.

metrics-proxy (DaemonSet)

$ bpfman-operator-component-override add metrics-proxy && kubectl delete daemonset -n bpfman bpfman-metrics-proxy
Added override: metrics-proxy
daemonset.apps "bpfman-metrics-proxy" deleted

$ sleep 10; kubectl get daemonset -n bpfman bpfman-metrics-proxy
Error from server (NotFound): daemonsets.apps "bpfman-metrics-proxy" not found

$ bpfman-operator-component-override delete metrics-proxy && sleep 5; kubectl get daemonset -n bpfman bpfman-metrics-proxy
Removed override: metrics-proxy
NAME                   DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
bpfman-metrics-proxy   1         1         1       1            1           <none>          5s

csi (CSIDriver)

$ bpfman-operator-component-override add csi && kubectl delete csidriver csi.bpfman.io
Added override: csi
csidriver.storage.k8s.io "csi.bpfman.io" deleted

$ sleep 10; kubectl get csidriver csi.bpfman.io
Error from server (NotFound): csidrivers.storage.k8s.io "csi.bpfman.io" not found

$ bpfman-operator-component-override delete csi && sleep 5; kubectl get csidriver csi.bpfman.io
Removed override: csi
NAME            ATTACHREQUIRED   PODINFOONMOUNT   STORAGECAPACITY   TOKENREQUESTS   REQUIRESREPUBLISH   MODES       AGE
csi.bpfman.io   false            true             false             <unset>         false               Ephemeral   5s

config (ConfigMap)

$ bpfman-operator-component-override add config && kubectl delete configmap -n bpfman bpfman-config
Added override: config
configmap "bpfman-config" deleted

$ sleep 10; kubectl get configmap -n bpfman bpfman-config
Error from server (NotFound): configmaps "bpfman-config" not found

$ bpfman-operator-component-override delete config && sleep 5; kubectl get configmap -n bpfman bpfman-config
Removed override: config
NAME            DATA   AGE
bpfman-config   3      5s

ALL (bulk operations)

$ bpfman-operator-component-override add ALL && bpfman-operator-component-override list
Added override: daemon
Added override: csi
Added override: metrics-proxy
Added override: config
Current overrides:
  DaemonSet/bpfman-daemon (unmanaged=true)
  CSIDriver/csi.bpfman.io (unmanaged=true)
  DaemonSet/bpfman-metrics-proxy (unmanaged=true)
  ConfigMap/bpfman-config (unmanaged=true)

$ bpfman-operator-component-override reset && bpfman-operator-component-override list
All overrides removed
No overrides configured

All component types (DaemonSet, CSIDriver, ConfigMap) work correctly with the override functionality.

[frobware]

LGTM

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (a519dfe) to head (f92cf6a).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@     Coverage Diff     @@
##   main   #475   +/-   ##
===========================
===========================

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.