Removing ACL extension from Shoot leaves ACL config in EnvoyFilter

[timebertt]

What happened:

When removing the ACL extension from the Shoot spec, it leaves the ACL config in the EnvoyFilter for the apiserver-proxy path (ingress-gateway/shoot--foo--bar) until the next shoot reconciliation.

What you expected to happen:

Removing the ACL extension from the Shoot spec should take effect on all paths immediately.

How to reproduce it (as minimally and precisely as possible):

1. Create a shoot with the ACL extension enabled, e.g.:

kind: Shoot
# ...
spec:
  extensions:
    - type: acl
      providerConfig:
        rule:
          action: ALLOW
          cidrs:
            - 1.2.3.4/32
          type: remote_ip

2. Remove the ACL extension from the shoot spec.
3. The shoot--foo--bar EnvoyFilter in istio-ingress namespace is unchanged and still contains the ACL config injected by the webhook.

Expand me

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: shoot--foo--bar
  namespace: istio-ingress
spec:
  configPatches:
  - applyTo: FILTER_CHAIN
    match:
      context: ANY
      listener:
        portNumber: 8443
    patch:
      operation: ADD
      value:
        filter_chain_match:
          destination_port: 443
          prefix_ranges:
          - address_prefix: 100.83.42.91
            prefix_len: 32
        filters:
        - name: acl-internal-remote_ip
          typed_config:
            '@type': type.googleapis.com/envoy.extensions.filters.network.rbac.v3.RBAC
            rules:
              action: ALLOW
              policies:
                acl-internal:
                  permissions:
                  - any: true
                  principals:
                  - remote_ip:
                      address_prefix: 1.2.3.4
                      prefix_len: 32
                  # always allowed CIDRs...
            stat_prefix: envoyrbac

4. Reconcile the shoot once again.
5. The ACL config is removed from the EnvoyFilter.

Anything else we need to know?:

When removing the ACL extension from the shoot, the Extension object is deleted after gardenlet applied the EnvoyFilter.
On extension deletion, the extension controller triggers the EnvoyFilter webhook with an empty patch.
However, the webhook doesn't actively remove the ACL config (it responds without a patch).
Hence, the config is left until gardenlet applies the desired state of the EnvoyFilter again and the webhook doesn't act on the object anymore.

Environment:

• Gardener version: v1.85
• Extension version: 72d7499