Skip to content

Watch and sync changes to related resources#149

Open
iakmc wants to merge 1 commit intokcp-dev:mainfrom
iakmc:related-kcp-resources-watch-2
Open

Watch and sync changes to related resources#149
iakmc wants to merge 1 commit intokcp-dev:mainfrom
iakmc:related-kcp-resources-watch-2

Conversation

@iakmc
Copy link

@iakmc iakmc commented Mar 17, 2026

Summary

Previously, changes to related resources with origin: kcp (e.g. a
ConfigMap created by a user in a kcp workspace) were ignored — only
changes to the primary object triggered reconciliation.

This adds a watch field to RelatedResourceSpec that allows service
providers to configure how the agent identifies the owning primary object
when a related resource changes. Two strategies are supported:

  • byOwner: inspects the OwnerReferences of the changed object to find
    the primary object by Kind.
  • byLabel: evaluates Go template expressions against the changed object
    to build a label selector, then lists primary objects matching that
    selector.

When watch is configured on an origin: kcp related resource, the
agent sets up a MultiClusterWatch for that resource type. When a change
is detected, the configured strategy is used to enqueue the owning
primary object for reconciliation, which then syncs the updated related
object to the service cluster.

What Type of PR Is This?

/kind feature
/kind api-change

Related Issue(s)

Fixes #118

Release Notes

Added `watch` field to `RelatedResourceSpec` to enable automatic syncing
of changes to `origin: kcp` related resources to the service cluster.

@kcp-ci-bot kcp-ci-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. kind/feature Categorizes issue or PR as related to a new feature. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Mar 17, 2026
@kcp-ci-bot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@kcp-ci-bot kcp-ci-bot added kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API dco-signoff: yes Indicates the PR's author has signed the DCO. labels Mar 17, 2026
@kcp-ci-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign embik for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kcp-ci-bot kcp-ci-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Mar 17, 2026
@iakmc iakmc force-pushed the related-kcp-resources-watch-2 branch 2 times, most recently from 7253588 to 14bea92 Compare March 17, 2026 09:01
@iakmc iakmc marked this pull request as ready for review March 17, 2026 09:02
@kcp-ci-bot kcp-ci-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 17, 2026
@iakmc iakmc changed the title Watch and sync changes to related resources WIP Watch and sync changes to related resources Mar 17, 2026
@kcp-ci-bot kcp-ci-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 17, 2026
@iakmc
Copy link
Author

iakmc commented Mar 17, 2026

/test pull-api-syncagent-verify

1 similar comment
@iakmc
Copy link
Author

iakmc commented Mar 17, 2026

/test pull-api-syncagent-verify

@iakmc
Copy link
Author

iakmc commented Mar 17, 2026

/retest pull-api-syncagent-verify

@kcp-ci-bot
Copy link
Contributor

@iakmc: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

/test pull-api-syncagent-build-image
/test pull-api-syncagent-lint
/test pull-api-syncagent-test
/test pull-api-syncagent-test-e2e-kcp-0.27
/test pull-api-syncagent-test-e2e-kcp-0.28
/test pull-api-syncagent-test-e2e-kcp-0.29
/test pull-api-syncagent-validate-prow-yaml
/test pull-api-syncagent-verify

Use /test all to run all jobs.

Details

In response to this:

/retest pull-api-syncagent-verify

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@iakmc
Copy link
Author

iakmc commented Mar 17, 2026

/test pull-api-syncagent-verify

@iakmc
Copy link
Author

iakmc commented Mar 17, 2026

/test pull-api-syncagent-test-e2e-kcp-0.28 pull-api-syncagent-validate-prow-yaml

change copyright year of related handlers

On-behalf-of: SAP <iskren.pertov@sap.com>
Signed-off-by: Iskren Petrov <iskren@kubermatic.com>

Watch and sync changes to related resources
@iakmc iakmc force-pushed the related-kcp-resources-watch-2 branch from 14bea92 to f929711 Compare March 17, 2026 11:00
@iakmc
Copy link
Author

iakmc commented Mar 17, 2026

/test pull-api-syncagent-test-e2e-kcp-0.28

@iakmc iakmc changed the title WIP Watch and sync changes to related resources Watch and sync changes to related resources Mar 17, 2026
@kcp-ci-bot kcp-ci-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 17, 2026
@iakmc
Copy link
Author

iakmc commented Mar 17, 2026

/test pull-api-syncagent-validate-prow-yaml

@xrstf
Copy link
Contributor

xrstf commented Mar 20, 2026

/retest


// RelatedResourceWatch configures how the watch handler maps a changed related resource
// back to its owning primary object.
// Exactly one of ByOwner or ByLabel must be set.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add an XValidation rule to ensure this. There are some examples in this file already.

// each value is a Go template expression evaluated with the changed object available as
// .watchObject (with fields .name, .namespace, .labels).
// +optional
ByLabel map[string]string `json:"byLabel,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we support a full-blown metav1.LabelSelector here?

// +optional
ByOwner *RelatedResourceWatchByOwner `json:"byOwner,omitempty"`

// ByLabel configures the watch handler to list primary objects matching a label selector
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename this to "ByLabels" or "BySelector"? "ByLabel" is singular and I think this feature supports a selector with multiple key-value pairs, right?

// RelatedResourceWatchByOwner configures reverse lookup via OwnerReferences.
type RelatedResourceWatchByOwner struct {
// Kind is the Kind to look for in the OwnerReferences of the changed related object.
Kind string `json:"kind"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should take a full GVK here and not just compare based on Kinds.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I think the opposite now. This is not required at all, is it?

We already know the GVK of the primary object on both sides (origin and destination). We also already know the GVK/GVR of every related resource (like ConfigMaps). So what the agent needs to do is to watch all the GVR of related resources on the origin side (ConfigMaps, for example) and check if they have an ownerRef to the GVK of the primary object (a Cluster object, or whatever). And if so, cool, enqueue the primary object (i.e. the owner).

We don't need any further configuration for the ByOwner functionality here, I think.

// of the owning primary object. Only related resources with a Watch config are covered.
watchedGVKs := sets.New[schema.GroupVersionKind]()
for _, relRes := range pubRes.Spec.Related {
if relRes.Origin != syncagentv1alpha1.RelatedResourceOriginKcp || relRes.Watch == nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't quite remember why we would only want to do this on the kcp side.. Couldn't we technically also have related objects on the service cluster side? It feels like this watching behaviour should work regardless of the origin side, no? 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I.e. it should always watch on the origin side, regardless where that side actually is.

}

// Use the local REST mapper to determine the Kind.
gvk, err := localManager.GetRESTMapper().KindFor(gvr)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the related resource originates in kcp (as per the first if statement in this loop), then why are we using the local (= service cluster) REST mapper to resolve it?

Related resources support projection (i.e. changing their GVK when syncing from one side to another), so a GVK that exists in kcp is not necessarily the same as it is on the service cluster.

Ideally this should use a restmapper of the origin side (whereever that might be).

// Use the local REST mapper to determine the Kind.
gvk, err := localManager.GetRESTMapper().KindFor(gvr)
if err != nil {
log.Warnw("Failed to determine Kind for origin:kcp related resource, skipping watch", "gvr", gvr, "error", err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I fear this log message will never be seen by anyone and during runtime, this state will also not fix itself (the agent won't try to re-establish this watch at a later time, unless you restart the entire agent).

To catch misconfigurations, wouldn't it make more sense to error out here?

}

default:
log.Warnw("origin:kcp related resource has Watch set but neither byOwner nor byLabel configured, skipping", "gvk", gvk)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should definitely be an error. Misconfigured PublishedResources should IMHO scream loudly. If we have the XValidation rule such errors will be much less likely, but I would still error out if the PublishedResource is broken.


func (h *byOwnerEventHandler) enqueue(obj *unstructured.Unstructured, q workqueue.TypedRateLimitingInterface[mcreconcile.Request]) {
for _, ref := range obj.GetOwnerReferences() {
if ref.Kind == h.ownerKind {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should compare G, V and K, I think.

"namespace": obj.GetNamespace(),
"labels": obj.GetLabels(),
},
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please create an explicit struct for this templating context in https://github.com/kcp-dev/api-syncagent/blob/main/internal/sync/templating/related.go, which helps in documenting the available template variables. Please also add a corresponding New...() func, to ensure that the newly introduced struct always gets built correctly.

You can then extend https://github.com/kcp-dev/api-syncagent/blob/main/docs/content/publish-resources/templating.md, which is trivial once the struct is setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/feature Categorizes issue or PR as related to a new feature. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: SyncAgent does not detect relatedResource changes

3 participants