Skip to content

Application Credential Support #317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-bot merged 1 commit into from
Feb 6, 2026
Merged

Application Credential Support #317

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions api/bases/watcher.openstack.org_watchers.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,6 +460,15 @@ spec:
type: string
type: object
type: object
auth:
description: Auth - Parameters related to authentication (shared by
all Watcher components)
properties:
applicationCredentialSecret:
description: ApplicationCredentialSecret - Secret containing Application
Credential ID and Secret
type: string
type: object
customServiceConfig:
description: |-
CustomServiceConfig - customize the service config using this parameter to change service defaults,
Expand Down
13 changes: 13 additions & 0 deletions api/v1beta1/common_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,11 @@ type WatcherSpecCore struct {
// APITimeout for Route and Apache
APITimeout *int `json:"apiTimeout"`

// +kubebuilder:validation:Optional
// +operator-sdk:csv:customresourcedefinitions:type=spec
// Auth - Parameters related to authentication (shared by all Watcher components)
Auth AuthSpec `json:"auth,omitempty"`

// +kubebuilder:validation:Optional
// NotificationsBusInstance is the name of the RabbitMqCluster CR to select
// the Message Bus Service instance used by the Watcher service to publish and consume notifications
Expand All @@ -139,6 +144,14 @@ type WatcherSpecCore struct {
NotificationsBusInstance *string `json:"notificationsBusInstance,omitempty"`
}

// AuthSpec defines authentication parameters
type AuthSpec struct {
// +kubebuilder:validation:Optional
// +operator-sdk:csv:customresourcedefinitions:type=spec
// ApplicationCredentialSecret - Secret containing Application Credential ID and Secret
ApplicationCredentialSecret string `json:"applicationCredentialSecret,omitempty"`
Copy link
Copy Markdown
Contributor

@amoralej amoralej Jan 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the current implementation of GetApplicationCredentialFromSecret in openstack-k8s-operators/keystone-operator#567 the, name of the secret is hardcoded to ac-{servicename}-secret , so there is no need to be able to parametrize it. We may need only a boolean ApplicationCredentialEnabled to specify that services should use an AC and fail if the secret do not exist.

Copy link
Copy Markdown
Collaborator

@SeanMooney SeanMooney Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isnt that a problem for roation.

the expected way to roate the applciation credital is to update the name of the Applaication credential CR or i guess the secret

ideally i owuld have prefered to have teh applciate credital CR name not the secret name here but either works.

hardcodeing however does not.

we have to be able to create a new application credential and update this to poitn to it to do the rotation.

that less imporant for watcher then for nova or other service on the edpm node because we need to do a edpm deployment and should not allow the applciation credital to be deleted until that happens.

Copy link
Copy Markdown
Collaborator

@SeanMooney SeanMooney Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

said another way i would expect to take the name of the CR here or the secreat and hardcodeing it is not somethign that shoudl be done in keystone. so i belive this crd change is valid.
removign it an realying on hardcoded name i dont think woudl be

Copy link
Copy Markdown
Contributor Author

@Deydra71 Deydra71 Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rotation mechanism works via the KeystoneApplicationCredential CR, not by changing secret names. Keystone-oeprator automatically rotates when the App cred is within the grace period gracePeriodDays.

Then it creates new AC in keystone, updates the same Secret with the new credentials. And old credential stays valid in keystone until its natural expiration.

There's always option to trigger rotation manually by patching expiration time to the past date, eg:

oc patch -n openstack keystoneapplicationcredential ac-barbican \
  --type=merge --subresource=status \
  -p '{"status":{"expiresAt":"2001-05-19T00:00:00Z"}}'

Copy link
Copy Markdown
Collaborator

@SeanMooney SeanMooney Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we cant asume its safe to deallocate teh old applcation credtial because edpm nodes cannot be rotated automiaticlly like that. we need to do a dataplane deployment

and it woudl be a security issue in my view to leak the old appclation credital without deleitng it in keystone ocne the refence to it is remvoed in openshift

}

// PasswordSelector to identify the DB and AdminUser password from the Secret
type PasswordSelector struct {
// +kubebuilder:validation:Optional
Expand Down
2 changes: 2 additions & 0 deletions api/v1beta1/conditions.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ const (
WatcherAPIReadyErrorMessage = "WatcherAPI error occured %s"
// WatcherPrometheusSecretErrorMessage -
WatcherPrometheusSecretErrorMessage = "Error with prometheus config secret"
// WatcherApplicationCredentialSecretErrorMessage -
WatcherApplicationCredentialSecretErrorMessage = "Error with application credential secret"
// WatcherApplierReadyInitMessage -
WatcherApplierReadyInitMessage = "WatcherApplier creation not started"
// WatcherApplierReadyRunningMessage -
Expand Down
16 changes: 16 additions & 0 deletions api/v1beta1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions config/crd/bases/watcher.openstack.org_watchers.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,6 +460,15 @@ spec:
type: string
type: object
type: object
auth:
description: Auth - Parameters related to authentication (shared by
all Watcher components)
properties:
applicationCredentialSecret:
description: ApplicationCredentialSecret - Secret containing Application
Credential ID and Secret
type: string
type: object
customServiceConfig:
description: |-
CustomServiceConfig - customize the service config using this parameter to change service defaults,
Expand Down
8 changes: 8 additions & 0 deletions config/manifests/bases/watcher-operator.clusterserviceversion.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,14 @@ spec:
- description: TLS - Parameters related to the TLS
displayName: TLS
path: apiServiceTemplate.tls
- description: Auth - Parameters related to authentication (shared by all Watcher
components)
displayName: Auth
path: auth
- description: ApplicationCredentialSecret - Secret containing Application Credential
ID and Secret
displayName: Application Credential Secret
path: auth.applicationCredentialSecret
version: v1beta1
description: The Watcher Operator project
displayName: Watcher Operator
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ require (
github.com/onsi/gomega v1.39.0
github.com/openshift/api v3.9.0+incompatible
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260115124008-0121df869109
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260116230254-f54dd51650ac
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260120112029-cd452f0497ba
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20251230215914-6ba873b49a35
github.com/openstack-k8s-operators/lib-common/modules/test v0.6.1-0.20251230215914-6ba873b49a35
github.com/openstack-k8s-operators/mariadb-operator/api v0.6.1-0.20260105160121-f7a8ef85ce8d
Expand Down
4 changes: 2 additions & 2 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,8 +120,8 @@ github.com/openshift/api v0.0.0-20250711200046-c86d80652a9e h1:E1OdwSpqWuDPCedyU
github.com/openshift/api v0.0.0-20250711200046-c86d80652a9e/go.mod h1:Shkl4HanLwDiiBzakv+con/aMGnVE2MAGvoKp5oyYUo=
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260115124008-0121df869109 h1:S+A67nntHZrL1lIL3qr91CpJj+A67M/G4t1cTKzeGdo=
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260115124008-0121df869109/go.mod h1:ZXwFlspJCdZEUjMbmaf61t5AMB4u2vMyAMMoe/vJroE=
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260116230254-f54dd51650ac h1:DZ/Cw3l4fQXTu2O78HAPIEhSYYZ7cR+QZv893Z+gvNU=
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260116230254-f54dd51650ac/go.mod h1:xqvebn9DqLavxp2z8Rz/7i1S6M9MJhxmZVHC+S1uHX0=
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260120112029-cd452f0497ba h1:4VaDkZFawGCkzwvfijnFLz0Gduxh17buj9fIwk0WULo=
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260120112029-cd452f0497ba/go.mod h1:xqvebn9DqLavxp2z8Rz/7i1S6M9MJhxmZVHC+S1uHX0=
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20251230215914-6ba873b49a35 h1:pF3mJ3nwq6r4qwom+rEWZNquZpcQW/iftHlJ1KPIDsk=
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20251230215914-6ba873b49a35/go.mod h1:kycZyoe7OZdW1HUghr2nI3N7wSJtNahXf6b/ypD14f4=
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.6.1-0.20251230215914-6ba873b49a35 h1:IdcI8DFvW8rXtchONSzbDmhhRp1YyO2YaBJDBXr44Gk=
Expand Down
8 changes: 8 additions & 0 deletions internal/controller/watcher_common.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ const (
tlsAPIPublicField = ".spec.tls.api.public.secretName"
topologyField = ".spec.topologyRef.Name"
memcachedInstanceField = ".spec.memcachedInstance"
authAppCredSecretField = ".spec.auth.applicationCredentialSecret" //nolint:gosec // G101: Not actual credentials, just field path
// service label for cinder endpoint
endpointCinder = "cinder"
)
Expand All @@ -60,6 +61,7 @@ var (
watcherWatchFields = []string{
passwordSecretField,
prometheusSecretField,
authAppCredSecretField,
}
decisionEngineWatchFields = []string{
passwordSecretField,
Expand Down Expand Up @@ -133,6 +135,12 @@ var (

// ErrTransportURLFieldMissing indicates that the TransportURL secret does not have the 'transport_url' field
ErrTransportURLFieldMissing = errors.New("the TransportURL secret does not have 'transport_url' field")

// ErrACSecretNotFound indicates that the ApplicationCredential secret was not found
ErrACSecretNotFound = errors.New("ApplicationCredential secret not found")

// ErrACSecretMissingKeys indicates that the ApplicationCredential secret is missing required keys
ErrACSecretMissingKeys = errors.New("ApplicationCredential secret missing required keys")
)

// GetLogger returns a logger object with a prefix of "controller.name" and additional controller context fields
Expand Down
72 changes: 70 additions & 2 deletions internal/controller/watcher_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,47 @@ func (r *WatcherReconciler) Reconcile(ctx context.Context, req ctrl.Request) (re
return ctrl.Result{}, ErrRetrievingTransportURLSecretData
}

// Try to get Application Credential from the secret specified in the CR
var acData *keystonev1.ApplicationCredentialData
if instance.Spec.Auth.ApplicationCredentialSecret != "" {
acSecretObj, _, err := secret.GetSecret(ctx, helper, instance.Spec.Auth.ApplicationCredentialSecret, instance.Namespace)
if err != nil {
if k8s_errors.IsNotFound(err) {
Log.Info("ApplicationCredential secret not found, waiting", "secret", instance.Spec.Auth.ApplicationCredentialSecret)
instance.Status.Conditions.Set(condition.FalseCondition(
condition.InputReadyCondition,
condition.RequestedReason,
condition.SeverityInfo,
watcherv1beta1.WatcherApplicationCredentialSecretErrorMessage))
return ctrl.Result{}, fmt.Errorf("%w: %s", ErrACSecretNotFound, instance.Spec.Auth.ApplicationCredentialSecret)
}
Log.Error(err, "Failed to get ApplicationCredential secret", "secret", instance.Spec.Auth.ApplicationCredentialSecret)
instance.Status.Conditions.Set(condition.FalseCondition(
condition.InputReadyCondition,
condition.ErrorReason,
condition.SeverityWarning,
watcherv1beta1.WatcherApplicationCredentialSecretErrorMessage))
return ctrl.Result{}, err
}
acID, okID := acSecretObj.Data[keystonev1.ACIDSecretKey]
acSecretData, okSecret := acSecretObj.Data[keystonev1.ACSecretSecretKey]
if okID && len(acID) > 0 && okSecret && len(acSecretData) > 0 {
acData = &keystonev1.ApplicationCredentialData{
ID: string(acID),
Secret: string(acSecretData),
}
Log.Info("Using ApplicationCredentials auth", "secret", instance.Spec.Auth.ApplicationCredentialSecret)
} else {
Log.Error(nil, "ApplicationCredential secret missing required keys", "secret", instance.Spec.Auth.ApplicationCredentialSecret)
instance.Status.Conditions.Set(condition.FalseCondition(
condition.InputReadyCondition,
condition.ErrorReason,
condition.SeverityWarning,
watcherv1beta1.WatcherApplicationCredentialSecretErrorMessage))
return ctrl.Result{}, fmt.Errorf("%w: %s", ErrACSecretMissingKeys, instance.Spec.Auth.ApplicationCredentialSecret)
}
}

// Prometheus config secret

hashPrometheus, _, prometheusSecret, err := ensureSecret(
Expand Down Expand Up @@ -354,7 +395,7 @@ func (r *WatcherReconciler) Reconcile(ctx context.Context, req ctrl.Request) (re

// End of Prometheus config secret

subLevelSecretName, err := r.createSubLevelSecret(ctx, helper, instance, transporturlSecret, notificationURLSecret, inputSecret, db)
subLevelSecretName, err := r.createSubLevelSecret(ctx, helper, instance, transporturlSecret, notificationURLSecret, inputSecret, db, acData)
if err != nil {
return ctrl.Result{}, nil
}
Expand All @@ -375,7 +416,7 @@ func (r *WatcherReconciler) Reconcile(ctx context.Context, req ctrl.Request) (re
// Generate config for dbsync
configVars := make(map[string]env.Setter)

err = r.generateServiceConfigDBJobs(ctx, instance, db, &transporturlSecret, helper, &configVars)
err = r.generateServiceConfigDBJobs(ctx, instance, db, &transporturlSecret, helper, &configVars, acData)
if err != nil {
instance.Status.Conditions.Set(condition.FalseCondition(
condition.ServiceConfigReadyCondition,
Expand Down Expand Up @@ -775,6 +816,7 @@ func (r *WatcherReconciler) generateServiceConfigDBJobs(
transporturlSecret *corev1.Secret,
helper *helper.Helper,
envVars *map[string]env.Setter,
acData *keystonev1.ApplicationCredentialData,
) error {
Log := r.GetLogger(ctx)
Log.Info("generateServiceConfigs - reconciling config for Watcher CR")
Expand Down Expand Up @@ -804,6 +846,12 @@ func (r *WatcherReconciler) generateServiceConfigDBJobs(
"APIPublicPort": fmt.Sprintf("%d", watcher.WatcherPublicPort),
}

// Add Application Credential data if provided
if acData != nil {
templateParameters["ACID"] = acData.ID
templateParameters["ACSecret"] = acData.Secret
}

return GenerateConfigsGeneric(ctx, helper, instance, envVars, templateParameters, customData, labels, true)
}

Expand Down Expand Up @@ -867,6 +915,7 @@ func (r *WatcherReconciler) createSubLevelSecret(
notificationURLSecret *corev1.Secret,
inputSecret corev1.Secret,
db *mariadbv1.Database,
acData *keystonev1.ApplicationCredentialData,
) (string, error) {
Log := r.GetLogger(ctx)
Log.Info(fmt.Sprintf("Creating SubCr Level Secret for '%s'", instance.Name))
Expand All @@ -884,6 +933,13 @@ func (r *WatcherReconciler) createSubLevelSecret(
watcher.GlobalCustomConfigFileName: instance.Spec.CustomServiceConfig,
NotificationURLSelector: string(notificationURLSecret.Data[TransportURLSelector]),
}

// Add Application Credential data if provided
if acData != nil {
data["ACID"] = acData.ID
data["ACSecret"] = acData.Secret
}

secretName := instance.Name

labels := labels.GetLabels(instance, labels.GetGroupLabel(watcher.ServiceName), map[string]string{})
Expand Down Expand Up @@ -1266,6 +1322,18 @@ func (r *WatcherReconciler) SetupWithManager(mgr ctrl.Manager) error {
return err
}

// index authAppCredSecretField
if err := mgr.GetFieldIndexer().IndexField(context.Background(), &watcherv1beta1.Watcher{}, authAppCredSecretField, func(rawObj client.Object) []string {
// Extract the secret name from the spec, if one is provided
cr := rawObj.(*watcherv1beta1.Watcher)
if cr.Spec.Auth.ApplicationCredentialSecret == "" {
return nil
}
return []string{cr.Spec.Auth.ApplicationCredentialSecret}
}); err != nil {
return err
}

return ctrl.NewControllerManagedBy(mgr).
For(&watcherv1beta1.Watcher{}).
Owns(&watcherv1beta1.WatcherAPI{}).
Expand Down
10 changes: 10 additions & 0 deletions internal/controller/watcherapi_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -479,6 +479,16 @@ func (r *WatcherAPIReconciler) generateServiceConfigs(
if string(secret.Data[NotificationURLSelector]) != "" {
templateParameters["NotificationURL"] = string(secret.Data[NotificationURLSelector])
}

// Application Credential data
if acID, ok := secret.Data["ACID"]; ok && len(acID) > 0 {
if acSecretData, ok := secret.Data["ACSecret"]; ok && len(acSecretData) > 0 {
templateParameters["ACID"] = string(acID)
templateParameters["ACSecret"] = string(acSecretData)
Log.Info("Using ApplicationCredentials auth")
}
}

// MTLS
if memcachedInstance.GetMemcachedMTLSSecret() != "" {
templateParameters["MemcachedAuthCert"] = fmt.Sprint(memcachedv1.CertMountPath())
Expand Down
9 changes: 9 additions & 0 deletions internal/controller/watcherapplier_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -442,6 +442,15 @@ func (r *WatcherApplierReconciler) generateServiceConfigs(
templateParameters["NotificationURL"] = string(secret.Data[NotificationURLSelector])
}

// Application Credential data
if acID, ok := secret.Data["ACID"]; ok && len(acID) > 0 {
if acSecretData, ok := secret.Data["ACSecret"]; ok && len(acSecretData) > 0 {
templateParameters["ACID"] = string(acID)
templateParameters["ACSecret"] = string(acSecretData)
Log.Info("Using ApplicationCredentials auth")
}
}

// MTLS
if memcachedInstance.GetMemcachedMTLSSecret() != "" {
templateParameters["MemcachedAuthCert"] = fmt.Sprint(memcachedv1.CertMountPath())
Expand Down
9 changes: 9 additions & 0 deletions internal/controller/watcherdecisionengine_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -593,6 +593,15 @@ func (r *WatcherDecisionEngineReconciler) generateServiceConfigs(
templateParameters["NotificationURL"] = string(secret.Data[NotificationURLSelector])
}

// Application Credential data
if acID, ok := secret.Data["ACID"]; ok && len(acID) > 0 {
if acSecretData, ok := secret.Data["ACSecret"]; ok && len(acSecretData) > 0 {
templateParameters["ACID"] = string(acID)
templateParameters["ACSecret"] = string(acSecretData)
Log.Info("Using ApplicationCredentials auth")
}
}

// MTLS
if memcachedInstance.GetMemcachedMTLSSecret() != "" {
templateParameters["MemcachedAuthCert"] = fmt.Sprint(memcachedv1.CertMountPath())
Expand Down
16 changes: 14 additions & 2 deletions templates/00-default.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,20 @@ memcache_tls_keyfile = {{ .MemcachedAuthKey }}
memcache_tls_cafile = {{ .MemcachedAuthCa }}
memcache_tls_enabled = true
{{ end }}
{{ if (index . "ACID") }}
auth_type = v3applicationcredential
application_credential_id = {{ .ACID }}
application_credential_secret = {{ .ACSecret }}
{{ else }}
project_domain_name = Default
project_name = service
user_domain_name = Default
password = {{ .ServicePassword }}
username = {{ .ServiceUser }}
auth_type = password
{{ end }}
auth_url = {{ .KeystoneAuthURL }}
interface = internal
auth_type = password
region_name = {{ .Region }}
{{ if .CaFilePath }}
cafile = {{ .CaFilePath }}
Expand All @@ -64,14 +70,20 @@ cafile = {{ .CaFilePath }}

{{ if (index . "KeystoneAuthURL") }}
[watcher_clients_auth]
{{ if (index . "ACID") }}
auth_type = v3applicationcredential
application_credential_id = {{ .ACID }}
application_credential_secret = {{ .ACSecret }}
{{ else }}
project_domain_name = Default
project_name = service
user_domain_name = Default
password = {{ .ServicePassword }}
username = {{ .ServiceUser }}
auth_type = password
{{ end }}
auth_url = {{ .KeystoneAuthURL }}
interface = internal
auth_type = password
{{ if .CaFilePath }}
cafile = {{ .CaFilePath }}
{{ end }}
Expand Down
Loading