OCPBUGS-65626: add service account to guard pod#2076
OCPBUGS-65626: add service account to guard pod#2076openshift-merge-bot[bot] merged 1 commit intoopenshift:masterfrom
Conversation
|
@ehearne-redhat: This pull request references Jira Issue OCPBUGS-65626, which is valid. The bug has been moved to the POST state. 3 validation(s) were run on this bug
Requesting review from QA contact: The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@ehearne-redhat: This pull request references Jira Issue OCPBUGS-65626, which is valid. 3 validation(s) were run on this bug
Requesting review from QA contact: The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@wangke19 @p0lyn0mial this PR is ready to review. Could you please take a look? :) |
|
/assign @ingvagabund @ingvagabund please take a look at this PR. I think you are the best person to take a look since you created the controller. Thanks! |
|
Hey @ingvagabund - thanks so much for your review! I have amended the changes and added unit tests. Tests are passing locally. Please take a look when you have the chance :) |
|
The PR looks good in overall. Thank you. Just few more nits. |
|
Can you also please open evidence PRs for the corresponding operators as wel? KCM-o, KA-o, KS-o. To see the CI goes green to avoid any hidden corners. |
|
@ingvagabund thanks so much for your review - I have completed the evidence PRs. I'll re-ping for review when the tests come back. :) |
|
Hey @ingvagabund the evidence PR's openshift/cluster-kube-controller-manager-operator#905 , openshift/cluster-kube-apiserver-operator#2026 , and openshift/cluster-kube-scheduler-operator#610 are now ready to review for evidence. I have attached proof of guard pods using their own service accounts in each. Please take a look when you have the chance. :) |
|
Brilliant :) Thank you. |
|
@p0lyn0mial for the final approval |
|
/lgtm cancel @ehearne-redhat can you please squash the commits before merging? |
ce096db to
1763302
Compare
1763302 to
2ebfe9b
Compare
9a4deb2 to
895b5de
Compare
4cad71f to
5c9bf94
Compare
| pdbGetter: pdbGetter, | ||
| pdbLister: kubeInformersForTargetNamespace.Policy().V1().PodDisruptionBudgets().Lister(), | ||
| saGetter: saGetter, | ||
| saLister: saLister, |
There was a problem hiding this comment.
let's take the lister from kubeInformersForTargetNamespace
There was a problem hiding this comment.
then let's add the informer to factory.New().WithInformers() below.
| if errors.IsNotFound(err) { | ||
| _, _, err = resourceapply.ApplyServiceAccount(ctx, c.saGetter, syncCtx.Recorder(), serviceAccount) | ||
| if err != nil { | ||
| klog.Errorf("Unable to create service account %v for Guard Pods: %v", serviceAccount.Name, err) |
There was a problem hiding this comment.
won't this ere be logged because we use WithSyncDegradedOnError ?
if yes, maybe we should skip the additional logging. (applies to the other places)
There was a problem hiding this comment.
OK, I will remove the redundant klog.Errorf from my changes.
| _, _, err = resourceapply.ApplyServiceAccount(ctx, c.saGetter, syncCtx.Recorder(), serviceAccount) | ||
| if err != nil { | ||
| klog.Errorf("Unable to create service account %v for Guard Pods: %v", serviceAccount.Name, err) | ||
| return fmt.Errorf("Unable to create service account %v for Guard Pods: %v", serviceAccount.Name, err) |
There was a problem hiding this comment.
also, I think that in go in general errors should not be capitalized.
There was a problem hiding this comment.
Yes, this is true. I was just following the convention from the other errors. I will remove those capital starts.
5c9bf94 to
eb3b963
Compare
p0lyn0mial
left a comment
There was a problem hiding this comment.
Maybe for consistency we should change the existing code so that it uses lower-case for errors, uses %w to wrap errors, and doesn't log and return the same error. We could do that in a new commit for easier review.
| pdbClient := b.kubeClient.PolicyV1() | ||
| saClient := b.kubeClient.CoreV1() | ||
| operandInformers := b.kubeNamespaceInformers.InformersFor(b.operandNamespace) | ||
| saLister := operandInformers.Core().V1().ServiceAccounts().Lister() |
There was a problem hiding this comment.
we take the lister from kubeInformersForTargetNamespace.Core().V1().ServiceAccounts().Lister() this is not needed.
| podClient, | ||
| pdbClient, | ||
| saClient, | ||
| saLister, |
There was a problem hiding this comment.
we take the lister from kubeInformersForTargetNamespace.Core().V1().ServiceAccounts().Lister() this is not needed.
| podGetter corev1client.PodsGetter, | ||
| pdbGetter policyclientv1.PodDisruptionBudgetsGetter, | ||
| saGetter corev1client.ServiceAccountsGetter, | ||
| saLister corelisterv1.ServiceAccountLister, |
There was a problem hiding this comment.
we take the lister from kubeInformersForTargetNamespace.Core().V1().ServiceAccounts().Lister() this is not needed.
| if err == nil { | ||
| _, _, err = resourceapply.DeleteServiceAccount(ctx, c.saGetter, syncCtx.Recorder(), serviceAccount) | ||
| if err != nil { | ||
| klog.Errorf("unable to delete Service Account: %v", err) |
There was a problem hiding this comment.
use %w for err wrapping.
There was a problem hiding this comment.
I'm getting this message when I try to use error wrapping
k8s.io/klog/v2.Errorf does not support error-wrapping directive %w
There was a problem hiding this comment.
won't this ere be logged because we use WithSyncDegradedOnError ?
if yes, maybe we should skip the additional logging. (applies to the other places
| errs = append(errs, err) | ||
| } | ||
| } else if !apierrors.IsNotFound(err) { | ||
| klog.Errorf("unable to get service account %v from lister: %v", serviceAccount.Name, err) |
There was a problem hiding this comment.
use %w for err wrapping.
There was a problem hiding this comment.
Same here
k8s.io/klog/v2.Errorf does not support error-wrapping directive %w
There was a problem hiding this comment.
won't this ere be logged because we use WithSyncDegradedOnError ?
if yes, maybe we should skip the additional logging. (applies to the other places)
There was a problem hiding this comment.
we could use fmt if we want to enhance the context and then add it to the errs
| if errors.IsNotFound(err) { | ||
| _, _, err = resourceapply.ApplyServiceAccount(ctx, c.saGetter, syncCtx.Recorder(), serviceAccount) | ||
| if err != nil { | ||
| return fmt.Errorf("Unable to create service account %v for Guard Pods: %v", serviceAccount.Name, err) |
There was a problem hiding this comment.
use %w for err wrapping.
There was a problem hiding this comment.
This one I can do. :)
| } | ||
| } | ||
|
|
||
| func TestGuardServiceAccountManifestStability(t *testing.T) { |
There was a problem hiding this comment.
could this test be updated to something like:
sa := resourceread.ReadServiceAccountV1OrDie(serviceAccountTemplate)
expected := &corev1.ServiceAccount{
ObjectMeta: metav1.ObjectMeta{
Name: "guard-sa",
Labels: map[string]string{"app": "guard"},
},
}
if !equality.Semantic.DeepEqual(sa, expected) {
t.Fatalf("guard-pod-sa.yaml manifest has changed. " +
"The controller only creates the SA, it does not update existing ones. " +
"If the manifest changed, add update logic and update the expected value in this test.")
}
?
eb3b963 to
9ec4053
Compare
|
@ehearne-redhat: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
6d44615 to
19a0412
Compare
| errs = append(errs, err) | ||
| } | ||
| } else if !apierrors.IsNotFound(err) { | ||
| klog.Errorf("unable to get service account %v from lister: %v", serviceAccount.Name, err) |
There was a problem hiding this comment.
won't this ere be logged because we use WithSyncDegradedOnError ?
if yes, maybe we should skip the additional logging. (applies to the other places)
| errs = append(errs, err) | ||
| } | ||
| } else if !apierrors.IsNotFound(err) { | ||
| klog.Errorf("unable to get service account %v from lister: %v", serviceAccount.Name, err) |
There was a problem hiding this comment.
we could use fmt if we want to enhance the context and then add it to the errs
| guardServiceAccount *corev1.ServiceAccount | ||
| createConditionalFunc func() (bool, bool, error) | ||
| withArbiter bool | ||
| expectSADeleted bool // true if service account should be deleted (cleanup mode) |
There was a problem hiding this comment.
let's rm the comment, not sure it is helpful.
| if err == nil { | ||
| _, _, err = resourceapply.DeleteServiceAccount(ctx, c.saGetter, syncCtx.Recorder(), serviceAccount) | ||
| if err != nil { | ||
| klog.Errorf("unable to delete Service Account: %v", err) |
There was a problem hiding this comment.
won't this ere be logged because we use WithSyncDegradedOnError ?
if yes, maybe we should skip the additional logging. (applies to the other places
| if errors.IsNotFound(err) { | ||
| _, _, err = resourceapply.ApplyServiceAccount(ctx, c.saGetter, syncCtx.Recorder(), serviceAccount) | ||
| if err != nil { | ||
| return fmt.Errorf("Unable to create service account %v for Guard Pods: %w", serviceAccount.Name, err) |
There was a problem hiding this comment.
isn't the go convention to start err with a lowercase letter ?
| // The guard pod uses automountServiceAccountToken: false. | ||
| // There's nothing meaningful to update. So update logic isn't needed right now. | ||
| _, err = c.saLister.ServiceAccounts(serviceAccount.Namespace).Get(serviceAccount.Name) | ||
| if errors.IsNotFound(err) { |
There was a problem hiding this comment.
let's use apierrors and remove"k8s.io/apimachinery/pkg/api/errors" from the imports.
| if err == nil { | ||
| _, _, err = resourceapply.DeleteServiceAccount(ctx, c.saGetter, syncCtx.Recorder(), serviceAccount) | ||
| if err != nil { | ||
| klog.Errorf("unable to delete Service Account: %v", err) |
There was a problem hiding this comment.
the other log msg use "service account" not "Service Account", let's be consistent.
There was a problem hiding this comment.
I believe this error log has been removed based on previous comment above. The consistency has been resolved anyway. :)
19a0412 to
546c960
Compare
This change add a bespoke service account to a guard pod, and introduces checks to ensure proper service account cleanup. Tests are included to test behaviour of service account in different scenarios. Test is included to ensure developers review code in case of Service Account manifest change.
546c960 to
c56f4c3
Compare
|
/approve /hold |
|
/lgtm |
|
/hold cancel |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ehearne-redhat, ingvagabund, p0lyn0mial The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@ehearne-redhat: Jira Issue OCPBUGS-65626: All pull requests linked via external trackers have merged: Jira Issue OCPBUGS-65626 has been moved to the MODIFIED state. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
This change adds a bespoke service account to the guard pod. It also handles service account cleanup, and includes additional fields in tests and basic service account testing.
The reason for the change is that we should opt to use a bespoke service account rather than default.