How to add kubeletConfiguration and weight to provisioners

[abierbaum]

It doesn't look like the current API has support for kubeletConfiguration and weight when adding provisioners.

Is there any way to override the provisioner spec as part of adding a new one?

[robertd]

Not at the moment (since frankly I did not have a need for that feature yet), but PRs are always welcome. 😊

…

On Thu, Feb 16, 2023 at 3:06 PM Allen ***@***.***> wrote: It doesn't look like the current API has support for kubeletConfiguration and weight when adding provisioners. Is there any way to override the provisioner spec as part of adding a new one? — Reply to this email directly, view it on GitHub <#38>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAHWN2KDKBE5JPITAPKR53WX2JEDANCNFSM6AAAAAAU6VHBXY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[abierbaum]

@robertd I can work on a PR that adds support.

I was thinking of adding an escape hatch to allow people to override the Provisioner and AWSNodeTemplate resources before they get passed to addManifest. (ex: https://github.com/robertd/cdk-karpenter/blob/main/src/karpenter.ts#L554)

The idea being that as karpenter moves forward there will always be some little things that aren't supported yet by the simplified API this package provides. So if there was a way to override things after they are built then that could help people add whatever they need.

Maybe something like:

addProvisioner(id: string,  provisionerSpecs?: ProvisionerSpecs, overrideProvisioner?: (r: any) => any, overrideNodeTemplate?: (r: any) => any,)

The code internally would be something like:

const nodeTemplateRes = {
   apiVersion:  'karpenter.k8s.aws/v1alpha1',
      kind: 'AWSNodeTemplate',
      metadata: {
        name: awsNodeTemplateId,
      },
     ...
}
const finalNodeTemplateRes = overrideNodeTemplate == null ? nodeTemplateRes : overrideNodeTemplate(nodeTemplateRes);
this.cluster.addManifest(awsNodeTemplateId, finalNodeTemplateRes)

What would you think about something like this concept? What would you like the addProvisioner method call and arguments to look like?

Note: alternatively, the callbacks could be defined in the ProvisionerSpecs instead.

If you are receptive to any of these options and let me know what path you like here, I could work up a PR.

[abierbaum]

@robertd See: #39 I put in some ideas that I think should work, but wanted to get some feedback from you on the direction before going further.

[robertd]

Thanks for the PR. I’m out on a vacation until 2/27. I’ll review this PR when I get back.

…

On Fri, Feb 17, 2023 at 3:38 PM Allen ***@***.***> wrote: @robertd <https://github.com/robertd> See: #39 <#39> I put in some ideas that I think should work, but wanted to get some feedback from you on the direction before going further. — Reply to this email directly, view it on GitHub <#38 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAHWN5OM32NGJSQDN4WFETWX7OT5ANCNFSM6AAAAAAU6VHBXY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[abierbaum]

@robertd Thanks. More than happy to take it a different directly. Really just trying to provide maximum flexibility for people to override anything they want as an escape hatch.