Feat: cilium routing mode setting

[tico88612]

What type of PR is this?

/kind feature

What this PR does / why we need it:

Some users want independent variables and hope to be able to configure them like tunnelProtocol.

Which issue(s) this PR fixes:

Fixes #12541

Does this PR introduce a user-facing change?:

Add cilium_routing_mode independent variable for Cilium routingMode

[tico88612]

/retest

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mhkarimi1383, tico88612

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tico88612]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[VannTen]

At this point can't we just have a values-overrides thing which we'd feed directly to cilium cli ?

[mhkarimi1383]

At this point can't we just have a values-overrides thing which we'd feed directly to cilium cli ?

If we have tunnel_mode we should have routing_mode too since they are related, But we need some place to be able to pass some values/params in raw

[mhkarimi1383]

I'm unable to use kubespray without custom patch applied every time

It is a needed thing to set in some cases and it makes the kubespray unusable

[ayoubfaouzi]

if cilium_routing_mode is native, we should skip routingMode: {{ cilium_routing_mode }} entirely, otherwise, cilium complains.

[mhkarimi1383]

I this should be allowed here and should be passed directly to cilium, for example if cilium introduce a new feature or something like that we can't handle, I think keeping this simple is a better approach

[tico88612]

@ayoubfaouzi Why? If you skip this, that's not working. routingMode default is tunnel.

[ayoubfaouzi]

Oh sorry, what I wanted to say was, when I deployed kubespray with cilium as CNI, I want to use native routing mode, so no tunneling. I modified cilium_tunnel_mode to disabled as in the doc.

However cilium kept complaining about disabled, looks like it did not get it (should be only vxlan or geneve), so I had to edit the cm and completely get rid of cilium_tunnel_mode and add routing-mode: native.

[tico88612]

Or maybe you try an empty string? the cilium docs said cilium_tunnel_mode only accepted "", "vxlan", "geneve"

[satishdotpatel]

Yes, I am on same boat. cilium_tunnel_mode only accept vxlan or geneve. I tried empty string which started throwing errors.

For now I just kept tunnel-protocol to vxlan. Hope its acceptable value with routing mode native.

[VannTen]

I don't understand why we'd need this when we already have cilium_extra_values. Isn't that enough to pass stuff to cilium-cli ?

[mhkarimi1383]

I don't understand why we'd need this when we already have cilium_extra_values. Isn't that enough to pass stuff to cilium-cli ?

It is needed as we have a separate variable for tunnelProtocol
They are needed together

[VannTen]

The fact that one variable exists is not justification enough to add another one.
Is there a dependency between the two ? (as in, the value of one depend on the other)

From what I can see, the values.yml.j2 has already started duplicating the values of the helm chart, which is a nightmare for maintainability.

@tico88612 Am I missing something here ? The file feeded to cilium-cli looks like it has very little logic in it, basically just translate ansible variables into cilium-cli values.

(In that case we should IMHO stop adding to it and just use the extra values stuff which is already there).

[tico88612]

To the best of my knowledge, for the existing variable cilium_tunnel_mode to be effective, cilium_routing_mode must be set to tunnel (which happens to be the official Cilium default).
However, some users set cilium_routing_mode to native, in which case cilium_tunnel_mode needs to be changed to an empty string. It can be said that there is a dependency between them.

That said, there are still some alternative approaches, which would end up looking like this:

cilium_tunnel_mode: ""

cilium_extra_values:
  routingMode: "native"

TBH, I'm actually quite hesitant about this point, and I don't want it to become another copied helm values.

[satishdotpatel]

I have used following setting to activate routing mode native.

cilium_config_extra_vars:
  routing-mode: "native"

This is what my current config view looks

# cilium config view | grep -e tunnel -e native
ipv4-native-routing-cidr                          10.233.0.0/16
routing-mode                                      native
tunnel-protocol                                   vxlan
tunnel-source-port-range                          0-0

Question, does this confuse cilium that routing mode is native but tunnel-protocol is vxlan so routing may create issue in future? I didn't find a way to delete tunnel-protocol flag using kubespray.