UDP RSC HW offload and vmswitch

[cg129]

TCP RSC HW offload is not compatible with vmswitch therefore as long as the NIC is attached to a vmswitch, HW RSC is disabled on the NIC. Does UDP RSC HW offload have the similar incompatibility with VMSwitch as TCP?

[nibanks]

I believe vmswitch will have to be modified to support UDP RSC HW offload. cc @mskhot to comment.

[mskhot]

TCP RSC HW offload is not compatible with vmswitch therefore as long as the NIC is attached to a vmswitch, HW RSC is disabled on the NIC. Does UDP RSC HW offload have the similar incompatibility with VMSwitch as TCP?

There is no general compatibility issue between vmswitch & HW TCP RSC. We disabled HW RSC sometimes, because there was not much CPU cost saving from doing HW RSC (vs. SW RSC), and we can maintain a more consistent implementation with SW RSC. There were compatibility issues with filter drivers in-between the NIC & vmswitch that could not handle RSC packets.

[mskhot]

I believe vmswitch will have to be modified to support UDP RSC HW offload. cc @mskhot to comment.

Yes, as a protocol driver to the NIC, vmswitch would need to pass OIDs to enable/disable/configure RSC on the NIC. I assume UDP RSC is similar to TCP RSC in that even if the property is enabled on the NIC, it should remain non-operational until an OID is issued to enable it.

[nibanks]

pass OIDs to enable/disable/configure RSC on the NIC

We are going with a simpler model that doesn't require the OIDs. There is the status indication from the NIC to indicate support or not, and then the OOB data on the datapath. In the OS, we will support SW resegmentation if any component is incompatible.

[mtfriesen]

In theory, TCPIP task offloads should be disabled until OID_OFFLOAD_ENCAPSULATION is sent down by a protocol driver. This requirement is typically ignored.

[mskhot]

We are going with a simpler model that doesn't require the OIDs. There is the status indication from the NIC to indicate support or not, and then the OOB data on the datapath. In the OS, we will support SW resegmentation if any component is incompatible.

Most of the compat issues came from older filter or IM drivers that sit between the NIC & the OS code. For example, the legacy L2 Bridge for Wifi is not supporting RSC packets, causing HW RSC to fail when vswitch binds to the wifi adapter.

[cg129]

The URO spec says "In the absence of hardware URO, the existing software URO feature will continue to be available."

Does that mean some entities of the OS will coalesce UDP segments, if coalescing is desirable and the underlying NIC does not support HW UDP RSC?

[nibanks]

The OS already has coalescing logic in SW, but we're trying to optimize this further by pushing this down into HW. The text you quoted just means that even in if the HW doesn't support URO, the SW will continue to support it.

[cg129]

The OS already has coalescing logic in SW, but we're trying to optimize this further by pushing this down into HW. The text you quoted just means that even in if the HW doesn't support URO, the SW will continue to support it.

For legacy HW that doesn't support HW UDP RSC, does it make sense to implement in SW in the driver (duplicate the OS's work)?

Also, what is the guideline on tunnel packets to be processed? Should tunnel packets be ignored(not coalescing at all). If not to ignore, I'm particular interested in the following 3 scenarios:

1. VxLAN packets with inner eth frame containing a UDP packet. Attempt to coalesce tunneled UDP payload?
2. VxLAN packets with inner eth frame containing a non-UDP packet. Ignore, not to coalesce?
3. NVGRE packets with inner eth frame containing a UDP packet. Attempt to coalesce tunneled UDP payload?

[anrossi]

To answer your first question, for legacy HW that doesn't support UDP RSC, it's better to let Windows coalesce it, rather than update the NIC driver to do coalescing.

@mskhot does it make sense for the NIC to coalesce tunnel encapsulated UDP packets? Ultimately, we'd like to make this offload simple enough to enable as often as possible, in as many scenarios as possible.