[bpf-ci-bot] tc_tunnel/ip6gre subtests flake on s390x due to connect timeout

[kernel-patches-review-bot]

Summary

The tc_tunnel test's IPv6 GRE subtests (ip6gre_none, ip6gre_eth, ip6gre_mpls) intermittently fail on s390x with EINPROGRESS (errno 115) because the 1000ms connect timeout is insufficient for IPv6 neighbor resolution on s390x QEMU after repeated address flush/re-add cycles.

Failure Details

• Test / Component: tc_tunnel/ip6gre_none, tc_tunnel/ip6gre_eth, tc_tunnel/ip6gre_mpls
• Frequency: Occasional on s390x — observed in 3 out of 3 independent s390x test_progs/test_progs_cpuv4 runs on May 3, 2026
• Failure mode: Flaky — connect() returns EINPROGRESS (errno 115) due to SO_SNDTIMEO expiry during initial "connect without any encap" phase
• Affected architectures: s390x (QEMU emulation)
• CI runs observed:
  • https://github.com/kernel-patches/bpf/actions/runs/25285044414 (tc_tunnel/ip6gre_none, test_progs on s390x)
  • https://github.com/kernel-patches/bpf/actions/runs/25283723465 (tc_tunnel/ip6gre_mpls, test_progs_cpuv4 on s390x)
  • https://github.com/kernel-patches/bpf/actions/runs/25282558731 (tc_tunnel/ip6gre_eth, test_progs_cpuv4 on s390x)

Root Cause Analysis

The tc_tunnel test iterates through 18 tunnel subtests. Each subtest cycle calls subtest_cleanup() which flushes all addresses (ip a flush), then subtest_setup() which re-adds IPv6 addresses with nodad. The ip6gre subtests (indices 10-12 in the subtest array) run after 9 prior subtests have each performed this flush/re-add cycle.

On s390x QEMU, after this repeated churn of the kernel's IPv6 neighbor state, resolving the link-layer address for the veth peer (fd::2) via Neighbor Solicitation occasionally takes longer than 1000ms. The connect_client_to_server() function (test_tc_tunnel.c:171) used a hardcoded 1000 instead of the TIMEOUT_MS macro, and the macro itself was only 1000ms.

Key observations:

• The failure always occurs in the "connect without any encap" phase (before any tunnel is configured), confirming it's a basic IPv6 connectivity timing issue, not a tunnel convergence problem
• Exactly one of the three ip6gre subtests fails per run (random selection), and the others pass — indicating the issue self-resolves within the time it takes to run the other subtests
• Earlier IPv6 subtests (ip6tnl at index 2, ip6vxlan at index 6) never fail because fewer flush/re-add cycles have occurred by that point
• Later IPv6 subtests (ip6udp at indices 16-18) never fail because by then neighbor state has stabilized

The existing commit 2790db208b44 ("selftests/bpf: Improve tc_tunnel test reliability") increased TIMEOUT_MS from 500ms to 1000ms but left connect_client_to_server() using a separate hardcoded value and didn't account for s390x QEMU overhead.

Proposed Fix

The patch (0001-selftests-bpf-Fix-tc_tunnel-ip6gre-timeout-on-s390x.patch) makes two changes:

1. Increase TIMEOUT_MS from 1000ms to 2000ms — provides sufficient margin for IPv6 neighbor resolution on emulated architectures after repeated address churn
2. Use TIMEOUT_MS in connect_client_to_server() instead of hardcoded 1000 — ensures all timeout values are consolidated and can be tuned via a single macro

Impact

Without the fix, the ip6gre subtests will continue to randomly fail on s390x, causing approximately 1 spurious test failure per 3 s390x CI runs. This creates noise that masks real regressions and wastes developer time investigating false failures.

References

• Issue [bpf-ci-bot] tc_tunnel/udp_mpls times out on aarch64 due to insufficient connect timeout #475: Similar timeout issue affecting tc_tunnel/udp_mpls on aarch64 (same root cause, different arch/subtest)
• Commit 2790db208b44: Previous reliability improvement that raised timeout from 500ms to 1000ms
• Patch on lore: "[PATCH bpf-next v1] selftests/bpf: Improve tc_tunnel test reliability" (Jiayuan Chen, 2026-03-12)

[kernel-patches-review-bot]

0001-selftests-bpf-Fix-tc_tunnel-ip6gre-timeout-on-s390x.patch

From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: BPF CI Bot <bot+bpf-ci@kernel.org>
Date: Sun, 4 May 2026 00:00:00 +0000
Subject: [PATCH] selftests/bpf: Fix tc_tunnel ip6gre timeout on s390x

The tc_tunnel test's ip6gre subtests (ip6gre_none, ip6gre_eth,
ip6gre_mpls) intermittently fail on s390x with EINPROGRESS (errno 115)
due to connect() timeout during the initial "connect without any encap"
phase.

On s390x QEMU, after multiple subtest iterations of address
flush/re-add cycles, IPv6 neighbor resolution for the veth peer
occasionally takes longer than 1000ms. This manifests specifically in
the ip6gre subtests (indices 10-12) because they follow 9 previous
subtests that each flush and re-add addresses, churning the kernel's
IPv6 neighbor state.

Fix this by:
1. Increasing TIMEOUT_MS from 1000ms to 2000ms to provide sufficient
   margin for IPv6 neighbor resolution on emulated architectures.
2. Using the TIMEOUT_MS macro in connect_client_to_server() instead of
   a hardcoded 1000, ensuring all timeout values are consolidated.

The 2000ms timeout is still well below the 600s watchdog and provides
fast failure detection for genuine connectivity issues.

Observed failures across independent PRs:
  - https://github.com/kernel-patches/bpf/actions/runs/25285044414
  - https://github.com/kernel-patches/bpf/actions/runs/25283723465
  - https://github.com/kernel-patches/bpf/actions/runs/25282558731

Generated-by: BPF CI Bot (claude-opus-4-6-v1) <bot+bpf-ci@kernel.org>
Signed-off-by: BPF CI Bot <bot+bpf-ci@kernel.org>
---
 .../selftests/bpf/prog_tests/test_tc_tunnel.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tools/testing/selftests/bpf/prog_tests/test_tc_tunnel.c b/tools/testing/selftests/bpf/prog_tests/test_tc_tunnel.c
index 1aa7c9463980..d2aa51d595ce 100644
--- a/tools/testing/selftests/bpf/prog_tests/test_tc_tunnel.c
+++ b/tools/testing/selftests/bpf/prog_tests/test_tc_tunnel.c
@@ -39,7 +39,7 @@
 #define DEFAULT_TEST_DATA_SIZE	100
 #define GSO_TEST_DATA_SIZE	BUFFER_LEN

-#define TIMEOUT_MS			1000
+#define TIMEOUT_MS			2000
 #define TEST_PORT			8000
 #define UDP_PORT			5555
 #define MPLS_UDP_PORT			6635
@@ -168,7 +168,7 @@ static int check_server_rx_data(struct subtest_cfg *cfg,

 static struct connection *connect_client_to_server(struct subtest_cfg *cfg)
 {
-	struct network_helper_opts opts = {.timeout_ms = 1000};
+	struct network_helper_opts opts = {.timeout_ms = TIMEOUT_MS};
 	int family = cfg->ipproto == 6 ? AF_INET6 : AF_INET;
 	struct connection *conn = NULL;
 	int client_fd, server_fd;
--
2.43.0