feat(sources): add gRPC max connection age#25660
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e9ac53ed0a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
e9ac53e to
04865ba
Compare
|
Updated the PR after review feedback. Changes pushed in
Validation rerun locally: fmt, targeted config/vector tests, targeted clippy, and Docker Compose E2E. The E2E now directly shows reconnects in debug logs after idle expiry while forwarded events continue and no Vector ERROR/panic lines appear. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 04865ba3c9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
04865ba to
9297bd8
Compare
|
Follow-up update for the remaining review thread:
Validation rerun on
A delegated code review pass was also clean after the final reconnect coverage was added. |
Summary
Adds optional gRPC server keepalive configuration for source-side gRPC servers:
keepalive.max_connection_age_secskeepalive.max_connection_age_grace_secsThe settings are wired into the Vector source and OpenTelemetry gRPC source, sharing the same gRPC server helper. Defaults preserve current behavior.
Closes #19457.
Design note
Tonic 0.11 does not expose native server
max_connection_age/ grace APIs. I tested a tonic 0.14 upgrade path, but it cascaded into broader hyper, generated-code, and Vector sink transport compatibility work. To keep this PR focused and reviewable, this implementation enforces the source-side gRPC connection lifetime at the accepted IO layer without changing sink behavior or upgrading tonic.A future tonic upgrade can replace this with native server-side max-age support once the wider tonic/hyper migration is handled separately.
Tests
make generate-component-docscargo fmt --all -- --checkcargo test --no-default-features --features sources-vector,sources-opentelemetry --lib config_keepalivecargo test --no-default-features --features sources-vector,sources-opentelemetry --lib config_grpc_keepalivecargo test --no-default-features --features sources-vector --lib sources::vector::test::max_connection_age_closes_idle_connectioncargo test --no-default-features --features sources-vector,sinks-vector --lib sources::vector::tests::receivecargo clippy --no-default-features --features sources-vector,sources-opentelemetry --lib -- -D warnings -A clippy::manual_option_zipdocker-buildx-built PR image withsources-vector,sinks-vector,sources-demo_logs,sinks-console: Vector source configured withkeepalive.max_connection_age_secs = 3received 76 forwarded events over 16 seconds with no client/server errors, confirming delivery continues past repeated server-side connection expiry.The Clippy allow is for an existing Rust 1.96 lint in unrelated config loading code.
Follow-up validation
After review feedback, the connection-age fallback was tightened so the local lifetime wrapper only closes an expired connection once that connection is idle. The active-request tracking is per accepted connection and is held until the response body is dropped, avoiding a global drain across unrelated connections and avoiding mid-response socket closure.
Additional local validation after the follow-up change:
cargo fmt --all -- --checkcargo test --no-default-features --features sources-vector --lib sources::util::grpc::tests::max_connection_age_service_trackscargo test --no-default-features --features sources-vector --lib sources::vector::test::max_connection_age_closes_idle_connectioncargo test --no-default-features --features sources-vector,sinks-vector --lib sources::vector::tests::receivecargo test --no-default-features --features sources-vector,sources-opentelemetry --lib config_keepalivecargo test --no-default-features --features sources-vector,sources-opentelemetry --lib config_grpc_keepalivecargo clippy --no-default-features --features sources-vector,sources-opentelemetry --lib -- -D warnings -A clippy::manual_option_zipdocker-buildx-built PR image withmax_connection_age_secs = 3andmax_connection_age_grace_secs = 1:ERROR/panic lines were observed