Skip to content

aggregator/pkg/handlers: fix post-teardown log panic#1167

Merged
makramkd merged 3 commits into
mainfrom
mk/CCIP-11671
Jun 9, 2026
Merged

aggregator/pkg/handlers: fix post-teardown log panic#1167
makramkd merged 3 commits into
mainfrom
mk/CCIP-11671

Conversation

@makramkd

@makramkd makramkd commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Description

TestBatchWriteCommitCCVNodeDataHandler_CancelledContextReturnsImmediately
was frequently panicking in CI with:

panic: Log in goroutine after TestBatchWriteCommitCCVNodeDataHandler_CancelledContextReturnsImmediately has completed

The batch handler returns early when the context is cancelled, but the
per-item worker goroutines keep running until they unblock from
CheckAggregation. Those goroutines then log "failed to trigger
aggregation" via the zaptest logger — which holds a reference to the
now-torn-down *testing.T and panics.

The fix is test-side only: replace logger.TestSugared(t) with
logger.Sugared(logger.Nop()) in this specific test. The nop logger is
safe to call after the test exits. The handler's error log is correct
production behaviour and is intentionally left unchanged.

The panic was seen on two separate CI runs on the same day
(PR #1166 and the merge queue), always with the same message ID,
confirming a deterministic race in the test setup rather than random
flakiness.

Testing

Ran the previously failing test 50 consecutive times with -count=50:

go test ./aggregator/pkg/handlers/...
  -run TestBatchWriteCommitCCVNodeDataHandler_CancelledContextReturnsImmediately
  -count=50

Checklist

  • Breaking changes documented in changelog (see changelog directory)
  • Cross link related PRs (in this or other repositories)
  • just lint fix - no new lint errors
  • just generate - mocks and protobufs are up to date

makramkd added 2 commits June 9, 2026 19:15
- Replace TestSugared(t) with Nop logger in cancelled-context
  test; worker goroutines outlive t in Go 1.26+ and panic if
  they log after teardown
@makramkd makramkd marked this pull request as ready for review June 9, 2026 16:28
@makramkd makramkd requested a review from a team as a code owner June 9, 2026 16:28
winder
winder previously approved these changes Jun 9, 2026
bukata-sa
bukata-sa previously approved these changes Jun 9, 2026
@makramkd makramkd dismissed stale reviews from bukata-sa and winder via bb14b55 June 9, 2026 16:41
@github-actions

github-actions Bot commented Jun 9, 2026

Copy link
Copy Markdown

Code coverage report:

Package main mk/CCIP-11671 Diff
github.com/smartcontractkit/chainlink-ccv/aggregator 49.35% 49.34% -0.01%
github.com/smartcontractkit/chainlink-ccv/bootstrap 54.14% 54.14% +0.00%
github.com/smartcontractkit/chainlink-ccv/cli 65.13% 65.13% +0.00%
github.com/smartcontractkit/chainlink-ccv/cmd 15.54% 15.54% +0.00%
github.com/smartcontractkit/chainlink-ccv/common 54.76% 54.76% +0.00%
github.com/smartcontractkit/chainlink-ccv/executor 45.97% 45.97% +0.00%
github.com/smartcontractkit/chainlink-ccv/indexer 36.32% 36.32% +0.00%
github.com/smartcontractkit/chainlink-ccv/integration 46.13% 46.13% +0.00%
github.com/smartcontractkit/chainlink-ccv/pkg 84.62% 84.62% +0.00%
github.com/smartcontractkit/chainlink-ccv/pricer 0.00% 0.00% +0.00%
github.com/smartcontractkit/chainlink-ccv/protocol 63.06% 63.06% +0.00%
github.com/smartcontractkit/chainlink-ccv/verifier 34.48% 34.48% +0.00%
Total 46.50% 46.50% +0.00%

@makramkd makramkd requested a review from bukata-sa June 9, 2026 16:56
@makramkd makramkd enabled auto-merge June 9, 2026 16:57
@makramkd makramkd added this pull request to the merge queue Jun 9, 2026
Merged via the queue into main with commit 1251b7d Jun 9, 2026
35 checks passed
@makramkd makramkd deleted the mk/CCIP-11671 branch June 9, 2026 17:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants