fix(ci): raise AVM check-circuit per-tx timeout to 120s (configurable)#24330
Draft
AztecBot wants to merge 2 commits into
Draft
fix(ci): raise AVM check-circuit per-tx timeout to 120s (configurable)#24330AztecBot wants to merge 2 commits into
AztecBot wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Raise the per-input
avm_check_circuittimeout inyarn-project/end-to-end/bootstrap.shfrom the hard-coded30sto a120sdefault, configurable viaAVM_CHECK_CIRCUIT_TIMEOUT.Root cause
The
avm-check-circuitjob onnextfailed with exit code124(timeout): run 28236450261. The job-level exit propagated from a single timed-out child command:Each check-circuit command runs under
ISOLATE=1:TIMEOUT=30s. Thee2e_multiple_blobstest emits more than a blob's worth of public data, producing a long AVM trace (~700k rows in a prior occurrence) whoseChecking circuit...step needs ~32–35s under the default 2-CPU isolated CI quota — over the 30s budget. There was no circuit assertion or exception; the check was progressing correctly and was killed bytimeoutat the limit. Every other tx passed, including a second, smallere2e_multiple_blobstx (2s). The in-codeWARNINGcomment already anticipated exactly this.This is not a regression from the triggering commit (#24305, "remove the TS AVM simulator"). That PR only touches the TS simulator package and C++ fuzzer entry points — it modifies neither the C++ AVM circuit checker (
barretenberg/cpp/.../vm, thebb-avmbinary) nor the check-circuit invocation/timeout inbootstrap.sh/ci.sh. The checker's performance and the timeout are unchanged by it; the heavy tx's check-circuit time has simply crept up to the limit.History (please merge promptly)
This exact fix has been proposed and lost three times without ever being technically rejected:
cb/avm-check-circuit-timeout-60sbranch from 2026-06-20 carries the same fix with no PR.The job will keep failing on every heavy run until this lands. This PR reuses #24137's already-reviewed approach (120s default +
AVM_CHECK_CIRCUIT_TIMEOUToverride) rebased onto currentnext. Please mark it ready and merge rather than let it go stale again.Verification
bash -O extglob -n bootstrap.shpasses.TIMEOUT=120s;AVM_CHECK_CIRCUIT_TIMEOUT=75semitsTIMEOUT=75s.Created by claudebox · group:
slackbot