Skip to content

Fix flaky collector tests#2079

Open
Dreamsorcerer wants to merge 1 commit into
mainfrom
sam/fix-flaky-collector
Open

Fix flaky collector tests#2079
Dreamsorcerer wants to merge 1 commit into
mainfrom
sam/fix-flaky-collector

Conversation

@Dreamsorcerer
Copy link
Copy Markdown
Collaborator

Problem

Some of the tests using the collector are flaky due to having more CPU contention with xdist.

Solution

Raise the timeouts further to handle this.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 13, 2026

Greptile Summary

Raises the default CallbackCollector timeout from 5.0 s to 15.0 s to prevent flaky test failures caused by CPU contention when tests run in parallel under pytest-xdist.

  • The __init__ default value for timeout in CallbackCollector is updated from 5.0 to 15.0; callers that pass an explicit timeout are unaffected.
  • The trade-off is that tests which genuinely hang (e.g. a message never arrives) will now take 15 s instead of 5 s before surfacing an AssertionError, tripling CI wait time on real failures — acceptable if flakiness is the bigger problem in practice.

Confidence Score: 5/5

Safe to merge — the only change is a larger default timeout that reduces test flakiness without affecting production code.

A single default argument is bumped from 5 s to 15 s in a test utility class. No logic, no data flow, and no production paths are touched. The only observable consequence is that genuinely broken tests take longer to report a timeout failure, which is an acceptable trade-off for eliminating false flakiness under heavy parallelism.

No files require special attention.

Important Files Changed

Filename Overview
dimos/utils/testing/collector.py Default timeout raised from 5.0 s to 15.0 s to reduce flakiness under xdist CPU contention — one-line change with no logic impact.

Sequence Diagram

sequenceDiagram
    participant Test as Test / Publisher
    participant CC as CallbackCollector
    participant EV as threading.Event

    Test->>CC: "CallbackCollector(n=3, timeout=15.0)"
    activate CC
    CC->>EV: Event()
    deactivate CC

    loop publish n messages
        Test->>CC: collector(msg, topic)
        CC->>CC: results.append((msg, topic))
        alt "len(results) >= n"
            CC->>EV: set()
        end
    end

    Test->>CC: wait()
    CC->>EV: "wait(timeout=15.0)"
    alt Event set within 15 s
        EV-->>CC: True
        CC-->>Test: returns (success)
    else Timeout after 15 s
        EV-->>CC: False
        CC-->>Test: raises AssertionError
    end
Loading

Reviews (1): Last reviewed commit: "Fix flaky collector tests" | Re-trigger Greptile

@codecov
Copy link
Copy Markdown

codecov Bot commented May 13, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant