Skip to content

chore: add CI, .gitignore, and 35 new tests#1

Open
SuperInstance wants to merge 1 commit intomainfrom
chore/add-ci-gitignore-tests
Open

chore: add CI, .gitignore, and 35 new tests#1
SuperInstance wants to merge 1 commit intomainfrom
chore/add-ci-gitignore-tests

Conversation

@SuperInstance
Copy link
Copy Markdown
Owner

@SuperInstance SuperInstance commented Apr 12, 2026

Summary

Adds CI, .gitignore, and expanded test coverage for the sandbox simulation framework.

Changes

  • .gitignore: Python bytecode, build artifacts, IDE files, test caches
  • GitHub Actions CI (ci.yml):
    • Test matrix: Python 3.10, 3.11, 3.12
    • Lint job: import structure check for all modules
  • 35 new tests in tests/test_fleet_compat.py:
    • Fleet compat (16 tests): failure-to-status mapping for all 5 FailureTypes, error code mapping, simulation_result_to_status (success/error/partial/cancelled), to_fleet_error wrapping, FleetError passthrough, simulation_error, enrich_step_record (2 skipped — require real fleet-stdlib)
    • Harness edge cases (8 tests): reset, agent removal, string failure type injection, empty queue step, DELAYED/DUPLICATE/CORRUPTED failure modes, max_steps limit
    • FailureInjector advanced (6 tests): has_planned_failures, multiple same-step injections, targeted injection with agent filter, consume-once semantics, rate boundary behavior
    • Trust scores (3 tests): increase on success response, decrease on error response, clamp to [0.0, 1.0]
    • Agent send (2 tests): send via harness returns True, send without harness returns False

Test Results

  • Before: 27 tests
  • After: 62 tests (60 passed, 2 skipped)

Files Added

  • .gitignore
  • .github/workflows/ci.yml
  • tests/test_fleet_compat.py

Staging: Open in Devin

- Add .gitignore for Python/IDE/build artifacts
- Add GitHub Actions CI (test matrix: 3.10, 3.11, 3.12 + import lint)
- Add tests/test_fleet_compat.py with 35 new tests covering:
  - Fleet compat (16 tests): failure-to-status mapping, error codes, simulation result status, FleetError wrapping, enrich_step_record
  - Harness edge cases (8 tests): reset, agent removal, string failure type, empty queue step, DELAYED/DUPLICATE/CORRUPTED failures, max_steps limit
  - FailureInjector advanced (6 tests): has_planned_failures, multiple same-step, targeted injection, consume-once, rate boundary
  - Trust scores (3 tests): increase on success, decrease on error, clamp to [0,1]
  - Agent send (2 tests): send via harness, send without harness
- Total: 60 passed, 2 skipped (require real fleet-stdlib)
Copy link
Copy Markdown

@beta-devin-ai-integration beta-devin-ai-integration bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View 4 additional findings in Devin Review.

Staging: Open in Devin

Comment on lines +16 to +17
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'src'))
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Wrong sys.path manipulation causes ImportError when running test file standalone

Line 17 adds src/ to sys.path (copied from test_extended.py), but lines 19-34 use from src.xxx imports (matching test_sandbox.py's style). These two patterns are incompatible: adding src/ to sys.path enables from harness.xxx imports (without the src. prefix), while from src.xxx imports require the project root on sys.path. Running python tests/test_fleet_compat.py fails immediately with ModuleNotFoundError: No module named 'src'. The file explicitly includes if __name__ == "__main__": unittest.main() at line 377 suggesting standalone execution is intended. Compare with tests/test_sandbox.py:14-17 which correctly adds the project root for the same from src.xxx import style.

Suggested change
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'src'))
# Ensure project root is on sys.path so ``src.xxx`` relative imports resolve.
_PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if _PROJECT_ROOT not in sys.path:
sys.path.insert(0, _PROJECT_ROOT)
Staging: Open in Devin

Was this helpful? React with 👍 or 👎 to provide feedback.

Debug

Playground

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant