feat: Structured JSON output from child runners + pre/post test hooks

[witqq]

Summary

Two related improvements for better orchestration and validation.

1. Structured JSON Output from Child Runners

Child test runners should output structured JSON that the orchestrator can parse directly, instead of relying on file-based communication.

Current flow:

orchestrator -> spawns child runner -> child writes to file -> orchestrator reads file

Proposed flow:

orchestrator -> spawns child runner -> child outputs JSON to stdout -> orchestrator parses directly

Benefits:

• No file I/O race conditions
• Easier to add custom metadata
• Child runners can report progress incrementally
• Cleaner error handling

Example child output:

{
  "suite": "unit",
  "passed": 150,
  "failed": 2,
  "skipped": 5,
  "duration": 12500,
  "failures": [...],
  "metadata": {
    "nodeVersion": "20.10.0",
    "testFramework": "jest@29.7.0"
  }
}

2. Pre/Post Test Hooks

Allow running custom scripts before all tests start and after all tests complete.

Use Case: Invariant Checking

Some code paths should NEVER be called during tests (e.g., production error handlers, deprecated functions). We can verify this with counters:

// In code being tested
let deprecatedCallCount = 0;
export function deprecatedFunction() {
  deprecatedCallCount++;
  // ...
}

// Exposed for testing
export function getDeprecatedCallCount() {
  return deprecatedCallCount;
}

Config:

export default {
  hooks: {
    beforeAll: './scripts/reset-counters.ts',
    afterAll: './scripts/check-invariants.ts',
  },
  suites: [...]
} satisfies Config;

Example afterAll script:

// scripts/check-invariants.ts
import { getDeprecatedCallCount } from '../src/deprecated.js';
import { getProductionErrorCount } from '../src/errors.js';

const issues: string[] = [];

if (getDeprecatedCallCount() > 0) {
  issues.push(`⚠️ Deprecated function called ${getDeprecatedCallCount()} times during tests`);
}

if (getProductionErrorCount() > 0) {
  issues.push(`⚠️ Production error handler triggered ${getProductionErrorCount()} times`);
}

if (issues.length > 0) {
  console.log('\n⚠️ POST-TEST WARNINGS:');
  issues.forEach(i => console.log(`  ${i}`));
  // Optionally fail the build
  // process.exit(1);
}

Hook Output in Report:

==========================================
  TEST SUMMARY REPORT
==========================================

Suite         Passed  Failed  Time
Unit             150       0  12.5s
E2E               45       0  30.2s

✓ ALL TESTS PASSED

⚠️ POST-TEST WARNINGS:
  - Deprecated function called 3 times during tests
  - Consider updating: tests/unit/legacy.test.ts

==========================================

Benefits:

• Catch unintended code paths being hit
• Track test coverage of deprecated APIs
• Validate test environment state
• Custom metrics collection (memory, connections, etc.)