Enhance test suites with Effect-TS for improved reliability and composability

[Soushi888]

Problem Statement

The current test infrastructure faces several challenges with async test coordination, error handling, and resource management in Holochain tryorama tests:

Current Pain Points

• Complex Async Coordination: Manual Promise chaining and callback patterns in multi-agent scenarios lead to difficult-to-debug test failures
• Resource Management: Manual conductor/player lifecycle management with inconsistent cleanup patterns
• Error Handling: Insufficient error context when DHT sync operations fail or timeout
• Test Flakiness: Inconsistent DHT synchronization timing causing intermittent failures
• Limited Observability: Debugging failing tests requires manual console logging and verbose mode
• Code Duplication: Repeated setup/teardown patterns across test files without proper abstraction

Technical Analysis

Current test architecture uses:

• Vitest 3.1.3 + @holochain/tryorama 0.18.2 for Holochain testing
• 4-layer testing strategy: Foundation → Integration → Scenarios → Performance
• Complex multi-agent test coordination with manual resource cleanup
• Async/Promise-based patterns with callback hell in scenario setup

Example of current complexity:

test("Complete user onboarding workflow", async () => {
  await runScenarioWithTwoAgents(
    async (_scenario: Scenario, alice: PlayerApp, bob: PlayerApp) => {
      // Manual Promise chaining
      const alicePersonResult = await createPerson(alice.cells[0], samplePerson({...}));
      await storePrivateData(alice.cells[0], samplePrivateData({...}));
      // ... complex nested async operations with limited error handling
    }
  );
});

Effect-TS Benefits

Effect-TS provides powerful patterns that directly address our testing challenges:

🛡️ Typed Error Handling

• Effect.tryPromise for converting Promise-based tryorama calls with typed error channels
• Structured error types replace generic Promise rejections
• Fail-fast behavior with detailed error context

🧹 Automatic Resource Cleanup

• Scope for guaranteed cleanup of conductors, players, and connections
• Automatic resource disposal even when tests fail
• Eliminates manual scenario.cleanUp() calls

⚡ Fiber-Based Concurrency

• Built-in concurrency primitives for multi-agent operations
• Structured concurrency prevents resource leaks and orphaned operations
• Timeout handling built into the Effect runtime

📊 Built-in Observability

• Automatic tracing and metrics collection
• Structured logging with correlation IDs
• Performance metrics collection without manual instrumentation

🧱 Composable Test Building Blocks

• Pure functions for test data generation
• Reusable service patterns for common operations
• Pipeline composition for complex test scenarios

Implementation Areas

1. Agent Management Service

interface AgentManagementService {
  createTestConductor: (config: ConductorConfig) => Effect.Effect<Conductor, AgentError, Scope>
  setupMultiAgentScenario: (agentCount: number) => Effect.Effect<PlayerApp[], AgentError, Scope>  
  syncDHT: (agents: PlayerApp[]) => Effect.Effect<void, SyncError>
}

2. DHT Sync Operations

interface DHTSyncService {
  waitForSync: (agents: PlayerApp[], timeout?: Duration) => Effect.Effect<void, SyncTimeout>
  retryableSync: (operation: Effect.Effect<T, E>) => Effect.Effect<T, E>
  batchOperations: <T>(operations: Effect.Effect<T, E>[]) => Effect.Effect<T[], E>
}

3. Test Data Factories

interface TestDataService {
  generatePerson: (overrides?: Partial<Person>) => Effect.Effect<Person, never>  
  generatePrivateData: (personId: string) => Effect.Effect<PrivateData, never>
  generateCommitment: (config: CommitmentConfig) => Effect.Effect<Commitment, never>
}

4. Performance Testing

interface MetricsService {
  measureOperation: <T, E>(operation: Effect.Effect<T, E>) => Effect.Effect<T & Metrics, E>
  collectSystemMetrics: () => Effect.Effect<SystemMetrics, MetricsError>
  generatePerformanceReport: (results: TestResults[]) => Effect.Effect<Report, ReportError>
}

5. Scenario Orchestration

interface ScenarioService {
  composeWorkflow: <T>(steps: Effect.Effect<T, E>[]) => Effect.Effect<T[], E>
  conditionalSteps: <T>(condition: boolean, step: Effect.Effect<T, E>) => Effect.Effect<Option<T>, E>
  parallelScenarios: <T>(scenarios: Effect.Effect<T, E>[]) => Effect.Effect<T[], E>
}

Migration Strategy

Phase 1: Foundation (1 week)

• Add Effect-TS dependencies to tests package.json
• Create basic service interfaces and error types
• Wrap existing runScenarioWithTwoAgents utility in Effect
• Implement AgentManagementService with Scope-based cleanup

Phase 2: Core Services (2 weeks)

• Implement DHTSyncService with retry policies and timeouts
• Create TestDataService with pure factory functions
• Convert person management tests to use Effect patterns
• Add structured logging and error context

Phase 3: Advanced Features (2 weeks)

• Implement MetricsService for performance testing
• Add ScenarioService for composable workflows
• Create advanced concurrency patterns for complex scenarios
• Add distributed tracing for multi-agent operations

Phase 4: Migration & Documentation (1 week)

• Convert remaining test suites to Effect patterns
• Create documentation and examples for team adoption
• Performance comparison between old and new approaches
• Team training on Effect-TS testing patterns

Acceptance Criteria

✅ Dependencies & Setup

• effect and related packages added to tests/package.json
• TypeScript configuration updated for Effect-TS compatibility
• Basic service architecture documented

✅ Core Infrastructure

• AgentManagementService implemented with automatic cleanup
• DHTSyncService with retry policies and structured timeouts
• Error types defined for all major failure modes
• TestDataService providing pure factory functions

✅ Migration Evidence

• At least one complete test file converted to Effect patterns
• Performance metrics showing improved reliability (reduced flaky test rate)
• Demonstrable improvement in error context and debugging experience
• Backward compatibility maintained during migration

✅ Documentation & Adoption

• README section explaining Effect-TS testing patterns
• Code examples showing common testing scenarios
• Migration guide for converting existing tests
• Team knowledge transfer completed

Technical Considerations

Integration Requirements

• Vitest Compatibility: Effect-TS tests must work seamlessly with existing Vitest runner
• Tryorama Integration: Effect services must wrap tryorama APIs without breaking functionality
• TypeScript Configuration: Ensure proper type inference and error reporting

Performance Impact

• Bundle Size: Effect-TS adds ~150KB to test bundle (acceptable for test environment)
• Runtime Overhead: Effect runtime adds <5ms per test (minimal impact)
• Memory Usage: Monitor memory usage with Scope-based resource management

Team Adoption

• Learning Curve: Effect-TS patterns require 1-2 week adoption period
• Documentation: Comprehensive examples and patterns documentation needed
• Pair Programming: Knowledge transfer through collaborative implementation

Backward Compatibility

• Gradual Migration: Existing tests continue working during transition
• Utility Compatibility: Current helper functions remain available
• Configuration: No breaking changes to test runner configuration

Related Issues

This enhancement supports several other initiatives:

• PPR System Testing: Complex multi-agent scenarios benefit from Effect composition
• Performance Testing: Built-in metrics collection for load testing
• CI/CD Reliability: Reduced test flakiness improves deployment confidence

Dependencies

• Requires Effect-TS packages: effect, @effect/platform, @effect/schema
• Compatible with current Vitest 3.1.3 and @holochain/tryorama 0.18.2
• No breaking changes to existing test infrastructure during migration

---

Priority: P2-high (testing infrastructure improvement)
Estimated Effort: 6 weeks (phased implementation)
Impact: Improved test reliability, better error handling, enhanced developer experience

[Soushi888]

Closing as superseded by the Sweettest migration (#61, merged in commit 3b4d0d0).

This issue proposed wrapping Tryorama with Effect-TS to solve async coordination, error handling, and resource cleanup problems. All of those problems are now addressed differently: Tryorama is being replaced by Sweettest (Holochain's official Rust-based test framework), which handles conductor lifecycle, multi-agent scenarios, and DHT sync natively in Rust without needing Effect-TS scaffolding on top.

Effect-TS remains the planned approach for the frontend service layer (#7) — that direction is unchanged. But it has no role in the backend test infrastructure going forward.

New NDO Layer 0 test coverage will be written directly in Sweettest. See the upcoming NDO Layer 0 milestone for specifics.