Skip to content

TEST_SPECS.md

Latest commit

 

History

History
433 lines (360 loc) · 10.6 KB

TEST_SPECS.md

File metadata and controls

433 lines (360 loc) · 10.6 KB

Test Specifications for VOIDSTUDIO Requirements

Bug Agent Testing Guide Last Updated: 2026-01-10

Testing Methodology

For each completed requirement, the Bug Agent will:

  1. Verify file/feature existence
  2. Test basic functionality
  3. Run syntax/validation checks
  4. Test error conditions
  5. Verify integration points
  6. Run any available unit tests
  7. Document results in BUG_FINDINGS.md

Phase 1.0: Project Setup Tests

REQ-001: pnpm-workspace.yaml

Test Cases:

  • File exists at /home/user/Tradingpoly/pnpm-workspace.yaml
  • Valid YAML syntax
  • Contains packages array
  • Includes apps/* and packages/* patterns

REQ-002: Root package.json

Test Cases:

  • File exists at /home/user/Tradingpoly/package.json
  • Valid JSON syntax
  • Contains "workspaces" field
  • Contains "name" field
  • Contains "private": true
  • Contains "scripts" object

REQ-003: Root tsconfig.json

Test Cases:

  • File exists at /home/user/Tradingpoly/tsconfig.json
  • Valid JSON syntax
  • Contains "compilerOptions"
  • Contains path aliases configuration
  • References workspace packages

REQ-004: .nvmrc

Test Cases:

  • File exists at /home/user/Tradingpoly/.nvmrc
  • Contains Node.js version 20.x
  • Version format is valid

REQ-005: .npmrc

Test Cases:

  • File exists at /home/user/Tradingpoly/.npmrc
  • Contains pnpm-specific settings
  • No secrets in file

REQ-006: .gitignore

Test Cases:

  • File exists at /home/user/Tradingpoly/.gitignore
  • Includes node_modules
  • Includes dist/build directories
  • Includes .env files
  • Includes OS-specific files

REQ-007: .env.example

Test Cases:

  • File exists at /home/user/Tradingpoly/.env.example
  • Contains all required API key placeholders
  • No actual secrets present
  • Includes comments for each variable

REQ-008: turbo.json

Test Cases:

  • File exists at /home/user/Tradingpoly/turbo.json
  • Valid JSON syntax
  • Contains "pipeline" configuration
  • Defines build dependencies

REQ-009: .prettierrc

Test Cases:

  • File exists at /home/user/Tradingpoly/.prettierrc
  • Valid JSON/YAML syntax
  • Contains formatting rules

REQ-010: .eslintrc.js

Test Cases:

  • File exists at /home/user/Tradingpoly/.eslintrc.js
  • Valid JavaScript syntax
  • Exports configuration object
  • Contains rules configuration

REQ-011-024: Directory Structure

Test Cases:

  • apps/ directory exists
  • apps/cli/ directory exists
  • apps/web/ directory exists
  • apps/mcp-server/ directory exists
  • packages/ directory exists
  • packages/core/ directory exists
  • packages/types/ directory exists
  • pipelines/ directory exists
  • remotion/ directory exists
  • templates/ directory exists
  • python/ directory exists
  • config/ directory exists
  • assets/ directory exists with .gitkeep
  • output/ directory exists with .gitkeep

REQ-025-035: Package Initialization

Test Cases:

  • packages/core/package.json exists and valid
  • packages/types/package.json exists and valid
  • apps/cli/package.json exists and valid
  • TypeScript installed (check node_modules)
  • tsx installed
  • vitest installed
  • zod installed
  • commander installed
  • All tsconfig.json files valid

Phase 1.1: Core Types Package Tests

REQ-036-045: Base Types

Test Cases:

  • All interfaces exported from packages/types
  • TypeScript compilation succeeds
  • No type errors
  • Interfaces have JSDoc comments

REQ-046-055: Generation Types

Test Cases:

  • All generation interfaces defined
  • Types match provider requirements
  • No circular dependencies

REQ-056-065: Pipeline Types

Test Cases:

  • All pipeline interfaces defined
  • Event types properly structured
  • Error types comprehensive

REQ-066-075: Zod Schemas

Test Cases:

  • All schemas export successfully
  • Schema validation works
  • Invalid data rejected correctly
  • Default values applied
  • Error messages clear

REQ-076-082: Export Configuration

Test Cases:

  • Barrel export works (index.ts)
  • ESM build succeeds
  • CJS build succeeds
  • Can import from package

Phase 1.2: Core Package Tests

REQ-083-092: Configuration Management

Test Cases:

  • loadConfig() loads .env correctly
  • saveConfig() writes files
  • getEnvVar() validates required vars
  • ConfigManager singleton works
  • YAML config parsing works
  • Config merging works correctly

REQ-093-102: Logger

Test Cases:

  • Logger instantiates
  • All log levels work
  • JSON format works
  • Pretty format works
  • File logging works
  • Child logger works
  • Silent mode works

REQ-103-112: File System Utilities

Test Cases:

  • ensureDir() creates directories
  • writeFile() atomic writes work
  • readFile() handles encodings
  • copyFile() works correctly
  • moveFile() works correctly
  • deleteFile() works correctly
  • listFiles() glob patterns work
  • Temp file utilities work

REQ-113-122: Asset Management

Test Cases:

  • AssetManager instantiates
  • registerAsset() works
  • getAsset() retrieves correctly
  • deleteAsset() works
  • listAssets() works
  • Hash generation works
  • Metadata extraction works

REQ-123-132: Error Handling

Test Cases:

  • All error classes extend base
  • Error serialization works
  • Error codes unique
  • isRetryable() works correctly

Phase 1.3: CLI Application Tests

REQ-133-142: CLI Framework

Test Cases:

  • CLI entry point executes
  • Commander parses commands
  • --version flag works
  • --help flag works
  • Global flags work
  • Error handling works

REQ-143-152: Init Command

Test Cases:

  • void init runs without errors
  • Project directory created
  • Config files created
  • Success message displayed
  • --template flag works
  • --force flag works

REQ-153-162: Config Command

Test Cases:

  • void config show displays config
  • void config set updates values
  • void config get retrieves values
  • Sensitive values masked
  • Validation works
  • Nested keys work

REQ-163-172: Voice Command

Test Cases:

  • void voice "text" works
  • --input file flag works
  • --provider flag works
  • --voice flag works
  • --output flag works
  • Progress indicator shows
  • Output file created
  • API errors handled

Phase 1.4: Voice Pipeline Tests

REQ-173-182: Provider Abstraction

Test Cases:

  • VoiceProvider interface defined
  • VoiceProviderFactory works
  • Provider registration works
  • Provider lookup works
  • Health check works

REQ-183-195: ElevenLabs Provider

Test Cases:

  • Provider instantiates
  • API key validation works
  • generateVoice() returns audio
  • listVoices() returns list
  • Rate limiting works
  • Error handling works
  • Audio buffer returned

REQ-196-205: OpenAI TTS Provider

Test Cases:

  • Provider instantiates
  • API key validation works
  • generateVoice() works
  • Voice selection works
  • Model selection works
  • Format selection works
  • Audio buffer returned

REQ-206-215: Voice Pipeline Integration

Test Cases:

  • VoicePipeline instantiates
  • Config input works
  • Provider selection works
  • Voice generation works
  • File save works
  • Progress events emit
  • Cancellation works

Phase 1.5: FFmpeg Integration Tests

REQ-216-225: FFmpeg Detection

Test Cases:

  • detectFFmpeg() finds ffmpeg
  • detectFFmpeg() finds ffprobe
  • Version check works
  • Error on missing ffmpeg
  • Installation instructions shown

REQ-226-236: FFmpeg Command Builder

Test Cases:

  • FFmpegCommand instantiates
  • input() adds input
  • output() adds output
  • Options work correctly
  • toString() formats command
  • toArgs() returns array

REQ-237-246: FFmpeg Execution

Test Cases:

  • executeFFmpeg() runs command
  • Progress parsing works
  • Progress events emit
  • Exit handling works
  • Error handling works
  • Cancellation works

REQ-247-256: FFmpeg Operations

Test Cases:

  • trimVideo() works
  • concatVideos() works
  • addAudioToVideo() works
  • extractAudio() works
  • convertFormat() works
  • resizeVideo() works
  • getMediaInfo() works

Phase 1.6: Remotion Setup Tests

REQ-257-266: Remotion Project

Test Cases:

  • remotion/ directory exists
  • package.json valid
  • Dependencies installed
  • Root.tsx exists
  • index.ts exists
  • remotion.config.ts exists
  • npx remotion studio starts

REQ-267-276: Basic Components

Test Cases:

  • All components export
  • TypeScript compiles
  • Components render without error
  • Props validation works

REQ-277-286: Composition Templates

Test Cases:

  • All compositions registered
  • Props work correctly
  • Dynamic duration works
  • Compositions render

REQ-287-296: Remotion Rendering

Test Cases:

  • renderVideo() function works
  • Composition ID accepted
  • Props passed correctly
  • Output file created
  • Progress reporting works
  • Error handling works

Phase 1.7: Testing Infrastructure Tests

REQ-297-306: Test Setup

Test Cases:

  • vitest runs
  • Test files discovered
  • Coverage works
  • Scripts in package.json work

REQ-307-346: Unit & Integration Tests

Test Cases:

  • All test files run
  • Tests pass
  • Coverage > 70%
  • No flaky tests

Phase 1.8: Documentation Tests

REQ-347-366: Documentation

Test Cases:

  • README.md exists and complete
  • Installation instructions work
  • Examples run correctly
  • API docs complete
  • No broken links
  • JSDoc comments present

Bug Severity Assignment

When bugs are found, assign severity:

  • CRITICAL: System won't build/run, data loss, security issue
  • HIGH: Feature completely broken, no workaround
  • MEDIUM: Feature partially works, workaround exists
  • LOW: Cosmetic, edge case, minor issue

Test Execution Priority

  1. File Existence: Always test first
  2. Syntax Validation: JSON/YAML/TS compilation
  3. Basic Functionality: Core feature works
  4. Error Handling: Graceful failures
  5. Integration: Works with other components
  6. Performance: No obvious bottlenecks

Notes

  • Each test creates detailed reproduction steps
  • All bugs logged with timestamp
  • Screenshots/logs attached when relevant
  • Bugs linked to requirements