-
Notifications
You must be signed in to change notification settings - Fork 0
Testing
This page describes how correctness and regressions are tested in the CHIP-8 emulator.
Unlike many classic systems, CHIP-8 is not a single, strictly specified platform: it exists in numerous variants (CHIP-8, SCHIP, XO-CHIP, and others), each with its own historical quirks and behavioral differences. Even within “plain” CHIP-8, many instructions have multiple real-world interpretations. As a result, emulator correctness cannot be reduced to “one true behavior.” The goal of the testing system is therefore twofold:
- Reliably detect regressions within a given configuration or variant.
- Make differences between variants explicit, reproducible, and intentional.
This makes emulator changes safe and predictable despite the inherent ambiguity of the platform.
The testing is mostly implemented using test ROMs. But many test ROMs are interactive and expose multiple execution paths for different CHIP-8 variants and quirks. The test system therefore setups the emulator to the intended variand and injects scripted input events to select the intended mode at runtime.
This allows a single ROM to validate several test cases by navigating its built-in menus automatically. The test harness deterministically drives these selections, ensuring that each variant is tested in a reproducible way without manual actions.
The test results are validated using Golden file comparison. This was chosen because many CHIP-8 behaviors manifest visually and because it provides a robust way to detect subtle differences across platforms. This approach allows the test suite to:
- Catch regressions early.
- Verify that variant-specific quirks behave as intended.
- Ensure that changes do not silently alter established behavior.
The emulator is validated by executing well-known CHIP-8 test ROMs that exercise specific aspects of the system. These ROMs are treated as part of the test contract and are executed automatically as part of the test suite.
Location: testdata/roms/
The project uses:
- Timendus test suite
- Octo test suite
- Custom ROMs
Expected behavior is validated via deterministic output comparison using golden files.
Golden files represent the expected output of the emulator for a given ROM and configuration. Typically stored as PNG framebuffer screenshots. Tests render a frame and compare it against the golden reference.
Location: testdata/golden/
Golden files are generated by running tests with an explicit output flag. Saving framebuffer output to testdata/golden/.
Golden generation is never automatic. Updating golden files is done only when:
- Emulator behavior changes intentionally
- A bug is fixed
- Output format is updated
During test runs:
- Emulator output is rendered
- The result is compared byte-for-byte with the golden file
- Any mismatch fails the test
This ensures visual correctness and prevents silent regressions.
Tests are executed in CI to ensure regressions are caught early.
CI steps:
- Set up Go toolchain
- Run unit and integration tests
- Validate golden files