test: add proptest-based real-IDL coverage tests by shahan-khatchadourian-anchorage · Pull Request #225 · anchorageoss/visualsign-parser

shahan-khatchadourian-anchorage · 2026-03-26T18:37:53Z

Summary

Adds 4 new property tests that run against real production IDLs (loaded via IDL_FILE env var):

real_idl_all_arg_names_in_parsed_output — after successful parse, every declared arg name appears in program_call_args
real_idl_overlong_data_is_rejected — valid bytes + 1 trailing byte → Err
real_idl_truncated_data_is_rejected — valid bytes with last byte removed → Err
real_idl_wrong_discriminator_not_dispatched_as_original — swapping discriminators between instructions doesn't cause misdispatch

These complement the existing real_idl_never_panics and real_idl_valid_data_always_parses_ok crash-safety tests with targeted correctness assertions.

Stacked on

test: add real-IDL structural validation tests #224 (structural validation tests)
Add fuzz targets, semantic tests, and CI for visualsign-solana #204 (fuzz targets)

Test plan

IDL_FILE=<path> cargo test --test fuzz_idl_parsing real_idl — all real-IDL tests pass
cargo test --test fuzz_idl_parsing — full suite passes (tests skip when IDL_FILE unset)

🤖 Generated with Claude Code

Two libFuzzer targets covering the full visualsign-solana stack: - fuzz_transaction_string: arbitrary bytes into transaction_string_to_visual_sign - fuzz_versioned_transaction: arbitrary bytes deserialized as VersionedTransaction then passed to versioned_transaction_to_visual_sign Run with: cargo +nightly fuzz run <target> (from src/chain_parsers/visualsign-solana/fuzz/) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- proptest label: runs cargo test -p visualsign-solana - fuzz label: installs nightly + cargo-fuzz, runs each fuzz target for 30s - ubuntu job: restricted to main push/PR to avoid triggering on label events Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

On crash, extracts the libFuzzer summary (everything after the ─── line) and posts it as a PR comment via gh. No artifacts or separate jobs needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Move proptest and fuzz jobs out of main.yml into dedicated workflow files (proptest.yml, fuzz.yml) so they appear as distinct named checks. Add pull-requests: write permission to fuzz job to allow posting crash comments via gh pr comment. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Shared composite action posts crash/failure output as a PR comment and tags @copilot to fix the issue. Fuzz and proptest workflows use it via extract steps that write output to GITHUB_OUTPUT. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Move build_transaction, options_with_idl, instruction_fields, find_text and related helpers from pipeline_integration.rs into tests/common/mod.rs so they can be reused by other test files without duplication. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Test the full visualization pipeline with realistic Borsh-serialized instruction data for Drift deposit, Lifinity swap, Raydium swapBaseInput, Orca swap (including u128 sqrtPriceLimit), Meteora swap, Kamino deposit, Stabble swap (Option<u64> arg), and OpenBook deposit. Each test uses assert_semantic() to verify decoded instruction name and arg values match what was serialized. Lives in its own file to keep pipeline_integration.rs focused on proptest-based property tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add 'labeled' to pull_request trigger types and allow the ubuntu job to run when the 'ci' label is present, not just for PRs targeting main. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Pin fuzz Cargo.lock to avoid spl-token-2022 breakage on nightly (spl-token-group-interface 0.7.2 pulled in solana-nullable which is incompatible with spl-token-2022 10.0.0) - Remove stale Cargo.lock gitignore rule from visualsign-solana - Remove post-failure-comment action and simplify fuzz/proptest workflows - Use continue-on-error on fuzz steps so crashes show as warnings (known pre-existing panics in instructions.rs bounds checking) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

On crash/failure, add fuzz-failure or proptest-failure label to the PR. On clean run, remove the label. This gives visible signal without blocking the check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add || true to --add-label calls so the step doesn't fail if the label operation itself has issues (e.g. label not yet created in the repo). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Extend the property-based testing section with cargo fuzz targets, semantic pipeline tests, CI workflow labels (proptest, fuzz, ci), and failure label behavior (fuzz-failure, proptest-failure). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Pin nightly toolchain to 2026-03-13 (known-good) in both rust-toolchain.toml and fuzz.yml - Use --locked for cargo install cargo-fuzz in CI and docs - Cache fuzz/target/ and key off fuzz/Cargo.lock instead of src/Cargo.lock Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

- Use ..VisualSignOptions::default() in test helpers to prevent breakage when new fields are added to the struct - Pass github.event.pull_request.number through env var instead of direct context interpolation in shell blocks Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…ng (#223) * fix: guard against panics on malformed transaction data Two panic sites in the legacy transaction path: 1. instructions.rs: unchecked indexing into account_keys with program_id_index and account indices from compiled instructions. Malformed transactions with out-of-bounds indices cause index-out-of-bounds panics. 2. accounts/decode.rs: arithmetic underflow when message header values (num_readonly_signed_accounts, num_readonly_unsigned_accounts) exceed the actual account keys array length. Fix: apply the same defensive patterns already used in the v0 transaction path — filter_map with bounds checks for instruction indices, saturating_sub for header arithmetic, and an explicit error for empty account keys. Verified with cargo-fuzz: ~930,000 runs across both fuzz targets (fuzz_transaction_string, fuzz_versioned_transaction) with zero crashes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address Copilot review: improve comments, error message, add tests - Clarify comment to distinguish skipped instructions (OOB program_id) from omitted accounts (OOB account index) - Use "Legacy transaction" in error message for consistency with v0 path - Add unit tests for decode_accounts and decode_v0_accounts with inconsistent header values to lock in saturating_sub behavior Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Extract load_idl_from_env into common/mod.rs and add real_idl_validation.rs with deterministic structural invariant tests for production IDLs: discriminator presence/uniqueness, instruction name uniqueness, and IDL hash stability. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add 4 new property tests against real production IDLs: - arg name completeness in parsed output - overlong data rejection (trailing byte) - truncated data rejection (missing byte) - discriminator isolation (cross-instruction swap) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

shahan-khatchadourian-anchorage and others added 17 commits March 25, 2026 10:28

Post fuzz crash summary as PR comment on failure

73c07ef

On crash, extracts the libFuzzer summary (everything after the ─── line) and posts it as a PR comment via gh. No artifacts or separate jobs needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Allow triggering CI on any PR via 'ci' label

6a667ad

Add 'labeled' to pull_request trigger types and allow the ubuntu job to run when the 'ci' label is present, not just for PRs targeting main. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add failure labels to fuzz and proptest workflows

b19f3d9

On crash/failure, add fuzz-failure or proptest-failure label to the PR. On clean run, remove the label. This gives visible signal without blocking the check. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Harden label steps in fuzz and proptest workflows

0b545b9

Add || true to --add-label calls so the step doesn't fail if the label operation itself has issues (e.g. label not yet created in the repo). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

shahan-khatchadourian-anchorage force-pushed the shahankhatch/real-idl-structural-tests branch 2 times, most recently from f65ebb5 to cdaf441 Compare April 2, 2026 22:10

Base automatically changed from shahankhatch/real-idl-structural-tests to main April 5, 2026 13:44

prasanna-anchorage approved these changes Apr 8, 2026

View reviewed changes

prasanna-anchorage merged commit fbc6053 into main Apr 8, 2026
7 checks passed

prasanna-anchorage deleted the shahankhatch/real-idl-proptest-coverage branch April 8, 2026 05:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: add proptest-based real-IDL coverage tests#225

test: add proptest-based real-IDL coverage tests#225
prasanna-anchorage merged 17 commits into
mainfrom
shahankhatch/real-idl-proptest-coverage

shahan-khatchadourian-anchorage commented Mar 26, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

shahan-khatchadourian-anchorage commented Mar 26, 2026

Summary

Stacked on

Test plan

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants