Add proptest-based fuzz and pipeline integration tests for Solana IDL parsing

docs/contributor-guides/testing-visualizations.mdx

@@ -126,6 +126,57 @@ Verify your fixture passes:
 cargo test -p visualsign-<chain> test_my_protocol
 ```
 
+## Property-based testing (Solana)
+
+Solana IDL parsing includes [proptest](https://proptest-rs.github.io/proptest/)-based fuzz tests that verify crash safety and correctness across randomly generated IDLs and instruction data. These tests live in:
+
+- `src/chain_parsers/visualsign-solana/tests/fuzz_idl_parsing.rs` — parser-level fuzz and roundtrip tests
+- `src/chain_parsers/visualsign-solana/tests/pipeline_integration.rs` — full-pipeline integration tests
+
+### Running fuzz tests
+
+```bash
+# Default 256 cases per property
+cargo test -p visualsign-solana --test fuzz_idl_parsing
+
+# More iterations for deeper fuzzing
+PROPTEST_CASES=5000 cargo test -p visualsign-solana --test fuzz_idl_parsing
+```
+
+### Testing against real IDLs
+
+The `scripts/fuzz_all_idls.sh` script runs fuzz tests against all embedded production IDLs in one pass:
+
+```bash
+./scripts/fuzz_all_idls.sh
+```
+
+You can also target a specific IDL:
+
+```bash
+IDL_FILE=/path/to/my_program.json cargo test -p visualsign-solana --test fuzz_idl_parsing real_idl
+```
+
+### Roundtrip tests
+
+A roundtrip test constructs an IDL and matching borsh-encoded instruction bytes, feeds them through the parser, and verifies the output matches expectations. "Roundtrip" refers to the encode-then-decode cycle: you know exactly what went in, so you can assert exactly what comes out.
+
+There are two kinds in use:
+
+- **Concrete roundtrips** (e.g., `roundtrip_single_u64_arg`) — Hand-crafted IDL JSON and hand-crafted byte payloads. These assert that specific parsed values match exactly (e.g., `amount == 42`). They serve as specification-by-example: each test documents one type scenario (no args, mixed primitives, `Option<T>`, `Vec<T>`, defined structs, multi-instruction dispatch).
+
+- **Property-based roundtrips** (e.g., `fuzz_valid_data_always_parses_ok`) — Randomly generated IDL shapes paired with machine-generated valid borsh bytes from `arb_valid_instruction_bytes`. These assert that parsing succeeds and the instruction name matches, without checking specific field values. They verify the parser's contract holds across all type combinations, not just the hand-picked examples.
+
+Both kinds complement each other: concrete roundtrips pin down known-good behavior, while property-based roundtrips explore the space of inputs you did not think to write by hand.
+
+### Adding a new test
+
+1. Write a strategy that generates the IDL shape you want to test (or use an existing one from `solana_parser_fuzz_core::proptest`)
+2. Add a `proptest!` test that exercises the parser with generated inputs
+3. Add a concrete roundtrip test for the same scenario to serve as specification-by-example
+4. Run the tests — if proptest finds a failure, it saves a regression seed to `.proptest-regressions`
+5. Commit the `.proptest-regressions` file so the failing case is reproduced in CI
+
 ## Validation checklist
 
 Before submitting your visualization:

scripts/fuzz_all_idls.sh

@@ -0,0 +1,150 @@
+#!/usr/bin/env bash
+# fuzz_all_idls.sh — run IDL fuzz tests against every embedded Solana IDL.
+#
+# The embedded IDLs live in the solana_parser git dependency:
+#
+#   ape_pro.json          6 instructions   4 types   (Ape Pro)
+#   cndy.json             7 instructions   4 types   (Metaplex Candy Machine)
+#   collision.json        1 instruction    2 types   (test fixture: duplicate type names)
+#   cyclic.json           1 instruction    2 types   (test fixture: cyclic type references)
+#   drift.json          199 instructions  81 types   (Drift Protocol V2)
+#   jupiter.json         34 instructions   8 types   (Jupiter Swap)
+#   jupiter_agg_v6.json  14 instructions   9 types   (Jupiter Aggregator V6)
+#   jupiter_limit.json    8 instructions  12 types   (Jupiter Limit)
+#   kamino.json          36 instructions  51 types   (Kamino)
+#   lifinity.json         3 instructions   4 types   (Lifinity Swap V2)
+#   meteora.json         64 instructions  38 types   (Meteora)
+#   openbook.json        29 instructions  32 types   (Openbook)
+#   orca.json            49 instructions  11 types   (Orca Whirlpool)
+#   raydium.json         10 instructions   5 types   (Raydium)
+#   stabble.json         17 instructions   8 types   (Stabble)
+#
+# For each IDL the script runs two test functions from fuzz_idl_parsing.rs:
+#
+#   real_idl_never_panics
+#     — 50/50 valid/random discriminator mix; on Ok asserts correct dispatch.
+#
+#   real_idl_valid_data_always_parses_ok
+#     — generates borsh-correct bytes for every instruction; asserts is_ok().
+#
+# Usage:
+#   ./scripts/fuzz_all_idls.sh
+#   PROPTEST_CASES=1000 ./scripts/fuzz_all_idls.sh
+#   ./scripts/fuzz_all_idls.sh /path/to/extra.json ...   # append extra IDLs
+#
+# Requirements: cargo, python3
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+WORKSPACE_TOML="$SCRIPT_DIR/../src/Cargo.toml"
+CASES="${PROPTEST_CASES:-256}"
+
+# ── Locate the solana_parser IDL directory via cargo metadata ─────────────────
+
+IDL_DIR="$(python3 - "$WORKSPACE_TOML" <<'PY'
+import json, os, subprocess, sys
+
+manifest = sys.argv[1]
+result = subprocess.run(
+    ["cargo", "metadata", "--manifest-path", manifest, "--format-version", "1"],
+    capture_output=True, text=True, check=True,
+)
+data = json.loads(result.stdout)
+for pkg in data["packages"]:
+    if pkg["name"] == "solana_parser":
+        idl_dir = os.path.join(os.path.dirname(pkg["manifest_path"]), "src", "solana", "idls")
+        if os.path.isdir(idl_dir):
+            print(idl_dir)
+            sys.exit(0)
+print("error: solana_parser IDL directory not found", file=sys.stderr)
+sys.exit(1)
+PY
+)"
+
+# ── Collect IDL files: embedded + any extras passed as arguments ──────────────
+
+IDL_FILES=("$IDL_DIR"/*.json)
+for extra in "${@}"; do
+    IDL_FILES+=("$extra")
+done
+
+# ── Build once so the loop doesn't pay compilation cost each iteration ─────────
+
+echo "Building test binary..."
+cargo test \
+    --manifest-path "$WORKSPACE_TOML" \
+    -p visualsign-solana \
+    --test fuzz_idl_parsing \
+    --no-run \
+    2>&1 | grep -E "^(   Compiling|    Finished|error)" || true
+echo ""
+
+# ── Run tests for each IDL ────────────────────────────────────────────────────
+
+PASS=0
+FAIL=0
+FAILED_IDLS=()
+
+printf "%-30s %13s %7s  %s\n" "IDL" "Instructions" "Types" "Result"
+printf "%-30s %13s %7s  %s\n" "───────────────────────────" "────────────" "─────" "──────"
+
+for idl_file in "${IDL_FILES[@]}"; do
+    name="$(basename "$idl_file" .json)"
+
+    # Get instruction/type counts
+    read -r inst_count type_count < <(python3 -c "
+import json, sys
+try:
+    d = json.load(open(sys.argv[1]))
+    print(len(d.get('instructions', [])), len(d.get('types', [])))
+except Exception:
+    print(0, 0)
+" "$idl_file")
+
+    printf "%-30s %13s %7s  " "$name" "$inst_count" "$type_count"
+
+    # Run both real_idl_* tests for this IDL.
+    output=$(IDL_FILE="$idl_file" PROPTEST_CASES="$CASES" \
+        cargo test \
+            --manifest-path "$WORKSPACE_TOML" \
+            -p visualsign-solana \
+            --test fuzz_idl_parsing \
+            real_idl \
+            --quiet \
+            2>&1)
+
+    # Extract "N passed; M failed" directly from cargo's summary line.
+    summary=$(echo "$output" | grep -oE "[0-9]+ passed; [0-9]+ failed" | head -1)
+
+    if [ -z "$summary" ]; then
+        echo "FAIL (no test result)"
+        FAIL=$(( FAIL + 1 ))
+        FAILED_IDLS+=("$name ($idl_file)")
+    else
+        failed_count=$(echo "$summary" | grep -oE "[0-9]+ failed" | grep -oE "[0-9]+")
+        if [ "${failed_count:-0}" -gt 0 ]; then
+            echo "FAIL ($summary)"
+            FAIL=$(( FAIL + 1 ))
+            FAILED_IDLS+=("$name ($idl_file)")
+        else
+            echo "PASS ($summary)"
+            PASS=$(( PASS + 1 ))
+        fi
+    fi
+done
+
+echo ""
+echo "Results: $PASS passed, $FAIL failed  (PROPTEST_CASES=$CASES)"
+
+if (( FAIL > 0 )); then
+    echo ""
+    echo "Failed:"
+    for entry in "${FAILED_IDLS[@]}"; do
+        echo "  $entry"
+    done
+    echo ""
+    echo "Re-run a single IDL with full output:"
+    echo "  IDL_FILE=<path> cargo test --manifest-path src/Cargo.toml -p visualsign-solana --test fuzz_idl_parsing real_idl"
+    exit 1
+fi

src/Cargo.toml

@@ -59,3 +59,4 @@ tracing = "0.1.41"
 
 # Pin visualsign dependencies
 visualsign = { path = "./visualsign" }
+

src/chain_parsers/visualsign-solana/Cargo.toml

@@ -6,10 +6,7 @@ edition = "2024"
 [dependencies]
 
 tracing = { workspace = true }
-# Using custom fork for IDL-based instruction parsing features not yet in upstream
-# Features: parse_instruction_with_idl, CustomIdlConfig, decode_idl_data
-# Tracking: https://github.com/tkhq/solana-parser/pull/20
-solana_parser = { git = "https://github.com/prasincs/solana-parser.git", rev = "8248d99e42ce8a56ad440ed9b2201607feb1a150" }
+solana_parser = { git = "https://github.com/anchorageoss/solana-parser.git", rev = "a0c554d" }
 visualsign = { workspace = true }
 generated = { path = "../../generated" }
 serde_json = { workspace = true }
@@ -28,8 +25,10 @@ spl-token-2022 = "10.0.0"
 spl-token-2022-interface = "2.1.0"
 
 [dev-dependencies]
+solana-parser-fuzz-core = { git = "https://github.com/anchorageoss/solana-parser.git", rev = "a0c554d", features = ["proptest"] }
 serde = { version = "1.0", features = ["derive"] }
 serde_json = "1.0"
 jupiter-swap-api-client = "0.2.0"
 base64 = "0.22.1"
 bs58 = "0.5"
+proptest = "1"

src/chain_parsers/visualsign-solana/tests/common/mod.rs

@@ -0,0 +1,44 @@
+//! Shared test helpers for IDL-based fuzz and integration tests.
+
+use solana_parser::decode_idl_data;
+use solana_parser::solana::structs::Idl;
+
+/// Decode an IDL JSON string, extract the discriminator for the instruction at
+/// `inst_idx`, and return `(idl, data)` where `data` = discriminator ++ `arg_bytes`.
+///
+/// Returns `None` if decoding fails, the IDL has no instructions, or the
+/// selected instruction has no discriminator.
+pub fn build_disc_data(
+    idl_json: &str,
+    inst_idx: usize,
+    arg_bytes: &[u8],
+) -> Option<(Idl, Vec<u8>)> {
+    let idl = decode_idl_data(idl_json).ok()?;
+    if idl.instructions.is_empty() {
+        return None;
+    }
+    let inst = &idl.instructions[inst_idx % idl.instructions.len()];
+    let disc = inst.discriminator.as_ref()?;
+    let mut data = disc.clone();
+    data.extend_from_slice(arg_bytes);
+    Some((idl, data))
+}
+
+/// Build instruction bytes using a 50/50 valid-discriminator / random-data split.
+///
+/// When `use_valid_disc` is true, attempts to prepend a real discriminator from
+/// the IDL instruction at `inst_idx`. Falls back to raw `data` if decoding
+/// fails, the IDL has no instructions, or the instruction has no discriminator.
+pub fn build_maybe_disc_bytes(
+    idl_json: &str,
+    use_valid_disc: bool,
+    inst_idx: usize,
+    data: Vec<u8>,
+) -> Vec<u8> {
+    if use_valid_disc {
+        if let Some((_idl, disc_data)) = build_disc_data(idl_json, inst_idx, &data) {
+            return disc_data;
+        }
+    }
+    data
+}

src/chain_parsers/visualsign-solana/tests/fuzz_idl_parsing.proptest-regressions

@@ -0,0 +1,7 @@
+# Seeds for failure cases proptest has generated in the past. It is
+# automatically read and these particular cases re-run before any
+# novel cases are generated.
+#
+# It is recommended to check this file in to source control so that
+# everyone who runs the test benefits from these saved cases.
+cc af260e671c772ce9f858d25d370f930405baa1e38d323d04a6b86f5b0982ae76 # shrinks to use_valid_disc = false, inst_idx = 0, data = [96, 247, 167, 54, 77, 60, 130, 112, 23, 135, 236, 197, 133, 142, 88, 108, 64, 218, 209, 129, 250, 150, 186, 54, 43, 160, 137, 32, 205, 132, 202, 26, 174, 203, 20, 98, 247, 80, 244, 152, 252, 101, 178, 140, 198, 102, 80, 54, 82, 62, 163, 135, 165, 173, 17, 174, 62, 190, 173, 224, 31, 58, 19, 128, 45, 138, 230, 242, 82, 195, 167, 81, 90, 82, 25, 97, 210, 136, 193, 160, 123, 110, 175, 153, 225, 104, 136, 247, 155, 179, 167, 210, 222, 171, 40, 97, 8, 141, 4, 14, 43, 115, 107, 233, 127, 59, 27, 2, 194, 98, 59, 150, 23, 29, 52, 44, 88, 15, 65, 56, 250, 8, 57, 66, 49, 202, 59, 199, 67, 225, 28, 14, 38, 143, 151, 214, 24, 113, 127, 232, 43, 190, 32, 251, 148, 232, 57, 198, 91, 208, 56, 22, 7, 4, 124, 46, 207, 80, 111, 189, 170, 87, 32, 112, 94, 190, 229, 196, 90, 8, 119, 225, 207, 253, 142, 30, 190, 126, 146, 167, 66, 84, 206, 44, 204, 201, 184, 253, 254, 142, 64, 4, 55, 14, 232, 209, 159, 254, 158, 169, 189, 165, 175, 224, 240, 0, 100, 71, 218, 71, 251, 223, 192, 80, 218, 160, 192, 100, 249, 248, 161, 48, 24, 255, 66, 234, 202, 197, 70, 206, 104, 107, 6, 175, 41, 188, 59, 233, 220, 44, 99, 83, 53, 176, 74, 155, 61, 236, 166, 45, 86, 131, 155, 3, 169, 117, 220, 78, 183, 254, 76, 245, 100, 128, 6, 254, 96, 26, 127, 42, 208, 93, 68, 243, 146, 25, 135, 58, 159, 136, 119, 89, 197, 176, 137, 91, 21, 238, 25, 211, 217, 58, 254, 191, 16, 79, 171, 26, 216, 133, 162, 241, 148, 223, 106, 196, 233, 200, 206, 159, 103, 63, 29, 98, 112, 116, 181, 85, 155, 170, 157, 236, 177, 132, 151, 149, 86, 2, 250, 114, 56, 131, 62, 109, 21, 238, 197, 38, 230, 144, 91, 200, 71, 253, 168, 139, 139, 47, 105, 248, 141, 236, 9, 146, 152, 25, 56, 191, 83, 60, 44, 158, 240, 203, 162, 94, 5, 16, 208, 44, 113, 50, 21, 35, 74, 227, 233, 55, 18, 2, 180, 12, 31, 22, 182, 133, 125, 109, 158, 81, 165, 27, 33, 232, 2, 69, 83, 15, 164, 148, 209, 186, 176, 24, 253, 215, 25, 143, 106, 4, 162, 39, 125, 166, 174, 147, 255, 238, 149, 78, 84, 127, 38, 87, 219, 41, 24, 113, 114, 112, 54, 211, 71, 85, 102, 111, 111, 18, 128, 137, 226, 237, 31, 206, 89, 193, 241, 238, 211, 33, 157, 73, 78, 223, 66, 85, 206, 120, 32, 76, 37, 248, 213, 218, 68, 35, 97, 169, 94, 168, 238, 183, 125, 180, 177, 206, 114, 198, 140, 37, 152, 134, 59, 52, 150, 60, 250, 4, 141, 174, 247, 5, 103, 179, 98, 254, 229, 219, 54, 94, 102, 158, 21, 105, 245, 142, 5, 83, 95, 114, 187, 234, 51, 254, 66, 68, 114, 240, 144, 193, 29, 133, 125, 34, 245, 119, 3, 191, 226, 105, 209, 12, 14, 71, 31, 139, 38, 240, 190, 19, 45, 176, 109, 149, 240, 219, 43, 94, 214, 186, 249, 241, 142, 213, 39, 240, 50, 241, 92, 65, 101, 150, 251, 10, 148, 196, 221, 70, 21, 38, 83, 109, 101, 168, 135, 164, 113, 132, 156, 89, 11, 244, 198, 214, 173, 113, 239, 241, 98, 8, 193, 3, 121, 108, 128, 72, 253, 131, 200, 197, 30, 213, 229, 135, 213, 90, 204, 249, 33, 185, 198, 1, 112, 164, 225, 254, 74, 46, 180, 239, 35, 223, 185, 72, 175, 56, 148, 102, 45, 117, 225, 190, 218, 41, 20, 225, 92, 121, 177, 84, 207, 151, 70, 166, 210, 183, 66, 232, 75, 5, 28, 70, 1, 30, 3, 123, 67, 90, 201, 202, 255, 78, 102, 105, 148, 239, 107, 78, 198, 114, 109, 249, 161, 44, 25, 52, 204, 80, 192, 190, 202, 26, 185, 249, 149, 254, 211, 76, 69, 227, 89, 254, 99, 252, 243, 155, 200, 81, 238, 149, 125, 181, 164, 108, 231, 173, 185, 242, 15, 193, 100, 142, 103, 213, 61, 191, 208, 6, 160, 7, 116, 15, 14, 243, 216, 80, 5, 15, 116, 182, 20, 51, 93, 190, 2, 16, 242, 142, 46, 94, 190, 199, 89, 216, 191, 108, 40, 25, 177, 68, 26, 225, 188, 193, 144, 126, 135, 118, 184, 220, 51, 59, 218, 8, 28, 55, 183, 220, 73, 230, 160, 181, 33, 6, 60, 23, 132, 249, 131, 89, 237, 64, 17, 104, 46, 160, 153, 215, 107, 76, 106, 218, 20, 255, 3, 12, 163, 189, 184, 39, 153, 46, 246, 143, 20, 233, 88, 199, 33, 244, 222, 30, 89, 19, 98, 11, 136, 133, 135, 204, 195, 185, 69, 169, 9, 71, 241, 233, 29, 218, 232, 228, 22, 234, 44, 52, 140, 250, 100, 168, 167, 242, 203, 43, 13, 50, 103, 205, 209, 203, 155, 101, 113, 245, 127, 78, 91, 157, 51, 53, 102, 60, 44, 15, 74, 161, 108, 101, 152, 163, 25, 21, 73, 102, 206, 31, 193, 6, 254, 71, 52, 179, 194, 242, 42, 222, 103, 244, 117, 117, 103, 87, 158, 205, 62, 208, 20, 152, 140, 7, 124, 42, 88, 105, 2, 117, 243, 212, 253, 35, 226, 247, 67, 73, 109, 160, 224, 76, 5, 112, 129, 199, 124, 193, 32, 13, 174, 198, 186, 7, 120, 245, 119, 40, 170, 33, 10, 217, 80, 119, 57, 234, 236, 245, 162, 44, 41, 244, 179, 118, 37, 175, 155, 38, 56, 124, 1, 123, 122, 4, 76, 101, 51, 218, 145, 190, 24, 113, 39, 44, 209, 96, 43, 153, 5, 38, 90, 169, 83, 11, 51, 219, 138, 231, 219, 239, 96, 171, 188, 150, 229, 20, 223, 49, 123, 58, 81, 170, 142, 157, 115, 134, 98, 56, 183, 174, 38, 178, 165, 227, 99, 216, 163, 169, 51, 48, 83, 225, 105, 212, 61, 171, 53, 92, 67, 174, 29, 4, 44, 163, 128, 158, 255, 251, 145, 37, 33, 206, 64, 83, 28, 104, 175, 200, 163, 57, 205, 175, 156, 155, 152, 13, 73, 120, 81, 69]