submission measurements seems to be broken

[michele-nuzzi]

while creating a PR for pebble (#138) I couldn't measure my submissions

for good measure, I tried to run the same comand on exsisting submissions, without success

for example, running in the root of the project

cape submission measure submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/

gives me

INFO: Measuring UPLC program: submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/fibonacci.uplc

ERROR: ✗ Failed to measure UPLC program (submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/fibonacci.uplc)

ERROR: Test execution details:

ERROR: Hint: Check test data format if you see parse errors. BuiltinData uses:
ERROR:   - Integers: 42, -123
ERROR:   - ByteStrings: #deadbeef
ERROR:   - Constructors: 0(), 1(42 #cafe)
ERROR:   - Lists: [1 2 3]
ERROR:   - Maps: {1:42}

[michele-nuzzi]

after reuilding ghc from scratch I can confirm I get the same error using nix for all submissions

[Unisay]

This is weird, I am trying to reproduce the failure but no luck so far:

 main  cape submission measure submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/
INFO: Measuring UPLC program: submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/fibonacci.uplc
SUCCESS: Updated submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/metrics.json
SUCCESS: All measurements completed successfully!

BTW, CI workflow run for your PR executed the same command without errors.

[Unisay]

Please try cape test from inside the nix shell noticing any possible errors, esp. wrt measure executable.

[michele-nuzzi]

@Unisay

sudo nix develop --extra-experimental-features nix-command --extra-experimental-features flakes
warning: Git tree '/home/michele/hlabs/UPLC-CAPE' is dirty
evaluating derivation 'git+file:///home/michele/hlabs/UPLC-CAPE#devShells.x86_64-linux.default'

  
   🚀 UPLC-CAPE Development Shell 
  
  💡 Quick Start:
  
  •  cape benchmark list : List available benchmarks
  •  cape submission new : Create new submission
  •  usage : View command cheat sheet

📦 Synchronizing Cabal package index...
root@ThinkPad:/home/michele/hlabs/UPLC-CAPE# cabe submission measure
bash: cabe: command not found
root@ThinkPad:/home/michele/hlabs/UPLC-CAPE# ca`e submission measure
> ^C
root@ThinkPad:/home/michele/hlabs/UPLC-CAPE# cape submission measure
ERROR: No mode specified. Provide a PATH or use --all
CAPE Submission Measure

USAGE:
    cape submission measure [OPTIONS] PATH
    cape submission measure --all

DESCRIPTION:
    Measure UPLC program(s) and generate metrics.json. This wrapper auto-discovers the
    scenario and cape-tests.json based on repository layout. No explicit scenario or test
    flags are supported here.

OPTIONS:
    -h, --help            Show this help message
    -a, --all             Measure all submissions under submissions/
        --no-color        Disable colored output
        --verbose         Verbose output

BEHAVIOR:
    - Scenario is inferred from the path: submissions/<scenario>/<submission>/...
    - Cape-tests.json is REQUIRED at scenarios/<scenario>/cape-tests.json
    - Test suite is passed to measure tool for comprehensive validation (-t)
    - metrics.json is created/updated alongside each .uplc file measured.

EXAMPLES:
    - Measure one submission folder:
        cape submission measure submissions/fibonacci/Plinth_1.52.0.0_User

    - Measure all submissions:
        cape submission measure --all
root@ThinkPad:/home/michele/hlabs/UPLC-CAPE# cape submission measure submissions/fibonacci_naive_recursion/pebble_0.1.2_michele-nuzzi/
INFO: Measuring UPLC program: submissions/fibonacci_naive_recursion/pebble_0.1.2_michele-nuzzi/fibonacci_naive_recursion.uplc

ERROR: ✗ Failed to measure UPLC program (submissions/fibonacci_naive_recursion/pebble_0.1.2_michele-nuzzi/fibonacci_naive_recursion.uplc)

ERROR: Test execution details:

ERROR: Hint: Check test data format if you see parse errors. BuiltinData uses:
ERROR:   - Integers: 42, -123
ERROR:   - ByteStrings: #deadbeef
ERROR:   - Constructors: 0(), 1(42 #cafe)
ERROR:   - Lists: [1 2 3]
ERROR:   - Maps: {1:42}
Error: /home/michele/hlabs/UPLC-CAPE/scripts/cape-subcommands/submission/measure.sh:282: command "return $has_errors" failed with exit code 1
root@ThinkPad:/home/michele/hlabs/UPLC-CAPE# cape submission measure submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/
INFO: Measuring UPLC program: submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/fibonacci.uplc

ERROR: ✗ Failed to measure UPLC program (submissions/fibonacci_naive_recursion/Aiken_1.1.17_KtorZ/fibonacci.uplc)

ERROR: Test execution details:

ERROR: Hint: Check test data format if you see parse errors. BuiltinData uses:
ERROR:   - Integers: 42, -123
ERROR:   - ByteStrings: #deadbeef
ERROR:   - Constructors: 0(), 1(42 #cafe)
ERROR:   - Lists: [1 2 3]
ERROR:   - Maps: {1:42}
Error: /home/michele/hlabs/UPLC-CAPE/scripts/cape-subcommands/submission/measure.sh:282: command "return $has_errors" failed with exit code 1
root@ThinkPad:/home/michele/hlabs/UPLC-CAPE# cape test
UPLC CAPE Test Suite
====================
INFO: Test environment initialized at: /tmp/nix-shell.27m9SP/tmp.UMd8ZW8L1A/sandbox
INFO: Testing environment requirements...
ERROR: ✗ measure tool buildable
  Command: (cd "/home/michele/hlabs/UPLC-CAPE" && cabal build exe:measure)
  Exit code: 0
  Expected: success (zero)
  Output (last 10 lines):
    Downloaded   plutus-tx-plugin-1.45.0.0
    Building     plutus-core-1.45.0.0 (lib:index-envs)
    Building     plutus-core-1.45.0.0 (lib:satint)
    Installing   plutus-core-1.45.0.0 (lib:satint)
    Completed    plutus-core-1.45.0.0 (lib:satint)
    Installing   plutus-core-1.45.0.0 (lib:index-envs)
    Completed    plutus-core-1.45.0.0 (lib:index-envs)
    Starting     plutus-core-1.45.0.0 (lib)
    Building     plutus-core-1.45.0.0 (lib)
    terminated
INFO: Testing core commands...
INFO: ✓ cape help
INFO: ✓ cape no args
INFO: ✓ cape test help
INFO: ✓ cape test -h
INFO: Testing benchmark commands...
INFO: ✓ benchmark help
INFO: ✓ benchmark list
INFO: ✓ benchmark list help
INFO: ✓ benchmark new help
INFO: ✓ benchmark fibonacci
INFO: ✓ benchmark no args fail
INFO: ✓ benchmark invalid fail
INFO: Testing submission commands...
INFO: ✓ submission help
INFO: ✓ submission list
INFO: ✓ submission list help
INFO: ✓ submission list fibonacci
INFO: ✓ submission no args fail
INFO: ✓ submission complete success
INFO: ✓ submission cleanup (remove template submission)
INFO: ✓ submission invalid fail
INFO: Testing variant functionality...
INFO: ✓ submission without variant
INFO: ✓ verify no-variant folder
INFO: ✓ verify no-variant metadata no variant field
INFO: ✓ submission with variant
INFO: ✓ verify variant folder
INFO: ✓ cleanup variant tests
INFO: Testing submission list filtering...
INFO: ✓ list all submissions
INFO: ✓ list fibonacci submissions
INFO: Testing verification & measurement...
INFO: ✓ verify help
INFO: ✓ measure help
INFO: Testing schema validation error reporting...
INFO: ✓ create test submission for schema errors
INFO: ✓ create invalid metadata (wrong type for source_available)
INFO: ✓ verify detects type error and shows details
INFO: ✓ restore metadata and add extra properties
INFO: ✓ add invalid extra properties to metadata
INFO: ✓ verify detects additional properties and shows details
INFO: ✓ cleanup schema test submissions
INFO: Testing reporting...
INFO: ✓ report help
INFO: ✓ report dry-run all
INFO: ✓ report dry-run fibonacci
INFO: ✓ aggregate
INFO: Testing global options...
INFO: ✓ verbose benchmark
INFO: ✓ no-color benchmark
INFO: ✓ verbose short
INFO: Testing integration workflow...
INFO: ✓ create benchmark
INFO: ✓ create submission
INFO: ✓ verify submission created
INFO: Testing directory structure...
INFO: ✓ scenarios dir
INFO: ✓ submissions dir
INFO: ✓ scripts dir
INFO: ✓ templates exist
INFO: Testing Haskell library tests...
ERROR: ✗ cabal test cape-tests
  Command: (cd "/home/michele/hlabs/UPLC-CAPE" && cabal test cape-tests)
  Exit code: 0
  Expected: success (zero)
  Output (last 10 lines):
    Starting     hspec-discover-2.11.14 (lib)
    Building     hspec-discover-2.11.14 (lib)
    Building     plutus-core-1.45.0.0 (lib)
    Installing   hspec-discover-2.11.14 (lib)
    Completed    hspec-discover-2.11.14 (lib)
    Starting     hspec-discover-2.11.14 (exe:hspec-discover)
    Building     hspec-discover-2.11.14 (exe:hspec-discover)
    Installing   hspec-discover-2.11.14 (exe:hspec-discover)
    Completed    hspec-discover-2.11.14 (exe:hspec-discover)
    terminated
INFO: Testing UPLC cape-tests validation...
ERROR: ✗ UPLC submissions validation
ERROR: UPLC submissions validation failed
Error: /home/michele/hlabs/UPLC-CAPE/scripts/cape-subcommands/test/test.sh:341: command "return 1" failed with exit code 1

========================================
Tests: 53 | Passed: 50 | Failed: 3
ERROR: SOME TESTS FAILED
root@ThinkPad:/home/michele/hlabs/UPLC-CAPE# ^C
root@ThinkPad:/home/michele/hlabs/UPLC-CAPE# 

[Unisay]

This is the main problem:

ERROR: ✗ measure tool buildable
  Command: (cd "/home/michele/hlabs/UPLC-CAPE" && cabal build exe:measure)
  Exit code: 0
  Expected: success (zero)
  Output (last 10 lines):
    Downloaded   plutus-tx-plugin-1.45.0.0
    Building     plutus-core-1.45.0.0 (lib:index-envs)
    Building     plutus-core-1.45.0.0 (lib:satint)
    Installing   plutus-core-1.45.0.0 (lib:satint)
    Completed    plutus-core-1.45.0.0 (lib:satint)
    Installing   plutus-core-1.45.0.0 (lib:index-envs)
    Completed    plutus-core-1.45.0.0 (lib:index-envs)
    Starting     plutus-core-1.45.0.0 (lib)
    Building     plutus-core-1.45.0.0 (lib)
    terminated

cape submission measure command uses this Haskell program measure for generating metrics.json
For some reason (terminated ?) the cabal build exe:measure failed.
Do you happen to also use some kind of an OOM-killer daemon?
Do you have a globally-installed cabal ?
If you have nix shell please try to cabal clean && cabal update && cabal build -v exe:measure manually.

[michele-nuzzi]

seems to work after running cabal clean && cabal update && cabal build -v exe:measure in the nix shell