NTV: Non-intrusive Trace-based Verification for RISC-V processors

Note: This is part of the ProcessorCI project and uses files generated by other repositories. Follow the example for a better understanding.

NTV is a verification tool that leverages simulation and trace comparison to verify the cores. The processor under test trace is compared to the trace generated by Spike (RISC-V ISA Sim).

Usage

There are two steps: generating the traces and then comparing them.

1. First step needs the makefile(which points to files outside this repository), the ELF file, the Cocotb path to the register file and the output folder:

python3 exec_trace.py -m tmp/rvx/rvx.mk -e tmp/000_addi.elf -s tmp/rvx/ -r Processor.integer_file -o output/

2. Second step is to generate the spike trace:

python3 spike_trace.py -e tmp/000_addi.elf -o output/

3. Third Step is to generate the speculative final processor trace, with the help of the spike trace and then compare it

python3 compare_traces.py -s output/000_addi.spike.json -d output/000_addi.fragmented.json -o output/

Generating traces using the Spike fork

spike_trace.py executes a single program and generate its respective execution trace. The flags are:

• -e or -E: path to the single ELF program (-e) or to the folder containing multiple ELF programs (-E).
• -o: path to the output folder were the trace will be stored.

An example command is:

$ python3 processor_ci_verification/spike_trace.py -e processor_ci_verification/example/sanity_check.elf -o processor_ci_verification/example

Generating traces using the Cocotb simulation

Cocotb is a simulation tool that allows the use of Python code to interact with the RTL simulation.

elf_reader.py reads the ELF file and stores it in an list. Used by exec_trace.py.

exec_trace.py is the testbench to execute the programs. It has the memory model and instantiates the processor under test. The fetch interface, the register file and the memory are monitored in order to generate the execution trace. The execution flags are:

• -m: path to the makefile. The Cocotb make commands will use it.
• -e or -E: path to the ELF file to execute. -e for single program and -E for folder with multiple programs.
• -r: path to the *_reg_file.json file containing information about the register file.
• -o: output folder to store the fragmented traces.
• -v: verbose option to show the full Cocotb output.

Example command:

$ python3 processor_ci_verification/exec_trace.py -m processor_ci_verification/example/tinyriscv.mk -e processor_ci_verification/example/sanity_check.elf -r processor_ci_verification/example/tinyriscv_reg_file.json -o processor_ci_verification/example -v

Comparing traces

The exec_trace.py testbench is not able to generate the full trace. It stores each part of the trace separately, in fragments:

{
 "comment": "Trace for sanity_check.elf on tinyriscv",
 "fetches": [
  [0,5244307],
  [4,18],
  [8,19]
 ],
 "regfile_commits": [
  [11,5]
 ],
 "memory_accesses": [
  [60,5]
 ]
}

The generate_final_trace function in compare_traces.py uses the spike trace to generate a speculative trace. If the fragmented trace is correct, the final trace will also be. If the fragmented trace is wrong, the final trace may be unaligned or present other errors.

Here is an example of what should be in the final trace:

  {
    "pc": 0,
    "instr": 5244307,
    "target_reg": 11,
    "reg_val": 5,
    "mem_addr": null,
    "mem_val": null,
    "speculative_commit": false,
    "speculative_fetch": false
  },

After the end of comparison, mismatches are shown:

Mismatch found:
Spike entry:     {'pc': 0x4, 'instr': 0x19, 'target_reg': None, 'reg_val': None, 'mem_addr': None, 'mem_val': None}
DUT entry:       {'pc': 0x4, 'instr': 0x18, 'target_reg': None, 'reg_val': None, 'mem_addr': None, 'mem_val': None}