[FEATURE] Support Running With Vitest Instead of Jest

[ckifer]

Is your feature request related to a problem? Please describe.
I'd rather not need to have a jest setup to use reassure when I currently use vitest for testing

Describe the solution you'd like
Support configuring reassure with vitest or jest at consumers preference

Describe alternatives you've considered
N/A

Additional context
There is already a TEST_RUNNER_PATH env variable, but pointing that to the vitest binary complains about needing WASM

"test:perf": "TEST_RUNNER_PATH=./node_modules/.bin/vitest reassure",

ReferenceError: WebAssembly is not defined
    at file:///workplace/ckifer/recharts/node_modules/vite/dist/node/chunks/dep-2b82a1ce.js:16497:1125
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:530:24)
    at async main (file:///workplace/ckifer/recharts/node_modules/vitest/dist/cli-wrapper.js:61:5)

[mdjastrzebski]

@ckifer this is a great idea, first step into making it work would be have to make sure that RNTL / RTL run correctly with vitest, as Reassure is built on top of these libraries. Are you interested in web React or React Native testing?

Do you have capacity to verify whether RTL / RNTL support vitest? And/or contribute missing config/etc?

[ckifer]

For me mostly web testing.

I have no "proof" atm that vitest is compatible with RTL but for me all functions and features work out of the box once vitest itself is set up. I think RTL is the main testing library people are using right now with vitest and react so should be pretty well supported

[ckifer]

RTL calls out in their docs that they recommend using vitest when making svelte applications
https://testing-library.com/docs/svelte-testing-library/setup/#vitest

That gives me enough confidence these are completely compatible

[mdjastrzebski]

@ckifer would you be able to confirm this empirically by running sample test with vitest & RTL? (Svelte probably uses Svelte Testing Lib or something).

Regarding UX how would you see such an option? Something like "reassure --vitest" / "reassure --runner vitest" or some other way?

[ckifer]

@mdjastrzebski recharts just moved to vitest from jest and we run all tests with RTL

https://github.com/recharts/recharts/actions/runs/6605228230/job/17940180193

The second option seems fine - add a flag for runner, default it to jest, and have vitest as an option

[mdjastrzebski]

@ckifer do you have capacity to contribute such feature as a PR?

[ckifer]

@mdjastrzebski no not at the moment 😅

Tagging a few people interested in this though: @nikolasrieble @PavelVanecek @akamfoad @branberry

[branberry]

@mdjastrzebski @ckifer I should have some bandwidth to work on this!

[mdjastrzebski]

@branberry go ahead!

[branberry]

Hey y'all! Quick update. I have been working on this issue, and I still ran into the WebAssembly import error. I narrowed this down to when we call spawnSync. Specifically, the --jitless flag seems to cause this error to occur when using Vitest. When removing it, it appears to run just fine, but I am not sure what the ramifications are if we remove this.

I can conditionally remove the --jitless flag for Vitest as a workaround.

Thoughts @mdjastrzebski?

[mdjastrzebski]

Does Vitest run with on v8 JS engine (node)?

If so the goal of --jitless flag is make all of the runs be comparable, instead of Just In Time compilation kicking in during later runs and cause distortions due to:

1. pausing for JIT compilation
2. faster execution when using compiled code

From that perspective using --jitless or achieving similar effect in some other way.

[ckifer]

Vitest is definitely running on node

At some point the code must get here and try to load WASM..

Would be interesting to debug the vitest/vite code or cut vite an issue

[mdjastrzebski]

Seems like WASM might need some compilation. Based on the V8 docs, it seems we might want to use Liftoff compiler (non-optimizing) vs Turbo Fan (optimizing) in order to keep execution time stable (potentially excluding the initial execution which we already discard with Jest as well): https://v8.dev/docs/wasm-compilation-pipeline#flags-for-experimentation

@branberry could you add proper flags flags depending on using vitest (probably --liftoff --no-wasm-tier-up but please experiment locally with stability and share your results).