Node 20 auto-loader not able to transpile TS to JS when using ts-node/esm

[TomasHubelbauer]

Prerequisites

• I have written a descriptive issue title
• I have searched existing issues to ensure the bug has not already been reported

Fastify version

4.10.0

Plugin version

No response

Node.js version

20.1.0

Operating system

macOS

Operating system version (i.e. 20.04, 11.3, 10)

13.3.1

Description

I think this is a Node issue, but only @fastify/autoloader is affected by it, so I am opening an issue with you good folks as well in case this is really a Fastify issue instead or if it is a Node issue but you are able to find a workaround!

  fastify.register(fastifyAutoload, {
    dir: join(__dirname, 'routes'),
    forceESM: true,
  });

This piece of code works in Node 19 when using node --loader=ts-node/esm (the route files are in TypeScript) but not on Node 20.

I have come up with a repro here: https://github.com/TomasHubelbauer/node-esm-loader-repro

Steps to Reproduce

Follow my repro repo: https://github.com/TomasHubelbauer/node-esm-loader-repro

Steps copied here for posterity.

Node 20:

1. nvm install 20 to install Node 20
2. node --version to ensure Node version (I get 20.1.0)
3. npm install to install dependencies
4. npm run test to run the health.test.ts script

Notice the test fails and Fastify's autoload is seemingly not aware of the
--loader option and attempts to load routes/health.ts without TypeScript to
JavaScript conversion via ts-node/esm.

npm run test

> repro@0.0.0 test
> node --loader=ts-node/esm --experimental-specifier-resolution=node --test health.test.ts

ℹ (node:95105) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
ℹ (Use `node --trace-warnings ...` to show where the warning was created)
✖ should be alive (32.750836ms)
  Error: "@fastify/autoload cannot import plugin at '/routes/health.ts'. To fix this error compile TypeScript to JavaScript or use 'ts-node' to run your app."
      at findPlugins (/node_modules/@fastify/autoload/index.js:224:15)
      at async autoload (/node-esm-loader-repro/node_modules/@fastify/autoload/index.js:35:22)

ℹ tests 1
ℹ suites 0
ℹ pass 0
ℹ fail 1
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 2208.335483

✖ failing tests:

✖ should be alive (32.750836ms)
  Error: "@fastify/autoload cannot import plugin at 'routes/health.ts'. To fix this error compile TypeScript to JavaScript or use 'ts-node' to run your app."
      at findPlugins (/node_modules/@fastify/autoload/index.js:224:15)
      at async autoload (/node_modules/@fastify/autoload/index.js:35:22)

Node 19:

1. nvm install 19 to install Node 20
2. node --version to ensure Node version (I get 19.9.0)
3. npm install to install dependencies
4. npm run test to run the health.test.ts script

Notice the test passes and Fastify's autoload is inherit the --loader option
and uses the ts-node/esm loader successfully to auto-load routes/health.ts.

npm run test

> repro@0.0.0 test
> node --loader=ts-node/esm --experimental-specifier-resolution=node --test health.test.ts

ℹ (node:95453) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
ℹ (Use `node --trace-warnings ...` to show where the warning was created)
✔ should be alive (472.711395ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 2679.742161

Expected Behavior

Works the same way in both 19 and 20.

[mcollina]

Moved to the autoload module.

[climba03003]

Works the same way in both 19 and 20.

I would just hold this issue and only fix it when --loader becomes stable.
As far as I know, there are numbers of issue about the experimental loader.
The error is mostly cause by nodejs/node#44710

[mcollina]

The APIs for loaders changed in v20 and it broke ts-node, mostly due to nodejs/node#47880.

Once the dust settle and there is an API for us to use, we'll need to make adaptation here.

[Joabesv]

Hi guys! I recently got bit by this, do we have any updates?

[mcollina]

Loaders are mostly stable now, so A PR would be highly welcomed, otherwise this will take a bit.

[Joabesv]

not saying that I can do it, but where in the code should I take a look to know about how autoload deal with the loader flag?

[Uzlopak]

I debugged it and it seems, that (Symbol.for('ts-node.register.instance') in process) is returning false. You can test it yourself:

clone the repo, open in vscode, install deps, set break point in node_modules/@fastifa/autoload/index.js in line 8.

run npm t in javascript debug terminal . In node 18 the symbol is set, in node 20 not.

[Uzlopak]

Possibly related to
TypeStrong/ts-node#1997

[Joabesv]

thank you for the very informative tips on how to test it @Uzlopak. was testing with node 20.6.0 and the problem happens with tsx too! I guess is happening with all ts-loaders

[Uzlopak]

But you could do it "manually" by setting e.g. the TS_NODE_DEV to a truthy value.

 "scripts": {
    "test": "TS_NODE_DEV=1 node --loader=ts-node/esm --experimental-specifier-resolution=node --test health.test.ts"
  },

Maybe we should add a environment variable, like FASTIFY_AUTOLOAD_TS to generally override it

[thaivlinh]

@Uzlopak 's tip work. In my case I'm using tsx.

"dev": "TS_NODE_DEV=1 node --no-warnings=ExperimentalWarning --watch --loader=tsx src/server.ts",

[Joabesv]

Can we label this issue as has workaround so others can see it has a solution?

[Uzlopak]

I created a PR #327

It is basically the same as TS_NODE_DEV but with the doubt, that it should not result in some side effects or whatever, because we use our own ENV variable.