Skip to content

lib: allow CJS source map cache to be reclaimed#51711

Merged
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
legendecas:source-maps/cjs
May 28, 2024
Merged

lib: allow CJS source map cache to be reclaimed#51711
nodejs-github-bot merged 1 commit into
nodejs:mainfrom
legendecas:source-maps/cjs

Conversation

@legendecas

Copy link
Copy Markdown
Member

Unifies the CJS and ESM source map cache map and allows the CJS cache
entries to be queried more efficiently with a source url without
iteration on an IterableWeakMap. Reclaims the CJS cache entry with
a FinalizationRegistry instead.

Add a regression test to verify that the CJS source map cache entry can be
reclaimed.

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/loaders

@nodejs-github-bot nodejs-github-bot added the needs-ci PRs that need a full CI run. label Feb 9, 2024
@legendecas legendecas added the source maps Issues and PRs related to source map support. label Feb 9, 2024
Comment thread lib/internal/source_map/source_map_cache.js
Comment thread lib/internal/source_map/source_map_cache.js Outdated
Comment thread lib/internal/source_map/source_map_cache.js Outdated
Comment thread lib/internal/source_map/source_map_cache_map.js Outdated
@legendecas

Copy link
Copy Markdown
Member Author

@aduh95 @joyeecheung would you mind taking a look again? Thank you!

run();

// Run until the source map is cleared by GC, or fail the test after determined iterations.
common.gcUntil('SourceMap of deleted CJS module is cleared', () => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this has the potential of flakes because the strategy used by common.gcUntil() isn't very sound (i.e. sometimes gc() is not aggressive enough to purge things). It would be safer to do it for a bunch more modules (say 30), and checks if any of them can be collected, which reduces the likelihood of the one single module that we are looking at just cannot be garbage collected in time because V8 just doesn't feel pressured enough to do that.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion, updated the test with more GC pressure to reduce the chance of flaky.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should launch a stress-test CI to make sure we're not introducing any flakiness

Comment thread lib/internal/source_map/source_map_cache.js Outdated
Comment thread lib/internal/source_map/source_map_cache.js Outdated
Comment thread lib/internal/modules/esm/translators.js Outdated
@legendecas legendecas force-pushed the source-maps/cjs branch 2 times, most recently from 9482198 to 611d7db Compare March 18, 2024 11:02
@legendecas legendecas added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 18, 2024
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 18, 2024
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@legendecas legendecas requested a review from aduh95 March 19, 2024 03:31
@legendecas

Copy link
Copy Markdown
Member Author

@joyeecheung would you mind taking another look at this? Thank you!

@joyeecheung joyeecheung left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but this needs a rebase

Unifies the CJS and ESM source map cache map with SourceMapCacheMap
and allows the CJS cache entries to be queried more efficiently with
a source url without iteration on an IterableWeakMap.

Add a test to verify that the CJS source map cache entry can be
reclaimed.
@legendecas legendecas added the request-ci Add this label to start a Jenkins CI on a PR. label May 27, 2024
@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label May 27, 2024
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@legendecas

Copy link
Copy Markdown
Member Author

CI is green. @joyeecheung @aduh95 would be great to have another review, thanks!

run();

// Run until the source map is cleared by GC, or fail the test after determined iterations.
common.gcUntil('SourceMap of deleted CJS module is cleared', () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should launch a stress-test CI to make sure we're not introducing any flakiness

@legendecas

legendecas commented May 28, 2024

Copy link
Copy Markdown
Member Author

Stress test: https://ci.nodejs.org/job/node-stress-single-test/508 (passed)

@legendecas legendecas added the commit-queue Add this label to land a pull request using GitHub Actions. label May 28, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label May 28, 2024
@joyeecheung

Copy link
Copy Markdown
Member

Backport in #56927

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. source maps Issues and PRs related to source map support.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants