Clarify when to use `rushx` over `rush-pnpm run`

[npetruzzelli]

I've been looking through the website to find some information on the binaries and am wondering if I am missing something obvious.

When do I use rushx <script-name> over rush-pnpm run <script-name>? What do they do differently?

(toggle) Motivation/Why?

This came about as I am trying to figure out whether I need to open a discussion in Rush's chat room or if I need to open an issue with the npm-run-all2 package to figure out how to correctly use npm-run-all2 in a Rush monorepo, including special consideration for pnpm if needed.

• use npm-run-all2 from a script in a workspace's package.json file
• use npm-run-all2 from a command line when the CWD is in a workspace
  • globally installed
  • locally installed
• use npm-run-all2 at the root level
  • custom command + auto installer?

While the need to use a specific package may be mine, I think the confusion about when to use which command with pnpm is not unique to me and could use clarity in the documentation.

Is there a part of the documentation that I am missing?

(toggle) I tried to follow the Rush source code, but got lost or burned out (probably both)

1. All of the the binaries import start.js as a side effect
2. Start callsRushCommandSelector.execute(launcherVersion, selectedRushLib, options)
   1. Calls RushCommandSelector._getCommandName()
   2. Which determines the command that is called
3. if rush-pnpm
   1. launch the internal API using rush-lib, which then runs like any other API would
   2. Process and load the Rush configuration
      1. Determine that the package manager is pnpm using the config
      2. Set the path to a local copy of the package manager
   3. Determine the pnpm command arguments (this includes the which pnpm command is being called)
   4. Among other things, it produces warnings and errors for commands known to be incompatible or otherwise contradictory to Rush. run and exec are known to be safe
4. if rushx
   1. launch the internal API using rush-lib, which then runs like any other API would

(toggle) What Rush Command or Practice Replaces What?

Workspace script

• use rushx <script-name> as a replacement for:
  • npm run-script <script-name> (aliased: npm run)
  • yarn <script-name> (Yarn Classic) CWD is in the workspace
  • yarn <script-name>
• pnpm: ???
  • use rush-pnpm run <script-name> or rushx <script-name>? as a replacement for pnpm run <script-name>

Arbitrary Command From A Package At The Root Level

• don't do this in a Rush monorepo?
• use custom commands / autoinstallers instead?

In A Workspace While At The Root Level

• don't do this in a Rush monorepo?
• replaces:
  • npm run --workspace=<workspace-name> <command-name> (see also: --workspaces and --include-workspace-root)
  • yarn workspace <workspace-name> <command-name> (Yarn Classic)
  • yarn workspace <workspace-name> <command-name>
  • pnpm filtering

Arbitrary REMOTE Command From A Package At The Workspace Level

• don't do this in a Rush monorepo?

rush-pnpm dlx <command-name> "works" (I think?), but I can't figure out why. dlx is not known to be a safe command according to Rush. It is always REMOTE and doesn't seem to use the command in the workspace's LOCAL node_modules directory.

Arbitrary LOCAL Command From A Package At The Workspace Level

What is the rush replacement for the following?

• npx <command-name>
• npm exec <command-name>
  • npx vs npm exec
• yarn <command-name> (Yarn classic: Locally installed CLIs)
• yarn dlx <command-name> (Yarn Berry)
• yarn exec <command-name> (Yarn Berry)
• pnpm dlx <command-name>
  • pnpx is an alias for pnpm dlx
• pnpm exec <command-name>
  • pnpm <command-name>

    The exec part is actually optional when the command is not in conflict with a builtin pnpm command

[octogonz]

Great writeup!

Some answers from Nick's Zulip thread:

rushx is the idiomatic recommended command for running package.json scripts.

pnpm-rush is a wrapper used to invoke PNPM commands that are outside of Rush's feature set. A classic example is pnpm patch -- Rush doesn't directly get involved with package patching, but also you cannot simply use pnpm patch (because Rush generates PNPM config files under common/temp/ that PNPM doesn't know how to manage directly). So you should use rush-pnpm patch instead, which validates the configuration and then invokes pnpm patch with the right context. The other benefit of rush-pnpm is that it can warn you if you try to invoke PNPM commands that are not reliable in a Rush repo.

(Ideally we would eventually implement a native Rush equivalent for every command like pnpm patch, and then you wouldn't need to use rush-pnpm. But that effort isn't justified in cases where rush-pnpm works well enough.)

__

"lint:scripts": "eslint \"src/**/*.{tsx,ts,js,jsx,mjs,cjs}\""

In a large monorepo, this way of invoking tools tends not to scale very well. Some things to consider:

• Would you copy+paste this list of file extensions into every package.json? What if you have 1000 projects?
• Usingnpm-run-all to invoke individual tasks treats them as a single action. What if some need to be serial, and some need to be parallel? How can we cache individual subsets of work?
• What if you want to customize the build with CLI parameters (e.g. --production or --skip-docs)? Passing arguments to full-build will just append them to a shell command, assuming that the command will handle it intelligently.

Rush's typical approach is for projects to be built by a toolchain, such as Heft. The toolchain has configurable recipes (Heft calls them "rigs") that can be reused across many projects of a given type. It can support watch mode. Its actions be intelligently divided into "phases" that get cached independently. And its CLI can accept predefined parameters and pass them along to the appropriate subtasks.

When you have a more formalized way of building, then what you tend to want is a relatively small set of well-documented standard commands, rather than "run any shell command with any parameters in any project folder". (In a repo with 1000 projects, 900 of them probably are unfamiliar to you, so how would you even know what shell command to run? It's much better for every project to participate in a standardized set of operations like "build", "test", etc.)

But for special cases, pnpm-rush run is there if you need it.

We should improve the website docs to address this better.

[npetruzzelli]

@octogonz - thank you for taking the time to respond to me here and in the Zulip thread.

The below is some of my background for context, but for those that don't need it, it sounds like Heft is the Rush way of doing what I was looking for, so I will need to take some time to learn a bit more about it.

(toggle wall of text)

Early days, I started with Gulp. Gulp is a task runner, not a build system, but each task can be made aware of its dependencies and its dependencies can be in parallel or serial as needed. One of the nice things was the design direction given for the scope of plugins. Do one thing and do it well. Don't try to do everything. I believe this is how it avoided falling into Grunt's trap of multiple tools trying to do the same thing, often repeating work or conflicting with each other. At least that was my impression of Grunt. I haven't used it myself, so take that with a grain of salt.

From there it was Webpack => Create React App => modify CRA => rolldown. I'll spare there details as most people in this space can probably imagine the ups and downs. During this I was exposed to monorepos. First Yarn workspaces, then Lerna, then npm picked up workspaces and I had to start thinking about more than one package at a time. As a contractor, sometimes I worked on teams that used multiple operating systems or shells and scripts needed to work on all of them, so I became more aware of tools like npm-run-all, concurrently, and any number of command line packages, often but not always, written by Sindre Sorhus. These teams didn't necessarily have CICD, but still needed to do a lot for build/test/lint. It couldn't always be a single command.

A couple years ago, I learned about Nx, which was great and has a number of the features you described. The project I am currently on is my first experience with Rush. Its so different that it felt like I took a step backwards, when really I didn't know what I was ignorant about.

Because I was ignorant about the Rush (or Heft) way of doing things, I began reaching for familiar tools that solved the problem ("work in any common command line and on any common operating system") for me in the past, and was why I thought npm-run-all2 would allow me to pull multiple instructions out of bash shell scripts that currently hold a lot of things and move them back into package.json.

I figured if Rush caching (once I eventually start trying to learn about that) is anything like Nx's, then having all the commands in a single shell script would prevent many benefits of local and remote caches. I understand npm-run-all2 may not be ideal for some cases (especially larger monorepos), but it seemed like it might be a step closer to preparing for cache related improvements.

I'll look into learning Heft and if appropriate, see if it is something my fellow team members are interested in using for managing tasks.

Once again, thank you.