-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Description
Is there an existing issue for this?
- I have searched the existing issues
This issue exists in the latest npm version
- I am using the latest npm
Current Behavior
If you kick off multiple npx
processes at the same time for the same non-local package(s), they can potentially install atop each other in the same npx cache directory. This can cause one or both processes to fail silently, or with various errors (e.g. TAR_ENTRY_ERROR
, ENOTEMPTY
, EJSONPARSE
, MODULE_NOT_FOUND
), depending on when/where something gets clobbered.
From my run-ins with this in the wild, I think this is probably most likely to occur in monorepos where you are running multiple similar tasks simultaneously (e.g. with turborepo).
There are workarounds, but they have some downsides:
- Install the package to the npx cache before kicking off the concurrent executions, e.g.
👎 extra complexity up front
npx -y -p <specifier> true # no-op, just make sure we have it
👎 sometimes this is not known at the top-level (e.g. the owners of specific workspaces have independently decided to npx the same tool, or it's an implementation detail of a transitive dependency) - Install the package locally so that npx doesn't need to install it
👎 not ideal if you always want to use the latest version - Serialize the npx calls with some kind of lock/mutex
👎 puts extra complexity on the user
👎 potentially serializes too much, i.e. it's just the install that needs to be made process-safe, not the entire execution
From a glance I think this could probably be fixed in libnpmexec, perhaps by using the mkdir locking strategy (e.g. see proper-lockfile for one such implementation)... i.e. might be a matter of creating/managing/awaiting a (temp) subdirectory to "lock" this section, so that only one process is installing at a time.
Expected Behavior
Concurrent calls to npx
should not risk clobbering each other's installs.
Steps To Reproduce
Basic repro (doesn't always fail, and results vary from one to the next 🙃):
rm -rf "$(npm config get cache)" # blow away everything between runs
npx -y tsup --version &
npx -y tsup --version &
wait
Note that the bug isn't specific to tsup, I just picked that package since it repros ~25% of the time.
Environment
- npm: 11.3.0
- Node.js: v22.14.0
- OS Name: Mac OS Sequoia 15.3
- System Model Name: Macbook Pro