Skip to content

[BUG] concurrent npx executions are not process-safe #8224

@jenseng

Description

@jenseng

Is there an existing issue for this?

  • I have searched the existing issues

This issue exists in the latest npm version

  • I am using the latest npm

Current Behavior

If you kick off multiple npx processes at the same time for the same non-local package(s), they can potentially install atop each other in the same npx cache directory. This can cause one or both processes to fail silently, or with various errors (e.g. TAR_ENTRY_ERROR, ENOTEMPTY, EJSONPARSE, MODULE_NOT_FOUND), depending on when/where something gets clobbered.

From my run-ins with this in the wild, I think this is probably most likely to occur in monorepos where you are running multiple similar tasks simultaneously (e.g. with turborepo).

There are workarounds, but they have some downsides:

  • Install the package to the npx cache before kicking off the concurrent executions, e.g.
    npx -y -p <specifier> true # no-op, just make sure we have it
    👎 extra complexity up front
    👎 sometimes this is not known at the top-level (e.g. the owners of specific workspaces have independently decided to npx the same tool, or it's an implementation detail of a transitive dependency)
  • Install the package locally so that npx doesn't need to install it
    👎 not ideal if you always want to use the latest version
  • Serialize the npx calls with some kind of lock/mutex
    👎 puts extra complexity on the user
    👎 potentially serializes too much, i.e. it's just the install that needs to be made process-safe, not the entire execution

From a glance I think this could probably be fixed in libnpmexec, perhaps by using the mkdir locking strategy (e.g. see proper-lockfile for one such implementation)... i.e. might be a matter of creating/managing/awaiting a (temp) subdirectory to "lock" this section, so that only one process is installing at a time.

Expected Behavior

Concurrent calls to npx should not risk clobbering each other's installs.

Steps To Reproduce

Basic repro (doesn't always fail, and results vary from one to the next 🙃):

rm -rf "$(npm config get cache)" # blow away everything between runs
npx -y tsup --version &
npx -y tsup --version &
wait

Note that the bug isn't specific to tsup, I just picked that package since it repros ~25% of the time.

Environment

  • npm: 11.3.0
  • Node.js: v22.14.0
  • OS Name: Mac OS Sequoia 15.3
  • System Model Name: Macbook Pro

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bugthing that needs fixingNeeds Triageneeds review for next steps

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions