Reliably determine @rescript/runtime path #7637

cknitt · 2025-07-12T06:18:11Z

(updated description after moving the runtime to @rescript/runtime and deciding to use a command line arg instead of an env var)

To determine the path of the runtime module @rescript/runtime in a reliable way that works with all package managers / package directory layouts, we need to use the JS module resolution algorithm.

In rewatch, we can use https://github.com/oxc-project/oxc-resolver.
For the legacy build system, we can just use require.resolve.

This should fix all issues with pnpm not finding the runtime module.

pkg-pr-new · 2025-07-12T06:26:30Z

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript-lang/rescript@7637

@rescript/darwin-arm64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/darwin-arm64@7637

@rescript/darwin-x64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/darwin-x64@7637

@rescript/linux-arm64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/linux-arm64@7637

@rescript/linux-x64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/linux-x64@7637

@rescript/runtime

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/runtime@7637

@rescript/win32-x64

npm i https://pkg.pr.new/rescript-lang/rescript/@rescript/win32-x64@7637

commit: 290f40c

cknitt · 2025-07-12T07:15:35Z

BTW this approach should also make it easier to move the runtime to @rescript/runtime.

cknitt · 2025-07-12T10:50:55Z

@zth This probably needs to be taken into account in the editor extension, too?

BTW this approach should also make it easier to move the runtime to @rescript/runtime.

So maybe the variable should actually be RESCRIPT_RUNTIME or RESCRIPT_RUNTIME_PATH or something like that.

zth · 2025-07-12T18:17:20Z

@cknitt this means that you have to supply that env variable always to be able to run bsc standalone...?

cknitt · 2025-07-12T18:55:51Z

Yes. I mean we could fall back to the previous algorithm if the variable is not set, but we need to be aware that there are cases where this algorithm will not work.

We could also use a command line argument instead of an env var if that's your concern - just more work to implement to get it passed through for both rewatch and bsb.

nojaf · 2025-07-12T19:13:00Z

Would like to have a fallback for this. I frequently run bsc.

cknitt · 2025-07-13T05:08:35Z

Note that that would only be an issue when directly running the platform-specific bsc.exe though.

When running bsc via yarn bsc or ./node_modules/bin/bsc, you would be running the node wrapper script that correctly supplies the env var.

nojaf · 2025-07-13T07:36:16Z

Note that that would only be an issue when directly running the platform-specific bsc.exe though.

That includes running dune exec bsc right? If so, could you please add a note which relative path to set from this repository.

zth · 2025-07-13T09:25:46Z

Would it be possible to make it an optional bsc arg (that uses the old algorithm if not supplied) and bake resolving that into the "compiler args" command? Rewatch could call a node script to get the path. Maybe in the future we could check if this resolution is needed at all by checking what package manager is used.

Bsc is already used standalone in a few places, and I envision it being used even more standalone in the future (Svelte integration, one off scripts, code blocks in docs etc). The overhead of calling a node process for each invocation is non-negligible, and the orchestration for figuring out how to call it shouldn't be too difficult (that's why compiler args is so nice).

cknitt · 2025-07-14T06:09:59Z

Yes, we could make it an optional bsc arg. There actually already exists an optional arg -runtime for the current "external stdlib" feature. To reduce confusion, maybe we should complete the work started in #7483 first. So that we get rid of @rescript/std, and the stdlib always comes from @rescript/runtime. The -runtime flag would then be (optionally) used to pass the location of that package to handle the various non-standard node_modules layouts.

If the flag is not set, the default could be the simple implementation we had earlier that just assumed npm's node_modules layout.

I would avoid rewatch calling node though, I'd prefer node/bun/whatever calling rewatch via the wrapper script, with the wrapper script passing the correct location of the runtime as implemented in this PR. rewatch can then directly call bsc.exe with the -runtime flag, or include the -runtime flag in the compiler-args output.

cknitt · 2025-09-01T07:51:28Z

Now using a -runtime-path arg in bsc, and oxc_resolver in rewatch to determine the path of @rescript/runtime.

nojaf · 2025-09-01T08:26:52Z

rewatch/src/build/build_types.rs

    pub module_names: AHashSet<String>,
    pub deleted_modules: AHashSet<String>,
    pub bsc_path: PathBuf,
+    pub runtime_path: Option<PathBuf>,


Not 100% sold that this should be part of the BuildState.
Computing it in compile::compile is probably fine.
The fact that is is optional is a bit of a smell to make.
Clean doesn't use this.

nojaf · 2025-09-01T08:28:29Z

rewatch/src/build.rs

        PathBuf::from(&interface_filename).exists()
    };
+
+    let runtime_path = helpers::get_runtime_path(&project_context).ok();


Should this not fail? Isn't this destined to throw a more cryptic error in bsc if it wasn't found?

Well, it falls back to the default (simple) implementation in bsc then, but it can still continue.

When building @rescript/runtime itself, the resolution will fail for example.

(.ok means "toOption", weird naming...)

Might be good to add that somewhere in the code as well, when it is fine to be None.

nojaf · 2025-09-01T08:31:29Z

rewatch/src/helpers.rs

-    };

-    rescript_legacy_path.unwrap_or_else(|_| panic!("Could not find rescript-legacy.exe"))
+    // Resolve <project> -> rescript


Could you elaborate in a code comment why you are resolving rescript package first? This seems like an unnecessary extra step. Why not search for @rescript/runtime from the get go?

Tried that, but didn't work, probably because there is no direct dependency on the runtime package. Or maybe I was doing something wrong. I can have another look.

No — that may just be how resolve_oxc and https://nodejs.org/api/esm.html#resolution-algorithm work. That’s fine, but we should document it.

There’s a tendency to assume oxc_resolver can solve everything; I’m all for leaning on it when it helps, but we need to understand and document what it can and cannot do.

Let’s leave clear "whys" for future contributors so they understand our decisions and don’t keep getting challenged.

rewatch/src/helpers.rs

hackwaly · 2025-09-08T07:28:16Z

I've confirmed this works perfectly.

BTW. Off topic: The rescript npm package declares it has dependency @rescript/runtime, it should in peerDependencies, not dependencies.

cknitt · 2025-09-08T08:26:57Z

I've confirmed this works perfectly.

Thanks for testing! That's great to hear!

However, we'll most probably not keep the oxc_resolver based implementation, see notes here: #7117 (comment)

I will get back to you when we have something to test again!

BTW. Off topic: The rescript npm package declares it has dependency @rescript/runtime, it should in peerDependencies, not dependencies.

I don't think so. @rescript/runtime needs to be installed automatically when installing rescript, just like the binary packages @rescript/linux-x64 etc.
(This is different from the @rescript/std we had in v11 which was an optional package.)

hackwaly · 2025-09-08T18:43:58Z

@rescript/runtime needs to be installed automatically when installing rescript, just like the binary packages @rescript/linux-x64 etc.

Yes, I agree.

But, "dependencies" only install "@rescript/runtime" package under "rescript" package's "node_modules" dir. The generated ".js" file can't resolve to there when using pnpm.

"@rescript/runtime" need to be installed at the project's "node_modules" dir. So they can be reached by project generated ".js" files.

cknitt · 2025-09-20T06:31:44Z

Superseded by #7858

cknitt changed the title ~~Stdlib dir~~ Reliably determine standard lib dir Jul 12, 2025

cknitt force-pushed the stdlib-dir branch from b864763 to 373c19a Compare July 12, 2025 06:20

cknitt requested a review from zth July 12, 2025 06:32

cknitt mentioned this pull request Jul 12, 2025

Fix finding the standard library for pnpm some more #7624

Closed

cknitt requested a review from cristianoc July 12, 2025 06:36

cknitt force-pushed the stdlib-dir branch 2 times, most recently from a18e180 to 2daa984 Compare July 12, 2025 13:27

cknitt force-pushed the stdlib-dir branch 2 times, most recently from b7afc8e to 56bb638 Compare July 13, 2025 05:36

cknitt marked this pull request as draft July 14, 2025 05:54

This was referenced Aug 26, 2025

Remove @rescript/std (in favor of @rescript/runtime) #7811

Merged

12.0.0-alpha.13: Pervasives not found when using pnpm #7526

Closed

cknitt force-pushed the stdlib-dir branch 2 times, most recently from d64d7ed to 0ccec74 Compare August 31, 2025 18:10

cknitt changed the title ~~Reliably determine standard lib dir~~ Reliably determine @rescript/runtime dir Aug 31, 2025

cknitt changed the title ~~Reliably determine @rescript/runtime dir~~ Reliably determine @rescript/runtime path Aug 31, 2025

cknitt force-pushed the stdlib-dir branch 3 times, most recently from 1817c77 to f6d8303 Compare September 1, 2025 07:19

nojaf reviewed Sep 1, 2025

View reviewed changes

rewatch/src/helpers.rs Show resolved Hide resolved

cknitt force-pushed the stdlib-dir branch 4 times, most recently from 944d91b to 3683436 Compare September 1, 2025 10:50

cknitt mentioned this pull request Sep 1, 2025

V12 beta doesn't work with Deno #7828

Closed

5 tasks

cknitt added 13 commits September 5, 2025 16:11

Move all rescript legacy js sources to rescript-legacy folder

ef3039b

Remove unused rewatch.cmd

87a55c9

New module Runtime_package

58f2ef2

bsc: add -runtime-path option

841a29e

Rewatch determine runtime path via oxc_resolver and pass to bsc

5bcb28c

Add -runtime-path handling to bsc.js

3160c0b

Perform installation test with new build system

891e238

Also resolve rescript legacy via oxc_resolver

66a167a

Use ESM way to determine runtime path

2670fe1

Set -runtime-path in legacy build (bsb), too

10ece02

Also test legacy build in installation test

714223d

Log warning if runtime path couldn't be determined

be31ab7

Fix legacy clean

290f40c

cknitt force-pushed the stdlib-dir branch from 3683436 to 290f40c Compare September 5, 2025 14:49

cknitt marked this pull request as ready for review September 5, 2025 16:09

nojaf mentioned this pull request Sep 10, 2025

Resolve @rescript/runtime #7858

Merged

cknitt closed this Sep 20, 2025

Reliably determine @rescript/runtime path #7637

Reliably determine @rescript/runtime path #7637

Uh oh!

Conversation

cknitt commented Jul 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pkg-pr-new bot commented Jul 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cknitt commented Jul 12, 2025

Uh oh!

cknitt commented Jul 12, 2025

Uh oh!

zth commented Jul 12, 2025

Uh oh!

cknitt commented Jul 12, 2025

Uh oh!

nojaf commented Jul 12, 2025

Uh oh!

cknitt commented Jul 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nojaf commented Jul 13, 2025

Uh oh!

zth commented Jul 13, 2025

Uh oh!

cknitt commented Jul 14, 2025

Uh oh!

cknitt commented Sep 1, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hackwaly commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cknitt commented Sep 8, 2025

Uh oh!

hackwaly commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cknitt commented Sep 20, 2025

Uh oh!

Uh oh!

cknitt commented Jul 12, 2025 •

edited

Loading

pkg-pr-new bot commented Jul 12, 2025 •

edited

Loading

cknitt commented Jul 13, 2025 •

edited

Loading

hackwaly commented Sep 8, 2025 •

edited

Loading

hackwaly commented Sep 8, 2025 •

edited

Loading