Skip to content

Conversation

@marcins
Copy link
Contributor

@marcins marcins commented Sep 12, 2025

As this change touches a lot of files, I've done a Loom overview to highlight the important parts: https://www.loom.com/share/432549b7b21545a49e78829c13bb2206?sid=4887a78a-df6d-4b77-82a4-803ea6393e55

Motivation

We want to interate / experiment with changes to source maps, and currently as an external library this is complicated. If we in-source the source map library we are free to make breaking changes that suit our use cases and Atlaspack more directly.

Changes

  • Bring in the Rust crate for parcel_sourcemap
  • Update usages in our Rust code to use our crate
  • Have to bring in upstream parcel_sourcemap for the native CSS Transformer as it expects a parcel_sourcemap as input
  • Create a new @atlaspack/source-map package that is what @parcel/source-map was
  • Update any references and tests to ensure everything works

Checklist

  • Existing or new tests cover this change
  • There is a changeset for this change, or one is not required
  • Added documentation for any new features to the docs/ folder

@changeset-bot
Copy link

changeset-bot bot commented Sep 12, 2025

🦋 Changeset detected

Latest commit: 566bde7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 115 packages
Name Type
@atlaspack/parcel-to-atlaspack Minor
@atlaspack/transformer-typescript-types Minor
@atlaspack/transformer-typescript-tsc Minor
@atlaspack/eslint-plugin-internal Minor
@atlaspack/optimizer-inline-requires Minor
@atlaspack/integration-tests Minor
@atlaspack/types-internal Minor
@atlaspack/feature-flags Minor
@atlaspack/optimizer-cssnano Minor
@atlaspack/transformer-babel Minor
@atlaspack/eslint-config Minor
@atlaspack/optimizer-terser Minor
@atlaspack/transformer-less Minor
@atlaspack/transformer-sass Minor
@atlaspack/transformer-css Minor
@atlaspack/source-map Minor
@atlaspack/transformer-js Major
@atlaspack/optimizer-css Minor
@atlaspack/optimizer-swc Minor
@atlaspack/packager-css Minor
@atlaspack/packager-js Minor
@atlaspack/utils Minor
@atlaspack/core Minor
@atlaspack/rust Minor
atlaspack Patch
@atlaspack/bundler-default Patch
@atlaspack/codeframe Patch
@atlaspack/fs Patch
@atlaspack/graph Patch
@atlaspack/logger Patch
@atlaspack/plugin Patch
@atlaspack/profiler Patch
@atlaspack/types Patch
@atlaspack/workers Patch
@atlaspack/reporter-bundle-buddy Patch
@atlaspack/resolver-tesseract Patch
@atlaspack/runtime-js Patch
@atlaspack/transformer-postcss Patch
@atlaspack/transformer-tokens Patch
@atlaspack/transformer-xml Patch
@atlaspack/ts-utils Patch
@atlaspack/bundler-experimental Patch
@atlaspack/cache Patch
@atlaspack/cli Patch
@atlaspack/inspector Patch
@atlaspack/packager-html Patch
@atlaspack/reporter-cli Patch
@atlaspack/reporter-conditional-manifest Patch
@atlaspack/runtime-react-refresh Patch
@atlaspack/transformer-compiled-css-in-js Patch
@atlaspack/transformer-html Patch
@atlaspack/transformer-react-refresh-wrap Patch
@atlaspack/node-resolver-core Patch
@atlaspack/eslint-config-browser Patch
@atlaspack/config-default Major
@atlaspack/package-manager Patch
@atlaspack/test-utils Patch
@atlaspack/link Patch
@atlaspack/bundle-stats Patch
@atlaspack/optimizer-blob-url Patch
@atlaspack/optimizer-data-url Patch
@atlaspack/optimizer-image Major
@atlaspack/optimizer-svgo Patch
@atlaspack/packager-raw-url Patch
@atlaspack/packager-svg Patch
@atlaspack/packager-webextension Patch
@atlaspack/packager-xml Patch
@atlaspack/reporter-build-metrics Patch
@atlaspack/reporter-bundle-analyzer Patch
@atlaspack/reporter-bundle-stats Patch
@atlaspack/reporter-dev-server-sw Patch
@atlaspack/reporter-dev-server Patch
@atlaspack/reporter-json Patch
@atlaspack/reporter-lsp Patch
@atlaspack/reporter-sourcemap-visualiser Patch
@atlaspack/reporter-tracer Patch
@atlaspack/resolver-glob Patch
@atlaspack/runtime-browser-hmr Patch
@atlaspack/runtime-service-worker Patch
@atlaspack/runtime-webextension Patch
@atlaspack/transformer-image Major
@atlaspack/transformer-posthtml Patch
@atlaspack/transformer-webextension Patch
@atlaspack/transformer-webmanifest Patch
@atlaspack/watcher-watchman-js Patch
@atlaspack/validator-eslint Patch
@atlaspack/validator-typescript Patch
@atlaspack/config-webextension Major
@atlaspack/register Patch
@atlaspack/query Patch
@atlaspack/inspector-frontend Patch
@atlaspack/transformer-svg Patch
@atlaspack/bundler-library Patch
@atlaspack/compressor-brotli Patch
@atlaspack/compressor-gzip Patch
@atlaspack/compressor-raw Patch
@atlaspack/namer-default Patch
@atlaspack/optimizer-htmlnano Patch
@atlaspack/packager-raw Patch
@atlaspack/packager-ts Patch
@atlaspack/packager-wasm Patch
@atlaspack/resolver-default Patch
@atlaspack/transformer-glsl Patch
@atlaspack/transformer-graphql Patch
@atlaspack/transformer-inline-string Patch
@atlaspack/transformer-inline Patch
@atlaspack/transformer-json Patch
@atlaspack/transformer-jsonld Patch
@atlaspack/transformer-mdx Patch
@atlaspack/transformer-pug Patch
@atlaspack/transformer-raw Patch
@atlaspack/transformer-svg-react Patch
@atlaspack/transformer-toml Patch
@atlaspack/transformer-worklet Patch
@atlaspack/transformer-yaml Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@ferris-atlaspack-bot
Copy link

ferris-atlaspack-bot bot commented Sep 12, 2025

🦀 Ferris' Rust Changeset Check

I can see you have now included @atlaspack/rust in your changeset. This means your Rust changes will be published.
Now I'm a happy crab 🦀🎉

Copy link
Contributor Author

@marcins marcins Sep 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

@marcins marcins Sep 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This whole folder is a copy of https://github.com/parcel-bundler/source-map/tree/master/parcel_sourcemap

Merging this into the existing atlaspack_sourcemap would be a next step, as that crate is (mostly) a wrapper around this one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The files in here are mostly copied from https://github.com/parcel-bundler/source-map/tree/master/src, with a conversion to TS and other minor changes to make things work with TS.

@marcins marcins force-pushed the mszczepanski/insource-source-map branch from d7904d4 to 17c0457 Compare September 15, 2025 21:02
@marcins marcins force-pushed the mszczepanski/insource-source-map branch from 17c0457 to 51ecd1d Compare September 24, 2025 23:16
@github-actions
Copy link
Contributor

github-actions bot commented Sep 24, 2025

📊 Type Coverage Report

Coverage Comparison

Metric Baseline Current Change
Coverage Percentage 92.05% 92.06% 📈 +0.01%
Correctly Typed 201,713 202,796 +1,083
Total Expressions 219,133 220,268 +1,135
Untyped Expressions 17,420 17,472 +52

Files with Most Type Issues (Top 15)

File Issues Affected Lines
packages/core/integration-tests/test/javascript.ts 983 648
packages/core/integration-tests/test/cache.ts 885 626
packages/core/integration-tests/test/scope-hoisting.ts 623 490
packages/utils/node-resolver-core/test/resolver.ts 476 177
packages/core/integration-tests/test/html.ts 468 294
packages/core/integration-tests/test/sourcemaps.ts 356 176
packages/core/test-utils/src/utils.ts 330 205
packages/core/integration-tests/test/incremental-bundling.ts 298 206
packages/core/core/src/dumpGraphToGraphViz.ts 251 108
packages/core/integration-tests/test/output-formats.ts 227 161
packages/transformers/webextension/src/WebExtensionTransformer.ts 210 80
packages/core/integration-tests/test/react-refresh.ts 197 71
packages/core/integration-tests/test/css-modules.ts 191 107
packages/core/core/src/requests/TargetRequest.ts 190 133
packages/core/integration-tests/test/babel.ts 187 115

This report was generated by the Type Coverage GitHub Action

@marcins marcins marked this pull request as ready for review September 25, 2025 01:50
@marcins marcins requested a review from a team as a code owner September 25, 2025 01:50
@marcins marcins force-pushed the mszczepanski/insource-source-map branch from 5779cd2 to 932473e Compare October 1, 2025 05:47
@marcins marcins force-pushed the mszczepanski/insource-source-map branch 2 times, most recently from c5cb7e0 to fd073fc Compare October 17, 2025 04:03
@marcins marcins force-pushed the mszczepanski/insource-source-map branch from fd073fc to 8b7ca45 Compare November 20, 2025 23:34
In sources the @parcel/source-map Rust and JavaScript code into Atlaspack,
and updates all of the usages.

Introduces a `safeToBuffer` method to ensure that any Parcel sourcemaps
don't cause problems from the outside.
@marcins marcins force-pushed the mszczepanski/insource-source-map branch from 8b7ca45 to 566bde7 Compare November 20, 2025 23:52
@github-actions
Copy link
Contributor

📊 Benchmark Results

🎉 Performance improvements detected!

📊 Benchmark Results

Overall Performance

Test Duration JS Memory Peak Native Memory Peak vs Baseline Status
Three.js Real Repository (JS) 16.07s 1.81GB 1.89GB -2.17% duration, -14.15% JS memory 🟡 Neutral
Three.js Real Repository (Native) 24.54s 3.23GB 3.54GB -18.46% duration, -13.49% JS memory 🟢 Improvement

🔍 Detailed Phase Analysis

Three.js Real Repository (JS)

Phase Duration (avg) Duration (p95) Memory Peak (avg) Memory Peak (p95)
resolving 8.48s 8.87s 1.17GB 1.31GB
transforming 8.48s 8.86s 1.17GB 1.31GB
bundling 8.27s 8.64s 1.18GB 1.31GB
bundled 7.86s 8.27s 1.28GB 1.43GB
packaging 5.69s 5.94s 1.39GB 1.53GB
optimizing 5.54s 5.80s 1.81GB 1.89GB

Three.js Real Repository (Native)

Phase Duration (avg) Duration (p95) Memory Peak (avg) Memory Peak (p95)
bundling 15.41s 18.48s 2.32GB 2.65GB
bundled 15.03s 18.05s 2.51GB 2.85GB
packaging 7.10s 7.37s 2.78GB 3.13GB
optimizing 6.91s 7.23s 3.23GB 3.54GB

💾 Unified Memory Analysis

Three.js Real Repository (JS) Memory Statistics

Memory Type Metric Min Mean Median P95 P99 Max Std Dev
JavaScript RSS 1.31GB 1.63GB 1.72GB 1.89GB 1.89GB 1.89GB 207.65MB
Heap Used 80.08MB 89.70MB 94.23MB 99.71MB 99.71MB 99.71MB 8.16MB
Heap Total 103.35MB 131.99MB 130.60MB 152.34MB 152.34MB 152.34MB 16.29MB
External 38.94MB 106.03MB 173.09MB 173.09MB 173.09MB 173.09MB 67.05MB
Native (Rust) Physical Memory 1.23GB 1.46GB 1.48GB 1.74GB 1.86GB 1.89GB 160.74MB
Virtual Memory 30.05GB 30.76GB 30.80GB 31.02GB 31.10GB 31.15GB 182.71MB

Sample Counts: JS: 6, Native: 262

Three.js Real Repository (Native) Memory Statistics

Memory Type Metric Min Mean Median P95 P99 Max Std Dev
JavaScript RSS 2.37GB 2.96GB 3.03GB 3.54GB 3.54GB 3.54GB 383.43MB
Heap Used 76.96MB 77.21MB 77.28MB 77.52MB 77.52MB 77.52MB 0.19MB
Heap Total 87.95MB 95.40MB 98.45MB 100.45MB 100.45MB 100.45MB 5.11MB
External 179.56MB 182.45MB 185.34MB 185.34MB 185.34MB 185.34MB 2.89MB
Native (Rust) Physical Memory 2.49GB 2.89GB 2.92GB 3.30GB 3.45GB 3.54GB 252.43MB
Virtual Memory 30.65GB 33.36GB 33.47GB 33.81GB 33.98GB 34.04GB 511.25MB

Sample Counts: JS: 6, Native: 510

🖥️ Environment

  • Node.js: v22.17.0
  • Platform: linux (x64)
  • CPU: AMD EPYC 7763 64-Core Processor
  • Total Memory: 15.62GB
  • Git SHA: 566bde7
  • Branch: mszczepanski/insource-source-map
  • Timestamp: 2025-11-21T00:35:18.326Z

@marcins marcins merged commit 73168c2 into main Nov 23, 2025
26 of 27 checks passed
@marcins marcins deleted the mszczepanski/insource-source-map branch November 23, 2025 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants