estree codegen

[leaysgur]

Background

Boshen:
I wonder whether we should start forking and combining things to get things printed correct on the js side 😰
To complete the user story for oxc-parser-> transfer to js -> modify ast -> print
https://discord.com/channels/1079625926024900739/1080712024683708466/1379710015539249273

I happened to be researching sveltejs/esrap for another task, so we thought it might be a good choice if it could be integrated easily.

Prerequisite

oxc-parser currently produces:

• ESTree AST for JavaScript
• TS-ESTree AST for TypeScript

And it also supports JSX and several Stage 3 proposals.

How could we implement this?

At first I planned to use esrap as an optionalDependency of oxc-parser.

However, after trying it I found that:

• It does NOT support JSX
• It even fails to print simple TS code such as let a: string | null = null
  • Some of them have been fixed recently, but it still has missing
• We would also need to attach comments to nodes manually

Another issue is that sveltejs/esrap expects the AST produced by sveltejs/acorn-typescript, which is similar to @typescript-eslint/typescript-estree(oxc-parser outputs), but it seems slightly different.

Look for other implementations?

In short, the most robust way to turn a (TS-)ESTree AST into code today is Prettier. 😅

But if we choose Prettier, the main question is runtime performance? And it might be too much(too pretty) for this use-case.

If we do not choose Prettier, we could:

• Fork(or use extend API if possible?) esrap, astring or estree-util-to-js
• Add TS and|or JSX support
• Update comment attaching logic

Of course, the best option depends on how much effort we want to invest, but in any case Prettier will likely give more intuitive results than the alternatives.

However, if the comments do not have to be handled so strictly, performance seems to be better with these.

What do you think?

[jycouet]

Thank you for writing all this down, I'm going in circle for the last few days on all this :).
I find it interesting that it's surfacing only now as the transform hook of vite expect to return some printed code. So you parse ✅, transform ✅ and you need to print it to return it somehow. 👀

Coming from the svelte community, I stop at esrap, and want to improve a few not handled case yet!
I'll be happy to look at let a: string | null = null! (in addition to the PR I already did yesterday)
Let's see how it's shaping 🚀

Side note, oxc-parser is removing comments, no ? Or I didn't see something ?

I'll be happy to help more!

[Boshen]

I think we can move forward with esrap if it covers jsx and conform to @typescript-eslint/typescript-estree.

Side note, oxc-parser is removing comments, no ? Or I didn't see something ?

oxc-parser has not implemented any comment attachment logics (yet).

[leaysgur]

BTW, I plan to look into the following:

• Read through esrap
  • I already know it does not yet support JSX
• Check gaps between acorn-typescript & @typescript-eslint/estree (oxc-parser)
• Check astring with astravel + astring-jsx + astring-ts-generator combo

---

As briefly mentioned on Discord, the ultimate(still-unseen!) future goal is likely:

• The AST on the JS side can be raw-transferred BACK to the Rust side
• On the Rust side, we can turn that AST into code with oxc_formatter(or oxc_codegen?) with various options

...that's the ideal?

So, until that becomes possible, for the time being:

• For now, advise using a custom parser with Prettier
  • A solid option that works today and supports both TS-ESTree and JSX
  • Comment handling and quality are also ensured(not verified yet
• Meanwhile, provide some lightweight implementation ourselves?
  • How to implement depends on what I find in the investigation
  • Assume we'll compromise on comment handling, —best effort only
  • Instead, it may be better in performance

[leaysgur]

I'll share my research results...

if it covers jsx

No, esrap does not support printing JSX.

Also, there is no API to extend its behavior.

This means we need to fork it (or continuously send PRs?) to base on esrap.

and conform to @typescript-eslint/typescript-estree.

Unfortunately, it does not.

esrap can print the AST generated by @sveltejs/acorn-typescript, but not all TS-prefixed node types are supported.

Their AST structure looks similar but is not the same.

For example:

• It does not produce TSEnumBody, TSAsExpression, TSNonNullExpression, etc
• It always produces TSParenthesizedType, preserveParens: false has no effect
• Leading comments do not update the Program node's span.start
• module A.B.C {} produces nested TSModuleDeclaration instead of a nested TSQualifiedName
• Our add_fields tweaks, like typeAnnotation: null, are not completely shown
• etc...

If you would like to see it in action, please try https://leaysgur.github.io/js-multi-ast-viewer/

[Boshen]

So there is no full solution for printing estree on the js side.

I added the "E-Help Wanted" label.

The core team does have the time and energy to maintain another codegen on the js side.

[leaysgur]

no full solution for printing estree on the js side.

Let me elaborate on this a bit.

• The JS ESTree AST can be printed with esrap, astring, etc
  • For JSX, astring + astring-jsx can handle it
• For TS, esrap and astring-ts-generator can handle the acorn-typescript AST
  • But acorn-typescript does not fully support all TS syntaxes
• Finally, for the typescript-estree AST, only Prettier can print it correctly
• Otherwise, you need to extend astring via generator options and implement printer for JSX and TS-ESTree
  • esrap may also become extendable in near future

[overlookmotel]

As briefly mentioned on Discord, the ultimate(still-unseen!) future goal is likely:

• The AST on the JS side can be raw-transferred BACK to the Rust side
• On the Rust side, we can turn that AST into code with oxc_formatter(or oxc_codegen?) with various options

...that's the ideal?

Just to say, yes, I think this is our ideal end goal. But we don't expect to get there any time soon.

[Rich-Harris]

Hey, Rich from the Svelte team here. Our plan for esrap is to make it pluggable (sveltejs/esrap#35) in such a way that it can print any AST that uses the { type: string, ... } shape, be it JS, TS, JSX, CSS or whatever else. It's been on our wishlist for a while but if being able to add things like JSX support would be valuable to you, I'm happy to bump it up the priority list