Skip to content

Conversation

@gabritto
Copy link
Member

@gabritto gabritto commented Dec 11, 2025

Also propagates token flags in the places where Strada did.
This could have helped prevent #2326, probably.

Copilot AI review requested due to automatic review settings December 11, 2025 01:17
>T2 : Symbol(T2, Decl(assignmentCompatWithObjectMembersStringNumericNames.ts, 9, 46))
->'1.0' : Symbol(T2['1.0'], Decl(assignmentCompatWithObjectMembersStringNumericNames.ts, 10, 18))
+>'1.0' : Symbol(T2["10"], Decl(assignmentCompatWithObjectMembersStringNumericNames.ts, 10, 18))
+>'1.0' : Symbol(T2['10'], Decl(assignmentCompatWithObjectMembersStringNumericNames.ts, 10, 18))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing you did in this PR, but I bet this is a very simple oops somewhere

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a TokenFlags parameter to token creation functions in the AST factory to properly propagate quote styles and other token-specific formatting information throughout the codebase. Previously, token flags were set after creation, which could lead to inconsistencies.

Key Changes

  • Modified token creation functions (NewStringLiteral, NewNumericLiteral, NewBigIntLiteral, NewRegularExpressionLiteral, NewNoSubstitutionTemplateLiteral) to accept a flags parameter
  • Token flags are now masked at construction time to preserve only relevant flags for each token type
  • Updated all call sites to either propagate existing flags or use TokenFlagsNone for synthetic tokens
  • Test baselines now show consistent quote style preservation (single vs double quotes) in symbol output

Reviewed changes

Copilot reviewed 76 out of 76 changed files in this pull request and generated no comments.

Show a summary per file
File Description
internal/ast/ast.go Core API changes: added flags parameter to token creation functions, updated Clone methods, and refactored createToken function to pass flags directly
internal/parser/parser.go Parser now passes full token flags from scanner to token creation functions, removing post-creation flag assignments
internal/transformers/tstransforms/runtimesyntax.go Propagates token flags when creating string/numeric literals from enum members
internal/transformers/tstransforms/utilities.go Uses TokenFlagsNone for synthetic constant expressions
internal/transformers/moduletransforms/utilities.go Preserves token flags when rewriting module specifiers
internal/transformers/moduletransforms/externalmoduleinfo.go Uses TokenFlagsNone for synthetic helper imports
internal/transformers/moduletransforms/esmodule.go Uses TokenFlagsNone for synthetic "module" string literal
internal/transformers/moduletransforms/commonjsmodule.go Uses TokenFlagsNone for synthetic "__esModule" string
internal/transformers/jsxtransforms/jsx.go Uses TokenFlagsNone for all synthetic JSX-related string literals
internal/transformers/inliners/constenum.go Uses TokenFlagsNone for inlined const enum values
internal/transformers/estransforms/objectrestspread.go Uses TokenFlagsNone for synthetic array index literals
internal/transformers/estransforms/namedevaluation.go Uses TokenFlagsNone for synthetic name evaluation strings
internal/transformers/declarations/transform.go Uses TokenFlagsNone for enum declaration values in .d.ts files
internal/printer/factory.go Updated helper methods to use TokenFlagsNone for synthetic literals
internal/printer/printer_test.go Updated test code to pass TokenFlagsNone
internal/ls/codeactions_importfixes.go Preserves token flags when rewriting import module specifiers
internal/ls/autoimports.go Uses quote preference to set appropriate token flags for new imports
internal/ls/autoimportfixes.go Uses quote preference to set appropriate token flags for new imports
internal/compiler/fileloader.go Uses TokenFlagsNone for synthetic helper imports
internal/checker/nodebuilderimpl.go Properly handles quote style when creating property name literals from symbols
internal/checker/emitresolver.go Uses TokenFlagsNone for literal constant values
internal/checker/checker.go Uses TokenFlagsNone for synthetic element access literals
testdata/baselines/reference/**/*.symbols Baseline updates showing consistent quote style preservation in symbol output
testdata/baselines/reference/**/enumWithQuotedElementName1.js Shows proper quote preservation in emitted enum code
testdata/baselines/reference/**/enumWithUnicodeEscape1.js Shows proper quote preservation in emitted enum code with unicode escapes

@gabritto gabritto marked this pull request as draft December 11, 2025 01:45
case ast.KindIdentifier:
return tx.Factory().NewStringLiteralFromNode(name)
case ast.KindStringLiteral:
return tx.Factory().NewStringLiteral(name.Text())
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could preserve flags here, but we didn't in Strada (I think because cloneNode doesn't copy the singleQuote property, but haven't looked deeply), so if we propagate flags here it ends up producing a couple diffs.

name = tryRenameExternalModule(factory, moduleName, sourceFile)
}
if name == nil {
name = factory.NewStringLiteral(moduleName.Text())
Copy link
Member Author

@gabritto gabritto Dec 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as other comment above, we could preserve flags here but since Strada didn't, we end up with 190 new diffs if we try and preserve them now.

@gabritto gabritto marked this pull request as ready for review December 11, 2025 02:15
@gabritto gabritto mentioned this pull request Dec 11, 2025
Copy link
Member

@jakebailey jakebailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth leaving comments about those cases?

Copy link
Member

@jakebailey jakebailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, all diff deletions

@gabritto gabritto enabled auto-merge December 11, 2025 17:19
@gabritto gabritto added this pull request to the merge queue Dec 11, 2025
Merged via the queue into main with commit 59a089d Dec 11, 2025
22 checks passed
@gabritto gabritto deleted the gabritto/token_flags branch December 11, 2025 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants