Tokenizer incorrectly iterprets a long expression resulting in a string as a literal string

[bernatagulloesbrina]

Description

in the format expression of a calculation group I have the following expression:

`SWITCH(
TRUE( ),
SELECTEDMEASURENAME( ) IN
VALUES(
'Time Intelligence Affected Measures'[Measure]
),
IF(
FIND(
UNICHAR( 8204 ),
SELECTEDMEASUREFORMATSTRING(

            ),
            1,
            -1
        )
            <> -1,
        SELECTEDMEASUREFORMATSTRING(

        ),
        "+#,##0.# %;-#,##0.# %;-"
    ),
ISSELECTEDMEASURE(
    [Label as format string]
),
    """"
        & /*YOY%*/ VAR ValueCurrentPeriod =
            SELECTEDVALUE( 'Date'[Year] )
        VAR ValuePreviousPeriod = /*PY*/
            IF(
                [ShowValueForDates],
                CALCULATE(
                    SELECTEDVALUE( 'Date'[Year] ),
                    CALCULATETABLE(
                        DATEADD(
                            'Date'[Date],
                            -1,
                            YEAR
                        ),
                        'Date'[DateWithSales] = TRUE
                    )
                )
            )
        VAR Result =
            IF(
                NOT ISBLANK( ValueCurrentPeriod )
                    && NOT ISBLANK( ValuePreviousPeriod ),
                ValueCurrentPeriod & " vs "
                    & ValuePreviousPeriod
                    & " (%)"
            )
        RETURN
            Result & """",
SELECTEDMEASUREFORMATSTRING(

)

)`

I'm finding that TE3 tokenizer is interpretting this whole chunk as a string literal

"""" & /*YOY%*/ VAR ValueCurrentPeriod = SELECTEDVALUE( 'Date'[Year] ) VAR ValuePreviousPeriod = /*PY*/ IF( [ShowValueForDates], CALCULATE( SELECTEDVALUE( 'Date'[Year] ), CALCULATETABLE( DATEADD( 'Date'[Date], -1, YEAR ), 'Date'[DateWithSales] = TRUE ) ) ) VAR Result = IF( NOT ISBLANK( ValueCurrentPeriod ) && NOT ISBLANK( ValuePreviousPeriod ), ValueCurrentPeriod & " vs " & ValuePreviousPeriod & " (%)" ) RETURN Result & """"

I know super niche, but still, there are measures and variables in there that I wanted to replace

Tabular Editor 3 Version

3.24.2

Screenshots

Steps to Reproduce

if needed contact me for the repro pbix and script

Expected behavior

it should correctly break down into varibles, measures and so on

Crash Report

No response

Windows Version

11