Use new Cache api in various places #18872

majocha · 2025-08-29T09:10:40Z

implicitYieldExpressions, argInfoCache, typeDefinitelySubsumesNoCoerceCache, typeFeasibleEquivCache

github-actions · 2025-08-29T09:11:22Z

❗ Release notes required

Caution

No release notes found for the changed paths (see table below).

Please make sure to add an entry with an informative description of the change as well as link to this pull request, issue and language suggestion if applicable. Release notes for this repository are based on Keep A Changelog format.

The following format is recommended for this repository:

* <Informative description>. ([PR #XXXXX](https://github.com/dotnet/fsharp/pull/XXXXX))

See examples in the files, listed in the table below or in th full documentation at https://fsharp.github.io/fsharp-compiler-docs/release-notes/About.html.

If you believe that release notes are not necessary for this PR, please add NO_RELEASE_NOTES label to the pull request.

You can open this PR in browser to add release notes: open in github.dev

Change path	Release notes path	Description
`src/Compiler`	docs/release-notes/.FSharp.Compiler.Service/10.0.100.md	No release notes found or release notes format is not correct
`vsintegration/src`	docs/release-notes/.VisualStudio/18.0.md	No release notes found or release notes format is not correct

majocha · 2025-08-29T09:23:15Z

src/Compiler/Optimize/Optimizer.fs

        // Is it quadratic or quasi-quadratic?
-        if ValueIsUsedOrHasEffect cenv (fun () -> (freeInExpr (CollectLocalsWithStackGuard()) bodyR).FreeLocals) (bindR, bindingInfo) then
+        let collect expr = (freeInExpr (CollectLocalsWithStackGuard()) expr).FreeLocals
+        if ValueIsUsedOrHasEffect cenv (fun () -> (getFreeLocalsCache cenv).GetOrAdd(bodyR, collect)) (bindR, bindingInfo) then


This gives a modest 20% hit ratio in tests. May or may not help somewhat IRL but needs benchmarking.

Looking at Optimization times when building FCS I don't see any speedup.

majocha · 2025-08-29T09:25:59Z

src/Compiler/Utilities/lib.fs

+    /// Usage:
+    ///   let getValueFor = WeakMap.getOrCreate (fun () -> expensiveInit())
+    ///   let v = getValueFor someKey
+    let getOrCreate valueFactory =


Allows to quickly and and-hoc attach a cache instance in various places. Currently we thread various caches along with context objects / records which is not that convenient when testing things out.

auduchinok · 2025-08-29T13:05:54Z

src/Compiler/Checking/Expressions/CheckExpressions.fs

+let getArgInfoCache =
+    let options = Caches.CacheOptions.getDefault() |> Caches.CacheOptions.withNoEviction
+    let factory _ = new Caches.Cache<_, ArgReprInfo>(options, "argInfoCache")
+    WeakMap.getOrCreate factory


Are these caches going to be cleared when asking FCS to clear its caches?

Currently not, I think since they're weakly attached to higher lever stuff, they'll just get GC'd with it. I captured some telemetry and the caches get disposed quite often during the test run.

majocha · 2025-08-30T07:40:18Z

Ahh, there's that memory leak in vs tests, again 😔

T-Gro · 2025-09-04T12:27:08Z

src/Compiler/Checking/Expressions/CheckExpressions.fs

-        env.eCachedImplicitYieldExpressions.Add(synExpr1.Range, (synExpr1, expr1Ty, cachedExpr))
-        try TcExpr cenv overallTy env tpenv otherExpr
-        finally env.eCachedImplicitYieldExpressions.Remove synExpr1.Range
+        (getImplicitYieldExpressionsCache cenv).AddOrUpdate(synExpr1, (expr1Ty, cachedExpr))


The semantics w.r.t. to error handling is different now, aren't they?

Also it feels like the previous version had a race condition (FindAll and later Add), possibly the Multi map was chosen to workaround it by storing a list and not a single value?

Previous version had some concurrency problems, because Vlad at some point added ConcurrentDictionary as HashMultiMap backing store to fix them.

Semantics are different. Originally this just always removed the value in finally. I have to revisit this, I remember the change made sense to me but I forgot why, oh lol. I guess my thinking was eviction will be sufficient here. Now I also notice this was attached to env, not cenv.

This is not really tested in the test suite but there is a benchmark. I'll run it later to see if this still works.

This reverts commit 689112d.

majocha · 2025-09-19T07:55:01Z

I added printing cache metrics when --times option is set, to see what's going on during one off compilation.

T-Gro · 2025-09-19T10:38:03Z

With aditional type relations caches there seem to be a slight speedup against main:

DecentlySizedStandAloneFileBenchmark

cache-2

| Method | Mean     | Error    | StdDev   | Gen0      | Gen1      | Gen2      | Allocated |
|------- |---------:|---------:|---------:|----------:|----------:|----------:|----------:|
| Run    | 184.8 ms | 10.26 ms | 29.76 ms | 2000.0000 | 1000.0000 | 1000.0000 | 227.98 MB |

| Method | Mean     | Error    | StdDev   | Gen0      | Gen1      | Gen2      | Allocated |
|------- |---------:|---------:|---------:|----------:|----------:|----------:|----------:|
| Run    | 192.5 ms | 10.54 ms | 31.07 ms | 2000.0000 | 1000.0000 | 1000.0000 | 227.95 MB |

| Method | Mean     | Error    | StdDev   | Gen0      | Gen1      | Gen2      | Allocated |
|------- |---------:|---------:|---------:|----------:|----------:|----------:|----------:|
| Run    | 189.5 ms | 10.63 ms | 30.84 ms | 2000.0000 | 1000.0000 | 1000.0000 |    228 MB |

main

| Method | Mean     | Error   | StdDev   | Gen0      | Gen1      | Gen2      | Allocated |
|------- |---------:|--------:|---------:|----------:|----------:|----------:|----------:|
| Run    | 203.5 ms | 8.23 ms | 24.01 ms | 3000.0000 | 2000.0000 | 1000.0000 | 227.19 MB |

| Method | Mean     | Error   | StdDev   | Gen0      | Gen1      | Gen2      | Allocated |
|------- |---------:|--------:|---------:|----------:|----------:|----------:|----------:|
| Run    | 204.7 ms | 8.61 ms | 24.99 ms | 3000.0000 | 2000.0000 | 1000.0000 |    227 MB |

Seeing it's for just 1MB increase in memory, this is very nice tradeoff 👍 .

(I assume the cache size counts into the allocated bucket)

majocha · 2025-09-20T09:16:55Z

(I assume the cache size counts into the allocated bucket)

Yes, it allocates the few caches that are there for each run, but they are probably not filled up very much. I guess HistoricalBenchmark is not great to test cache improvements, it just compiles a single not very large file, and does not use Transparent Compiler.

But I did run it for some older commits from around April, to see if things are not regressing unnoticed:

detached at f487d45 

| Method | Mean     | Error    | StdDev   | Median   | Gen0      | Gen1      | Gen2      | Allocated |
|------- |---------:|---------:|---------:|---------:|----------:|----------:|----------:|----------:|
| Run    | 269.1 ms | 18.50 ms | 53.98 ms | 241.1 ms | 5000.0000 | 3000.0000 | 2000.0000 | 293.12 MB |

detached at c549ebf

| Method | Mean     | Error    | StdDev   | Median   | Gen0      | Gen1      | Gen2      | Allocated |
|------- |---------:|---------:|---------:|---------:|----------:|----------:|----------:|----------:|
| Run    | 254.0 ms | 14.47 ms | 37.87 ms | 242.6 ms | 5000.0000 | 3000.0000 | 2000.0000 | 292.85 MB |

Thankfully, it's all good. We are better now than we were, both in allocations and speed.

ETA: this is not a benchmark to assess performance reliably. We can have a serious degraded typecheck times in real build that will not show up here at all.

majocha · 2025-09-20T13:01:56Z

MemoizationTable uses ConcurrentDictionary backing, we can swap this with non-evicting Cache, to see stats of how the various memoization tables perform in the compiler.

For example compiling FSharp.Compiler.Service\Release\net10.0

    ----------------------------------------------------------------------------------------------------------------------------
    | Cache name                          | hit-ratio |     adds |  updates |     hits |   misses | evictions | eviction-fails |
    ----------------------------------------------------------------------------------------------------------------------------
    |                     methodInfoCache |    76.99% |     5624 |        0 |    18820 |     5624 |         0 |              0 |
    | mostSpecificOverrideMethodInfoCache |    73.24% |      365 |        0 |      999 |      365 |         0 |              0 |
    |                      eventInfoCache |     0.00% |        1 |        0 |        0 |        1 |         0 |              0 |
    |                   propertyInfoCache |    38.66% |     1764 |        0 |     1112 |     1764 |         0 |              0 |
    |            implicitYieldExpressions |     0.00% |        0 |        0 |        0 |   717864 |         0 |              0 |
    |             implicitConversionCache |    92.72% |     1368 |        0 |    17436 |     1368 |         0 |              0 |
    |                    ilFieldInfoCache |    75.19% |       32 |        0 |       97 |       32 |         0 |              0 |
    |            entireTypeHierarchyCache |    90.04% |     3428 |        0 |    30983 |     3428 |         0 |              0 |
    |                      v_memoize_file |    99.90% |      380 |        0 |   383211 |      380 |         0 |              0 |
    |           recdOrClassFieldInfoCache |    46.21% |      795 |        0 |      683 |      795 |         0 |              0 |
    |              typeFeasibleEquivCache |    26.66% |    53985 |        0 |    19625 |    53985 |         0 |              0 |
    |                typeSubsumptionCache |    50.88% |   252652 |        0 |   261756 |   252652 |         0 |              0 |
    | typeDefinitelySubsumesNoCoerceCache |    63.60% |    46590 |        0 |    81421 |    46590 |         0 |              0 |
    |                     namedItemsCache |    56.63% |    15838 |        0 |    20681 |    15838 |         0 |              0 |
    |           primaryTypeHierarchyCache |    50.00% |       14 |        0 |       14 |       14 |         0 |              0 |
    |           rawDataValueTypeGenerator |    96.40% |       39 |        0 |     1045 |       39 |         0 |              0 |
    |                        argInfoCache |    26.43% |    42013 |        0 |    15096 |    42013 |         0 |              0 |
    ----------------------------------------------------------------------------------------------------------------------------

majocha · 2025-09-20T20:37:16Z

I'll document here how I estimate the performance, because the existing benchmarks and not suitable.
The steps are:

make some changes
rebuild boostrap
build FCS net10.0 single threaded
search for |Typecheck in the binlog to view the result generated with --times switch:

dotnet publish .\proto.proj -c Proto

dotnet build  .\src\Compiler\ -c Release -f net10.0 -m:1 --no-incremental -bl

.\msbuild.binlog

T-Gro · 2025-09-25T14:05:30Z

For example compiling FSharp.Compiler.Service\Release\net10.0

I like how this brings in clarity into the various caches.
Even if some of them end up performing roughly the same wo/ an improvement, I would propose to keep them (with non-evicting config) simply for the added insights 👍

T-Gro · 2025-09-25T14:07:07Z

Also globalizing them could help tooling scenarios with multiproject solutions, especially if there are many shared dependencies 👍 .

majocha · 2025-09-25T14:14:41Z

Globalizing is something to try, currently the stats that show up are cumulative, simply by cache name.
There can be many instances, some caches are one per TcGlobals, some MemoizeTables are recreated per checked file IIRC. This PR is a testing ground of various things, but I could extract the MemoizationTable stuff. It should be very safe and negligible on performance, because no eviction cache is just a ConcurrentDictionary with some metrics.

github-project-automation bot added this to F# Compiler and Tooling Aug 29, 2025

github-project-automation bot moved this to New in F# Compiler and Tooling Aug 29, 2025

majocha commented Aug 29, 2025

View reviewed changes

auduchinok reviewed Aug 29, 2025

View reviewed changes

majocha force-pushed the cache-2 branch from 3c7d950 to 0bd3211 Compare August 29, 2025 17:47

majocha mentioned this pull request Aug 30, 2025

Type subsumption cache, fix keys to reduce memory use #18875

Merged

majocha added 2 commits September 3, 2025 15:48

optimizer

6b0f7d5

argInfo + implicit yield

208362a

majocha force-pushed the cache-2 branch from 476dcb6 to 208362a Compare September 3, 2025 13:50

majocha added 2 commits September 4, 2025 09:35

fix comment

4925224

Merge branch 'main' into cache-2

59bb141

T-Gro reviewed Sep 4, 2025

View reviewed changes

majocha added 12 commits September 4, 2025 17:30

Merge branch 'main' into cache-2

bd5a72f

see if this helps

8d32412

wip

689112d

Revert "wip"

13d4dd1

This reverts commit 689112d.

log min threads

7543b43

?

859fdac

force immediate eviction in some test runs

873a945

.

03a606d

Merge branch 'main' into cache-2

dedb3ee

.

e6fc866

format

4d69ae5

.

942640b

majocha force-pushed the cache-2 branch from 94879d3 to 942640b Compare September 5, 2025 15:40

majocha added 2 commits September 8, 2025 21:53

Merge branch 'main' into cache-2

d3fd87b

add a test

aac2a46

majocha added 3 commits September 15, 2025 23:22

Merge branch 'main' into cache-2

3d6c92c

Merge branch 'main' into cache-2

d3dbcd1

Merge branch 'main' into cache-2

14a7822

majocha force-pushed the cache-2 branch from 3033da3 to 14a7822 Compare September 17, 2025 13:46

majocha added 8 commits September 17, 2025 17:28

don't unload ad

f6c3319

server gc in netcore tests

62dc77f

collect stats better

930d871

show stats in vs output pane

af601bb

fix method missing at runtime - ns2.0 haha

66a5ba7

format

bd7c24d

Merge branch 'main' into cache-2

8f550ae

better formatting of stats

c308343

majocha force-pushed the cache-2 branch from 1b4f0e9 to c308343 Compare September 19, 2025 07:51

Merge branch 'main' into cache-2

db6ff66

majocha added 2 commits September 20, 2025 12:26

use immutable array

0e32a4b

MemoizationTable backed by Cache

8b4738e

majocha added 4 commits September 21, 2025 21:02

fix perf

2e96d76

format

314a6e4

Merge branch 'main' into cache-2

3b1124e

merge main

58c0e6b

majocha added 2 commits September 30, 2025 16:25

merge main

600f619

fix merge

e5ba962

Use new Cache api in various places #18872

Are you sure you want to change the base?

Use new Cache api in various places #18872

Uh oh!

Conversation

majocha commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❗ Release notes required

Uh oh!

majocha Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

majocha Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

majocha Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

auduchinok Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

majocha Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

majocha commented Aug 30, 2025

Uh oh!

T-Gro Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

majocha Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

majocha commented Sep 19, 2025

Uh oh!

T-Gro commented Sep 19, 2025

Uh oh!

majocha commented Sep 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

majocha commented Sep 20, 2025

Uh oh!

majocha commented Sep 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

T-Gro commented Sep 25, 2025

Uh oh!

T-Gro commented Sep 25, 2025

Uh oh!

majocha commented Sep 25, 2025

Uh oh!

Uh oh!

majocha commented Aug 29, 2025 •

edited

Loading

github-actions bot commented Aug 29, 2025 •

edited

Loading

majocha Aug 29, 2025 •

edited

Loading

majocha Sep 4, 2025 •

edited

Loading

majocha commented Sep 20, 2025 •

edited

Loading

majocha commented Sep 20, 2025 •

edited

Loading