Consider WF of coroutine witness when proving outlives assumptions #143545

compiler-errors · 2025-07-06T21:58:20Z

TL;DR

This PR introduces an unstable flag -Zhigher-ranked-assumptions which tests out a new algorithm for dealing with some of the higher-ranked outlives problems that come from auto trait bounds on coroutines. See:

Tracking issue for incorrect lifetime bound errors in async #110338

While it doesn't fix all of the issues, it certainly fixed many of them, so I'd like to get this landed so people can test the flag on their own code.

Background

Consider, for example:

use std::future::Future;

trait Client {
    type Connecting<'a>: Future + Send
    where
        Self: 'a;
    
    fn connect(&self) -> Self::Connecting<'_>;
}

fn call_connect<C>(c: C) -> impl Future + Send
where
    C: Client + Send + Sync,
{
    async move { c.connect().await }
}

Due to the fact that we erase the lifetimes in a coroutine, we can think of the interior type of the async block as something like: exists<'r, 's> { C, &'r C, C::Connecting<'s> }. The first field is the c we capture, the second is the auto-ref that we perform on the call to .connect(), and the third is the resulting future we're awaiting at the first and only await point. Note that every region is uniquified differently in the interior types.

For the async block to be Send, we must prove that both of the interior types are Send. First, we have an exists<'r, 's> binder, which needs to be instantiated universally since we treat the regions in this binder as unknown¹. This gives us two types: { &'!r C, C::Connecting<'!s> }. Proving &'!r C: Send is easy due to a Send impl for references.

Proving C::Connecting<'!s>: Send can only be done via the item bound, which then requires C: '!s to hold (due to the where Self: 'a on the associated type definition). Unfortunately, we don't know that C: '!s since we stripped away any relationship between the interior type and the param C. This leads to a bogus borrow checker error today!

Approach

Coroutine interiors are well-formed by virtue of them being borrow-checked, as long as their callers are invoking their parent functions in a well-formed way, then substitutions should also be well-formed. Therefore, in our example above, we should be able to deduce the assumption that C: '!s holds from the well-formedness of the interior type C::Connecting<'!s>.

This PR introduces the notion of coroutine assumptions, which are the outlives assumptions that we can assume hold due to the well-formedness of a coroutine's interior types. These are computed alongside the coroutine types in the CoroutineWitnessTypes struct. When we instantiate the binder when proving an auto trait for a coroutine, we instantiate the CoroutineWitnessTypes and stash these newly instantiated assumptions in the region storage in the InferCtxt. Later on in lexical region resolution or MIR borrowck, we use these registered assumptions to discharge any placeholder outlives obligations that we would otherwise not be able to prove.

How well does it work?

I've added a ton of tests of different reported situations that users have shared on issues like #110338, and an (anecdotally) large number of those examples end up working straight out of the box! Some limitations are described below.

How badly does it not work?

The behavior today is quite rudimentary, since we currently discharge the placeholder assumptions pretty early in region resolution. This manifests itself as some limitations on the code that we accept.

For example, tests/ui/async-await/higher-ranked-auto-trait-11.rs continues to fail. In that test, we must prove that a placeholder is equal to a universal for a param-env candidate to hold when proving an auto trait, e.g. '!1 = 'a is required to prove T: Trait<'!1> in a param-env that has T: Trait<'a>. Unfortunately, at that point in the MIR body, we only know that the placeholder is equal to some body-local existential NLL var '?2, which only gets equated to the universal 'a when being stored into the return local later on in MIR borrowck.

This could be fixed by integrating these assumptions into the type outlives machinery in a more first-class way, and delaying things to the end of MIR typeck when we know the full relationship between existential and universal NLL vars. Doing this integration today is quite difficult today.

tests/ui/async-await/higher-ranked-auto-trait-11.rs fails because we don't compute the full transitive outlives relations between placeholders. In that test, we have in our region assumptions that some '!1 = '!2 and '!2 = '!3, but we must prove '!1 = '!3.

This can be fixed by computing the set of coroutine outlives assumptions in a more transitive way, or as I mentioned above, integrating these assumptions into the type outlives machinery in a more first-class way, since it's already responsible for the transitive outlives assumptions of universals.

Moving forward

I'm still quite happy with this implementation, and I'd like to land it for testing. I may work on overhauling both the way we compute these coroutine assumptions and also how we deal with the assumptions during (lexical/nll) region checking. But for now, I'd like to give users a chance to try out this new -Zhigher-ranked-assumptions flag to uncover more shortcomings.

Instantiating this binder with infer regions would be incomplete, since we'd be asking for some instantiation of the interior types, not proving something for all instantiations of the interior types. ↩

rustbot · 2025-07-06T21:58:24Z

r? @spastorino

rustbot has assigned @spastorino.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2025-07-06T21:58:27Z

Some changes occurred to the core trait solver

cc @rust-lang/initiative-trait-system-refactor

compiler-errors · 2025-07-06T21:58:36Z

@bors2 try @rust-timer queue

rust-bors · 2025-07-06T21:58:39Z

⌛ Trying commit 9613c58 with merge c9e8f2f…

To cancel the try build, run the command @bors2 try cancel.

[experiment] Consider WF of coroutine witness when proving outlives assumptions This needs to be majorly cleaned up --- cc #110338 Consider, for example: ```rust use std::future::Future; trait Client { type Connecting<'a>: Future + Send where Self: 'a; fn connect(&'_ self) -> Self::Connecting<'_>; } fn call_connect<C>(c: &'_ C) -> impl Future + Send where C: Client + Send + Sync, { async move { c.connect().await } } ```

compiler/rustc_traits/src/coroutine_witnesses.rs

rust-bors · 2025-07-07T00:14:20Z

☀️ Try build successful (CI)
Build commit: c9e8f2f (c9e8f2fc63f9e854f5dc2b7f8dbffc5523b3138d, parent: de031bbcb161b0b7fc0eb16f77b02ce9fbdf4c9e)

rust-timer · 2025-07-07T09:28:28Z

Finished benchmarking commit (c9e8f2f): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.1%, 0.5%]	57
Regressions ❌ (secondary)	0.2%	[0.0%, 0.4%]	32
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.4%	[-1.0%, -0.0%]	8
All ❌✅ (primary)	0.2%	[0.1%, 0.5%]	57

Max RSS (memory usage)

Results (primary 1.1%, secondary -0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.1%	[0.7%, 1.7%]	3
Regressions ❌ (secondary)	2.6%	[1.1%, 3.4%]	4
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.0%	[-4.7%, -2.8%]	3
All ❌✅ (primary)	1.1%	[0.7%, 1.7%]	3

Cycles

Results (primary 2.2%, secondary 0.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.2%	[2.2%, 2.2%]	1
Regressions ❌ (secondary)	2.7%	[2.1%, 3.0%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-5.4%	[-5.6%, -5.3%]	2
All ❌✅ (primary)	2.2%	[2.2%, 2.2%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 460.528s -> 463.281s (0.60%)
Artifact size: 372.13 MiB -> 371.71 MiB (-0.11%)

bors · 2025-07-09T23:41:13Z

☔ The latest upstream changes (presumably #143538) made this pull request unmergeable. Please resolve the merge conflicts.

compiler-errors · 2025-07-10T00:33:19Z

@bors2 try

rust-bors · 2025-07-10T00:33:23Z

⌛ Trying commit a7ec91c with merge 3c75d58…

To cancel the try build, run the command @bors2 try cancel.

[experiment] Consider WF of coroutine witness when proving outlives assumptions This needs to be majorly cleaned up --- cc #110338 Consider, for example: ```rust use std::future::Future; trait Client { type Connecting<'a>: Future + Send where Self: 'a; fn connect(&'_ self) -> Self::Connecting<'_>; } fn call_connect<C>(c: &'_ C) -> impl Future + Send where C: Client + Send + Sync, { async move { c.connect().await } } ```

rust-bors · 2025-07-10T02:50:05Z

☀️ Try build successful (CI)
Build commit: 3c75d58 (3c75d5844aa65b4aa19243ac44ceda49fe4e61ca, parent: e43d139a82620a268d3828a73e12a8679339e8f8)

compiler-errors · 2025-07-10T02:52:33Z

Don't expect this to fix any code, but I want to see whether it causes new issues.

@craterbot check

craterbot · 2025-07-10T02:53:03Z

👌 Experiment pr-143545 created and queued.
🤖 Automatically detected try build 3c75d58
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

craterbot · 2025-07-10T02:53:26Z

🚧 Experiment pr-143545 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

craterbot · 2025-07-11T05:20:46Z

🎉 Experiment pr-143545 is completed!
📊 6 regressed and 5 fixed (662328 total)
📰 Open the summary report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

rustbot · 2025-07-13T21:02:20Z

There are changes to the tidy tool.

cc @jieyouxu

…interiors

compiler-errors · 2025-07-14T01:33:56Z

@bors2 try @rust-timer queue

[experiment] Consider WF of coroutine witness when proving outlives assumptions This needs to be majorly cleaned up --- cc #110338 Consider, for example: ```rust use std::future::Future; trait Client { type Connecting<'a>: Future + Send where Self: 'a; fn connect(&'_ self) -> Self::Connecting<'_>; } fn call_connect<C>(c: &'_ C) -> impl Future + Send where C: Client + Send + Sync, { async move { c.connect().await } } ```

rust-bors · 2025-07-14T01:34:00Z

⌛ Trying commit 6999485 with merge fd967a6…

To cancel the try build, run the command @bors2 try cancel.

rust-bors · 2025-07-14T03:52:34Z

☀️ Try build successful (CI)
Build commit: fd967a6 (fd967a649c6055db9ae4bcdab4313834d16bcb4b, parent: e9182f195b8505c87c4bd055b9f6e114ccda0981)

rust-timer · 2025-07-14T05:19:10Z

Finished benchmarking commit (fd967a6): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.2%	[0.1%, 0.5%]	47
Regressions ❌ (secondary)	0.2%	[0.0%, 0.5%]	30
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.1%	[-0.1%, -0.0%]	3
All ❌✅ (primary)	0.2%	[0.1%, 0.5%]	47

Max RSS (memory usage)

Results (primary 0.8%, secondary -2.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.8%	[0.5%, 1.0%]	3
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.9%	[-4.3%, -1.4%]	2
All ❌✅ (primary)	0.8%	[0.5%, 1.0%]	3

Cycles

Results (primary -2.5%, secondary 2.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.1%	[2.5%, 7.7%]	8
Improvements ✅ (primary)	-2.5%	[-2.5%, -2.5%]	1
Improvements ✅ (secondary)	-3.2%	[-3.4%, -2.9%]	2
All ❌✅ (primary)	-2.5%	[-2.5%, -2.5%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 464.056s -> 465.213s (0.25%)
Artifact size: 374.70 MiB -> 374.71 MiB (0.00%)

rustbot assigned spastorino Jul 6, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver) labels Jul 6, 2025

compiler-errors unassigned spastorino Jul 6, 2025

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 6, 2025

This comment has been minimized.

Sign in to view

compiler-errors commented Jul 6, 2025

View reviewed changes

compiler/rustc_traits/src/coroutine_witnesses.rs Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jul 7, 2025

compiler-errors force-pushed the coroutine-obl branch 2 times, most recently from 69dda47 to a7ec91c Compare July 10, 2025 00:33

craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 10, 2025

craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Jul 11, 2025

compiler-errors mentioned this pull request Jul 13, 2025

Region constraint nits #143901

Open

compiler-errors force-pushed the coroutine-obl branch from a7ec91c to b5d740d Compare July 13, 2025 21:02

rustbot added A-tidy Area: The tidy tool T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jul 13, 2025

compiler-errors added 5 commits July 13, 2025 23:21

Add outlives to CoroutineWitnessTypes

0de911a

Deduce outlives obligations from WF of coroutine interior types

f370519

Consider outlives assumptions when proving auto traits for coroutine …

35e11ea

…interiors

Gate things properly

e5c394b

Add tests, some of them fail...

6999485

compiler-errors force-pushed the coroutine-obl branch from b5d740d to 6999485 Compare July 13, 2025 23:27

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 14, 2025

compiler-errors changed the title ~~[experiment] Consider WF of coroutine witness when proving outlives assumptions~~ Consider WF of coroutine witness when proving outlives assumptions Jul 14, 2025

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jul 14, 2025

Consider WF of coroutine witness when proving outlives assumptions #143545

Are you sure you want to change the base?

Consider WF of coroutine witness when proving outlives assumptions #143545

Conversation

compiler-errors commented Jul 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TL;DR

Background

Approach

How well does it work?

How badly does it not work?

Moving forward

Footnotes

Uh oh!

rustbot commented Jul 6, 2025

Uh oh!

rustbot commented Jul 6, 2025

Uh oh!

compiler-errors commented Jul 6, 2025

Uh oh!

This comment has been minimized.

rust-bors bot commented Jul 6, 2025

Uh oh!

This comment has been minimized.

Uh oh!

rust-bors bot commented Jul 7, 2025

Uh oh!

This comment has been minimized.

rust-timer commented Jul 7, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

bors commented Jul 9, 2025

Uh oh!

compiler-errors commented Jul 10, 2025

Uh oh!

rust-bors bot commented Jul 10, 2025

Uh oh!

rust-bors bot commented Jul 10, 2025

Uh oh!

compiler-errors commented Jul 10, 2025

Uh oh!

craterbot commented Jul 10, 2025

Uh oh!

craterbot commented Jul 10, 2025

Uh oh!

craterbot commented Jul 11, 2025

Uh oh!

rustbot commented Jul 13, 2025

Uh oh!

compiler-errors commented Jul 14, 2025

Uh oh!

This comment has been minimized.

rust-bors bot commented Jul 14, 2025

Uh oh!

rust-bors bot commented Jul 14, 2025

Uh oh!

This comment has been minimized.

rust-timer commented Jul 14, 2025

Overall result: ❌✅ regressions and improvements - please read the text below

Uh oh!

Uh oh!

compiler-errors commented Jul 6, 2025 •

edited

Loading