Work with stdbuf environment variables

[tbu-]

Check _STDBUF_O and _STDBUF_E environment variables for setting up stdout and stderr. This makes Rust programs compatible with coreutils's (and uutils's) stdbuf program. It incidentally also solves uutils/coreutils#7967.

This lays ground work for solving #60673 because it lets the user of a Rust program decide the default buffering, offering the opportunity to change the current default, letting stdout buffer blockwise if it's not connected to a terminal, because the user can now override the behavior using stdbuf if it's needed.

[rustbot]

r? @ChrisDenton

rustbot has assigned @ChrisDenton.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

⚠️ Warning ⚠️

• There are issue links (such as #123) in the commit messages of the following commits.
  Please move them to the PR description, to avoid spamming the issues with references to the commit, and so this bot can automatically canonicalize them to avoid issues with subtree.
  • baf0f1a

[tbu-]

Based on #148274 because it makes some of the changes I need here, as well.

[Mark-Simulacrum]

Nominating for libs-api to decide whether we want to support this configuration interface.

[workingjubilee]

Bugs
On GLIBC platforms, specifying a buffer size, i.e. using fully buffered mode will result in undefined operation.

uh?

...what actually happens, here?

[hanna-kruppe]

I don't think this set of environment variables should be the first interface that Rust exposes to control stdio buffering. It's an undocumented, platform-specific interface (though this PR apparently generalizes it to all platforms) and would be insta-stable. Furthermore, the way it's implemented in this PR probably means (haven't tried it) that programs can choose their own buffering strategy by using env::set_var before the first call to io::{stdout,stderr}. As much as I'd like to see a solution to #60673, introducing (only) this hacky and limited way to do it would be disappointing.

[the8472]

Also see #78515 (comment) for the last time the team discussed this.

[ChrisDenton]

I think a decision on the first steps should be made in an issue (or ACP) before the implementation PR. Tbh, I'd be minded to close this until such a decision is made but as it's been nominated for libs-api discussion. I'll defer to that.

[tbu-]

I don't think this set of environment variables should be the first interface that Rust exposes to control stdio buffering. It's an undocumented, platform-specific interface (though this PR apparently generalizes it to all platforms)

I think stdbuf is pretty pervasive, except maybe on Windows. I could exclude Windows from the set of platforms, but I'd guess someone ported stdbuf even there, maybe uutils?

would be insta-stable.

Doesn't sound like a big problem in this case, as this is not something documented right now, but just a quality of implementation thing, which can be reverted within the stability guarantees of Rust.

Furthermore, the way it's implemented in this PR probably means (haven't tried it) that programs can choose their own buffering strategy by using env::set_var before the first call to io::{stdout,stderr}.

This could be fixed if wanted, by looking at the environment variables earlier in the process, maybe before main.

As much as I'd like to see a solution to #60673, introducing (only) this hacky and limited way to do it would be disappointing.

I don't want this to be the only way to set it, but a first way. Adding APIs for Rust programs to control their own buffering would also be very nice, but is relatively independent from this. A PR adding such APIs is also made easier by merging this, because of enum StdioBufWriter which already implements the required semantics.

Also see #78515 (comment) for the last time the team discussed this.

Seems to be about Rust programs controlling their own buffering, not setting the buffering from outside the Rust program.

[hanna-kruppe]

After a quick experiment and some digging about the error I got, I now know more than I ever wanted to about how GNU coreutils stdbuf works:

$ cat hello.c
#include <stdio.h>

void main() {
    printf("hello\n");
}
$ musl-gcc hello.c -o hello
$ ./hello
hello
$ stdbuf -oL ./hello
Error relocating /usr/libexec/coreutils/libstdbuf.so: __isoc23_strtoumax: symbol not found
Error relocating /usr/libexec/coreutils/libstdbuf.so: __fprintf_chk: symbol not found

It turns out that the environment variables read by this PR are intended as internal implementation detail for a dynamic library that the stdbuf command injects via LD_PRELOAD or similar mechanisms. The dynamic library has a __attribute__((constructor)) function that overrides the stdio buffers with freshly allocated buffers based on the environment variable values. Thus, the whole thing falls apart for statically linked binaries, which is somewhat common for Rust projects that distribute pre-built Linux binaries.

[tbu-]

@hanna-kruppe Thanks for the idea and testing it. I can confirm it also doesn't work on my machine.

Hmm. If stdbuf fails hard for non-glibc binaries, then this probably won't work… May I close this PR or should I wait for libs-api discussion?

[Noratrieb]

libs-api discussion is not needed if even the PR author thinks it's a bad idea now :D

[ChrisDenton]

Closing for that reason.