Rust produces unusable binaries

[jullanggit]

Issue

When installing fd-find (smallest crate I could reliably reproduce this on), rustic-rs, broot, jj-cli, ripgrep, bacon or presenterm, trying to run the resulting binary results in an error. The specific error seems to alternate randomly betweeen 3 different ones.

This only happens with both the mold linker and a global target dir set in .cargo/config.toml (entire contents):

[build]
rustflags = [
  "-Clinker=clang",
  "-Clink-arg=-fuse-ld=/usr/bin/mold",
]
target-dir = "/home/user/.cache/cargo/target/"

It also doesn't occur on all binaries, for example atem-dots and uwuify work fine.
All the below errors occur on both the stable and nightly compiler

Zeroed header

$ cargo install <crate>
$ <binary>
bash: <binary>: cannot execute binary file: Exec format error

Here one, but maybe not the entire, issue is, that the headers of the binaries are for some reason completely zeroed:

$ hexdump -n 16 -C <binary>
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00000010

Zeroed verneed

$ cargo install <crate>
$ <binary>
<binary>: error while loading shared libraries: <binary>: unsupported version 0 of Verneed record

Here, the binaries do contain a valid ELF header:

$ hexdump -n 16 -C <binary>
00000000  7f 45 4c 46 02 01 01 00  00 00 00 00 00 00 00 00  |.ELF............|
00000010

Instead, the version‑need section is completely zeroed:

$ readelf -x .gnu.version_r ./fd


Hex dump of section '.gnu.version_r':
  0x00001c18 00000000 00000000 00000000 00000000 ................
  0x00001c28 00000000 00000000 00000000 00000000 ................
  0x00001c38 00000000 00000000 00000000 00000000 ................
  0x00001c48 00000000 00000000 00000000 00000000 ................
  0x00001c58 00000000 00000000 00000000 00000000 ................
  0x00001c68 00000000 00000000 00000000 00000000 ................
  0x00001c78 00000000 00000000 00000000 00000000 ................
  0x00001c88 00000000 00000000 00000000 00000000 ................
  0x00001c98 00000000 00000000 00000000 00000000 ................
  0x00001ca8 00000000 00000000 00000000 00000000 ................
  0x00001cb8 00000000 00000000 00000000 00000000 ................
  0x00001cc8 00000000 00000000 00000000 00000000 ................
  0x00001cd8 00000000 00000000 00000000 00000000 ................
  0x00001ce8 00000000 00000000 00000000 00000000 ................
  0x00001cf8 00000000 00000000 00000000 00000000 ................
  0x00001d08 00000000 00000000 00000000 00000000 ................
  0x00001d18 00000000 00000000 00000000 00000000 ................
  0x00001d28 00000000 00000000 00000000 00000000 ................
  0x00001d38 00000000 00000000 00000000 00000000 ................
  0x00001d48 00000000 00000000 00000000 00000000 ................
  0x00001d58 00000000 00000000 00000000 00000000 ................
  0x00001d68 00000000 00000000 00000000 00000000 ................
  0x00001d78 00000000 00000000 00000000 00000000 ................
  0x00001d88 00000000 00000000 00000000 00000000 ................

SIGSEGV

This one is the hardest to reproduce and only happened once during my testing and I sadly wasn't able to capture a lot of details, but the binary instantly crashed with a SIGSEGV

Meta

rustc +nightly --version --verbose:

rustc 1.90.0-nightly (da58c0513 2025-07-03)
binary: rustc
commit-hash: da58c051315268a197ce280f6ba07bbd03c66535
commit-date: 2025-07-03
host: x86_64-unknown-linux-gnu
release: 1.90.0-nightly
LLVM version: 20.1.7

rustc +stable --version --verbose:

rustc 1.88.0 (6b00bc388 2025-06-23)
binary: rustc
commit-hash: 6b00bc3880198600130e1cf62b8f8a93494488cc
commit-date: 2025-06-23
host: x86_64-unknown-linux-gnu
release: 1.88.0
LLVM version: 20.1.5

mold --version --verbose

mold 2.40.1 (96c798d7be595d833f3f7d33ca32fd43f2731b89; compatible with GNU ld)

I feel like I still don't really have a grasp on the real issue, if there is something, please let me know what I can try to narrow it down.

[bjorn3]

Which filesystem are you using? Does this reproduce when you try to link a non-rust project with mold?

[jullanggit]

Thank you for the questions, I am using bcachefs. Linking htop with mold works fine. So does linking most rust projects I work on privately. The errors also don't show up if I disable either linking with mold or the global target dir. I sadly(?) can't reproduce the errors on my laptop with identical cargo config file and rust & mold versions. I use btrfs on my laptop, although if the filesystem is at fault here it would be a very weird interaction as it works without the global target dir.

[bjorn3]

I know mold does a trick where once it is done linking it forks a child process and then once the parent has exited the child process exits. This to prevent blocking the exit of the parent process (and thus the rest of the build process) on unmapping all mmapped files. I wonder if bcachefs has trouble with that.

[jullanggit]

That could actually very well be the cause, when setting the target dir to /tmp/target (a tmpfs), the issue does not occur. Interestingly, it also doesn't happen if I set it to a directory on the same subvolume as rustc/cargo/mold (/home/user/target), so I suspect it has something to do with .cache being a separate subvolume (I also tried making ~/target a subvolume and the error occurs, so I think this actually is the issue). Thank you for your help!

[the8472]

Interestingly, it also doesn't happen if I set it to a directory on the same subvolume as rustc/cargo/mold (/home/user/target)

copy_file_range or similar might be involved if they're on the same filesystem. mixing that and mmap writes, perhaps even to unallocated extents, sure would be exciting exercise of fs api surface. But semantically the page cache is coherent, so once writes have finished the binaries should be valid.

[workingjubilee]

Hello, @koverstreet, rustc paging. We've encountered a very weird bug and we aren't sure what caused it. We're pretty sure it's not ours. Would you be willing to lend your expertise to help determine whether the root cause was

• the Linux kernel
• bcachefs
• mold
• something else?

[koverstreet]

Yeah, what do you need?

…

On Fri, Jul 4, 2025, 5:18 PM Jubilee ***@***.***> wrote: *workingjubilee* left a comment (rust-lang/rust#143428) <#143428 (comment)> Hello, @koverstreet <https://github.com/koverstreet>, rustc paging. We've encountered a very weird bug and we aren't sure what caused it. We're pretty sure it's not ours. Would you be willing to lend your expertise to help determine whether the root cause was - the Linux kernel - bcachefs - mold - something else? — Reply to this email directly, view it on GitHub <#143428 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPGX3U7D6HRVEAFADX23M33G34VFAVCNFSM6AAAAACAZKG6CWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMZXGMYTMOJSGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[workingjubilee]

I'm wondering if bjorn3's hypothesis is correct? #143428 (comment)

[koverstreet]

Not reproducing so far with rustc 1.87 + mold 2.40.1.

Can you check if that rustc reproduces it? And what kernel version are you on?

[koverstreet]

Can any of you get me more info on how to reproduce? Or check if it's the rustc version?

…

On Fri, Jul 4, 2025, 5:28 PM Jubilee ***@***.***> wrote: *workingjubilee* left a comment (rust-lang/rust#143428) <#143428 (comment)> I'm wondering if bjorn3's hypothesis is correct? #143428 (comment) <#143428 (comment)> — Reply to this email directly, view it on GitHub <#143428 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAPGX3UEUBZ7N2MPMBSEX5L3G35YBAVCNFSM6AAAAACAZKG6CWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMZXGMZDKMZVGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>