3.15.0a1 rewrites sysconfig-saved flags on `make install`

[hroncok]

Bug report

Bug description:

In Fedora, we have built the Python 3.15.0a1 release as we normally do. It results in broken sysconfig flags that prevent building 3rd-party extension modules.

There are some RPM-specific CFLAGS and LDFLAGS that we use to build Python, but we don't want them exposed to our users via sysconfig because it would break their build. One of such flags is -specs=/usr/lib/rpm/redhat/redhat-package-notes, but there are others. When such flags are used outside of RPM build, it fails:

$ git clone https://github.com/fedora-python/crypt_r.git && cd crypt_r
$ tox -e py315
...
      building '_crypt_r' extension
      creating build/temp.linux-x86_64-cpython-315/src
      gcc -Werror=incompatible-pointer-types -fPIC -I/home/churchyard/.cache/uv/builds-v0/.tmpPUr6Gz/include -I/usr/include/python3.15 -c src/_crypt_r.c -o build/temp.linux-x86_64-cpython-315/src/_crypt_r.o
      gcc -shared -Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
      -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -Werror=incompatible-pointer-types build/temp.linux-x86_64-cpython-315/src/_crypt_r.o -L/usr/lib64 -lcrypt -o
      build/lib.linux-x86_64-cpython-315/_crypt_r.cpython-315-x86_64-linux-gnu.so
...
      gcc: fatal error: environment variable ‘RPM_ARCH’ not defined
      compilation terminated.
      error: command '/usr/bin/gcc' failed with exit code 1

That's why we set those flags in the CFLAGS_NODIST, LDFLAGS_NODIST only. However:

$ python3.15
...
>>> for k, v in sysconfig.get_config_vars().items():
...     if '-specs=/usr/lib/rpm/redhat/redhat-package-notes' in str(v):
...         print(k, v)
...         
BLDSHARED gcc -shared -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes
CONFIGURE_LDFLAGS_NODIST -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g  -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -g
LDCXXSHARED g++ -shared -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes
LDFLAGS -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes
LDSHARED gcc -shared -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes
PY_CORE_LDFLAGS -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g  -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -g
PY_LDFLAGS -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes
PY_LDFLAGS_NODIST -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g  -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -g
PY_LDFLAGS_NOLTO -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes -fno-lto

Compare with:

$ python3.14
...
>>> for k, v in sysconfig.get_config_vars().items():
...     if '-specs=/usr/lib/rpm/redhat/redhat-package-notes' in str(v):
...         print(k, v)
...         
CONFIGURE_LDFLAGS_NODIST -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g  -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -g
PY_CORE_LDFLAGS -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g  -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -g -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g
PY_LDFLAGS_NODIST -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g  -fno-semantic-interposition -flto -fuse-linker-plugin -ffat-lto-objects -g -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g
PY_LDFLAGS_NOLTO -fno-lto -Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g

I was not yet able to reproduce this behavior outside of our RPM build of Python, which makes bisecting this harder. We set the flags like this:

+ CFLAGS='-fcf-protection -fexceptions '
+ export 'CFLAGS_NODIST= -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wno-complain-wrong-lang -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -D_GNU_SOURCE -fPIC -fwrapv'
+ CFLAGS_NODIST=' -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wno-complain-wrong-lang -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -D_GNU_SOURCE -fPIC -fwrapv'
+ export 'CXXFLAGS=-fcf-protection -fexceptions '
+ CXXFLAGS='-fcf-protection -fexceptions '
++ pkg-config --cflags-only-I libffi
+ export CPPFLAGS=
+ CPPFLAGS=
+ export 'OPT=-fcf-protection -fexceptions '
+ OPT='-fcf-protection -fexceptions '
++ pkg-config --cflags openssl
+ export 'CFLAGS=-fcf-protection -fexceptions  '
+ CFLAGS='-fcf-protection -fexceptions  '
++ pkg-config --libs-only-L openssl
+ export 'LDFLAGS= '
+ LDFLAGS=' '
++ pkg-config --libs-only-L openssl
+ export 'LDFLAGS_NODIST=-Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g '
+ LDFLAGS_NODIST='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -specs=/usr/lib/rpm/redhat/redhat-package-notes  -g '
...
+ /builddir/build/BUILD/python3.15-3.15.0_a1-build/Python-3.15.0a1/configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --runstatedir=/run --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-platlibdir=lib64 --enable-ipv6 --enable-shared --with-computed-gotos=yes --with-dbmliborder=gdbm:ndbm:bdb --with-system-expat --with-system-libmpdec --enable-loadable-sqlite-extensions --with-dtrace --with-lto --with-ssl-default-suites=openssl --without-static-libpython --with-wheel-pkg-dir=/usr/share/python-wheels --with-valgrind --without-ensurepip --enable-experimental-jit=yes-off --enable-optimizations
...
+ /usr/bin/make -O -j32 V=1 VERBOSE=1 EXTRA_CFLAGS=-O3 'CFLAGS_NODIST= -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wno-complain-wrong-lang -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -D_GNU_SOURCE -fPIC -fwrapv -O3' regen-all PYTHON_FOR_REGEN=python3.15
...
+ /usr/bin/make -O -j32 V=1 VERBOSE=1 EXTRA_CFLAGS=-O3 'CFLAGS_NODIST= -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wno-complain-wrong-lang -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -mtls-dialect=gnu2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -D_GNU_SOURCE -fPIC -fwrapv -O3'

Now I'm trying to see if reverting 9e3542a makes a difference.

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Linked PRs

• GH-140454: Remove pyconfig.h from Makefile JIT_DEPS #140491
• gh-140454: Normalize the JIT stencils filename on Linux to avoid mismatches between the Makefile and the generator #140823

[hroncok]

Now I'm trying to see if reverting 9e3542a makes a difference.

No difference.

[hroncok]

The installed /usr/lib64/python3.15/_sysconfigdata__linux_x86_64-linux-gnu.py is already busted.

[hroncok]

The build/lib.linux-x86_64-3.15/_sysconfigdata__linux_x86_64-linux-gnu.py is created as expected (e.g. empty PY_LDFLAGS) and later during make install or shortly after that, the files is updated (e.g. PY_LDFLAGS is populated) :/

[hroncok]

The difefrence between 3.14 and 3.15 is that Written build/lib.linux-x86_64-3.15/_sysconfigdata__linux_x86_64-linux-gnu.py happens in make install as well. And here we did not bother setting the flags. I still am unable to reproduce this locally somehow.

[hroncok]

Indeed, when exporting the same set of flags for make install, the leak does not happen. However, I was not yet able to determine what exactly triggers the re-generation of _sysconfigdata__linux_x86_64-linux-gnu.py during make install.

[hroncok]

OK, I isolated the problem to enabling the JIT.

$ export LDFLAGS=' '
$ export LDFLAGS_NODIST='-Wl,--as-needed -g '
$ ./configure --config-cache --without-ensurepip --enable-experimental-jit=yes-off
$ make -j12  # or your CPU count
$ cp build/lib.linux-x86_64-3.15/_sysconfigdata__linux_x86_64-linux-gnu.py make._sysconfigdata__linux_x86_64-linux-gnu.py

Then, in a fresh shell:

# set a  slightly different set of flags, e.g.
$ export LDFLAGS='-Wl,--as-needed -g '
$ make install DESTDIR=/tmp/python
python3.15 ./Tools/jit/build.py x86_64-pc-linux-gnu --output-dir . --pyconfig-dir . --cflags="" --llvm-version=""

===============================================================
JIT support for x86_64-unknown-linux-gnu is still experimental!
            Please report any issues you encounter.            
===============================================================

gcc ...

I did not expect make install to recompile the JIT. It did not do that on 3.14 -- I'll try bisecting that.

$ diff -u make._sysconfigdata__linux_x86_64-linux-gnu.py build/lib.linux-x86_64-3.15/_sysconfigdata__linux_x86_64-linux-gnu.py

--- make._sysconfigdata__linux_x86_64-linux-gnu.py	2025-10-22 23:47:47.613945167 +0200
+++ build/lib.linux-x86_64-3.15/_sysconfigdata__linux_x86_64-linux-gnu.py	2025-10-22 23:48:15.259119800 +0200
@@ -19,7 +19,7 @@
     'BINDIR': '/usr/local/bin',
     'BINLIBDEST': '/usr/local/lib/python3.15',
     'BLDLIBRARY': 'libpython3.15.a',
-    'BLDSHARED': 'gcc -shared',
+    'BLDSHARED': 'gcc -shared -Wl,--as-needed -g',
     'BOOTSTRAP_HEADERS': '\\',
     'BUILDEXE': '',
     'BUILDPYTHON': 'python',
@@ -642,11 +642,12 @@
     'IO_OBJS': '\\',
     'IPHONEOS_DEPLOYMENT_TARGET': '',
     'JIT_DEPS': '\\',
-    'LDCXXSHARED': 'g++ -shared',
-    'LDFLAGS': '',
+    'LDCXXSHARED': 'g++ -shared -Wl,--as-needed -g',
+    'LDFLAGS': '-Wl,--as-needed -g',
+    'LDFLAGS_NODIST': '',
     'LDLIBRARY': 'libpython3.15.a',
     'LDLIBRARYDIR': '',
-    'LDSHARED': 'gcc -shared',
+    'LDSHARED': 'gcc -shared -Wl,--as-needed -g',
     'LDVERSION': '3.15',
     'LIBC': '',
     'LIBDEST': '/usr/local/lib/python3.15',
@@ -982,8 +983,8 @@
     'PY_CPPFLAGS': '-I. -I./Include',
     'PY_ENABLE_SHARED': 0,
     'PY_HAVE_PERF_TRAMPOLINE': 1,
-    'PY_LDFLAGS': '',
-    'PY_LDFLAGS_NODIST': '-Wl,--as-needed -g -Wl,--as-needed -g',
+    'PY_LDFLAGS': '-Wl,--as-needed -g',
+    'PY_LDFLAGS_NODIST': '-Wl,--as-needed -g',
     'PY_LDFLAGS_NOLTO': '-Wl,--as-needed -g',
     'PY_SQLITE_ENABLE_LOAD_EXTENSION': 0,
     'PY_SQLITE_HAVE_SERIALIZE': 0,

[hroncok]

git bisect says 85c1ef6

The nature of that change makes me believe that's very much the likely cause.

cc @savannahostrowski

[hroncok]

Note that in Fedora, we have solved this by ensuring the same CFLAGS/LDFLAGS environment variables are set during make and make install. Yet I believe this might be considered problematic anyway.

[savannahostrowski]

Okay, so after much spelunking, I think I've found the issue...

In #137265, I introduced a dual-target rule to prevent stale platform-specific stencil files. This exposed an issue with including pyconfig.h in JIT_DEPS. With the dual-target rule, Make checks whether both stencil files are newer than all dependencies in JIT_DEPS:

JIT_DEPS = \
    $(srcdir)/Tools/jit/*.c \
    $(srcdir)/Tools/jit/*.py \
    $(srcdir)/Python/executor_cases.c.h \
    pyconfig.h

jit_stencils.h @JIT_STENCILS_H@: $(JIT_DEPS)
	@REGEN_JIT_COMMAND@

If pyconfig.h has a newer timestamp than either stencil file during make install, Make rebuilds the JIT stencils. This triggers rebuilds that ultimately cause _sysconfigdata to regenerate. When _sysconfigdata regenerates during install, it picks up build-only flags like LDFLAGS_NODIST that shouldn't be in the installed configuration.

I think we can remove pyconfig.h from JIT_DEPS. We don't need it there since Python/jit.c already includes Python.h, which pulls in pyconfig.h.