Skip to content

[Unix] Go back to only checking the runtime resource path for swiftrt.o #1822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

finagolfin
Copy link
Member

This is made possible by the swift-frontend now setting this instead in swiftlang/swift#79621. More changes incoming...

@finagolfin
Copy link
Member Author

Ready for review along with the required linked frontend pull, which passed the linux CI when tested with this pull (this pull has no effect on Darwin and Windows). Comments should mostly go there, since this is just a consequence of that pull.

@finagolfin
Copy link
Member Author

@swift-ci please test

@finagolfin
Copy link
Member Author

Alright, this is ready to go, a partial revert of #1667, independent of the linked changes in swiftlang/swift#79621. This pull alone passed CI, along with a full Windows toolchain build. I'd like to get this into 6.1 also next.

Some background: the prior change checked the -sdk alone for swiftrt.o, which sometimes works if the -sdk has one and it isn't too different, but usually doesn't. Under the planned new cross-compilation model, all Swift runtime resources like libswiftCore.so will be looked for in -sdk/usr/lib/swift/<os>, but this reverted code only looks for the single file swiftrt.o, which means the runtime libraries are not looked for in the same -sdk directory! This mixing of Swift runtime resources from two different resource directories in the new model may sometimes work, but will often break.

As for the original cross-compilation model referenced in that doc, which is still the one commonly used, this change I'm reverting often breaks builds because -sdk refers to a C/C++ sysroot alone, like the Android NDK, which usually contains no Swift resources. Since the prior change is broken in both cross-compilation models, I'm reverting it and proposing swiftlang/swift#79621 instead to implement it properly for both the original and new model.

@artemcm, please review.

@compnerd, let us know what you think.

@finagolfin
Copy link
Member Author

Ping @artemcm, mind approving this? I'd like to get it into 6.1 next and I don't think Saleem will respond.

Ever since I argued against this change in the linked pull last year, I have occasionally raised the problems that this caused, eg swiftlang/swift#76381, and got no response. I think they were probably experimenting with new SDK layouts at the time, tried this hack, then moved on to other things.

Let's fix this now, otherwise it is going to start breaking 6.1 SDK bundles and toolchain builds on Unix, ie the two main scenarios where an -sdk is now explicitly specified.

@finagolfin
Copy link
Member Author

@etcwilde, mind reviewing this? I suspect I'm not going to get a response from those above, since they approved the original pull that broke this, and I'd like to get this fixed before it finally ships in 6.1.

Copy link
Member

@etcwilde etcwilde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

swiftrt.o is associated with the SDK content though so that the sections are set up correctly for the implementation of swift_addNewDSOImage coming from the stdlib.

During the hybrid phase, this means that we should be grabbing swiftrt from the same place that we found libswiftCore (either via the sdk or resource-dir), though for expedience I'm fine with not hooking that exact logic up today.

As a compromise, can we continue to search the SDK first, and if it's not found there, fall back on the resource-dir?

@finagolfin
Copy link
Member Author

"SDK" can mean different things under the different cross-compilation models, so I try to be more specific.

Most compilation does not set the -sdk flag explicitly, so this code is ignored. We primarily set it explicitly in two scenarios:

  1. cross-compilation
  2. Building the Swift compiler and stdlib itself from source

The problem with this previous Unix change for most cross-compilation SDK bundles is that they use the old model of -sdk having a C/C++ sysroot alone and a separate -resource-dir containing Swift modules/libraries. Despite the bundle pitch doc saying otherwise, both flags are required, because of a separate bug in this swift-driver. This code I'm changing then fails as the -sdk contains only C/C++ files and no swiftrt.o, see swiftlang/swift#76381 for a concrete example.

As for searching both, the problem is it might find a swiftrt.o in -sdk and still fail because it's the wrong version. I'm trying to fix all this more comprehensively for Swift 6.2 in swiftlang/swift#79621, but I'd like to revert this broken swift-driver code first for 6.1, as we probably won't want to rush swiftlang/swift#79621 into 6.1.

@finagolfin
Copy link
Member Author

Ping @etcwilde or @artemcm, I think this should be merged before the 6.1 release, as SDK bundles and building the Swift toolchain itself with Swift 6.1 is going to start breaking on non-Darwin Unix otherwise, as shown in swiftlang/swift#76381. I get these runtime resources from an external -sdk properly in the frontend pull swiftlang/swift#79621 instead, and intend to implement this more completely that way instead.

@finagolfin
Copy link
Member Author

finagolfin commented Mar 28, 2025

@shahmishal, could you take a look at this, as it's been a couple weeks and I haven't really been able to get a review here? I tried building the 6.1 compiler with a prebuilt 6.1 on linux to see if the change this is reverting broke anything and it didn't, as the compiler build is careful to use clang for linking Swift executables instead of swift-driver, but the moment that changes, this previous change that I'm reverting will break the compiler build. It has already broken cross-compiling with SDKs on my Android CI with the -sdk/-resource-dir flags.

I'm asking you since you're the review manager for linux and I'd like to get this into 6.1 next.

@etcwilde
Copy link
Member

@al45tair, I know that this will affect the static linux SDK, do you have feedback?
@compnerd, do you have feedback here? Does this break your SDKs?

As for searching both, the problem is it might find a swiftrt.o in -sdk and still fail because it's the wrong version.

This is true. But we are also moving away from the model where the SDK content is shoved in the compiler resource directory.

@compnerd, you were driving the push to split the system C sysroot flag from the SDK flag. What is missing in
swiftlang/swift#76381? At least at first glance, the compiler invocation is only using the -sdk flag and not the -sysroot flag. I would expect the situation where you're using the C runtimes from the system and a custom runtime to look something like this swiftc -sdk <path to Swift runtimes> -sysroot /.

@finagolfin
Copy link
Member Author

finagolfin commented Mar 28, 2025

But we are also moving away from the model where the SDK content is shoved in the compiler resource directory.

Essentially nothing has been done to make that move, other than this change looking for a single file alone, which means currently under your new model, the 6.1 compiler always looks in two different directories for Swift runtime resources like swiftrt.o or Swift.swiftmodule when a -sdk is specified, which is a recipe for compilation failures when they don't match versions.

At least at first glance, the compiler invocation is only using the -sdk flag and not the -sysroot flag. I would expect the situation where you're using the C runtimes from the system and a custom runtime to look something like this swiftc -sdk -sysroot /.

Yes, the invocation in swiftlang/swift#76381 is using the current -sdk/-resource-dir model, because the planned new -sdk/-sysroot model doesn't work yet. Particularly, a glaring hole in the new model with the current compiler implementation is that the frontend will not look in -sdk or -sysroot for the Swift stdlib modules! I detailed that for you here a couple weeks ago.

@finagolfin
Copy link
Member Author

Sigh, with all the delays in reviewing this, 6.1 has now shipped with this regression, so we'll have to fix it in a patch release.

@finagolfin
Copy link
Member Author

@swift-ci test

@finagolfin
Copy link
Member Author

@swift-ci test windows

@finagolfin
Copy link
Member Author

Forced another Windows CI run because the prior one was not registering with github, which means it would have been useless for the merge.

@finagolfin
Copy link
Member Author

@etcwilde and @compnerd, this previous change I'm reverting broke compiling Unix SDK bundles using external platform C/C++ SDKs with the -sdk flag, because such platform C/C++ SDKs do not contain a swiftrt.o. Of course, SDK bundle support for using external C/C++ SDKs is half-baked- see swiftlang/swift-package-manager#8584 and the second blocker from this comment by @marcprux- but Marc is in the process of fixing those, and this is the only other major issue he found.

Can we go ahead and get this in now?

@marcprux
Copy link

This is the final blocker for being able to have the Swift Android SDK use a completely external NDK sysroot without any gruesome hacks.

Is there hope for getting this PR in? Or, at least, are there any objections or concerns that we can assuage? There were no responses from @al45tair or @compnerd to @etcwilde's inquiry about whether this might affect the existing SDKs — perhaps they could take another look and offer their thoughts?

@finagolfin
Copy link
Member Author

@swift-ci test

@finagolfin
Copy link
Member Author

This issue is breaking tests on the community Android CI:

/home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/bin/swiftc -target aarch64-unknown-linux-android -sdk /home/swiftci/android/ndk/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot -Xclang-linker --target=aarch64-unknown-linux-android21 -tools-directory /home/swiftci/android/ndk/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/bin -resource-dir /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/lib/swift -module-cache-path /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/swift-test-results/aarch64-unknown-linux-android/clang-module-cache -swift-version 4 -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.1:macOS 15.4, iOS 18.4, watchOS 11.4, tvOS 18.4, visionOS 2.4' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 6.1:macOS 15.4, iOS 18.4, watchOS 11.4, tvOS 18.4, visionOS 2.4' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.2:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 6.2:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftStdlib 6.3:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -Xfrontend -define-availability -Xfrontend 'StdlibDeploymentTarget 6.3:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -Xfrontend -define-availability -Xfrontend 'SwiftCompatibilitySpan 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2, visionOS 1.1' -Xfrontend -define-availability -Xfrontend 'SwiftCompatibilitySpan 6.2:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/test-android-aarch64/Interop/Cxx/function/Output/default-arguments-multifile.swift.tmp/main.swift /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/test-android-aarch64/Interop/Cxx/function/Output/default-arguments-multifile.swift.tmp/b.swift /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/test-android-aarch64/Interop/Cxx/function/Output/default-arguments-multifile.swift.tmp/c.swift -cxx-interoperability-mode=upcoming-swift -I /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/swift/test/Interop/Cxx/function/Inputs -o /home/swiftci/jenkins/workspace/oss-swift-RA-linux-ubuntu-24.04-android-arm64/buildbot_linux/swift-linux-x86_64/test-android-aarch64/Interop/Cxx/function/Output/default-arguments-multifile.swift.tmp/artifacts/out
error: link command failed with exit code 1 (use -v to see invocation)
clang++: error: no such file or directory: '/home/swiftci/android/ndk/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/lib/swift/android/aarch64/swiftrt.o

@compnerd, if you think you need this change being reverted for the Android SDKs in your Windows toolchain, can you apply this patch and let us know what error you're seeing?

@etcwilde, there is a bug in this new swift-driver, where it does not detect Swift runtime libraries and this swiftrt.o object file in a -sdk path. The way most remedy this long-standing bug is to explicitly specify the -resource-dir as sdkPath/usr/lib/swift/, but it appears Saleem does not want to do that, whether because he wants to stop using that flag or they have some other new organizational layout in their TBC SDKs.

But by hacking this in explicitly here, he has broken the build of those who do specify both -sdk/-resource-dir flags when cross-compiling to Unix platforms, particularly when done for external C/C++ SDKs that contain no Swift files, eg the Android NDK.

It may be possible to reconcile all this with additional info, as I just asked from Saleem, but we've gotten silence on the matter since it was merged last summer in #1667, despite my repeatedly raising the issue since then.

If someone needs that change, they should be able to explain why. If they cannot or just aren't using these Android SDKs anymore, we should merge this and go back to the way things were.

Pinging @shahmishal too, as this is breaking the upcoming Android SDK bundle with an external NDK, and this pull has received no real review for the last four months.

@finagolfin
Copy link
Member Author

@swift-ci test windows

@finagolfin
Copy link
Member Author

Ping @etcwilde, can we go ahead and get this in? Given the lack of response from @compnerd for months now, I doubt this matters to him, and if this pull breaks their CI, I have a proper fix queued up to go in swiftlang/swift#79621.

To summarize, the way this was done breaks cross-compilation to Unix platforms where the C/C++ SDK and Swift resource directory are separate, which will be the case for all proprietary C/C++ SDKs like the Android NDK, because it requires that an explicit -sdk have a swiftrt.o, giving it precedence over an implicit or explicit -resource-dir. But that is not the way Swift compilation works everywhere else in the compiler, -resource-dir always takes precedence over -sdk, so this Unix-only change that was made last year for when -sdk is specified broke that required cross-compilation mode.

If applying this revert breaks some external CI- it works with all official CI- a version of swiftlang/swift#79621 is the correct way to fix that, while still maintaining the proper precedence.

@compnerd
Copy link
Member

This definitely will break our CI. Note that we package up the registrar only in the SDK by design.

@finagolfin
Copy link
Member Author

@compnerd, OK, can you apply this pull in a branch and let us know what error you see? My goal is to keep your CI working while not breaking other valid cross-compilation scenarios, as the current swift-driver does. If you have one of the TBC people work with us, I think we can get a version of swiftlang/swift#79621, perhaps scoped only to Unix, to accomodate both your and others' needs.

@marcprux
Copy link

marcprux commented Jul 24, 2025

@compnerd We need this to move forward; it is blocking progress on the Android SDK. This pull has been languishing for 5 months now. I believe everyone agrees that @finagolfin's observations and suggested solution is correct (@etcwilde, agree/disagree?).

If it truly breaks Windows CI, then isn't that something you can fix? If not, would you accept a stopgap measure where we guard this change in #if !os(Windows)?

@compnerd
Copy link
Member

I disagree. I'm fine with windows continuing to use the right path for the cross compilation. The registrar is meant to be in the SDK, not the resource directory. We should be moving the other way - moving out of the world where we go to the resource directory. Code that is shipped to the client should reside in the SDK.

@finagolfin
Copy link
Member Author

I'll see if I can get the linked draft frontend pull swiftlang/swift#79621 to resolve both your concerns and mine, will let you know later.

@Steelskin
Copy link

Steelskin commented Jul 24, 2025

I ran a local build with this PR and tried to compile a CMake project with the resulting toolchain. This is the error I got:

cmake -B out -G Ninja -S . -D CMAKE_SYSTEM_NAME=Android -D CMAKE_SYSTEM_PROCESSOR=aarch64 -D "CMAKE_ANDROID_NDK=C:/Users/steel/src/toolchain/b/android-ndk-r27c" -D CMAKE_ANDROID_ARCH_ABI="arm64-v8a" -D CMAKE_C_COMPILER=clang -D CMAKE_CXX_COMPILER=clang++ -D CMAKE_C_COMPILER_TARGET=aarch64-unknown-linux-android21 -D CMAKE_CXX_COMPILER_TARGET=aarch64-unknown-linux-android21 -D "SWIFT_ANDROID_NDK_PATH=C:/Users/steel/src/toolchain/b/android-ndk-r27c" -D CMAKE_Swift_COMPILER_TARGET=aarch64-unknown-linux-android21
-- The Swift compiler identification is Apple 6.2
-- Check for working Swift compiler: S:/Program Files/Swift/Toolchains/0.0.0+Asserts/usr/bin/swiftc.exe
-- Check for working Swift compiler: S:/Program Files/Swift/Toolchains/0.0.0+Asserts/usr/bin/swiftc.exe - broken
CMake Error at C:/Users/steel/AppData/Local/Programs/cmake-3.29.9-windows-x86_64/share/cmake-3.29/Modules/CMakeTestSwiftCompiler.cmake:40 (message):
  The Swift compiler

    "S:/Program Files/Swift/Toolchains/0.0.0+Asserts/usr/bin/swiftc.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: 'C:/Users/steel/src/swift-firebase/out/CMakeFiles/CMakeScratch/TryCompile-q7tpp3'

    Run Build Command(s): C:/ProgramData/chocolatey/bin/ninja.exe -v cmTC_22d0e
    [1/2] S:\PROGRA~1\Swift\TOOLCH~1\000611~1.0_A\usr\bin\swiftc.exe -target aarch64-unknown-linux-android21 -j 32 -num-threads 32 -c  -module-name cmTC_22d0e -O -g -incremental -output-file-map CMakeFiles\cmTC_22d0e.dir\RelWithDebInfo\output-file-map.json  C:\Users\steel\src\swift-firebase\out\CMakeFiles\CMakeScratch\TryCompile-q7tpp3\main.swift
    <unknown>:0: warning: libc not found for 'aarch64-unknown-linux-android21'; C stdlib may be unavailable
    [2/2] C:\WINDOWS\system32\cmd.exe /C "cd . && S:\PROGRA~1\Swift\TOOLCH~1\000611~1.0_A\usr\bin\swiftc.exe -target aarch64-unknown-linux-android21 -j 32 -num-threads 32 -emit-executable -o cmTC_22d0e -O -g CMakeFiles\cmTC_22d0e.dir\main.swift.o    && cd ."
    FAILED: [code=1] cmTC_22d0e
    C:\WINDOWS\system32\cmd.exe /C "cd . && S:\PROGRA~1\Swift\TOOLCH~1\000611~1.0_A\usr\bin\swiftc.exe -target aarch64-unknown-linux-android21 -j 32 -num-threads 32 -emit-executable -o cmTC_22d0e -O -g CMakeFiles\cmTC_22d0e.dir\main.swift.o    && cd ."
    error: link command failed with exit code 1 (use -v to see invocation)
    clang: error: no such file or directory: 'S:\PROGRA~1\Swift\TOOLCH~1\000611~1.0_A\usr\lib\swift\android\aarch64\swiftrt.o'
    ninja: build stopped: subcommand failed.





  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:4 (project)


-- Configuring incomplete, errors occurred!

Specifically, the issue is here:

    clang: error: no such file or directory: 'S:\PROGRA~1\Swift\TOOLCH~1\000611~1.0_A\usr\lib\swift\android\aarch64\swiftrt.o'

This file is not there in the installation, it is in Platforms/Android.platform/Developer/SDKs/Android.sdk/usr/lib/swift/android/aarch64.

@finagolfin
Copy link
Member Author

Thanks @Steelskin, would it be possible to apply this pull on your CI, that @compnerd says this would break, and let me know what error that shows? This error you've shown is a bit unrelated to this pull, as you'll notice that the Swift command that CMake ran did not pass a -sdk flag, whereas this pull and discussion is all about how to handle compilation when an explicit -sdk is specified.

Regardless of your example, let me emphasize again that this swift-driver has a long-standing bug where it does not look in an explicit -sdk for runtime objects like swiftrt.o. Saleem has remedied that with this code I'm reverting here, whereas I think the more layered frontend change in swiftlang/swift#79621 is better, which is just copying over Saleem's earlier legacy driver approach from swiftlang/swift#25990 to the frontend and then to this swift-driver.

So this discussion is really all about which of Saleem's two different earlier approaches we should now use! 😄 Let me work on that frontend pull this week and once it passes the official CI, we can check if it fixes your TBC issues also.

@compnerd
Copy link
Member

@finagolfin I don't have a strong objection to the frontend approach. However, I do wonder if we need the frontend to learn about this. The driver is the appropriate place to know the registrar is. This pattern also matches what clang does for the C runtime initializers.

@finagolfin
Copy link
Member Author

Great, 😄 let me implement the frontend approach fully, then we can compare. Regarding whether the frontend is the right place to put this registrar logic, this swift-driver already delegates to the frontend to calculate the correct runtime resource and other target compilation paths, centralizing that logic there: that is the main reason I'm putting this -sdk detection there too.

That has its own benefits, like the frontend easily being able to find platform runtime modules in an explicit -sdk if needed, but that layered approach has its own edge cases also. Let me see if I can address all of those in that pull, then we can come back to this one.

@Steelskin
Copy link

@finagolfin

Here is the result from our CI run with your PR. The toolchain build succeeds but the Android smoke tests fails:

  swift build `
    --package-path D:\r\_work\swift-build\swift-build/SourceCache/cassowary `
    --triple x86_64-unknown-linux-android28 `
    --sdk "C:\Users\runner\.setup-ndk\r27c\toolchains\llvm\prebuilt\windows-x86_64\sysroot" `
    -Xswiftc -sdk -Xswiftc "C:\Users\runner\AppData\Local\Programs\Swift\Platforms\0.0.0\Android.platform\Developer\SDKs\Android.sdk" `
    -Xswiftc -sysroot -Xswiftc "C:\Users\runner\.setup-ndk\r27c\toolchains\llvm\prebuilt\windows-x86_64\sysroot" `
    -Xswiftc -I -Xswiftc "C:\Users\runner\AppData\Local\Programs\Swift\Platforms\0.0.0\Android.platform\Developer\SDKs\Android.sdk\usr\include" `
    -Xswiftc -Xclang-linker -Xswiftc -resource-dir -Xswiftc -Xclang-linker -Xswiftc C:/Users/runner/.setup-ndk/r27c/toolchains/llvm/prebuilt/windows-x86_64/lib/clang/18

[...]

[71/73] Write Objects.LinkFileList
error: link command failed with exit code 1 (use -v to see invocation)

clang: error: no such file or directory: 'C:\Users\runner\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\lib\swift\android\x86_64\swiftrt.o'

@finagolfin
Copy link
Member Author

Thanks @Steelskin, since that failing path is not passed in to your SwiftPM invocation, I assume your Swift compiler is installed at C:\Users\runner\AppData\Local\Programs\Swift\Toolchains\0.0.0+Asserts\usr\bin\swiftc.exe? If so, I think the eventual frontend pull combined with this swift-driver pull will work on your CI.

Let me clean up the frontend pull and I will let you know when it is ready to try.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants