Backports for julia 1.11.7 #59336
Open
Backports for julia 1.11.7 #59336
+104
−17
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…54840) Like #54671, but for `speccache_eq`. Saw another segfault with this in the stack trace, hence this fix. I also looked for other uses of `jl_smallintset_lookup` and there's one in `idset.c`. That doesn't appear to be racy but I'm not familiar with the code, so maybe you can take a look at it in case we need to push a fix for that one too @gbaraldi or @vtjnash? (cherry picked from commit dd1ed17)
This was in DAECompiler.jl code found by @serenity4. He also mentioned that writing up how one might go and fix a bug like this so i'll give a quick writeup (this was a very simple bug so it might not be too interesting) The original crash which looked something like > %19 = alloca [10 x i64], align 8 %155 = insertelement <4 x ptr> poison, ptr %19, i32 0 Unexpected instruction > [898844] signal 6 (-6): Aborted in expression starting at /home/gbaraldi/DAECompiler.jl/test/reflection.jl:28 pthread_kill at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) RecursivelyVisit<llvm::IntrinsicInst, LateLowerGCFrame::PlaceRootsAndUpdateCalls(llvm::ArrayRef<int>, int, State&, std::map<llvm::Value*, std::pair<int, int> >)::<lambda(llvm::AllocaInst*&)>::<lambda(llvm::Use&)> > at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:803 operator() at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:2560 [inlined] PlaceRootsAndUpdateCalls at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:2576 runOnFunction at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:2638 run at /home/gbaraldi/julia4/src/llvm-late-gc-lowering.cpp:2675 run at /home/gbaraldi/julia4/usr/include/llvm/IR/PassManagerInternal.h:91 which means it was crashing inside of late-gc-lowering, so the first thing I did was ran julia and the same test with LLVM_ASSERTIONS=1 and FORCE_ASSERTIONS=1 to see if LLVM complained about a malformed module, and both were fine. Next step was trying to get the failing code out for inspection. Easiest way is to do `export JULIA_LLVM_ARGS="--print-before=LateLowerGCFrame --print-module-scope"` and pipe the output to a file. The file is huge, but since it's a crash in LLVM we know that the last thing is what we want, and that gave me the IR I wanted. To verify that this is failing I did `make -C src install-analysis-deps` to install the LLVM machinery (opt...). That gets put in the `tools` directory of a julia build. Then I checked if this crashed outside of julia by doing `./opt -load-pass-plugin=../lib/libjulia-codegen.dylib --passes=LateLowerGCFrame -S test.ll -o tmp3.ll `. This is run from inside the tools dir so your paths might vary (the -S is so LLVM doesn't generate bitcode) and my code did crash, however it was over 500 lines of IR which makes it harder to debug and to write a test. Next step then is to minimize the crash by doing [`llvm-reduce`](https://llvm.org/docs/CommandGuide/llvm-reduce.html) over it (it's basically creduce but optimized for LLVM IR) which gave me a 2 line reproducer (in this case apparently just having the insertelement was enough for the pass to fail). One thing to be wary is that llvm-reduce will usually make very weird code, so it might be useful to modify the code slightly so it doesn't look odd (it will have unreachable basic-blocks and such). After the cleanup fixing the bug here wasn't interesting but this doesn't apply generally. And also always transform your reduced IR into a test to put in llvmpasses. (cherry picked from commit 906d348)
When this API was added, this function inlined, which is important, because the API relies on the allocation of the `Ref` being elided. At some point (I went back to 1.8) this regressed. For example, it is currently responsible for substantially all non-Expr allocations in JuliaParser. Before (parsing all of Base with JuliaParser): ``` │ Memory estimate: 76.93 MiB, allocs estimate: 719922. ``` After: ``` │ Memory estimate: 53.31 MiB, allocs estimate: 156. ``` Also add a test to make sure this doesn't regress again. (cherry picked from commit d6294ba)
Co-authored-by: Jameson Nash <[email protected]> (cherry picked from commit 3ed13ea)
Currently, `similar(::CodeUnits)` works as expected by going through the
generic `AbstractArray` method. However, the fallback method hit by
`similar(::Type{<:CodeUnits}, dims)` does not work, as it assumes the
existence of a constructor that accepts an `UndefInitializer`. This can
be made to work by defining a corresponding `similar` method that
returns an `Array`.
One could make a case that this is a bugfix since it was arguably a bug
that this method didn't work given that `CodeUnits` is an
`AbstractArray` subtype and the other `similar` methods work. If anybody
buys that argument, it could be nice to backport this; it came up in
some internal code that uses Arrow.jl and JSON3.jl together.
(cherry picked from commit 8e524c7)
would fix #57170, fix #54623 @nanosoldier `runbenchmarks("array", vs=":master")` (cherry picked from commit e853a4f)
(cherry picked from commit c0cc1e1)
(cherry picked from commit 2349f43)
(cherry picked from commit 107e1ac)
(cherry picked from commit fde79f5)
(cherry picked from commit 1f6eff1)
|
I have a Distributed.jl bugfix to backport, but I first need to do the backport in the Distributed repo, and then I'll bump Distributed here. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backported PRs:
using Foo: Foo, ...in package code (was: "using considered harmful") #42080Ptrvalues static-show w/ type-information #58584similarmethod forType{<:CodeUnits}#57826Need manual backport:
@time_imports@trace_compile@trace_dispatch#58011exp_impltopow#58062onefor theSizedArraytest helper #58209Coremethods from newly-inferred list #58510IteratorSizemethod forGenerator#58110hygienic-scopes in inner macro expansions #58965Contains multiple commits, manual intervention needed:
structto hard #56755--project=@scriptwhen outside script directory #56351jl_type_hashis concrete evaluated) #58401Non-merged PRs with backport label:
--trace-compile#58535transcode: prevent Windows sysimage invalidation #58038@nospecializeforstring_index_err#57604displaysizeto theIOContextused by the REPL #55499isfile_casesensitivefixes on Windows #55220@inboundsandBase.@propagate_inbounds#50157