Fused Broadcasting kills constant folding?

[oxinabox]

Consider two implementations of the same function

julia> function foo2(T, c, f)
       d = T.(c)
       return sum(f .* d)
       end
foo2 (generic function with 2 methods)

julia> bar2(T, c, f) = sum(f .* T.(c))
bar2 (generic function with 3 methods)

bar2 seems like it is just better,
it avoids creating the intermidairy variable, so allows the broadcase to fuse,
and indeed with a plain vector it is:

julia> @btime foo2(Float32, [0.1,0.2], [1.0,2.0]);
  473.802 ns (9 allocations: 512 bytes)

julia> @btime bar2(Float32, [0.1,0.2], [1.0,2.0]);
  369.920 ns (8 allocations: 416 bytes)

But with a StaticArray foo2 is much better.

julia> @btime foo2(Float32, @SVector[0.1,0.2], @SVector[1.0,2.0]);
  0.031 ns (0 allocations: 0 bytes)

julia> @btime bar2(Float32, @SVector[0.1,0.2], @SVector[1.0,2.0]);
  946.536 ns (22 allocations: 528 bytes)

I think what is happening is that something about foo2 is constant fold-able for StaticArrays.
and that bar2 isn't.

Those number were taken on 1.5., but i saw similar on master

[mbauman]

I believe this all stems from the fact that we use the mutable Ref{T} to make types broadcastable — combined (perhaps, depending on how minimal your real example is) with the fact that you don't force specialization on your T within your own function. We really should come back to #18379 and use a non-mutable wrapper here internally.

[Seelengrab]

For reference, there's also #35675 where some bikeshedding on the name of an immutable variant of Ref took place.

From reading those, it seems like a lot of people agree that this would be useful, but no name has been decided on and that's why it's stalled?

[JeffreySarnoff]

As @MasonProtter mentioned on Slack, instead of Ref(x) for some scalar x in a broadcast expression (x,) works better:

Ref is a mutable container and can block constant propagation. In many cases, this isn’t a big deal, but I think an immutable transparent container like a single argument Tuple is better.

With that information, I suggested using

macro nodot(x)
    :(($x,))
end

[MasonProtter]

Unfortunately, that would be inappropriate for the broadcast internals themselves because single element tuples a 1 dimensional containers instead of zero dimensional like Ref.

There’s some discussion of alternatives here #35778

[adienes]

note that the original issue seems to have been resolved now

julia> @btime foo2(Float32, @SVector[0.1,0.2], @SVector[1.0,2.0]);
  1.416 ns (0 allocations: 0 bytes)

julia> @btime bar2(Float32, @SVector[0.1,0.2], @SVector[1.0,2.0]);
  1.458 ns (0 allocations: 0 bytes)

although while testing this, I think there is a regression for both implementations in the non-static case. I'll open that as a separate issue