Skip to content

Add SmallTag type for more compact Dual types #748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 73 additions & 17 deletions src/config.jl
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,50 @@ end

Tag(::Nothing, ::Type{V}) where {V} = nothing


@inline function ≺(::Type{Tag{F1,V1}}, ::Type{Tag{F2,V2}}) where {F1,V1,F2,V2}
tagcount(Tag{F1,V1}) < tagcount(Tag{F2,V2})
end

"""
HashTag{Hash}

HashTag is similar to a Tag, but carries just a small UInt64 hash,
instead of the full type, which makes stacktraces / types easier to
read while still providing good resilience to perturbation confusion.
"""
struct HashTag{H}
end

@generated function tagcount(::Type{HashTag{H}}) where {H}
:($(Threads.atomic_add!(TAGCOUNT, UInt(1))))
end

function HashTag(f::F, ::Type{V}) where {F,V}
H = if F <: Tuple
# no easy way to check Jacobian tag used with Hessians as multiple functions may be used
# see checktag(::Type{Tag{FT,VT}}, f::F, x::AbstractArray{V}) where {FT<:Tuple,VT,F,V}
nothing
else
hash(F) ⊻ hash(V)
end
tagcount(HashTag{H}) # trigger generated function
HashTag{H}()
end

HashTag(::Nothing, ::Type{V}) where {V} = nothing

@inline function ≺(::Type{HashTag{H1}}, ::Type{Tag{F2,V2}}) where {H1,F2,V2}
tagcount(HashTag{H1}) < tagcount(Tag{F2,V2})
end

@inline function ≺(::Type{Tag{F1,V1}}, ::Type{HashTag{H2}}) where {F1,V1,H2}
tagcount(Tag{F1,V1}) < tagcount(HashTag{H2})
end

@inline function ≺(::Type{HashTag{H1}}, ::Type{HashTag{H2}}) where {H1,H2}
tagcount(HashTag{H1}) < tagcount(HashTag{H2})
end

struct InvalidTagException{E,O} <: Exception
end

Expand All @@ -36,13 +75,22 @@ checktag(::Type{Tag{FT,VT}}, f::F, x::AbstractArray{V}) where {FT,VT,F,V} =

checktag(::Type{Tag{F,V}}, f::F, x::AbstractArray{V}) where {F,V} = true

# HashTag is a smaller tag, that only confirms the hash
function checktag(::Type{HashTag{HT}}, f::F, x::AbstractArray{V}) where {HT,F,V}
H = hash(F) ⊻ hash(V)
if HT == H || HT === nothing
true
else
throw(InvalidTagException{HashTag{H},HashTag{HT}}())
end
end

# no easy way to check Jacobian tag used with Hessians as multiple functions may be used
checktag(::Type{Tag{FT,VT}}, f::F, x::AbstractArray{V}) where {FT<:Tuple,VT,F,V} = true

# custom tag: you're on your own.
checktag(z, f, x) = true


##################
# AbstractConfig #
##################
Expand All @@ -55,6 +103,14 @@ Base.eltype(cfg::AbstractConfig) = eltype(typeof(cfg))

@inline (chunksize(::AbstractConfig{N})::Int) where {N} = N

function maketag(tagstyle::Union{Symbol,Nothing}, f, X)
if HASHTAG_MODE_ENABLED
return HashTag(f, X)
else
return Tag(f, X)
end
end

####################
# DerivativeConfig #
####################
Expand Down Expand Up @@ -108,9 +164,9 @@ vector `x`.
The returned `GradientConfig` instance contains all the work buffers required by
`ForwardDiff.gradient` and `ForwardDiff.gradient!`.

If `f` is `nothing` instead of the actual target function, then the returned instance can
be used with any target function. However, this will reduce ForwardDiff's ability to catch
and prevent perturbation confusion (see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).
If `f` or `tag` is `nothing`, then the returned instance can be used with any target function.
However, this will reduce ForwardDiff's ability to catch and prevent perturbation confusion
(see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).

This constructor does not store/modify `x`.
"""
Expand Down Expand Up @@ -145,9 +201,9 @@ The returned `JacobianConfig` instance contains all the work buffers required by
`ForwardDiff.jacobian` and `ForwardDiff.jacobian!` when the target function takes the form
`f(x)`.

If `f` is `nothing` instead of the actual target function, then the returned instance can
be used with any target function. However, this will reduce ForwardDiff's ability to catch
and prevent perturbation confusion (see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).
If `f` or `tag` is `nothing`, then the returned instance can be used with any target function.
However, this will reduce ForwardDiff's ability to catch and prevent perturbation confusion
(see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).

This constructor does not store/modify `x`.
"""
Expand All @@ -170,9 +226,9 @@ The returned `JacobianConfig` instance contains all the work buffers required by
`ForwardDiff.jacobian` and `ForwardDiff.jacobian!` when the target function takes the form
`f!(y, x)`.

If `f!` is `nothing` instead of the actual target function, then the returned instance can
be used with any target function. However, this will reduce ForwardDiff's ability to catch
and prevent perturbation confusion (see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).
If `f!` or `tag` is `nothing`, then the returned instance can be used with any target function.
However, this will reduce ForwardDiff's ability to catch and prevent perturbation confusion
(see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).

This constructor does not store/modify `y` or `x`.
"""
Expand Down Expand Up @@ -212,9 +268,9 @@ configured for the case where the `result` argument is an `AbstractArray`. If
it is a `DiffResult`, the `HessianConfig` should instead be constructed via
`ForwardDiff.HessianConfig(f, result, x, chunk)`.

If `f` is `nothing` instead of the actual target function, then the returned instance can
be used with any target function. However, this will reduce ForwardDiff's ability to catch
and prevent perturbation confusion (see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).
If `f` or `tag` is `nothing`, then the returned instance can be used with any target function.
However, this will reduce ForwardDiff's ability to catch and prevent perturbation confusion
(see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).

This constructor does not store/modify `x`.
"""
Expand All @@ -236,9 +292,9 @@ type/shape of the input vector `x`.
The returned `HessianConfig` instance contains all the work buffers required by
`ForwardDiff.hessian!` for the case where the `result` argument is an `DiffResult`.

If `f` is `nothing` instead of the actual target function, then the returned instance can
be used with any target function. However, this will reduce ForwardDiff's ability to catch
and prevent perturbation confusion (see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).
If `f` or `tag` is `nothing`, then the returned instance can be used with any target function.
However, this will reduce ForwardDiff's ability to catch and prevent perturbation confusion
(see https://github.com/JuliaDiff/ForwardDiff.jl/issues/83).

This constructor does not store/modify `x`.
"""
Expand Down
5 changes: 5 additions & 0 deletions src/prelude.jl
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
const NANSAFE_MODE_ENABLED = @load_preference("nansafe_mode", false)
const DEFAULT_CHUNK_THRESHOLD = @load_preference("default_chunk_threshold", 12)

# On ≤1.10, the hash of a type cannot be computed at compile-time,
# making `HashTag(...)` type-unstable, so `Tag(...)` is left as
# as the default.
const HASHTAG_MODE_ENABLED = @load_preference("hashtag_mode", VERSION ≥ v"1.11")

const AMBIGUOUS_TYPES = (AbstractFloat, Irrational, Integer, Rational, Real, RoundingMode)

const UNARY_PREDICATES = Symbol[:isinf, :isnan, :isfinite, :iseven, :isodd, :isreal, :isinteger]
Expand Down