Added a new Subpackage for Augmented Lagrangian

[ParamThakkar123]

Checklist

• Appropriate tests were added
• Any code changes were done in a way that does not break public API
• All documentation related to code changes were updated
• The new code follows the
  contributor guidelines, in particular the SciML Style Guide and
  COLPRAC.
• Any new documentation only uses public API

Additional context

Add any other context about the problem here.

[ChrisRackauckas]

should be able to just need optimizationbase?

[ParamThakkar123]

Okay will replace with that one

[ParamThakkar123]

@ChrisRackauckas got the following error while switching to just OptimizationBase:

OptimizationAuglag.jl: Error During Test at D:\Optimization.jl\lib\OptimizationAuglag\test\runtests.jl:10
  Got exception outside of a @test
  MethodError: no method matching Optimization.OptimizationState(; u::Vector{Float64}, objective::Float64, p::Tuple{Vector{Float64}, Vector{Float64}})
  This error has been manually thrown, explicitly, so the method may exist but be intentionally marked as unimplemented.

  Closest candidates are:
    Optimization.OptimizationState(; iter, u, objective, grad, hess, original) got unsupported keyword argument "p"  
     @ Optimization D:\JuliaDepot\packages\Optimization\hfEs4\src\state.jl:24
    Optimization.OptimizationState(::Int64, ::X, ::O, ::G, ::H, ::S) where {X, O, G, H, S} got unsupported keyword arguments "u", "objective", "p"
     @ Optimization D:\JuliaDepot\packages\Optimization\hfEs4\src\state.jl:16
  
  Stacktrace:
    [1] kwerr(kw::@NamedTuple{u::Vector{Float64}, objective::Float64, p::Tuple{Vector{Float64}, Vector{Float64}}}, args::Type)
      @ Base .\error.jl:165
    [2] (::OptimizationAuglag.var"#4#10"{OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing}, Vector{Float64}, BitVector, Vector{Bool}})(θ::Vector{Float64}, p::Tuple{Vector{Float64}, Vector{Float64}})
      @ OptimizationAuglag D:\Optimization.jl\lib\OptimizationAuglag\src\OptimizationAuglag.jl:113
    [3] (::OptimizationFunction{true, SciMLBase.NoAD, OptimizationAuglag.var"#4#10"{OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing}, Vector{Float64}, BitVector, Vector{Bool}}, OptimizationBase.var"#grad#204"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, SciMLBase.NoAD, OptimizationAuglag.var"#4#10"{OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing}, Vector{Float64}, BitVector, Vector{Bool}}, OptimizationAuglag.var"#aug_grad#13"{OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing}, Vector{Float64}, BitVector, Vector{Bool}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing})(::Vector{Float64}, ::Vararg{Any})
      @ SciMLBase D:\JuliaDepot\packages\SciMLBase\dld4W\src\scimlfunctions.jl:4251
    [4] macro expansion
      @ D:\JuliaDepot\packages\OptimizationOptimisers\vlc0v\src\OptimizationOptimisers.jl:105 [inlined]
    [5] macro expansion
      @ D:\JuliaDepot\packages\Optimization\hfEs4\src\utils.jl:32 [inlined]
    [6] __solve(cache::OptimizationCache{OptimizationFunction{true, SciMLBase.NoAD, OptimizationAuglag.var"#4#10"{OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing}, Vector{Float64}, BitVector, Vector{Bool}}, OptimizationBase.var"#grad#204"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, SciMLBase.NoAD, OptimizationAuglag.var"#4#10"{OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing}, Vector{Float64}, BitVector, Vector{Bool}}, OptimizationAuglag.var"#aug_grad#13"{OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing}, Vector{Float64}, BitVector, Vector{Bool}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Nothing, Nothing, Nothing, Nothing, Nothing, Adam{Float64, Tuple{Float64, Float64}, Float64}, Bool, OptimizationOptimisers.var"#3#5", Nothing})
      @ OptimizationOptimisers D:\JuliaDepot\packages\OptimizationOptimisers\vlc0v\src\OptimizationOptimisers.jl:92  
    [7] solve!
      @ D:\JuliaDepot\packages\SciMLBase\dld4W\src\solve.jl:232 [inlined]
    [8] #solve#741
      @ D:\JuliaDepot\packages\SciMLBase\dld4W\src\solve.jl:130 [inlined]
    [9] __solve(cache::OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing})
      @ OptimizationAuglag D:\Optimization.jl\lib\OptimizationAuglag\src\OptimizationAuglag.jl:162
   [10] solve!(cache::OptimizationCache{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, var"#loss#3", OptimizationBase.var"#grad#92"{Tuple{Vector{Float64}, Vector{Float64}}, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}}, Nothing, Nothing, Nothing, Nothing, OptimizationBase.var"#85#102"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}}, OptimizationBase.var"#cons_j!#105"{AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, DifferentiationInterfaceSparseMatrixColoringsExt.SMCPushforwardSparseJacobianPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoSparse{AutoForwardDiff{nothing, Nothing}, SparseConnectivityTracer.TracerSparsityDetector{SparseConnectivityTracer.GradientTracer{Int64, BitSet}, SparseConnectivityTracer.HessianTracer{Int64, BitSet, Dict{Int64, BitSet}, SparseConnectivityTracer.NotShared}}, SparseMatrixColorings.GreedyColoringAlgorithm{:direct, SparseMatrixColorings.NaturalOrder}}, Vector{Float64}, Tuple{}}, DifferentiationInterface.BatchSizeSettings{2, true, true}, SparseArrays.SparseMatrixCSC{Bool, Int64}, SparseMatrixColorings.ColumnColoringResult{SparseArrays.SparseMatrixCSC{Bool, Int64}, Int64, SparseMatrixColorings.BipartiteGraph{Int64}, Vector{Int64}, Vector{SubArray{Int64, 1, Vector{Int64}, Tuple{UnitRange{Int64}}, true}}, Vector{Int64}, Nothing}, Matrix{Float64}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, Vector{Tuple{Vector{Float64}, Vector{Float64}}}, DifferentiationInterfaceForwardDiffExt.ForwardDiffOneArgPushforwardPrep{Tuple{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, AutoForwardDiff{nothing, Nothing}, Vector{Float64}, Tuple{Vector{Float64}, Vector{Float64}}, Tuple{}}, ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{OptimizationBase.var"#cons_oop#103"{OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Tuple{Vector{Float64}, Vector{Float64}}, Int64}, Float64}, Float64, 2}}, Tuple{}}}}, Nothing, Nothing, Nothing, Nothing, SparseArrays.SparseMatrixCSC{Bool, Int64}, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Vector{Int64}, Nothing, Nothing, Nothing}, OptimizationBase.ReInitCache{Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}}, Vector{Float64}, Vector{Float64}, Vector{Float64}, Vector{Int64}, Nothing, OptimizationAuglag.AugLag, Bool, var"#2#6", Nothing})
      @ SciMLBase D:\JuliaDepot\packages\SciMLBase\dld4W\src\solve.jl:232
   [11] solve(::OptimizationProblem{true, OptimizationFunction{true, AutoSparse{AutoForwardDiff{nothing, Nothing}, ADTypes.NoSparsityDetector, ADTypes.NoColoringAlgorithm}, var"#loss#3", Nothing, Nothing, Nothing, Nothing, Nothing, var"#cons1#5", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}, DataLoader{BatchView{Tuple{Vector{Float64}, Vector{Float64}}, ObsView{Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Vector{Int64}}, Val{nothing}}, Bool, :serial, Val{nothing}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, Vector{Float64}}, Random.TaskLocalRNG}, Vector{Float64}, Vector{Float64}, Nothing, Vector{Float64}, Vector{Int64}, Nothing, @Kwargs{}}, ::OptimizationAuglag.AugLag; kwargs::@Kwargs{maxiters::Int64, callback::var"#2#6"})
      @ SciMLBase D:\JuliaDepot\packages\SciMLBase\dld4W\src\solve.jl:130
   [12] macro expansion
      @ D:\Optimization.jl\lib\OptimizationAuglag\test\runtests.jl:33 [inlined]
   [13] macro expansion
      @ C:\Users\Hp\.julia\juliaup\julia-1.11.5+0.x64.w64.mingw32\share\julia\stdlib\v1.11\Test\src\Test.jl:1704 [inlined]
   [14] top-level scope
      @ D:\Optimization.jl\lib\OptimizationAuglag\test\runtests.jl:11
   [15] include(fname::String)
      @ Main .\sysimg.jl:38
   [16] top-level scope
      @ none:6
   [17] eval
      @ .\boot.jl:430 [inlined]
   [18] exec_options(opts::Base.JLOptions)
      @ Base .\client.jl:296
   [19] _start()
      @ Base .\client.jl:531
Test Summary:         | Error  Total   Time
OptimizationAuglag.jl |     1      1  10.5s
ERROR: LoadError: Some tests did not pass: 0 passed, 0 failed, 1 errored, 0 broken.
in expression starting at D:\Optimization.jl\lib\OptimizationAuglag\test\runtests.jl:10
ERROR: Package OptimizationAuglag errored during testing

[ParamThakkar123]

This works now

[ParamThakkar123]

@ChrisRackauckas I tried using just a single package OptimizationBase but Optimization state used in the code is present in Optimization so needed to use that one

[ChrisRackauckas]

Ahh that should be moved to Base I think

[ParamThakkar123]

Okay will start a PR on Base to add this up

[ParamThakkar123]

@ChrisRackauckas started a PR to add OptimizationState to Base: SciML/OptimizationBase.jl#155