Introduction of R2N Solver in JSOSolvers

[farhadrclass]

Overview

This PR introduces the new R2N solver, a first-order quadratic regularization method for unconstrained optimization, along with the implementation of a shifted system solver function. These contributions provide robust and efficient tools for solving challenging optimization tasks.

Key Additions

1. Introduction of R2N Solver:
   • Implements a first-order quadratic regularization approach.

Usage Example

solver = R2NSolver(nlp)
solve!(solver, nlp; kwargs...)

Parameters

• nlp::AbstractNLPModel{T, V}: The model to solve, as defined in NLPModels.jl.
• Keyword Arguments:
  • x::V = nlp.meta.x0: Initial guess.
  • atol::T = √ε(T): Absolute tolerance.
  • rtol::T = √ε(T): Relative tolerance, where the algorithm stops when ‖∇f(xᵏ)‖ ≤ atol + rtol * ‖∇f(x⁰)‖.
  • η1 = ε(T)^(1/4), η2 = T(0.95): Step acceptance parameters.
  • γ1 = T(1/2), γ2 = 1/γ1: Regularization update parameters.
  • σmin = ε(T): Step parameter.
  • max_eval::Int = -1: Maximum number of objective function evaluations.
  • max_time::Float64 = 30.0: Maximum time limit in seconds.
  • max_iter::Int = typemax(Int): Maximum number of iterations.
  • β = T(0) ∈ [0,1]: Momentum term constant. No momentum if β == 0.
  • verbose::Int = 0: Display iteration details every verbose iteration.
  • subsolver_type::Union{Type{<:KrylovSolver}, Type{ShiftedLBFGSSolver}} = ShiftedLBFGSSolver: Subsolver for the shifted system, defaulting to ShiftedLBFGSSolver.
  • subsolver_verbose::Int = 0: Displays iteration information for the subsolver when subsolver_verbose > 0.

[tmigot]

Thanks @farhadrclass for the PR. A first batch of comments.
The tests fail because of extra allocations. Any idea why?

What are the differences with FOMO?