Skip to content

Conversation

@tbng
Copy link
Contributor

@tbng tbng commented Apr 30, 2022

Objective: adding solver implemented in julia from https://github.com/gowerrobert/StochOpt.jl . Based on Nidham Gazagnadou, Robert M. Gower, Joseph Salmon (2019). Optimal mini-batch and step sizes for SAGA.

TODO list:

  • Write a julia solver that calls and returns the fitted weights (solvers/julia_saga_batch.jl)
  • Write a python wrapper for the file, including installing the required packages, then run the wrapper function ( solvers/julia_saga_batch.py)
  • Benchmarks this solver with multiple other solvers (sklearn l-bfgs + SGD + SAGA, SVRG, etc.) with different datasets from libsvm)

@tbng
Copy link
Contributor Author

tbng commented Apr 30, 2022

Maybe also pinging @ngazagna

Copy link
Member

@tomMoral tomMoral left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some comments.

Also, could you share the result you got? I am curious about what it look like for now.

@tbng
Copy link
Contributor Author

tbng commented May 3, 2022

Subopt curves are definitely not sexy yet:

Screenshot_2022-05-03_23-54-22
Screenshot_2022-05-03_23-54-01

@tbng tbng changed the title [WIP] Mini-batch SAGA solver from StochOpt.jl [WIP] Mini-batch solvers from StochOpt.jl May 9, 2022
@tbng
Copy link
Contributor Author

tbng commented May 9, 2022

Some plots on 2 reps simulated data, the delay time in the beginning for StochOpt solvers is probably due to julia wrapper initializing the function

Screenshot_2022-05-09_11-29-30
Screenshot_2022-05-09_11-29-11
Screenshot_2022-05-09_12-18-16
Screenshot_2022-05-09_12-17-55

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants