-
Notifications
You must be signed in to change notification settings - Fork 547
add cuopt direct solver #3620
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
add cuopt direct solver #3620
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3620 +/- ##
==========================================
- Coverage 89.05% 85.72% -3.34%
==========================================
Files 890 891 +1
Lines 102617 102802 +185
==========================================
- Hits 91384 88122 -3262
- Misses 11233 14680 +3447
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are working on making cuopt available in our testing infrastructure; can you please add tests to this PR?
@Iroy30 - We've been able to make cuopt available on our internal testing machines. Can you please add tests to this PR? |
5dbf9dd
to
6f64094
Compare
@mrmundt Thanks! We have added tests by enabling testing cuopt with LP and MILP capabilities in tests/solvers.py. Let us know if:
The following is the testing output I get relevant to cuOpt
|
t0 = time.time() | ||
self.solution = cuopt.linear_programming.solver.Solve(self._solver_model) | ||
t1 = time.time() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is fine, but just so you're aware, we have this lovely little utility called TicTocTimer
that you may want to consider using: https://pyomo.readthedocs.io/en/latest/api/pyomo.common.timing.TicTocTimer.html
@mrmundt What do you think about including this solver interface in pyomo.contrib.solvers? Would it make sense to pull-in new solver interfaces there, since that's where the new solver API is evolving? |
@whart222 - I am evenly split. Because we are still messing with what the new solver interfaces are going to actually do / how they will handle input and present output, I don't know if we want to put "new" solvers there or just "well-established" ones that we can robustly test / really know what they are supposed to do and return. |
@Iroy30 - I forgot to post this last week, but all of the failures are of the variety:
|
@Iroy30 - Two more things:
|
Co-authored-by: Miranda Mundt <[email protected]>
"Integers": "I", | ||
"Reals": "C", | ||
"Binary": "I", | ||
} # NonNegativeReals ? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am fairly certain this is necessary. I got the tests running on our test machine, and I keep getting these errors:
__________________________________________________________ Test_LP_block.test_cuopt_direct_python_nonsymbolic_labels ___________________________________________________________
self = <pyomo.solvers.tests.checks.test_pickle.Test_LP_block testMethod=test_cuopt_direct_python_nonsymbolic_labels>
def return_test(self):
> return pickle_test(self)
pyomo/solvers/tests/checks/test_pickle.py:141:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pyomo/solvers/tests/checks/test_pickle.py:48: in pickle_test
opt, status = model_class.solve(
pyomo/solvers/tests/models/base.py:123: in solve
results = opt.solve(
pyomo/solvers/plugins/solvers/direct_solver.py:130: in solve
self._presolve(*args, **kwds)
pyomo/solvers/plugins/solvers/direct_solver.py:68: in _presolve
self._set_instance(model, kwds)
pyomo/solvers/plugins/solvers/cuopt_direct.py:140: in _set_instance
self._add_block(model)
pyomo/solvers/plugins/solvers/cuopt_direct.py:143: in _add_block
self._add_var(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pyomo.solvers.plugins.solvers.cuopt_direct.CUOPTDirect object at 0x7f91e6fc5050>, variables = <generator object BlockData.component_data_objects at 0x7f91ec180a60>
def _add_var(self, variables):
# Map vriable to index and get var bounds
var_type_dict = {
"Integers": "I",
"Reals": "C",
"Binary": "I",
} # NonNegativeReals ?
self.var_name_dict = {}
v_lb, v_ub, v_type = [], [], []
for i, v in enumerate(variables):
> v_type.append(var_type_dict[str(v.domain)])
E KeyError: 'NonNegativeReals'
pyomo/solvers/plugins/solvers/cuopt_direct.py:100: KeyError
I am also seeing errors like this:
pyomo/solvers/tests/checks/test_no_solution_behavior.py:92:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pyomo/solvers/tests/checks/test_no_solution_behavior.py:61: in failed_solve_test
opt, results = model_class.solve(
pyomo/solvers/tests/models/base.py:123: in solve
results = opt.solve(
pyomo/solvers/plugins/solvers/direct_solver.py:130: in solve
self._presolve(*args, **kwds)
pyomo/solvers/plugins/solvers/direct_solver.py:68: in _presolve
self._set_instance(model, kwds)
pyomo/solvers/plugins/solvers/cuopt_direct.py:140: in _set_instance
self._add_block(model)
pyomo/solvers/plugins/solvers/cuopt_direct.py:143: in _add_block
self._add_var(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pyomo.solvers.plugins.solvers.cuopt_direct.CUOPTDirect object at 0x7f05bc563e10>
variables = <generator object _component_data_objects at 0x7f043a8d6890>
def _add_var(self, variables):
# Map vriable to index and get var bounds
var_type_dict = {
"Integers": "I",
"Reals": "C",
"Binary": "I",
} # NonNegativeReals ?
self.var_name_dict = {}
v_lb, v_ub, v_type = [], [], []
for i, v in enumerate(variables):
> v_type.append(var_type_dict[str(v.domain)])
E AttributeError: property 'domain' of 'variable' object has no getter
pyomo/solvers/plugins/solvers/cuopt_direct.py:100: AttributeError
Plus some other errors:
self = <pyomo.solvers.tests.checks.test_pickle.Test_LP_duals_maximize testMethod=test_cuopt_direct_python_nonsymbolic_labels>
def return_test(self):
> return pickle_test(self)
pyomo/solvers/tests/checks/test_pickle.py:141:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pyomo/solvers/tests/checks/test_pickle.py:48: in pickle_test
opt, status = model_class.solve(
pyomo/solvers/tests/models/base.py:123: in solve
results = opt.solve(
pyomo/solvers/plugins/solvers/direct_solver.py:168: in solve
result = self._postsolve()
pyomo/solvers/plugins/solvers/cuopt_direct.py:246: in _postsolve
self._load_rc()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pyomo.solvers.plugins.solvers.cuopt_direct.CUOPTDirect object at 0x7f909a5f4dd0>
vars_to_load = KeysView(<pyomo.common.collections.component_map.ComponentMap object at 0x7f9099137af0>)
def _load_rc(self, vars_to_load=None):
if not hasattr(self._pyomo_model, "rc"):
self._pyomo_model.rc = Suffix(direction=Suffix.IMPORT)
rc = self._pyomo_model.rc
var_map = self._pyomo_var_to_ndx_map
if vars_to_load is None:
vars_to_load = var_map.keys()
> reduced_costs = self.solution.get_reduced_costs()
E AttributeError: 'Solution' object has no attribute 'get_reduced_costs'
pyomo/solvers/plugins/solvers/cuopt_direct.py:261: AttributeError
________________________________________________________ Test_LP_unbounded.test_cuopt_direct_python_nonsymbolic_labels _________________________________________________________
self = <pyomo.solvers.tests.checks.test_pickle.Test_LP_unbounded testMethod=test_cuopt_direct_python_nonsymbolic_labels>
def return_test(self):
> return pickle_test(self)
pyomo/solvers/tests/checks/test_pickle.py:141:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pyomo/solvers/tests/checks/test_pickle.py:48: in pickle_test
opt, status = model_class.solve(
pyomo/solvers/tests/models/base.py:123: in solve
results = opt.solve(
pyomo/solvers/plugins/solvers/direct_solver.py:168: in solve
result = self._postsolve()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pyomo.solvers.plugins.solvers.cuopt_direct.CUOPTDirect object at 0x7f909927cbd0>
def _postsolve(self):
extract_duals = False
extract_slacks = False
extract_reduced_costs = False
for suffix in self._suffixes:
flag = False
if re.match(suffix, "rc"):
extract_reduced_costs = True
flag = True
if not flag:
raise RuntimeError(
"***The cuopt_direct solver plugin cannot extract solution suffix="
+ suffix
)
solution = self.solution
status = solution.get_termination_status()
self.results = SolverResults()
soln = Solution()
self.results.solver.name = "CUOPT"
self.results.solver.wallclock_time = self._wallclock_time
prob_type = solution.problem_category
if status in [1]:
self.results.solver.status = SolverStatus.ok
self.results.solver.termination_condition = TerminationCondition.optimal
soln.status = SolutionStatus.optimal
elif status in [3]:
self.results.solver.status = SolverStatus.warning
self.results.solver.termination_condition = TerminationCondition.unbounded
soln.status = SolutionStatus.unbounded
elif status in [8]:
self.results.solver.status = SolverStatus.ok
self.results.solver.termination_condition = TerminationCondition.feasible
soln.status = SolutionStatus.feasible
elif status in [2]:
self.results.solver.status = SolverStatus.warning
self.results.solver.termination_condition = TerminationCondition.infeasible
soln.status = SolutionStatus.infeasible
elif status in [4]:
self.results.solver.status = SolverStatus.aborted
self.results.solver.termination_condition = (
TerminationCondition.maxIterations
)
soln.status = SolutionStatus.stoppedByLimit
elif status in [5]:
self.results.solver.status = SolverStatus.aborted
self.results.solver.termination_condition = (
TerminationCondition.maxTimeLimit
)
soln.status = SolutionStatus.stoppedByLimit
elif status in [7]:
self.results.solver.status = SolverStatus.ok
self.results.solver.termination_condition = TerminationCondition.other
soln.status = SolutionStatus.other
else:
self.results.solver.status = SolverStatus.error
self.results.solver.termination_condition = TerminationCondition.error
soln.status = SolutionStatus.error
if self._solver_model.maximize:
self.results.problem.sense = maximize
else:
self.results.problem.sense = minimize
self.results.problem.upper_bound = None
self.results.problem.lower_bound = None
try:
self.results.problem.upper_bound = solution.get_primal_objective()
self.results.problem.lower_bound = solution.get_primal_objective()
except Exception as e:
pass
var_map = self._pyomo_var_to_ndx_map
primal_solution = solution.get_primal_solution().tolist()
for i, pyomo_var in enumerate(var_map.keys()):
> pyomo_var.set_value(primal_solution[i], skip_validation=True)
E IndexError: list index out of range
pyomo/solvers/plugins/solvers/cuopt_direct.py:243: IndexError
----------------------------------------------------------------------------- Captured stdout call -----------------------------------------------------------------------------
Setting parameter log_file to
Error in solve_lp: {"CUOPT_ERROR_TYPE": "ValidationError", "msg": "A_offsets must be set before calling the solver."}
Hi @Iroy30 - I got the tests running - the name in the It would be a good idea for you to create a |
I am looking at the "Attempting to use an unavailable solver" error currently. The rest should be fixed.
Is this resolved? |
Yes, I fixed that. Please look at the other errors I posted in different comments above. |
we are investigating certain Termination condition mismatches on our end |
Fixes #3626
Summary/Motivation:
Add cuOpt math optimization (includes LP and MILP) solver backend to Pyomo so users can solve pyomo models with cuOpt
Changes proposed in this PR:
Legal Acknowledgement
By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution: