What settings to choose to ensure reproducibility?

First off, thank you for making OSQP available as open source. We’ve recently started to experiment with it for use with some contact finite element problems, and for our first few (small-scale) tests it has worked very well. However, we have one issue: the results produced are not reproducible in the sense that if we re-run the same program several times, the results are not the same. When coupling this with relatively loose tolerances (which is desirable), we can observe wildly different behavior in our simulations between runs (due to the inherent sensitivity to perturbations in the contact problems).

From reading the OSQP paper, we suspect that perhaps this non-determinism comes from the fact that the decision of whether or not to re-compute a factorization depends on the time elapsed. Is this right? Are there also other significant sources of non-determinism? And what can we do about them? It seems that the setting adaptive_rho_fraction is relevant here, but it’s not entirely clear if we can disable this behavior.

We are using the Rust bindings, if this matters.

This seems tangentially related to what was discussed in this forum post, but our problem is rerunning the same application with the exact same problem, not reusing the solver across different but related problems.

I think it is very likely to be adaptive_rho_fraction that is the culprit. The solver tries to make a guess about the relative time required for updating the factor \rho based on the initial factorisation time, which will vary slightly when solving instances of the same problem. If you want deterministic behaviour you should set this value to 0, which will make rho update using a deterministic updating rule. You can then set something like adaptive_rho_interval = 50 or thereabout.

I do not think there is any other non-deterministic behaviour in the solver.

Thank you for the quick response! Will give your suggestions a try.

Hi again. I finally got back to actually trying out your suggestion (been a bit busy with other things). When I try to set adaptive_rho_fraction to 0.0, I get:

ERROR in validate_settings: adaptive_rho_fraction must be positive

What else can I do to ensure reproducibility?

Set adaptive_rho_interval to something non-zero, so that the automatic selection of this interval based in timing will be disabled. Something like adaptive_rho_interval = 25 or 50 is a good starting point.

Thanks for clarifying! It was not clear to me if setting only adaptive_rho_interval alone would be sufficient.

Perhaps it would be worth writing a little about this non-deterministic behavior in the documentation? I can imagine it’s an issue many will run into, especially in science, where determinism for running experiments is of crucial importance. I didn’t find anything about this in the docs at least, sorry if I missed something.

I agree it is not clear, and you are not the first user to raise this issue. I have added this to our to-do list for the 1.0 release.