Error Using L-bfgs-b In Scipy

I get some puzzling result when using the 'L-BFGS-B' method in scipy.optimize.minimize: import scipy.optimize as optimize import numpy as np def testFun(): prec = 1e3 fun

Solution 1:

BFGS method is one of those method that relies on not only the function value, but also the gradient and Hessian (think of it as first and second derivative if you wish). In your func1(), once you have round() in it, the gradient is no longer continuous. BFGS method therefore fails right after the 1st iteration (think of as this: BFGS searched around the starting parameter and found the gradient is not changed, so it stopped). Similarly, I would expect other methods requiring gradient fail as BGFS.

You may be able to get it working by precondition or rescaling X. But better yet, you should try gradient free method such as 'Nelder-Mead' or 'Powell'

Solution 2:

round and int create step functions, which are not differentiable. The l-bfgs-b method is for solving smooth optimization problems. It uses an approximate gradient (if you don't give it an explicit one), and that will be garbage if the function has steps.