Fixes 2/3 of problems with Nelder Mead related traces

[eirikbrandsaas]

Fixes the first two the issues raised in #941

I tried to fix the last one, but that was beyond my capabilities. If you give me a pointer I should be able to figure it out.

[codecov]

Codecov Report

Merging #942 (ce651c1) into master (a1b957d) will increase coverage by 2.34%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #942      +/-   ##
==========================================
+ Coverage   83.60%   85.95%   +2.34%     
==========================================
  Files          42       42              
  Lines        3019     3182     +163     
==========================================
+ Hits         2524     2735     +211     
+ Misses        495      447      -48     

Impacted Files	Coverage Δ
src/api.jl	85.14% <100.00%> (+23.14%)	⬆️
src/multivariate/solvers/first_order/bfgs.jl	90.66% <0.00%> (-3.46%)	⬇️
src/univariate/printing.jl	17.64% <0.00%> (-1.11%)	⬇️
src/multivariate/optimize/interface.jl	81.13% <0.00%> (-0.25%)	⬇️
src/utilities/maxdiff.jl	100.00% <0.00%> (ø)
src/univariate/solvers/brent.jl	100.00% <0.00%> (ø)
src/utilities/assess_convergence.jl	100.00% <0.00%> (ø)
src/univariate/solvers/golden_section.jl	100.00% <0.00%> (ø)
src/multivariate/solvers/first_order/cg.jl	100.00% <0.00%> (ø)
src/multivariate/solvers/first_order/l_bfgs.jl	100.00% <0.00%> (ø)
... and 23 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 01b49c7...ce651c1. Read the comment docs.

[pkofod]

Thanks for looking into this. It looks fine. I would appreciate if you could add a test for the missing tr thing, just to make sure that the code path is run as part of the tests.

[eirikbrandsaas]

Sure, at least if you can slightly more specific on what you want. You just want me write a test that simply checks that the command works? In which of the many test/ files should it go?

[pkofod]

Sure, at least if you can slightly more specific on what you want. You just want me write a test that simply checks that the command works? In which of the many test/ files should it go?

Yes, just checking that what you changed does what you intended. Here would be a good place https://github.com/JuliaNLSolvers/Optim.jl/blob/master/test/multivariate/solvers/zeroth_order/nelder_mead.jl. Just add a testset with title #942

[eirikbrandsaas]

Okay, I'll get it done sometime "soon".

EDIT: Accidentally closed it when I posted the last comment. Sorry.

[eirikbrandsaas]

"Soon" became 6 months, but hey, who is counting?

I see now that you wanted me to post a separate pull request, but hope this works.

Note that, as mentioned in #941 that the stored trace is wrong, it replaces all simplex values with the last one. (That's why there are two @test_broken in the pull request.

Too see this:

using Optim
f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
x0 = [0.0,0.0]
res = optimize(f, x0, NelderMead(),Optim.Options(store_trace=true,trace_simplex=true,extended_trace=true,iterations=2,show_trace=true))
display(res.trace)

and compare the output of the simplexes (both y-values and x-simplexes) from the "running" trace and the stored res.trace.

[eirikbrandsaas]

@pkofod Just pinging.

(See comment above)