Fitting not working in sasview with sasmodels (Trac #534)

[ajj]

The models load from sasmodels and calculate in sasview.

But fitting does not fit. Need to see why and fix.

Migrated from http://trac.sasview.org/ticket/534

{
    "status": "closed",
    "changetime": "2016-06-19T15:43:14",
    "_ts": "2016-06-19 15:43:14.035179+00:00",
    "description": "The models load from sasmodels and calculate in sasview.\n\nBut fitting does not fit. Need to see why and fix.",
    "reporter": "ajj",
    "cc": "",
    "resolution": "fixed",
    "workpackage": "SasModels Redesign",
    "time": "2016-03-18T16:04:05",
    "component": "SasView",
    "summary": "Fitting not working in sasview with sasmodels",
    "priority": "blocker",
    "keywords": "",
    "milestone": "SasView 4.0.0",
    "owner": "pkienzle",
    "type": "task"
}

[ajj]

Trac update at 2016/03/18 16:09:24: ajj changed workpackage from "SasView Bug Fixing" to "SasModels Redesign"

[butlerpd]

Trac update at 2016/03/19 11:59:50:

• butler changed owner from "" to "ajj"
• butler changed status from "new" to "assigned"
• butler changed type from "defect" to "task"

[ajj]

Trac update at 2016/03/19 21:48:55:

• ajj changed _comment0 from:

Notes on testing:

Working : DREAM, Nelder-Mead, Differential evolution
Not returning a fit : Levenberg-Marquardt
Completely broken (see below) : QuasiNewton

{{{
2016-03-19 21:05:07,746 ERROR Traceback (most recent call last):
2016-03-19 21:05:07,746 ERROR File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py
", line 345, in run_bumps
2016-03-19 21:05:07,746 ERROR best, fbest = fitdriver.fit()
2016-03-19 21:05:07,746 ERROR File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitter
s.py", line 823, in fit
2016-03-19 21:05:07,748 ERROR **self.options)
2016-03-19 21:05:07,748 ERROR File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitter
s.py", line 322, in solve
2016-03-19 21:05:07,749 ERROR steptol=options['xtol'],
2016-03-19 21:05:07,749 ERROR File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/quasin
ewton.py", line 261, in quasinewton
2016-03-19 21:05:07,750 ERROR return dict(status=termcode, x=xf, fx=ff, H=H, L=L,
2016-03-19 21:05:07,750 ERROR UnboundLocalError: local variable 'H' referenced before assignment
2016-03-19 21:05:07,757 ERROR Traceback (most recent call last):
File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 82, in c
ompute
result = map(map_apply, inputs)
File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 14, in map_apply
return apply(arguments[0], arguments[1:])
File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 11, in map_getattr
return getattr(classInstance, classFunc)(*args)
File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py", line 274, in fit
result = run_bumps(problem, handler, curr_thread)
File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py", line 345, in run_bumps
best, fbest = fitdriver.fit()
File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitters.py", line 823, in fit
**self.options)
File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitters.py", line 322, in solve
steptol=options['xtol'],
File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/quasinewton.py", line 261, in quasinewton
return dict(status=termcode, x=xf, fx=ff, H=H, L=L,
UnboundLocalError: local variable 'H' referenced before assignment

None }}}

to:

1458424241778532

• ajj changed _comment1 from:

Notes on testing:

Working : DREAM, Nelder-Mead, Differential evolution
Not returning a fit : Levenberg-Marquardt
Completely broken (see below) : ?QuasiNewton

{{{
2016-03-19 21:05:07,746 ERROR Traceback (most recent call last):
2016-03-19 21:05:07,746 ERROR File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py
", line 345, in run_bumps
2016-03-19 21:05:07,746 ERROR best, fbest = fitdriver.fit()
2016-03-19 21:05:07,746 ERROR File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitter
s.py", line 823, in fit
2016-03-19 21:05:07,748 ERROR **self.options)
2016-03-19 21:05:07,748 ERROR File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitter
s.py", line 322, in solve
2016-03-19 21:05:07,749 ERROR steptol=options['xtol'],
2016-03-19 21:05:07,749 ERROR File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/quasin
ewton.py", line 261, in quasinewton
2016-03-19 21:05:07,750 ERROR return dict(status=termcode, x=xf, fx=ff, H=H, L=L,
2016-03-19 21:05:07,750 ERROR UnboundLocalError: local variable 'H' referenced before assignment
2016-03-19 21:05:07,757 ERROR Traceback (most recent call last):
File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 82, in c
ompute
result = map(map_apply, inputs)
File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 14, in map_apply
return apply(arguments[0], arguments[1:])
File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 11, in map_getattr
return getattr(classInstance, classFunc)(*args)
File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py", line 274, in fit
result = run_bumps(problem, handler, curr_thread)
File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py", line 345, in run_bumps
best, fbest = fitdriver.fit()
File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitters.py", line 823, in fit
**self.options)
File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitters.py", line 322, in solve
steptol=options['xtol'],
File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/quasinewton.py", line 261, in quasinewton
return dict(status=termcode, x=xf, fx=ff, H=H, L=L,
UnboundLocalError: local variable 'H' referenced before assignment

None }}}

to:

1458424257973580

• ajj commented:

Notes on testing:

Working : DREAM, Nelder-Mead, Differential evolution
Not returning a fit : Levenberg-Marquardt
Completely broken (see below) : !QuasiNewton

2016-03-19 21:05:07,746 ERROR Traceback (most recent call last):
2016-03-19 21:05:07,746 ERROR   File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py
", line 345, in run_bumps
2016-03-19 21:05:07,746 ERROR     best, fbest = fitdriver.fit()
2016-03-19 21:05:07,746 ERROR   File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitter
s.py", line 823, in fit
2016-03-19 21:05:07,748 ERROR     **self.options)
2016-03-19 21:05:07,748 ERROR   File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitter
s.py", line 322, in solve
2016-03-19 21:05:07,749 ERROR     steptol=options['xtol'],
2016-03-19 21:05:07,749 ERROR   File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/quasin
ewton.py", line 261, in quasinewton
2016-03-19 21:05:07,750 ERROR     return dict(status=termcode, x=xf, fx=ff, H=H, L=L,
2016-03-19 21:05:07,750 ERROR UnboundLocalError: local variable 'H' referenced before assignment
2016-03-19 21:05:07,757 ERROR Traceback (most recent call last):
  File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 82, in c
ompute
    result = map(map_apply, inputs)
  File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 14, in map_apply
    return apply(arguments[0], arguments[1:])
  File "/Users/andrewjackson/Devel/sasview/src/sas/sasgui/perspectives/fitting/fit_thread.py", line 11, in map_getattr
    return  getattr(classInstance, classFunc)(*args)
  File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py", line 274, in fit
    result = run_bumps(problem, handler, curr_thread)
  File "/Users/andrewjackson/Devel/sasview/src/sas/sascalc/fit/BumpsFitting.py", line 345, in run_bumps
    best, fbest = fitdriver.fit()
  File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitters.py", line 823, in fit
    **self.options)
  File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/fitters.py", line 322, in solve
    steptol=options['xtol'],
  File "/Users/andrewjackson/anaconda/lib/python2.7/site-packages/bumps/quasinewton.py", line 261, in quasinewton
    return dict(status=termcode, x=xf, fx=ff, H=H, L=L,
UnboundLocalError: local variable 'H' referenced before assignment

 None ```


* **ajj** changed owner from "ajj" to "pkienzle"

[butlerpd]

Trac update at 2016/04/16 23:53:51: butler commented:

Mathieu says "Works, but you have to specify bumps 0.7.5.7." Not sure on what platform or what test. I have pulled the latest sasmodels and !SasView, including my commits that fix the copy to clipboard and move the new models into categories. I am using my windows 10 laptop. check_packages.py reports I am using bumps 0.7.5.7. I used the cyl_400_40.txt test file from our 1D test folder and fit with the cylinder model starting with the default parameters and setting both sld values to be fixed - everything else varying. I found that the same three algorithms that worked in Andrews tests still work fine while both the LM and !QuasiNewton do some processing as the chi^2^ goes from 21000 to 23,000 and error bars appear after the fit. However the parameters returned do not seem to change (except for rounding errors in !QuasiNewton which displays background as 0.000998xx instead of 0.001 it starts with and which is displayed when using LM.

NOTE: while radius and length are allowed to float their initial values are what was used to create the test data so they are not expected to change.

Note 2: there seem to be several local minimum that are quite close but have large changes in the dimensions.

[butlerpd]

Trac update at 2016/04/16 23:56:45: butler commented:

Note 3: the errors that Andrew reports above are not thrown when I was running it. All logs, including the eclipse output console are clean.

[smk78]

Trac update at 2016/04/26 14:18:38: smk78 commented:

Have just pulled on everything, built, and tried fitting on W7 using

Common Packages
reportlab Version Installed: 3.1.44
lxml Version Installed: 3.4.2.0
PIL Version Installed: 1.1.7
xhtml2pdf NOT INSTALLED
unittest-xml-reporting Version Installed: 1.10.0
pylint Installed (Unknown version)
matplotlib Version Installed: 1.4.3
bumps Version Installed: 0.7.5.7
scipy Version Installed: 0.15.1
periodictable Version Installed: 1.4.1
setuptools Version Installed: 18.4
sphinx Version Installed: 1.3.6
pyparsing Version Installed: 2.0.3
numpy Version Installed: 1.9.2
html5lib Version Installed: 0.999
wx Version Installed: 3.0.0.0

Windows Specific Packages:
pywin Version Installed: 219
py2exe Version Installed: 0.6.9
comtypes Version Installed: 0.6.2

I concur with Paul B above, fitting seems to do something but parameters and plot do not update. Also, you can't change optimiser (you can select a different one and click apply, but the dialog does not go away).

[smk78]

Trac update at 2016/04/26 15:17:36: smk78 commented:

Word on the street was that bumps 0.7.5.9 (not earlier) now needed.

Having updated bumps...

Common Packages
reportlab Version Installed: 3.1.44
lxml Version Installed: 3.4.2.0
PIL Version Installed: 1.1.7
xhtml2pdf NOT INSTALLED
unittest-xml-reporting Version Installed: 1.10.0
pylint Installed (Unknown version)
matplotlib Version Installed: 1.4.3
bumps Version Installed: 0.7.5.9
scipy Version Installed: 0.15.1
periodictable Version Installed: 1.4.1
setuptools Version Installed: 18.4
sphinx Version Installed: 1.3.6
pyparsing Version Installed: 2.0.3
numpy Version Installed: 1.9.2
html5lib Version Installed: 0.999
wx Version Installed: 3.0.0.0

Windows Specific Packages:
pywin Version Installed: 219
py2exe Version Installed: 0.6.9
comtypes Version Installed: 0.6.2

...and re-building, it does seem that '''fitting is now working''' on W7.

[butlerpd]

Trac update at 2016/06/19 15:43:14:

• butler commented:

This I think has been resolved a while back and certainly is working in 4.00 alpha2. There are a number of other issues but they are subject of their own tickets. Am thus closing this ticket as resolved.

• butler changed resolution from "" to "fixed"
• butler changed status from "assigned" to "closed"