Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store objects in WorkerConfig rather than nested dictionaries #2426

Merged
merged 20 commits into from
Feb 14, 2025

Conversation

kif
Copy link
Member

@kif kif commented Feb 6, 2025

The idea is to store:

  • Polarization description (actually a namedtuple)
  • Units (actually a dedicated class)
  • PoniFile (actually a dedicated class)
  • ErrorModel (actually an enum)

in a transparent way and have the serialization/deserialization managed for the user

@kif kif requested a review from t20100 February 6, 2025 16:43
@kif kif added the proposal Something which have to be tested label Feb 6, 2025
@kif
Copy link
Member Author

kif commented Feb 7, 2025

@t20100 Hi Thomas, I believe you will love this PR:
Now attributes on the dataclass like poni, unit, polarization_description and error_model are stored in memory as their native python objects and serialized as needed.
This is not completely trivial since:

  • PoniFile is a dataclass-like object
  • Unit is also a dedicated class
  • ErrorModel is an Enum
  • PolarizationDescription is a named-tuple.

@kif kif added ready to merge Please review and removed proposal Something which have to be tested labels Feb 7, 2025
@kif
Copy link
Member Author

kif commented Feb 7, 2025

New depreciation warning in pyFAI-integrate when importing a JSON file:

(py311) jerome@antarctica:/tmp/tutorial$ pyFAI-integrate 
INFO:numexpr.utils:NumExpr defaulting to 16 threads.
ERROR:pyFAI.gui.IntegrationDialog:No such file: .azimint.json
WARNING:pyFAI.io.integration_config:Those are the parameters which have not been converted !polarization_factor: None
WARNING:pyFAI.DEPRECATION:Function __contains__ is deprecated since pyFAI version 2025.02. Reason: Ponifile should not be used as a dict.
  File "/home/jerome/.venv/py311/bin/pyFAI-integrate", line 8, in <module>
    sys.exit(main())
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 855, in main
    result = _main(args)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 848, in _main
    result = integrate_gui(options, args)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 152, in integrate_gui
    result = app.exec_()
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 560, in __selectFile
    self.loadFromJsonFile(filename)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 611, in loadFromJsonFile
    self.setConfig(config)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 332, in setConfig
    self.setWorkerConfig(integration_config.WorkerConfig.from_dict(dico, inplace=False))
WARNING:pyFAI.DEPRECATION:Function __getitem__ is deprecated since pyFAI version 2025.02. Reason: Ponifile should not be used as a dict.
  File "/home/jerome/.venv/py311/bin/pyFAI-integrate", line 8, in <module>
    sys.exit(main())
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 855, in main
    result = _main(args)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 848, in _main
    result = integrate_gui(options, args)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 152, in integrate_gui
    result = app.exec_()
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 560, in __selectFile
    self.loadFromJsonFile(filename)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 611, in loadFromJsonFile
    self.setConfig(config)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 332, in setConfig
    self.setWorkerConfig(integration_config.WorkerConfig.from_dict(dico, inplace=False))
WARNING:pyFAI.DEPRECATION:Function get is deprecated since pyFAI version 2025.02. Reason: Ponifile should not be used as a dict.
  File "/home/jerome/.venv/py311/bin/pyFAI-integrate", line 8, in <module>
    sys.exit(main())
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 855, in main
    result = _main(args)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 848, in _main
    result = integrate_gui(options, args)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/app/integrate.py", line 152, in integrate_gui
    result = app.exec_()
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 560, in __selectFile
    self.loadFromJsonFile(filename)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 611, in loadFromJsonFile
    self.setConfig(config)
  File "/home/jerome/.venv/py311/lib/python3.11/site-packages/pyFAI/gui/widgets/WorkerConfigurator.py", line 332, in setConfig
    self.setWorkerConfig(integration_config.WorkerConfig.from_dict(dico, inplace=False))

Geometry is not properly loaded

@kif
Copy link
Member Author

kif commented Feb 7, 2025

Other issues found in json file saved from pyFAI integrate/pyFAI-calib2:

  • poni_version absent from PoniFile serialisation
  • polarization_description is still wrong (contains null or is a list of lists)
  • Normalization should not be null but 1.0 when absent
  • calib2 version has extra deprecated options like do_polarization, polarization_factor, do_mask, do_flat, do_dark

@kif
Copy link
Member Author

kif commented Feb 7, 2025

integrate and diffmap are working as expected.

@kif kif merged commit 9b5e20e into silx-kit:main Feb 14, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready to merge Please review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants