Implement reading experimental data and parameters from Scipp CIF (#157)

* Draft a new repo structure for easydiffraction

* Remove some unused files

* Rename Job.py to job.py

* Rename InterfaceFactory to WrapperFactory

* Move cryspy.py to easydiffraction/calculators/cryspy/calculator.py

* Rename CryspyV2 to CryspyWrapper

* Rename CFML to Pycrysfml

* Move pdffit2 to a dedicated subfolder

* Rename InterfaceTemplate to WrapperBase

* Remove cryspy V1

* Move download_from_repository to utils.py

* Move cryspy_parser.py to easydiffraction/calculators/cryspy/parser.py

* Move cifV2ToV1 from cif.py to cryspy/parser.py

* Remove io/cif.py

* Fix file headers

* Move easydiffraction to src/easydiffraction

* Move Interfaces/types.py to .../calculators/wrapper_types.py

* Ruff fixes

* Bump version to 0.1.8 for test-ipynb step to install the current package

* Implement reading experimental data and parameters from Scipp CIF

* Fix ruff issues

* Fix scipp test

* Fixes after discussion and review comments

* Apply `ruff format`

* Use versioningit to create the package version based on git tags

* Update release-drafter. Use RESOLVED_VERSION

* Adjust the length of comments to be < 72

* Reduce number of approved release labels

* Fetch all tags in actions/checkout for versioningit to work

* Add default tag if versioningit doesn't fine one

* Get full history with tags to get the version number by versioningit

* Try after merging master into develop

* Add comments

* fix pypi publish workflow to get version by versioningit

Author: AndrewSazonov

.github/release-drafter.yml

@@ -1,32 +1,36 @@
+# This file is used to configure the Release Drafter GitHub Action
 # https://github.com/marketplace/actions/release-drafter
-name-template: v$NEXT_PATCH_VERSION 🌈
-tag-template: v$NEXT_PATCH_VERSION
+
+name-template: 'v$RESOLVED_VERSION 🌈'
+tag-template: 'v$RESOLVED_VERSION'
 categories:
-- title: 🚀 Features
-  labels:
-  - feature
-  - enhancement
-- title: 🐛 Bug Fixes
-  labels:
-  - fix
-  - bugfix
-  - bug
-- title: 🧰 Maintenance
-  labels:
-  - chore
-  - documentation
+- title: '🚀 Features'
+  labels:  # Labels to use to categorize a pull request as a feature
+    - 'enhancement'
+- title: '🐛 Bug Fixes'
+  labels:  # Labels to use to categorize a pull request as a bug fix
+    - 'bug'
+- title: '🧰 Maintenance'
+  labels:  # Labels to use to categorize a pull request as a maintenance task
+    - 'chore'
+    - 'documentation'
+    - 'refactor'
 change-template: '- $TITLE @$AUTHOR (#$NUMBER)'
+change-title-escapes: '\<*_&' # You can add # and @ to disable mentions, and add ` to disable code blocks.
 version-resolver:
   major:
     labels:
-    - major
+      - 'major'
   minor:
     labels:
-    - minor
+      - 'enhancement'
   patch:
     labels:
-    - patch
+      - 'chore'
+      - 'documentation'
+      - 'refactor'
   default: patch
 template: |
   ## Changes
+
   $CHANGES

.github/workflows/delete-old-runs.yml

@@ -1,4 +1,5 @@
-# This workflow will delete old workflow runs based on the input parameters.
+# This workflow will delete old workflow runs based on the input
+# parameters.
 # https://github.com/Mattraks/delete-workflow-runs
 
 name: Delete old workflow runs

.github/workflows/ossar-analysis.yml

@@ -1,4 +1,5 @@
-# Integrates a collection of open source static analysis tools with GitHub code scanning.
+# Integrates a collection of open source static analysis tools with
+# GitHub code scanning.
 # https://github.com/github/ossar-action
 
 name: Run security static analysis

.github/workflows/pypi-publish.yml

@@ -1,4 +1,5 @@
-# Builds a Python package and publish it to PyPI when a new tag is created.
+# Builds a Python package and publish it to PyPI when a new tag is
+# created.
 
 name: Upload release to PyPI
 
@@ -19,6 +20,8 @@ jobs:
     steps:
     - name: Check-out repository
       uses: actions/checkout@v4
+      with:
+        fetch-depth: '0'  # full history with tags to get the version number by versioningit
 
     - name: Set up Python
       uses: actions/setup-python@v5

.github/workflows/release-drafter.yml

@@ -1,4 +1,5 @@
-# Drafts your next Release notes as pull requests are merged into default branch
+# Drafts your next Release notes as pull requests are merged into
+# default branch
 
 name: Update release draft
 
@@ -14,7 +15,7 @@ jobs:
 
     steps:
       - uses: release-drafter/release-drafter@v6
-        id: create_release
+        id: create-release
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
@@ -31,7 +32,7 @@ jobs:
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         with:
-          upload_url: ${{ steps.create_release.outputs.upload_url }} # This pulls from the create_release step above
+          upload_url: ${{ steps.create-release.outputs.upload_url }} # This pulls from the create-release step above
           asset_path: ./examples.zip
           asset_name: examples.zip
           asset_content_type: application/zip

.github/workflows/test-code.yaml

@@ -44,6 +44,8 @@ jobs:
     steps:
       - name: Checkout repository
         uses: actions/checkout@v4
+        with:
+          fetch-depth: '0'  # full history with tags to get the version number by versioningit
 
       - name: Set up Python
         uses: actions/setup-python@v5
@@ -83,6 +85,8 @@ jobs:
     steps:
     - name: Checkout repository
       uses: actions/checkout@v4
+      with:
+        fetch-depth: '0'  # full history with tags to get the version number by versioningit
 
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v5

.github/workflows/test-ipynb.yml

@@ -34,6 +34,8 @@ jobs:
     steps:
       - name: Checkout repository
         uses: actions/checkout@v4
+        with:
+          fetch-depth: '0'  # full history with tags to get the version number by versioningit
 
       - name: Set up Python
         uses: actions/setup-python@v5
@@ -69,6 +71,8 @@ jobs:
     steps:
       - name: Check-out repository
         uses: actions/checkout@v4
+        with:
+          fetch-depth: '0'  # full history with tags to get the version number by versioningit
 
       - name: Set up Python environment
         uses: actions/setup-python@v5

.github/workflows/test-package.yaml

@@ -51,6 +51,8 @@ jobs:
     steps:
     - name: Checkout repository
       uses: actions/checkout@v4
+      with:
+        fetch-depth: '0'  # full history with tags to get the version number by versioningit
 
     - name: Set up Python ${{ matrix.python-version }}
       uses: actions/setup-python@v5
@@ -104,6 +106,10 @@ jobs:
         name: EasyDiffractionLib_py${{ matrix.python-version }}_${{ matrix.os }}_${{ runner.arch }}
         path: .  # directory to extract downloaded zipped artifacts
 
+    # The local version must be higher than the PyPI version for pip to
+    # prefer the local version. So, after a new release and a new tag,
+    # remember to merge the master branch with the develop branch,
+    # and then create a new feature branch from the develop branch.
     - name: Install Python package from previous job with 'dev' extras
       run: pip install 'easydiffraction[dev]' --find-links=dist
 
@@ -117,7 +123,7 @@ jobs:
       shell: bash
       run: >
         pytest 
-        --nbmake examples 
+        --nbmake examples/
         --ignore-glob='examples/*emcee*'
         --nbmake-timeout=300 
         --color=yes

.github/workflows/verify-pr-labels.yml

@@ -1,4 +1,5 @@
-# Verifies if a pull request has at least one label from a set of valid labels before it can be merged.
+# Verifies if a pull request has at least one label from a set of valid
+# labels before it can be merged.
 
 name: Verify pull request labels
 
@@ -18,6 +19,6 @@ jobs:
       uses: jesusvasquez333/[email protected]
       with:
         github-token: ${{ secrets.GITHUB_TOKEN }}
-        valid-labels: chore, fix, bugfix, bug, enhancement, feature, dependencies, documentation
+        valid-labels: 'enhancement, bug, chore, documentation, refactor'
         pull-request-number: ${{ github.event.pull_request.number }}
         disable-reviews: false

DEVELOPMENT.md

@@ -1,6 +1,6 @@
 # Development
 
-This is an example of a workflow that describes the development process. 
+This is an example of a workflow that describes the development process.
 
 * Clone EasyDiffractionLib repository
   ```console
@@ -27,18 +27,19 @@ This is an example of a workflow that describes the development process.
   ```console
   python -m pip install --upgrade pip
   ```
-* Install easydiffraction from root with `dev` extras for development 
+* Install easydiffraction from root with `dev` extras for development
   ```console
   pip install '.[dev]'
   ```
 * Make changes in the code
 * Run Ruff - Python linter and code formatter (configuration is in pyproject.toml)
   ```console
-  ruff check . --fix 
+  ruff check --fix  # Linting (overwriting files)
+  ruff format       # Formatting (overwriting files)
   ```
 * Run python tests
   ```console
-  pytest tests/ --color=yes -n auto 
+  pytest tests/ --color=yes -n auto
   ```
 * Clear all Jupyter notebooks output
   ```console

docs/dev/diagrams/classes.md

@@ -0,0 +1,95 @@
+* src
+  * easydiffraction (as before)
+    * calculators (as before)
+      * cryspy
+        * calculator.py (instead of calculators/cryspy.py)
+          * class Cryspy (as before)
+        * wrapper.py (instead of Interfaces/cryspyV2.py)
+          * class CryspyWrapper (instead of CryspyV2)
+      * pdffit2
+        * calculator.py (instead of calculators/pdffit2.py)
+          * class Pdffit2 (as before)
+        * wrapper.py (instead of Interfaces/pdffit2.py)
+          * class Pdffit2Wrapper (instead of Pdffit2)
+      * pycrysfml
+        * calculator.py (instead of calculators/CFML.py)
+          * class Pycrysfml (instead of CFML)
+        * wrapper.py (instead of Interfaces/CFML.py)
+          * class PycrysfmlWrapper (instead of CFML)
+      * \_\_init\_\_.py
+      * calculator_base.py (no equivalent in old API? Do we need it?)
+        * class CalculatorBase (no equivalent in old API? Do we need it?)
+      * wrapper_factory.py ??? (instead of interface.py)
+        * class WrapperFactory ??? (instead of InterfaceFactory)
+      * wrapper_base.py (instead of Interfaces/interfaceTemplate.py)
+        * class WrapperBase (instead of InterfaceTemplate)
+
+
+
+
+## Old structure
+
+* easydiffraction - As before
+  * calculators - As before
+    * \_\_init\_\_.py - As before
+    * CFML.py - Move to src/easydiffraction/calculators/pycrysfml/calculator.py
+      * class CFML - Rename to Pycrysfml
+    * cryspy.py - Move to src/easydiffraction/calculators/cryspy/calculator.py
+      * class Cryspy - As before
+    * GSASII.py - Remove
+    * pdffit2.py - Move to src/easydiffraction/calculators/pdffit2/calculator.py
+      * class Pdffit2 - As before
+  * components - Temporarily as before
+    * \_\_init\_\_.py
+    * phase.py
+    * polarization.py
+    * site.py
+  * elements - Temporarily as before
+    * Backgrounds
+      * \_\_init\_\_.py
+      * Background.py
+      * Factorial.py
+      * Point.py
+    * Experiments
+      * \_\_init\_\_.py
+      * Experiment.py
+      * Pattern.py
+    * \_\_init\_\_.py
+  * Interfaces
+    * \_\_init\_\_.py
+    * CFML.py - Move to src/easydiffraction/calculators/pycrysfml/wrapper.py
+      * class CFML - Rename to PycrysfmlWrapper
+    * cryspy.py - Remove
+    * cryspyV2.py - Move to src/easydiffraction/calculators/cryspy/wrapper.py
+      * class CryspyV2 - Rename to CryspyWrapper
+    * GSASII.py - Remove
+    * interfaceTemplate.py - Move to src/easydiffraction/calculators/wrapper_base.py
+      * class InterfaceTemplate - Rename to WrapperBase
+    * pdffit2.py - Move to src/easydiffraction/calculators/pdffit2/wrapper.py
+      * class Pdffit2 - Rename to Pdffit2Wrapper
+    * types.py
+  * io - Remove
+    * \_\_init\_\_.py - Remove
+    * cif.py - Remove
+    * cif_reader.py - From EDB. Move to EasyCrystallography
+    * cryspy_parser.py - Move to calculators/cryspy/...
+    * download.py - Remove
+      * def download_from_repository -> Move to easydiffraction/utils.py
+    * helpers.py - Remove
+  * Profiles - Temporarily as before
+    * \_\_init\_\_.py
+    * Analysis.py
+    * common.py
+    * Container.py
+    * Experiment.py
+    * JobType.py
+    * P1D.py
+    * Sample.py
+  * \_\_init\_\_.py - As before
+  * interfaces.py - Move to src/easydiffraction/calculators/wrapper_factory.py
+    * class InterfaceFactory - Rename to WrapperFactory
+  * Job.py - Rename to job.py
+  * Jobs.py - Remove
+  * main.py - As before
+  * Runner.py - Remove
+  * sample.py - Temporarily as before