Refactor core objects, improve validation, and update API (#106)

* Simplifies branch processing and installs badgery

* Adds flag for unit test coverage in .badgery.yaml

* Updates dependencies and easydiffraction shortcuts

* Adds dashboard build trigger post-tutorial tests

* Updates tutorial command names in workflow script

* Simplifies installation by using all extras

* Adds dashboard build trigger after PyPI tests

* Refactors sample model creation architecture

* Updates sample model API in tests

* Updates sample model API in tutorials

* Refactors argument validation and simplifies CIF reading

* Update SampleModel API and adjust linked phases check

* Increase McCabe complexity limit and add dependency

* Fixes dev-install command in pixi.toml

* Adds centralized logging utility

* Adds function to parse CIF-style numeric strings to ufloat

* Refactors core object model for improved structure

* Refactors ID attributes to use entry_name consistently

* Refactors alias and constraint classes

* Refactors core objects to enhance category handling

* Refactors core components and structures for enhanced clarity

* Adds CIF alias utility functions to enhance data handling

* Add test suite for SampleModels components and fix imports

* Updates package versions in dependency lock file

* Introduce allowed attributes and update collection initialization

* Adds allowed attributes to mixins for parameter handling

* Refactors CIF conversion by removing method calls

* Enhances attribute validation in CryspyCalculator

* Allow editing of uncertainty and free attributes

* Refines CIF data parsing and handling logic

* Modifies model initialization and linking

* Refines instrument and peak attribute handling

* Fixes bug in parameter setting logic

* Refactors parameter attributes and access methods

* Enhances background and model flexibility

* Rename error handling methods and adjust datablock management

* Refactors entry name handling in core classes

* Standardizes 'entry_name' to 'category_entry_name'

* Enhances sample model and plotting configuration

* Enhances item management in CategoryCollection

* Adds allowed attributes to SingleCrystalExperiment

* Replaces human-readable ID generation with property access

* Comments out unit test step from pre-push hook

* Disables physical parameter bounds

* Refines physical constraints and simplifies code

* Refactors method calls to use add_from_args for clarity

* Refactors method calls and updates numeric constants

* Updates parameter bound handling and method naming

* Refactors method calls to improve clarity

* Refines attribute access and updates test methods

* Refactors item addition in CategoryCollection

* Refactors unit tests to use full_cif_names

* Refactors parameter access and method calls

* Refines attribute naming for clarity in background model

* Adjusts default renderer for PyCharm environment

* Updates attribute checks, adds data files, renames scripts

* Adds extra test configuration and refines unit tests

* Refactors attribute access and control logic

* Enhances logging and error handling mechanism

* Simplifies error handling in logging with 'raise from'

* Refactors core objects for modular design

* Improves logging behavior during pytest

* Simplifies py-format-fix command

* Refactors parameter descriptor hierarchy

* Enhances logging and documents parameter abstraction

* Refactors test cases to align with new API and data structures

* Cleans up test imports and simplifies assertions

* Refactors singleton tests for consistency

* Refines descriptor/parameter handling logic

* First attempt to add CifMixin to parameter classes

* Updates CifMixin.cif_uid to use full_name

* Refactors parameter abstraction layer

* Enhances Logger configuration with environment variables

* Refactors lazy loading in __init__.py for efficiency

* Consolidates attribute guard mixins and refactors logic

* Move mixins to guarded base class

* Refactors core component architecture

* Replace enforce_type with typeguard for runtime type checking

* Remove legacy signature from add method and update tests

* Enhances category item renaming logic (ugly temporary workaround)

* Refactors attribute handling in parameter classes

* Enhances Project class with type-checking and encapsulation

* Refines __str__ method and fixes super call

* Enhances collection handling by implementing AbstractCollection

* Enhances type safety with generics for collections

* Add SPDX license headers to source files

* Initialize base class in CollectionBase constructor

* Refactors parameter access for consistency

* Removes unnecessary Generic typing

* Work in progress

* Enhances identity resolution and string representation

* Improves parameter retrieval in collections

* Adds `full_name` property to multiple classes

* Simplifies attribute handling in GuardedBase

* Renames full_name property to unique_name

* Refines validation logging messages

* Adds configurable error reaction in Logger

* Enhances type error diagnostics in checktype decorator

* Giant refactoring

* Refactors parameter handling in FitResults

* Refactors descriptor properties and enhances validation

* Removes unused TODO comments and corrects assertions

* Adds SPDX license headers to source files

* Enhances space group validation logic

* Refactors validation logic for atom site properties

* Adds missing newline in atom_sites.py

* Refactors CIF conversion and updates test format

* Reformats code for consistency and clarity

* Refines parameter handling and improves validation

* Refactors tests to improve readability and maintainability

* Enhances atom site configuration in tutorials

* Enhances x-axis data selection for plotting

* Refactors atom site addition for clarity and consistency

* Fixes parameter limit handling

* Fixes attribute assignment and model access

* Refactors data extraction with list comprehensions

* Refactors constraint handling and data access methods

* Enhances peak profile type handling in experiments

* Removes unused public attributes method

* Improves logging settings and Python path resolution

* Refines validators with enhanced logging

* Adds parameter validation and logging enhancements

* Replaces diagnostics and validators with an enhanced implementation

* Adds Diagnostics for Type Override and Enhances Collections

* Enhances validation and logging mechanisms

* Refactors validator and identity handling

* Replaces RegexValidator with AttributeSpec for validation

* Maintain parent linkage for nested objects

* Corrects attribute access in identity resolution

* Refactors validation types to use DataTypes enum

* Refines type validation logic in diagnostics

* Replace UUID with secure random UID generation

* Refines atom site validations and logging

* Enhances validation logic and diagnostics handling

* Relocates CifHandler to new IO module for clarity

* Refactors experiment enums structure

* Updates peak mixin imports for better organization

* Refactors instrument setup for improved modularity

* Refactors background module implementation

* Refactors CIF serialization methods

* Refactors CIF serialization to centralized functions

* Removes obsolete CIF conversion methods

* Restructures experiment type classes for modular design

* Refactors instrument handling and restructures sample models

* Reorganizes import paths to category_items

* Refactors component naming for better clarity

* Refactors code to categorize collections

* Reorganizes import paths to use category_collections modules

* Refactors background type handling for clarity

* Refactors background handling and reorganizes enums

* Refactors background and instrument modules organization

* Refactors instrument and peak base class imports

* Refactors argument validation in ExperimentFactory

* Refactors import paths for consistency

* Refactors datastore architecture for segregation of types

* Fixes property getter for uncertainty

* Add and update SPDX license headers across files

* Adds package structure documentation generator

* Renames datastore type modules for clarity

* Refactors package structure for clarity

* Renames and restructures package directories for clarity

* Renames core modules for consistency

* New unit tests

* Refactors factory method usage in test suite

* Add unit tests for various components in easydiffraction

* Adds unit test for crystallographic and experimental data

* Enables macOS h5py dependency for validation

* Replaces scaffold tests with concrete test implementations

* Replaces test scaffolds with concrete tests

* Adds comprehensive test coverage for core modules

* Format tests

* Enhances SPDX header update for tests directory

* Cleans up tests and refines pytest configuration

* Refines config and reorganizes files for tmp exclusion

* Adds comprehensive docstrings to Analysis classes and descriptors

* Enhances experiment class documentation

* Updates .gitignore for test and linter caches

* Removes Python cache before packaging artifacts

* Updates python_socketio and mkdocs-jupyter dependencies

* Adds multiple Python cache removal steps

* Adds dev-install step in GitHub workflow

* Removes redundant dev-install command from workflow

* Reorganizes dependency configuration

* Refactors Python cache removal in workflow

* Refactors to use Factory classes for model and experiment creation

* Enhances enums with descriptive methods

* Enhances documentation with docstrings

* Add missing documentation for core modules

* Adds docstrings for improved code documentation

* Updates pytest configuration for test management

* Refactors CryspyCalculator and updates SPDX compliance

* Remove commented code and add documentation placeholders

* Adds environment variable for CI branch

Author: AndrewSazonov

.badgery.yaml

@@ -52,6 +52,7 @@ cards:
   - group: Coverage
     type: codecov
     title: Unit test coverage (Codecov)
+    flag: unittests
     enabled: true
 
   - group: Coverage

.github/workflows/dashboard.yaml

@@ -41,33 +41,34 @@ jobs:
 
       - name: Install and setup development dependencies
         shell: bash
-        run: pixi run dev
+        run: |
+          pixi run dev
+          pixi add --pypi --git https://github.com/enhantica/badgery badgery
 
       - name: Run docstring coverage and code complexity/maintainability checks
         run: |
           for BRANCH in ${{ env.DEFAULT_BRANCH }} ${{ env.DEVELOP_BRANCH }} ${{ env.CI_BRANCH }}; do
             echo "=== Processing branch $BRANCH ==="
-            if [ -d "../worktree-$BRANCH" ]; then
+            if [ -d "../$BRANCH" ]; then
               echo "Branch $BRANCH already processed, skipping"
               continue
             fi
-            git worktree add ../worktree-$BRANCH origin/$BRANCH
+            git worktree add ../$BRANCH origin/$BRANCH
             mkdir -p reports/$BRANCH
             echo "Docstring coverage for branch $BRANCH"
-            pixi run interrogate -c pyproject.toml --fail-under=0 ../worktree-$BRANCH/src > reports/$BRANCH/coverage-docstring.txt
+            pixi run interrogate -c pyproject.toml --fail-under=0 ../$BRANCH/src > reports/$BRANCH/coverage-docstring.txt
             echo "Cyclomatic complexity for branch $BRANCH"
-            pixi run radon cc -s -j ../worktree-$BRANCH/src --exclude ../worktree-$BRANCH/src/easydiffraction/crystallography/space_group_lookup_table.py > reports/$BRANCH/cyclomatic-complexity.json
+            pixi run radon cc -s -j ../$BRANCH/src > reports/$BRANCH/cyclomatic-complexity.json
             echo "Maintainability index for branch $BRANCH"
-            pixi run radon mi -j ../worktree-$BRANCH/src --exclude ../worktree-$BRANCH/src/easydiffraction/crystallography/space_group_lookup_table.py > reports/$BRANCH/maintainability-index.json
+            pixi run radon mi -j ../$BRANCH/src > reports/$BRANCH/maintainability-index.json
             echo "Raw metrics for branch $BRANCH"
-            pixi run radon raw -s -j ../worktree-$BRANCH/src --exclude ../worktree-$BRANCH/src/easydiffraction/crystallography/space_group_lookup_table.py > reports/$BRANCH/raw-metrics.json
+            pixi run radon raw -s -j ../$BRANCH/src > reports/$BRANCH/raw-metrics.json
           done
 
       - name: Generate dashboard HTML
-        run: |
-          pixi add --pypi --git https://github.com/enhantica/badgery badgery
-          pixi run pip show badgery
-          pixi run python -m badgery --config .badgery.yaml --repo ${{ github.repository }} --branch ${{ env.CI_BRANCH }} --output index.html
+        run: >
+          pixi run python -m badgery --config .badgery.yaml --repo ${{
+          github.repository }} --branch ${{ env.CI_BRANCH }} --output index.html
 
       - name: Prepare publish directory
         run: |

.github/workflows/pypi-test.yaml

@@ -20,6 +20,7 @@ env:
   DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
 
 jobs:
+  # Job 1: Test installation from PyPI on multiple OS
   pypi-package-tests:
     strategy:
       matrix:
@@ -54,10 +55,8 @@ jobs:
       - name: Create the environment and install dependencies
         run: pixi install
 
-      - name: Install package from PyPI with dev and visualization extras
-        run: |
-          pixi add --pypi "easydiffraction[dev,visualization]"
-          pixi run easydiffraction --version
+      - name: Install package from PyPI with all extras
+        run: pixi add --pypi "easydiffraction[all]"
 
       - name: Run unit tests to verify the installation
         run: pixi run unit-tests
@@ -70,8 +69,31 @@ jobs:
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
-          pixi run tutorials-list
-          pixi run tutorials-fetch
+          pixi run easydiffraction --version
+          pixi run easydiffraction list-tutorials
+          pixi run easydiffraction fetch-tutorials
 
       - name: Test tutorials as notebooks
         run: pixi run notebook-tests
+
+  # Job 2: Trigger dashboard build
+  dashboard-build-trigger:
+    needs: pypi-package-tests
+
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check-out repository
+        uses: actions/checkout@v5
+
+      - name: Trigger dashboard build
+        uses: actions/github-script@v7
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            await github.rest.actions.createWorkflowDispatch({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              workflow_id: "dashboard.yaml",
+              ref: "${{ env.CI_BRANCH }}"
+            });

.github/workflows/test.yaml

@@ -147,6 +147,10 @@ jobs:
         shell: bash
         run: pixi remove --pypi easydiffraction
 
+      - name: Remove Python cache files before uploading
+        shell: bash
+        run: pixi run clean-pycache
+
       # More than one file/dir need to be specified in 'path', to preserve the
       # structure of the dist/ directory, not only its contents.
       - name: Upload Python package for the next job
@@ -156,6 +160,7 @@ jobs:
           path: |
             dist/
             tests/
+            pytest.ini
             pixi.toml
             pixi.lock
           if-no-files-found: 'error'

.github/workflows/tutorial-tests.yaml

@@ -22,7 +22,12 @@ concurrency:
     ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
   cancel-in-progress: true
 
+# Set the environment variables to be used in all jobs defined in this workflow
+env:
+  CI_BRANCH: ${{ github.head_ref || github.ref_name }}
+
 jobs:
+  # Job 1: Test tutorials as scripts and notebooks on multiple OS
   tutorial-tests:
     strategy:
       fail-fast: false
@@ -60,3 +65,25 @@ jobs:
       - name: Test tutorials as notebooks
         shell: bash
         run: pixi run notebook-tests
+
+  # Job 2: Trigger dashboard build
+  dashboard-build-trigger:
+    needs: tutorial-tests
+
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Check-out repository
+        uses: actions/checkout@v5
+
+      - name: Trigger dashboard build
+        uses: actions/github-script@v7
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            await github.rest.actions.createWorkflowDispatch({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              workflow_id: "dashboard.yaml",
+              ref: "${{ env.CI_BRANCH }}"
+            });

.gitignore

@@ -4,9 +4,18 @@ __pycache__
 .coverage
 .pyc
 
+# PyTest
+.pytest_cache
+
+# MyPy
+.mypy_cache
+
 # Pixi
 .pixi
 
+# Ruff
+.ruff_cache
+
 # Jupyter Notebooks
 .ipynb_checkpoints
 

.pre-commit-config.yaml

@@ -49,9 +49,9 @@ repos:
         pass_filenames: false
         stages: [pre-push]
 
-      - id: pixi-unit-tests
-        name: pixi run unit-tests
-        entry: pixi run unit-tests
-        language: system
-        pass_filenames: false
-        stages: [pre-push]
+      #- id: pixi-unit-tests
+      #  name: pixi run unit-tests
+      #  entry: pixi run unit-tests
+      #  language: system
+      #  pass_filenames: false
+      #  stages: [pre-push]

.prettierignore

@@ -1,2 +1,6 @@
 # Pixi
+.pixi
 pixi.lock
+
+# MyPy
+.mypy_cache