Absorb astronomy example + projects/cosmology from autofit_workspace

projects/README.rst

@@ -0,0 +1,7 @@
+The ``projects`` contains example projects which use **PyAutoFit** to perform modeling.
+
+They show how one would build a structure a project around **PyAutoFit** to best use its tools for model-composition, model-fitting, visualization, etc.
+
+They also demonstrate advanced functionality using real-world examples.
+
+- ``cosmology``: Example **PyAutoFit** project using a model-fitting example from Astronomy. This highlights **multi-level models**.

projects/cosmology/config/.gitignore

@@ -0,0 +1 @@
+!.gitignore

projects/cosmology/config/README.rst

@@ -0,0 +1,15 @@
+The ``config`` folder contains configuration files which customize default **PyAutoLens**.
+
+Folders
+-------
+
+- ``priors``: Configs defining default priors assumed on every model component and set of parameters.
+
+Files
+-----
+
+- ``general.yaml``: Customizes general **PyAutoLens** settings.
+- ``non-linear.yaml``: Configs for default non-linear search (e.g. MCMC, nested sampling) settings.
+- ``logging.yaml``: Customizes the logging behaviour of **PyAutoLens**.
+- ``visualize.yaml``: Configs defining what images are output by a lens model fit.
+- ``notation.yaml``: Configs defining labels and formatting of model parameters when used for visualization.

projects/cosmology/config/general.yaml

@@ -0,0 +1,33 @@
+updates:
+  iterations_per_quick_update: 1e99 # Non-linear search iterations between every quick update, which just displays the maximum likelihood model fit.
+  iterations_per_full_update: 1e99  # Non-linear search iterations between every full update, which outputs all visuals and result fits (e.g. model.result, search.summary), this exits the search and can be slow.  
+hpc:
+  hpc_mode: false                   # If True, use HPC mode, which disables GUI visualization, logging to screen and other settings which are not suited to running on a super computer.
+  iterations_per_quick_update: 10000 #  Non-linear search iterations between every quick update, which just displays the maximum likelihood model fit.  
+  iterations_per_full_update: 1e99  # Non-linear search iterations between every full update, which outputs all visuals and result fits (e.g. model.result, search.summary), this exits the search and can be slow.
+inversion:
+  check_reconstruction: true        # If True, the inversion's reconstruction is checked to ensure the solution of a meshs's mapper is not an invalid solution where the values are all the same.
+  reconstruction_vmax_factor: 0.5   # Plots of an Inversion's reconstruction use the reconstructed data's bright value multiplied by this factor.  
+output:
+  force_pickle_overwrite: false     #   force_pickle_overwrite: false     # If True, pickle files output by a search (e.g. samples.pickle) are recreated when a new model-fit is performed.
+  force_visualize_overwrite: false # If True, visualization images output by a search (e.g. subplots of the fit) are recreated when a new model-fit is performed.
+  info_whitespace_length: 80        # Length of whitespace between the parameter names and values in the model.info / result.info
+  log_level: INFO                   # The level of information output by logging.
+  log_to_file: false                # If True, outputs the non-linear search log to a file (and not printed to screen).
+  log_file: output.log              # The name of the file the logged output is written to (in the non-linear search output folder)
+  model_results_decimal_places: 3   # Number of decimal places estimated parameter values / errors are output in model.results.
+  remove_files: false               # If True, all output files of a non-linear search (e.g. samples, visualization, etc.) are deleted once the model-fit has completed, such that only the .zip file remains.
+  search_internal : false    # If True, the search internal folder which contains a saved state of the non-linear search is delete, saving on hard-disk space.
+  samples_to_csv: true              # If True, non-linear search samples are written to a .csv file.
+  unconverged_sample_size : 100     # If outputting results of an unconverged search, the number of samples used to estimate the median PDF values and errors.
+parallel:
+  warn_environment_variables: true  # If True, a warning is displayed when the search's number of CPU > 1 and enviromment variables related to threading are also > 1.
+
+profiling:
+  parallel_profile: false           # If True, the parallelization of the fit is profiled outputting a cPython graph.
+  should_profile: false             # If True, the ``profile_log_likelihood_function()`` function of an analysis class is called throughout a model-fit, profiling run times.
+  repeats: 1                        # The number of repeat function calls used to measure run-times when profiling.
+test:
+  check_preloads: false
+  exception_override: false
+  parallel_profile: false

projects/cosmology/config/logging.yaml

@@ -0,0 +1,20 @@
+version: 1
+disable_existing_loggers: false
+handlers:
+  console:
+    class: logging.StreamHandler
+    level: INFO
+    stream: ext://sys.stdout
+    formatter: formatter
+  file:
+    class: logging.FileHandler
+    level: INFO
+    filename: root.log
+    formatter: formatter
+root:
+  level: INFO
+  handlers:
+  - console
+formatters:
+  formatter:
+    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'

projects/cosmology/config/non_linear/GridSearch.yaml

@@ -0,0 +1,5 @@
+# The settings of a parallelized grid search of non-linear searches.
+
+parallel:
+  number_of_cores: 3 # The number of cores the search is parallelized over by default, using Python multiprocessing.
+  step_size: 0.1     # The default step size of each grid search parameter, in terms of unit values of the priors.

projects/cosmology/config/non_linear/README.rst

@@ -0,0 +1,9 @@
+The ``non_linear`` folder contains configuration files which customize the default behaviour of non-linear searches in
+**PyAutoLens**.
+
+Files
+-----
+
+- ``mcmc.yaml``: Settings default behaviour of MCMC non-linear searches (e.g. Emcee).
+- ``nest.yaml``: Settings default behaviour of nested sampler non-linear searches (e.g. Dynesty).
+- ``mle.yaml``: Settings default behaviour of maximum likelihood estimator (mle) searches (e.g. PySwarms).

projects/cosmology/config/notation.yaml

@@ -0,0 +1,41 @@
+# The notation configs define the labels of every model parameter and its derived quantities, which are used when
+# visualizing results (for example labeling the axis of the PDF triangle plots output by a non-linear search).
+
+
+# label: The label given to the each parameter, for plots like PDF corner plots.
+
+# For example, if `centre=x`, the plot axis will be labeled 'x'.
+
+
+# superscript: the superscript used on certain plots that show the results of different model-components.
+
+# For example, if `Gaussian=g`, plots where the parameters of the Gaussian model-component have superscript `g`.
+
+label:
+  label:
+    sigma: \sigma
+    centre: x
+    intensity: norm
+    parameter0: a
+    parameter1: b
+    parameter2: c
+    rate: \lambda
+  superscript:
+    Exponential: e
+    Gaussian: g
+    ModelComponent0: M0
+    ModelComponent1: M1
+
+# label_format: The format certain parameters are output as in output files like the `model.results` file.
+
+# For example, if `centre={:.2d}`, the format of the centre parameter in results files will use this Python format.
+
+label_format:
+  format:
+    sigma: '{:.2f}'
+    centre: '{:.2f}'
+    intensity: '{:.2f}'
+    parameter0: '{:.2f}'
+    parameter1: '{:.2f}'
+    parameter2: '{:.2f}'
+    rate: '{:.2f}'

projects/cosmology/config/priors/galaxy.yaml

@@ -0,0 +1,11 @@
+Redshift:
+  redshift:
+    type: Uniform
+    lower_limit: 0.0
+    upper_limit: 3.0
+    width_modifier:
+      type: Absolute
+      value: 1.0
+    limits:
+      lower: 0.0
+      upper: inf

projects/cosmology/config/priors/light_profiles.yaml

@@ -0,0 +1,61 @@
+LightDevVaucouleurs:
+  centre_0:
+    type: Gaussian
+    mean: 0.0
+    sigma: 0.3
+    width_modifier:
+      type: Absolute
+      value: 0.05
+    limits:
+      lower: -inf
+      upper: inf
+  centre_1:
+    type: Gaussian
+    mean: 0.0
+    sigma: 0.3
+    width_modifier:
+      type: Absolute
+      value: 0.05
+    limits:
+      lower: -inf
+      upper: inf
+  axis_ratio:
+    type: Uniform
+    lower_limit: 0.2
+    upper_limit: 1.0
+    width_modifier:
+      type: Relative
+      value: 1.0
+    limits:
+      lower: 0.0
+      upper: inf
+  angle:
+    type: Uniform
+    lower_limit: 0.0
+    upper_limit: 180.0
+    width_modifier:
+      type: Relative
+      value: 1.0
+    limits:
+      lower: 0.0
+      upper: inf
+  effective_radius:
+    type: Uniform
+    lower_limit: 0.0
+    upper_limit: 30.0
+    width_modifier:
+      type: Relative
+      value: 1.0
+    limits:
+      lower: 0.0
+      upper: inf
+  intensity:
+    type: LogUniform
+    lower_limit: 1.0e-06
+    upper_limit: 1000000.0
+    width_modifier:
+      type: Relative
+      value: 0.5
+    limits:
+      lower: 0.0
+      upper: inf

projects/cosmology/config/priors/mass_profiles.yaml

@@ -0,0 +1,51 @@
+MassIsothermal:
+  centre_0:
+    type: Gaussian
+    mean: 0.0
+    sigma: 0.3
+    width_modifier:
+      type: Absolute
+      value: 0.05
+    limits:
+      lower: -inf
+      upper: inf
+  centre_1:
+    type: Gaussian
+    mean: 0.0
+    sigma: 0.3
+    width_modifier:
+      type: Absolute
+      value: 0.05
+    limits:
+      lower: -inf
+      upper: inf
+  axis_ratio:
+    type: Uniform
+    lower_limit: 0.2
+    upper_limit: 1.0
+    width_modifier:
+      type: Relative
+      value: 1.0
+    limits:
+      lower: 0.0
+      upper: inf
+  angle:
+    type: Uniform
+    lower_limit: 0.0
+    upper_limit: 180.0
+    width_modifier:
+      type: Relative
+      value: 1.0
+    limits:
+      lower: 0.0
+      upper: inf
+  mass:
+    type: Uniform
+    lower_limit: 0.0
+    upper_limit: 4.0
+    width_modifier:
+      type: Relative
+      value: 1.0
+    limits:
+      lower: 0.0
+      upper: inf

projects/cosmology/config/visualize.yaml

@@ -0,0 +1,28 @@
+general:
+  general:
+    backend: default         # The matploblib backend used for visualization. `default` uses the system default, can specifiy specific backend (e.g. TKAgg, Qt5Agg, WXAgg).
+plots_search:
+  dynesty:
+    corner: true             # Output corner figure (using anestetic) during a non-linear search fit?
+    cornerpoints: false      # Output Dynesty cornerpoints figure during a non-linear search fit?
+    runplot: true            # Output Dynesty runplot figure during a non-linear search fit?
+    traceplot: true          # Output Dynesty traceplot figure during a non-linear search fit?
+  emcee:
+    corner: true             # Output corner figure (using corner.py) during a non-linear search fit?
+    likelihood_series: true  # Output likelihood series figure during a non-linear search fit?
+    time_series: true        # Output time series figure during a non-linear search fit?
+    trajectories: true       # Output trajectories figure during a non-linear search fit?
+  pyswarms:
+    contour: true            # Output contour figure during a non-linear search fit?
+    cost_history: true       # Output cost_history figure during a non-linear search fit?
+    time_series: true        # Output time_series figure during a non-linear search fit?
+    trajectories: true       # Output trajectories figure during a non-linear search fit?
+  ultranest:
+    corner: true             # Output corner figure (using anestetic) during a non-linear search fit?
+    runplot: true            # Output Ultranest runplot figure during a non-linear search fit?
+    traceplot: true          # Output Ultranest traceplot figure during a non-linear search fit?
+  zeus:
+    corner: true             # Output corner figure (using corner.py) during a non-linear search fit?
+    likelihood_series: true  # Output likelihood series figure during a non-linear search fit?
+    time_series: true        # Output time series figure during a non-linear search fit?
+    trajectories: true       # Output trajectories figure during a non-linear search fit?