Skip to content

Ability to pass an environment file when running a study #474

@nicholaslivingstone

Description

@nicholaslivingstone

Desired Enhancement

It'd be nice to have the ability pass an environment file to fill in environment variables rather than modifying the pure spec. Something like:

maestro run --env-file .env my_spec.yml

Motivation

I'm currently crafting spec files that will be shared with members of my group ideally for reproducibility via a github repo. There are some environment variables that I'm setting in the spec that should be fixed, but others that will be user specific. For example, here's a small example from one of the current specs:

env:
  variables:
    # Fixed Configuration that shouldn't be changed
    CALI_CONFIG: "runtime-profile,output=amg_caliper_data.cali"
    MPICH_GPU_SUPPORT_ENABLED: 1
    OUTPUT_PATH: studies/
    
    # Environment Paths that need to be set based on user
    SPACK_ENV:  # <-- Path to user's pack environment
    PYTHON_ENV: # <-- Path to user's target python environment

I know the trivial solution would just be to tell users to only modify that section of the spec. But I'd argue it'd be cleaner to provide a .env.example in my repo containing the necessary variables which can be changed to .env be passed like the initial example.

SPACK_ENV=/path/to/my/spack/env
PYTHON_ENV=/path/to/my/python/env

That way it's clear which variables should and shouldn't be changed by the user, and the .env file can be ignored by git, preventing the likelihood of accidentally pushing a spec with hard-coded user specific paths.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions