Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

42 dev config #43

Draft
wants to merge 72 commits into
base: main
Choose a base branch
from
Draft

42 dev config #43

wants to merge 72 commits into from

Conversation

JhanSrbinovsky
Copy link
Collaborator

@JhanSrbinovsky JhanSrbinovsky commented Feb 12, 2025

see issue #42


🚀 The latest prerelease access-esm1p6/pr43-18 at 94083d9 is here: #43 (comment) 🚀

@JhanSrbinovsky JhanSrbinovsky linked an issue Feb 12, 2025 that may be closed by this pull request
Copy link

The model version in the spack.yaml has not been updated.
Either update it manually, or comment the following to have it updated and committed automatically:

  • !bump major for feature releases
  • !bump minor for bugfixes

@JhanSrbinovsky
Copy link
Collaborator Author

@blimlim arrggghhh - why is this failing? its just a straight up copy of SpinUp_b which has always worked, including today.

@blimlim
Copy link
Contributor

blimlim commented Feb 12, 2025

Hi @JhanSrbinovsky!

The deployment system has different requirements depending on whether a pull request is marked as a draft or not, which I think is bringing up this error.

When a PR is not a draft, the deployment assumes that you're making a new release or new version of the model. In this case it requires the version number in

specs:
    - [email protected]_2024.12.0
...
projections:
          access-esm1p6: '{name}/dev_2024.12.0'

to be updated to something new.

For pull requests marked as drafts, this requirement doesn't apply and it will happily deploy prerelease executables without changes to the version number.

Is the goal to merge this PR into main? In this case, I'd try updating the version to dev_2025.02.0.

Otherwise if you're building some executables for testing, marking this PR as a draft should then let the build go ahead.

@JhanSrbinovsky JhanSrbinovsky marked this pull request as draft February 12, 2025 23:55
Copy link

The model version in the spack.yaml has not been updated.
Either update it manually, or comment the following to have it updated and committed automatically:

  • !bump major for feature releases
  • !bump minor for bugfixes

@blimlim
Copy link
Contributor

blimlim commented Feb 13, 2025

!redeploy

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr43-1 with commit 087b6f2

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-1 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-1

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-1 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2025.01.1

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2025.01.1

If this is not what was expected, commit changes to config/versions.json.

@JhanSrbinovsky JhanSrbinovsky self-assigned this Feb 13, 2025
@JhanSrbinovsky
Copy link
Collaborator Author

@blimlim - looks like the same error

@blimlim
Copy link
Contributor

blimlim commented Feb 13, 2025

Hi @JhanSrbinovsky, I think the deployment itself was successful despite the messages from github! I've been able to load the modules listed:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-1

I think the Some checks were not successful message is just carrying over from earlier – @CodeGat is this correct?

@CodeGat
Copy link
Member

CodeGat commented Feb 13, 2025

Yeah. The failing check is from the initial deployment of the commit. The !redeployed commit succeeded.

The reason you're still seeing the red cross is because GitHub treats previous failed checks as a higher priority than new successes...bit of a quirk of GitHub!

@blimlim
Copy link
Contributor

blimlim commented Feb 13, 2025

Thanks @CodeGat!

@JhanSrbinovsky
Copy link
Collaborator Author

Thanks for your help @blimlim & @CodeGat.

what did you to to get the redeploy to work @blimlim

@blimlim
Copy link
Contributor

blimlim commented Feb 13, 2025

Redeploying is quite simple thanks to @CodeGat! You just have to add a comment saying "!redeploy"

@CodeGat
Copy link
Member

CodeGat commented Feb 13, 2025

Indeed. Its main use case was for redeploying changes to a git.BRANCH component, but since that isn't working at the moment the isn't as much reason to use it!

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr43-2 with commit 8f70ab1

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-2 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-2

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-2 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2025.01.1

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2025.01.1

If this is not what was expected, commit changes to config/versions.json.

@JhanSrbinovsky
Copy link
Collaborator Author

!redeploy

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr43-3 with commit 8f70ab1

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-3 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-3

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-3 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2025.01.1

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2025.01.1

If this is not what was expected, commit changes to config/versions.json.

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr43-4 with commit f987b9e

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-4 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-4

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-4 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2025.01.1

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2025.01.1

If this is not what was expected, commit changes to config/versions.json.

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr43-5 with commit d2622bd

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-5 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-5

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-5 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2025.01.1

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2025.01.1

If this is not what was expected, commit changes to config/versions.json.

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr43-6 with commit 4ac0c6a

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-6 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-6

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-6 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2025.01.1

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2025.01.1

If this is not what was expected, commit changes to config/versions.json.

@JhanSrbinovsky JhanSrbinovsky mentioned this pull request Feb 19, 2025
@JhanSrbinovsky
Copy link
Collaborator Author

JhanSrbinovsky commented Feb 20, 2025

Active runs from dev-access-esm1,6 with varying runtime configs:
all have soil_thermal_fix =true

  1. LUH3restart - revised LUH3 based restart
  2. PI-case2e. - soil/veg params thru nmls
  3. PI-case-l_rev - L_rev_corr=true

@JhanSrbinovsky
Copy link
Collaborator Author

"LUH3 restart" output is AT

/scratch/p66/jxs599/access-esm/archive/dev-LUH3-restart-expt-5eed0c42

Copy link

🚀 Deploying access-esm1.6 dev_2024.12.0 as prerelease pr43-7 with commit 8083584

Details and usage instructions

This access-esm1.6 model will be deployed as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-7 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-7

where the binaries shall be on your $PATH.
This Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-7 environment.

🛠️ Using: spack 0.22, spack-packages 2024.12.0, spack-config 2025.02.1

Details

It will be deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2024.12.0
  • access-nri/spack-config version 2025.02.1

If this is not what was expected, commit changes to config/versions.json.

@CodeGat
Copy link
Member

CodeGat commented Mar 20, 2025

@JhanSrbinovsky - yeah, it is a bit annoying having to rebase due to a conflict when there is no intention to merge...unfortunately that's a restriction set by GitHub, where they don't run checks (and therefore deployments) if there is a merge conflict.

I can have a look at the spack logs to see what is being built - which commits/how many commits back do you want me to check?

@JhanSrbinovsky
Copy link
Collaborator Author

However, @blimlim - the conflicts may prevent the checks (and therefore the deployments). Might have to rebase and just accept this branches side of all the changes.
@CodeGat This has been a problem I have encountered and I have been unsure of whether main contains an appropriate update I should accept or whether it is the older version

@JhanSrbinovsky
Copy link
Collaborator Author

@JhanSrbinovsky you should be good to go once the latest deployment is complete

@blimlim Can you please let me know when this happens. The "conflict" on our end cant be resolved so we've decided to run side by side simulations (at least for a few days). Thanks

@blimlim
Copy link
Contributor

blimlim commented Mar 21, 2025

Hi @JhanSrbinovsky, the deployment after resolving the conflict is available here: #43 (comment)

@JhanSrbinovsky
Copy link
Collaborator Author

I've just learnt that @blimlim (who was helping me co-ordinate these things) is away today

@CodeGat - maybe we'll worry about checking the logs later. I honestly dont think there's a problem there. We are going to run comparable side-by-side simulations which should answer that question anyway. On the build side we wanted to be incorporating Pears'slatest ocean, Dave Bi's CICE update. I'm not sure of the status of the spack build and what I need to do on my end?

We were hoping to use the new sapphire (@manodeep was involved with this) rapids config. which I thought was being added to the default config? But when I payu clone-d yesterday I didnt pick up any notmalsr changes or anything?

@JhanSrbinovsky
Copy link
Collaborator Author

Hi @JhanSrbinovsky, the deployment after resolving the conflict is available here: #43 (comment)

Thanks @blimlim - I just heard that you were away?

Copy link

🚀 Attempted to deploy access-esm1p6 Prerelease pr43-15 with commit ede3f2a

🖥️ Gadi Deployment ✔️

Usage Instructions

This access-esm1.6 model will be deployed to Gadi as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-15 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-15

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-15 environment.

Configuration Information

This Prerelease is deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2025.03.002
  • access-nri/spack-config version 2025.02.2

If the above was not what was expected, commit changes to config/versions.json in this PR.

@JhanSrbinovsky
Copy link
Collaborator Author

oh no - how did a conflict re-emerge. Following @blimlim fixing the conflict, I git pull-ed the latest into my local copy. no conflicts. changed the commit hash/ ci/push and now there is a conflict again.

@dougiesquire
Copy link
Contributor

dougiesquire commented Mar 21, 2025

It's because there have been new changes to main that modify lines that you have also modified here. Am I correct that this PR is only for testing? If so, I don't think you need to worry too much about conflicts with main. As long as your branch (42-dev-config) has what you want, you're good. That said, I'm happy to resolve the current conflicts if you'd like?

ADDED: In case it wasn't clear: you didn't do anything wrong.

@JhanSrbinovsky
Copy link
Collaborator Author

yes it is for testing. Im fine with resolving the conflicts but the problem is that the differences are outside of the UM7 confines of my domain and not lines that I have ever modified. I didnt rebase from main either so IDK why the other side of the conflict is from main. At any rate main seems to have a later tag for cice or whatever which would be what I want - I think. I basically want to pick up the same Ocean, WOMBAT , tracers, ice etc that Pearse is using. I will update the UM7 location and continue a run from where he is now.

@dougiesquire
Copy link
Contributor

The warning is just telling you that there are conflicts between this branch and main that would need to be resolved if you wanted merge this branch into main (which you don't). The reason the comparison is being done to main is because the PR is set up as a pull request into main.

@pearseb could you please point us to the prerelease PR that you are using so that we can makes sure @JhanSrbinovsky has the right versions for non-UM components?

@JhanSrbinovsky
Copy link
Collaborator Author

OK thanks Dougie - makes sense.

@JhanSrbinovsky
Copy link
Collaborator Author

OK so 👍
From /scratch/p66/pjb581/access-esm/archive/pi_concentrations-expt-c55f7217/restart858/config.yaml

modules:
use:
- /g/data/vk83/prerelease/modules
load:
- access-esm1p6/pr31-3

So everything I need is here:

c214308

@JhanSrbinovsky
Copy link
Collaborator Author

JhanSrbinovsky commented Mar 21, 2025

it looks like my spack.yaml is already ok. I am now using (updated by @blimlim it must be) a commit hash to cice4 but I suspect this is actually DaveBi's branch head?" I cant see cice4 to verify

@dougiesquire
Copy link
Contributor

I am now using (updated by @blimlim it must be) a commit hash to cice4 but I suspect this is actually DaveBi's branch head?" I cant see cice4 to verify

Yup that's correct.

Copy link

🚀 Attempted to deploy access-esm1p6 Prerelease pr43-16 with commit 8b0536b

🖥️ Gadi Deployment ✔️

Usage Instructions

This access-esm1.6 model will be deployed to Gadi as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-16 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-16

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-16 environment.

Configuration Information

This Prerelease is deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2025.03.002
  • access-nri/spack-config version 2025.02.2

If the above was not what was expected, commit changes to config/versions.json in this PR.

@JhanSrbinovsky
Copy link
Collaborator Author

Thanks Dougie - do you know about the saphirerapids payu config?

@JhanSrbinovsky
Copy link
Collaborator Author

its OK - I think ive picked up the branch directly. It isnt merged to anywhere yet, but I think/hope it will work from here. comments suggest that there's a good chance :)

Copy link

🚀 Attempted to deploy access-esm1p6 Prerelease pr43-17 with commit 7c916b9

🖥️ Gadi Deployment ✔️

Usage Instructions

This access-esm1.6 model will be deployed to Gadi as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-17 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-17

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-17 environment.

Configuration Information

This Prerelease is deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2025.03.002
  • access-nri/spack-config version 2025.02.2

If the above was not what was expected, commit changes to config/versions.json in this PR.

@manodeep
Copy link

@JhanSrbinovsky Could you figure out the changes necessary to run on the sapphirerapids queue?

@JhanSrbinovsky JhanSrbinovsky deployed to Gadi Prerelease March 21, 2025 08:29 — with GitHub Actions Active
@JhanSrbinovsky
Copy link
Collaborator Author

JhanSrbinovsky commented Mar 21, 2025

@manodeep I think so. appears to be humming along anyway

payu clone https://github.com/ACCESS-NRI/access-esm1.6-configs -B preindustrial-sapphire-rapids

Actually there were some extra things I added/kept from my old config.yaml

Copy link

🚀 Attempted to deploy access-esm1p6 Prerelease pr43-18 with commit 94083d9

🖥️ Gadi Deployment ✔️

Usage Instructions

This access-esm1.6 model will be deployed to Gadi as:

  • dev_2024.12.0 as a Release (when merged).
  • pr43-18 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-esm1p6/pr43-18

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-esm1p6-pr43-18 environment.

Configuration Information

This Prerelease is deployed using:

  • access-nri/spack on branch 0.22
  • access-nri/spack-packages version 2025.03.002
  • access-nri/spack-config version 2025.02.2

If the above was not what was expected, commit changes to config/versions.json in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dev-config
7 participants