Skip to content

Commit 89afa74

Browse files
bmhan12victorapmBubuschDENEL Bertrand
authored
Spack 1.0.2 (#324)
* Update uberenv commit * Lock in main spack repo and spack-packages version & hash * Dockerfile, environment file, recipe changes for toss_4/Ubuntu/Rockylinux/pangea * Add uberenv_libs to .gitignore * Update Sherlock specs * Fix gcc version * Fill in missing cxx line for gcc@11 * use upstream hdf5 recipe * Generalize .gitignore for uberenv testing * Toolchain changes for toss 4 - CUDA is untested * toss 4 cray rocm toolchain changes - hypre+magma unexpected concretization failure to work through * Fix target arch * bump up build_jobs * Add --no-permissions flag * Update scripts/setupLC-TPL-uberenv-helper.bash Co-authored-by: Brian Han <[email protected]> * Fix ATS flags for Dane and Matrix * Add bison, flex, and gcc-runtime to toss_4_x86_64_ib * Disable Sherlock jobs * Add bison, flex, and ninja to toss_4_x86_64_ib_cray * Add --clean option to setupLC-TPL scripts * Add workarounds to get %llvm-amdgpu, simplify spec for rocm to fix magma issue * Use newer external blas - let spack build flex (scotch compilation failure otherwise) * Add gcc-runtime to toss_4_x86_64_ib_cray * Add ninja to toss_4_x86_64_ib * vtk recipe correction - depends_on c and cxx * Remove noAVX builds * Minor tweaks * Bug fix for running on compute node * Bug fix * Small fix - Use toolchain for Ubuntu clang images * upgrade pangea-4 config * Add back +unified-memory variant for hypre when +cuda or +rocm * Updated Pine config * clang --> llvm in LC uberenv bash script for chmod path --------- Co-authored-by: Victor A. P. Magri <[email protected]> Co-authored-by: Victor A. P. Magri <[email protected]> Co-authored-by: Gaetan FUSS <[email protected]> Co-authored-by: DENEL Bertrand <[email protected]>
1 parent bef2425 commit 89afa74

File tree

30 files changed

+1078
-1401
lines changed

30 files changed

+1078
-1401
lines changed

‎.github/workflows/docker_build_tpls.yml‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -67,20 +67,20 @@ jobs:
6767
DOCKER_ROOT_IMAGE: 7g8efcehpff/pangea-almalinux8-gcc9.4-openmpi4.1.2-cuda11.5.0-openblas0.3.18:4
6868
HOST_ARCH: ppc64le
6969
RUNS_ON: streak2
70-
- name: Sherlock CPU (centos 7.9.2009, gcc 10.1.0, open-mpi 4.1.2, openblas 0.3.10, zlib 1.2.11)
71-
DOCKER_REPOSITORY: geosx/sherlock-gcc10.1.0-openmpi4.1.2-openblas0.3.10-zlib1.2.11
72-
TPL_DOCKERFILE: docker/Stanford/Dockerfile
73-
DOCKER_ROOT_IMAGE: matteofrigo5/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11-cmake3.31.4-no-geosx:0.0.1
74-
SPEC: "%[email protected]~pygeosx~openmp"
75-
INSTALL_DIR_ROOT: /oak/stanford/groups/tchelepi/geos-sherlock/CPU
76-
RUNS_ON: ubuntu-latest
77-
- name: Sherlock GPU (centos 7.9.2009, gcc 10.1.0, open-mpi 4.1.2, cuda 12.4.0, openblas 0.3.10, zlib 1.2.11)
78-
DOCKER_REPOSITORY: geosx/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11
79-
TPL_DOCKERFILE: docker/Stanford/Dockerfile
80-
DOCKER_ROOT_IMAGE: matteofrigo5/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11-cmake3.31.4-no-geosx:0.0.1
81-
SPEC: "%[email protected]~pygeosx~openmp cuda_arch=70 ^[email protected]+allow-unsupported-compilers"
82-
INSTALL_DIR_ROOT: /oak/stanford/groups/tchelepi/geos-sherlock/GPU
83-
RUNS_ON: ubuntu-latest
70+
# - name: Sherlock CPU (centos 7.9.2009, gcc 10.1.0, open-mpi 4.1.2, openblas 0.3.10, zlib 1.2.11)
71+
# DOCKER_REPOSITORY: geosx/sherlock-gcc10.1.0-openmpi4.1.2-openblas0.3.10-zlib1.2.11
72+
# TPL_DOCKERFILE: docker/Stanford/Dockerfile
73+
# DOCKER_ROOT_IMAGE: matteofrigo5/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11-cmake3.31.4-no-geosx:0.0.1
74+
# SPEC: "~pygeosx~openmp %[email protected]"
75+
# INSTALL_DIR_ROOT: /oak/stanford/groups/tchelepi/geos-sherlock/CPU
76+
# RUNS_ON: ubuntu-latest
77+
# - name: Sherlock GPU (centos 7.9.2009, gcc 10.1.0, open-mpi 4.1.2, cuda 12.4.0, openblas 0.3.10, zlib 1.2.11)
78+
# DOCKER_REPOSITORY: geosx/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11
79+
# TPL_DOCKERFILE: docker/Stanford/Dockerfile
80+
# DOCKER_ROOT_IMAGE: matteofrigo5/sherlock-gcc10.1.0-openmpi4.1.2-cuda12.4.0-openblas0.3.10-zlib1.2.11-cmake3.31.4-no-geosx:0.0.1
81+
# SPEC: "~pygeosx~openmp cuda_arch=70 %[email protected] ^[email protected]+allow-unsupported-compilers"
82+
# INSTALL_DIR_ROOT: /oak/stanford/groups/tchelepi/geos-sherlock/GPU
83+
# RUNS_ON: ubuntu-latest
8484

8585
steps:
8686
- name: Checkout

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,4 @@
66
.vscode/
77
spack-*.txt
88
__pycache__
9+
uberenv_*

‎.uberenv_config.json‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@
44
"package_final_phase": "lvarray_hostconfig",
55
"package_source_dir": "../..",
66
"spack_url": "https://github.com/spack/spack",
7-
"spack_commit": "2bfcc69fa870d3c6919be87593f22647981b648a",
7+
"spack_commit": "734c5db2121b01c373eed6538e452f18887e9e44",
88
"spack_configs_path": "scripts/spack_configs",
9-
"spack_packages_path": "scripts/spack_packages/packages"
9+
"spack_packages_path": "scripts/spack_packages/packages",
10+
"spack_packages_commit": "a75a7f75182ffc7a51c6ca7f0fec4bf9b2705be8"
1011
}

‎docker/TotalEnergies/Pangea3.Dockerfile‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,9 +48,11 @@ RUN --mount=src=.,dst=$SRC_DIR,readwrite cd ${SRC_DIR} && \
4848
ln -s /usr/lib64/libnsl.so.2.0.0 /usr/lib64/libnsl.so && \
4949
ln -s /usr/lib64/librdmacm.so.1.3.48.0 /usr/lib64/librdmacm.so && \
5050
ln -s /usr/lib64/liblustreapi.so.1 /usr/lib64/liblustreapi.so && \
51+
ln -s /data_local/sw/lsf/10.1/linux3.10-glibc2.17-ppc64le/lib/libbat.so /usr/lib64/libbat.so && \
52+
ln -s /data_local/sw/lsf/10.1/linux3.10-glibc2.17-ppc64le/lib/liblsf.so /usr/lib64/liblsf.so && \
5153
# Run uberenv
5254
./scripts/uberenv/uberenv.py \
53-
--spec "%[email protected]+cuda~uncrustify~openmp~pygeosx cuda_arch=70 ^[email protected]+allow-unsupported-compilers ^caliper~gotcha~sampler~libunwind~libdw~papi" \
55+
--spec "+cuda~uncrustify~openmp~pygeosx cuda_arch=70 %gcc-9 ^[email protected]+allow-unsupported-compilers ^caliper~gotcha~sampler~libunwind~libdw~papi" \
5456
--spack-env-file=${SRC_DIR}/docker/pangea-spack.yaml \
5557
--project-json=.uberenv_config.json \
5658
--prefix ${GEOSX_TPL_DIR} \
@@ -61,7 +63,7 @@ RUN --mount=src=.,dst=$SRC_DIR,readwrite cd ${SRC_DIR} && \
6163
cp *.cmake /spack-generated-wave-solver-only.cmake && \
6264
# Remove extraneous spack files
6365
cd ${GEOSX_TPL_DIR} && \
64-
rm -rf bin/ build_stage/ misc_cache/ spack/ spack_env/ .spack-db/
66+
rm -rf bin/ build_stage/ builtin_spack_packages_repo/ misc_cache/ spack/ spack_env/ .spack-db/
6567

6668
# Build only the wave solver for Pangea 3
6769
RUN echo 'set ( GEOS_ENABLE_CONTACT OFF CACHE BOOL "" FORCE )' >> /spack-generated-wave-solver-only.cmake && \

‎docker/pangea-spack.yaml‎

Lines changed: 33 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -17,31 +17,41 @@ spack:
1717
- ../defaults.yaml
1818
- ../versions.yaml
1919

20-
compilers::
21-
- compiler:
22-
spec: gcc@=9.4.0
23-
paths:
24-
cc: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gcc
25-
cxx: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/g++
26-
f77: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gfortran
27-
fc: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gfortran
28-
flags: {}
29-
operating_system: almalinux8
30-
target: ppc64le
31-
modules: []
32-
environment:
33-
prepend_path:
34-
LD_LIBRARY_PATH: /data_local/sw/lsf/10.1/linux3.10-glibc2.17-ppc64le/lib
35-
extra_rpaths: []
20+
toolchains:
21+
gcc-9:
22+
- spec: '%[email protected]'
23+
when: '%c'
24+
- spec: '%[email protected]'
25+
when: '%cxx'
26+
- spec: '%[email protected]'
27+
when: '%fortran'
28+
- spec: '%[email protected]'
29+
when: '%mpi'
3630

3731
packages:
3832
all:
3933
target: [x86_64]
40-
compiler: [gcc]
41-
providers:
42-
blas: [openblas]
43-
lapack: [openblas]
44-
mpi: [openmpi]
34+
35+
mpi:
36+
require:
37+
- openmpi
38+
39+
blas:
40+
require:
41+
- "openblas"
42+
lapack:
43+
require:
44+
- "openblas"
45+
46+
gcc:
47+
externals:
48+
- spec: [email protected] languages:='c,c++,fortran'
49+
prefix: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v
50+
extra_attributes:
51+
compilers:
52+
c: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gcc
53+
cxx: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/g++
54+
fortran: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-8.4.1/gcc-9.4.0-xe5cqnyajaqz75up3gflln5zlj2rue5v/bin/gfortran
4555

4656
autoconf:
4757
externals:
@@ -94,10 +104,6 @@ spack:
94104
- spec: [email protected]
95105
prefix: /data_local/sw/spack/0.17.0/opt/spack/linux-rhel8-power9le/gcc-9.4.0/openblas-0.3.18-vk36pzksytuhylqesg4cca7667np5sjp
96106
buildable: false
97-
blas:
98-
buildable: false
99-
lapack:
100-
buildable: false
101107

102108
libtool:
103109
externals:
@@ -110,13 +116,9 @@ spack:
110116
prefix: /usr
111117
buildable: false
112118

113-
# Spack may grab for mpi & we don't want to use them
114-
mpi:
115-
buildable: false
116-
117119
openmpi:
118120
externals:
119-
- spec: [email protected]%[email protected] +cuda~cxx~cxx_exceptions~java+lustre~memchecker+pmi+static~wrapper-rpath schedulers=lsf
121+
- spec: [email protected] +cuda~cxx~cxx_exceptions~java+lustre~memchecker+pmi+static~wrapper-rpath schedulers=lsf %[email protected]
120122
prefix: /data_local/sw/openmpi/4.1.2/env/gcc-8.4.1
121123

122124
# Built-in ncurses causes spack failure
@@ -137,7 +139,7 @@ spack:
137139
buildable: false
138140
python:
139141
externals:
140-
- spec: [email protected]+bz2+crypt+ctypes+dbm+lzma+nis+pyexpat~pythoncmd+readline+sqlite3+ssl~tkinter+uuid+zlib
142+
- spec: [email protected]+bz2+crypt+ctypes+dbm+lzma+pyexpat~pythoncmd+readline+sqlite3+ssl~tkinter+uuid+zlib
141143
prefix: /usr
142144
buildable: false
143145
readline:

‎docker/rocky-spack.yaml‎

Lines changed: 54 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -17,44 +17,62 @@ spack:
1717
- ../defaults.yaml
1818
- ../versions.yaml
1919

20-
compilers::
21-
- compiler:
22-
extra_rpaths: []
23-
flags:
24-
cflags: -pthread
25-
cxxflags: -pthread
26-
modules: []
27-
operating_system: rocky8
28-
paths:
29-
cc: /usr/bin/clang
30-
cxx: /usr/bin/clang++
31-
f77: /usr/bin/gfortran
32-
fc: /usr/bin/gfortran
33-
spec: [email protected]
34-
target: x86_64
35-
- compiler:
36-
extra_rpaths: []
37-
flags:
38-
cflags: -pthread
39-
cxxflags: -pthread
40-
modules: []
41-
operating_system: rocky8
42-
paths:
43-
cc: /opt/rh/gcc-toolset-13/root/usr/bin/gcc
44-
cxx: /opt/rh/gcc-toolset-13/root/usr/bin/g++
45-
f77: /opt/rh/gcc-toolset-13/root/usr/bin/gfortran
46-
fc: /opt/rh/gcc-toolset-13/root/usr/bin/gfortran
47-
spec: [email protected]
48-
target: x86_64
20+
toolchains:
21+
gcc-13:
22+
- spec: cxxflags='-pthread'
23+
- spec: cflags='-pthread'
24+
- spec: '%[email protected]'
25+
when: '%c'
26+
- spec: '%[email protected]'
27+
when: '%cxx'
28+
- spec: '%[email protected]'
29+
when: '%fortran'
30+
- spec: '%[email protected]'
31+
when: '%mpi'
32+
clang-17:
33+
- spec: cxxflags='-pthread'
34+
- spec: cflags='-pthread'
35+
- spec: '%[virtuals=c][email protected]+clang~flang~lld~lldb'
36+
when: '%c'
37+
- spec: '%[virtuals=cxx][email protected]+clang~flang~lld~lldb'
38+
when: '%cxx'
39+
- spec: '%[virtuals=fortran][email protected]'
40+
when: '%fortran'
41+
- spec: '%[email protected]'
42+
when: '%mpi'
4943

5044
packages:
5145
all:
5246
target: [x86_64]
53-
compiler: [gcc, clang]
54-
providers:
55-
blas: [netlib-lapack]
56-
lapack: [netlib-lapack]
57-
mpi: [openmpi]
47+
48+
mpi:
49+
require:
50+
- openmpi
51+
52+
blas:
53+
require:
54+
- "netlib-lapack"
55+
lapack:
56+
require:
57+
- "netlib-lapack"
58+
59+
llvm:
60+
externals:
61+
- spec: [email protected]+clang~flang~lld~lldb
62+
prefix: /usr
63+
extra_attributes:
64+
compilers:
65+
c: /usr/bin/clang
66+
cxx: /usr/bin/clang++
67+
gcc:
68+
externals:
69+
- spec: [email protected] languages:='c,c++,fortran'
70+
prefix: /opt/rh/gcc-toolset-13/root/usr
71+
extra_attributes:
72+
compilers:
73+
c: /opt/rh/gcc-toolset-13/root/usr/bin/gcc
74+
cxx: /opt/rh/gcc-toolset-13/root/usr/bin/g++
75+
fortran: /opt/rh/gcc-toolset-13/root/usr/bin/gfortran
5876

5977
autoconf:
6078
version: [2.71]
@@ -77,20 +95,16 @@ spack:
7795
cuda:
7896
buildable: False
7997
externals:
80-
- spec: [email protected]%[email protected] +allow-unsupported-compilers
98+
- spec: [email protected] +allow-unsupported-compilers %[email protected]
8199
prefix: /usr/local/cuda
82-
- spec: [email protected]%[email protected] +allow-unsupported-compilers
100+
- spec: [email protected] +allow-unsupported-compilers %[email protected]
83101
prefix: /usr/local/cuda
84102
m4:
85103
buildable: false
86104
externals:
87105
- spec: [email protected]
88106
prefix: /usr
89107

90-
# Spack may grab for mpi & we don't want to use them
91-
mpi:
92-
buildable: false
93-
94108
netlib-lapack:
95109
buildable: false
96110
externals:

0 commit comments

Comments
 (0)