-
Notifications
You must be signed in to change notification settings - Fork 25
Add support for wasm
#852
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for wasm
#852
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,183 @@ | ||
name: Haskell CI (WASM) | ||
|
||
on: | ||
merge_group: | ||
pull_request: | ||
push: | ||
# we need this to populate cache for `master` branch to make it available to the child branches, see | ||
# https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache | ||
branches: | ||
- master | ||
# GH caches are removed when not accessed within 7 days - this schedule runs the job every 6 days making | ||
# sure that we always have some caches on master | ||
# schedule: | ||
# - cron: '0 0 */6 * *' | ||
|
||
jobs: | ||
build: | ||
runs-on: ${{ matrix.sys.os }} | ||
|
||
strategy: | ||
fail-fast: false | ||
matrix: | ||
sys: | ||
- { os: ubuntu-latest, shell: bash } | ||
# - { os: macos-latest, shell: bash } | ||
|
||
defaults: | ||
run: | ||
shell: ${{ matrix.sys.shell }} | ||
|
||
env: | ||
# Modify this value to "invalidate" the cabal cache. | ||
CABAL_CACHE_VERSION: "2025-05-29" | ||
|
||
concurrency: | ||
group: > | ||
wasm | ||
a+${{ github.event_name }} | ||
b+${{ github.workflow_ref }} | ||
c+${{ github.job }} | ||
f+${{ matrix.sys.os }} | ||
g+${{ (startsWith(github.ref, 'refs/heads/gh-readonly-queue/') && github.run_id) || github.event.pull_request.number || github.ref }} | ||
cancel-in-progress: true | ||
|
||
steps: | ||
- name: Concurrency group | ||
run: > | ||
echo | ||
wasm | ||
a+${{ github.event_name }} | ||
b+${{ github.workflow_ref }} | ||
c+${{ github.job }} | ||
f+${{ matrix.sys.os }} | ||
g+${{ (startsWith(github.ref, 'refs/heads/gh-readonly-queue/') && github.run_id) || github.event.pull_request.number || github.ref }} | ||
|
||
- uses: actions/checkout@v4 | ||
|
||
- uses: cachix/install-nix-action@v30 | ||
with: | ||
nix_path: nixpkgs=channel:nixos-unstable | ||
extra_nix_config: | | ||
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= | ||
substituters = https://cache.iog.io/ https://cache.nixos.org/ | ||
|
||
- uses: rrbutani/use-nix-shell-action@v1 | ||
with: | ||
devShell: .#wasm | ||
|
||
- name: Cabal update | ||
run: | | ||
wasm32-wasi-cabal update | ||
|
||
# A dry run `build all` operation does *NOT* downlaod anything, it just looks at the package | ||
# indices to generate an install plan. | ||
- name: Build dry run | ||
run: | | ||
wasm32-wasi-cabal build cardano-wasm --dry-run | ||
|
||
# From the install plan we generate a dependency list. | ||
- name: Record dependencies | ||
id: record-deps | ||
run: | | ||
cat dist-newstyle/cache/plan.json | jq -r '."install-plan"[] | select(.style != "local") | .id' | sort | uniq > dependencies.txt | ||
|
||
- name: Store month number as environment variable used in cache version | ||
run: | | ||
cat <<EOF >> $GITHUB_ENV | ||
MONTHNUM=$(date -u '+%m') | ||
GHC=$(ghc --numeric-version) | ||
STORE=$(wasm32-wasi-cabal path --store | tail -n 1) | ||
EOF | ||
|
||
# Cache is disabled because GHA default builders are not able to build all dependencies | ||
# because they lack RAM, so having the cache expire would break the CI check. | ||
# For this reason, we are providing a build of the dependencies instead in | ||
# the "Restore cached deps" step, and we make the check not required. | ||
# When we are able to make this CI check self-sufficient, we should reenable the | ||
# caching and remove the manual restoring of cached deps. | ||
|
||
# From the dependency list we restore the cached dependencies. | ||
# We use the hash of `dependencies.txt` as part of the cache key because that will be stable | ||
# until the `index-state` values in the `cabal.project` file changes. | ||
# - name: Restore cached dependencies | ||
palas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# uses: actions/cache/restore@v4 | ||
# id: cache | ||
# with: | ||
# path: | | ||
# ${{ env.STORE }} | ||
# dist-newstyle | ||
# key: | ||
# wasm-cache-${{ env.CABAL_CACHE_VERSION }}-${{ runner.os }}-${{ env.GHC }}-${{ hashFiles('cardano-wasm/dependencies.txt') }} | ||
# restore-keys: | | ||
# wasm-cache-${{ env.CABAL_CACHE_VERSION }}-${{ runner.os }}-${{ env.GHC }}- | ||
|
||
- name: Restore cached deps | ||
run: | | ||
wget "https://agrius.feralhosting.com/palas/wasm-cache/4c200033737be4736cd2a363d64c49a385937d5ea57d8e52773f65d08bbd1342.tar.bz2" | ||
tar -jxf 4c200033737be4736cd2a363d64c49a385937d5ea57d8e52773f65d08bbd1342.tar.bz2 | ||
rm -fr ~/.ghc-wasm/.cabal/store/ | ||
mv store ~/.ghc-wasm/.cabal/ | ||
|
||
# Now we install the dependencies. If the cache was found and restored in the previous step, | ||
# this should be a no-op, but if the cache key was not found we need to build stuff so we can | ||
# cache it for the next step. | ||
- name: Install dependencies | ||
run: | | ||
wasm32-wasi-cabal build cardano-wasm --only-dependencies --no-semaphore -j1 --ghc-options="-j1" | ||
|
||
# Always store the cabal cache. | ||
# - name: Cache Cabal store | ||
# uses: actions/cache/save@v4 | ||
# if: always() | ||
# with: | ||
# path: | | ||
# ${{ env.STORE }} | ||
# dist-newstyle | ||
# key: | ||
# ${{ steps.cache.outputs.cache-primary-key }} | ||
|
||
# Now we build. | ||
- name: Build all | ||
run: | | ||
wasm32-wasi-cabal build cardano-wasm --no-semaphore -j1 --ghc-options="-j1" | ||
|
||
# - name: Run tests | ||
# env: | ||
# TMPDIR: ${{ runner.temp }} | ||
# TMP: ${{ runner.temp }} | ||
# KEEP_WORKSPACE: 1 | ||
# run: cabal test all --enable-tests --test-show-details=direct | ||
|
||
# Uncomment the following back in for debugging. Remember to launch a `pwsh` from | ||
# the tmux session to debug `pwsh` issues. And be reminded that the `/msys2` and | ||
# `/msys2/mingw64` paths are not in PATH by default for the workflow, but tmate | ||
# will put them in. | ||
# You may also want to run | ||
# | ||
# $env:PATH=("C:\Program Files\PowerShell\7;{0}" -f $env:ORIGINAL_PATH) | ||
# | ||
# to restore the original path. Do note that some test might need msys2 | ||
# and will silently fail if msys2 is not in path. See the "Run tests" step. | ||
# | ||
# - name: Setup tmate session | ||
# if: ${{ failure() }} | ||
# uses: mxschmitt/action-tmate@v3 | ||
# with: | ||
# limit-access-to-actor: true | ||
|
||
wasm-builds-complete: | ||
needs: [build] | ||
if: ${{ always() }} | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Check if any previous job failed | ||
run: | | ||
if [[ "${{ needs.build.result }}" == "failure" ]]; then | ||
# this ignores skipped dependencies | ||
echo 'Required jobs failed to build.' | ||
exit 1 | ||
else | ||
echo 'Build complete' | ||
fi | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,6 +19,7 @@ index-state: | |
packages: | ||
cardano-api | ||
cardano-api-gen | ||
cardano-wasm | ||
|
||
extra-packages: Cabal, process | ||
|
||
|
@@ -59,3 +60,94 @@ if impl (ghc >= 9.12) | |
-- https://github.com/kapralVV/Unique/issues/11 | ||
, Unique:hashable | ||
|
||
-- WASM compilation specific | ||
|
||
if arch(wasm32) | ||
source-repository-package | ||
type: git | ||
location: https://github.com/amesgen/plutus.git | ||
tag: dc1edea4458d6fb794b245a26c730620265645f3 | ||
subdir: | ||
plutus-core | ||
plutus-ledger-api | ||
plutus-tx | ||
--sha256: sha256-QBtLmoS54b5QMAKIDOJIM6lmRC+1leBpuGKaFc7QQos= | ||
|
||
package plutus-core | ||
flags: +do-not-build-plutus-exec | ||
palas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
source-repository-package | ||
type: git | ||
location: https://github.com/haskell-wasm/hs-memory.git | ||
tag: a198a76c584dc2cfdcde6b431968de92a5fed65e | ||
--sha256: sha256-LRC3L+J921+/moZS7F17vCfM/4usYy/eMR+w/mXsjeA= | ||
|
||
source-repository-package | ||
type: git | ||
location: https://github.com/palas/ouroboros-network.git | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should be able to get our changes easily merged for IOE libraries. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is not a very nice patch: IntersectMBO/ouroboros-network@ef3e306 |
||
tag: ef3e30603e4e45dac336a085114ee22b7aa8c9ed | ||
subdir: | ||
ouroboros-network | ||
ouroboros-network-framework | ||
--sha256: sha256-+IdAmWJqzRy+erKONywtk+5YLrm63q942nZavoEA4E4= | ||
|
||
source-repository-package | ||
type: git | ||
location: https://github.com/palas/criterion.git | ||
tag: dd160d2b5f051e918e72fe1957d77905682b8d6c | ||
subdir: | ||
criterion-measurement | ||
--sha256: sha256-wzEwOUTeFL0C3QnS25/3X1ue2tUuedrLqtT0h1JZW6c= | ||
|
||
source-repository-package | ||
type: git | ||
location: https://github.com/palas/haskell-lmdb-mock.git | ||
tag: c8d61e6eee03ee271e7768c0576110da885aec48 | ||
--sha256: sha256-+gB1MmM6qRApz1p7tFsdvKoAWDrYB4a+bJ9Djm6ieYI= | ||
|
||
source-repository-package | ||
type: git | ||
location: https://github.com/palas/double-conversion.git | ||
tag: b2030245727ee56de76507fe305e3741f6ce3260 | ||
--sha256: sha256-kzwHHQzHPfPnIDtnSDAom7YGSzWjr0113x0zsfI/Tb0= | ||
|
||
source-repository-package | ||
type: git | ||
location: https://github.com/amesgen/cborg | ||
tag: 2dff24d241d9940c5a7f5e817fcf4c1aa4a8d4bf | ||
subdir: cborg | ||
--sha256: sha256-yuz1apKQ0EB9LtJkc/I1EEtB4oZnURMvCckvdFbT6qM= | ||
|
||
source-repository-package | ||
type: git | ||
palas marked this conversation as resolved.
Show resolved
Hide resolved
|
||
location: https://github.com/Jimbo4350/foundation.git | ||
tag: b3cb78484fe6f6ce1dfcef59e72ceccc530e86ac | ||
subdir: | ||
basement | ||
foundation | ||
--sha256: sha256-QKKHl/XocxGD7bwAoGe7VaIg9o8x4dA20j3sJOgiTBw= | ||
|
||
source-repository-package | ||
type: git | ||
location: https://github.com/palas/mempack.git | ||
tag: 0211addbbbf51011e5348d3696566eb12ccbef07 | ||
--sha256: sha256-iLc+foF2AM3vG6deuZ51+faI6buMkubMP75md51hMe8= | ||
|
||
source-repository-package | ||
type: git | ||
location: https://github.com/haskell-wasm/network | ||
tag: ab92e48e9fdf3abe214f85fdbe5301c1280e14e9 | ||
--sha256: sha256-U+ln/gbXoQZpNjZHydNa0FG/9GdJFgL1+T3+7KTzDWo= | ||
|
||
package cardano-crypto-praos | ||
flags: -external-libsodium-vrf | ||
|
||
package atomic-counter | ||
flags: +no-cmm | ||
|
||
constraints: time installed | ||
allow-newer: time | ||
|
||
package crypton | ||
ghc-options: -optc-DARGON2_NO_THREADS | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand that nix shell isn't caching those dependencies?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is missing a lot https://ci.iog.io/build/8189984#tabs-buildsteps
I think we should try bugging devx for help. Maybe building cardano-wasm with nix could populate IOG cache. But I guess it may be hard to do so with haskell.nix.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the nix shell is only caching the build tools. I don't know how to build it for wasm using
haskell.nix
, but that would probably solve the issue. There is an effort towards achieving that, but I don't think they got there yet. I can ask devx more directly, and see what they say