Skip to content

Add support for wasm #852

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/bin/check-git-dependencies
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ commits () {
done
}

grep '\(^source-repository-package\|^ *location:\|^ *tag:\)' cabal.project | sed 's|^source-repository-package|-|g' | \
cat cabal.project | sed '/-- WASM compilation specific/q' | grep '\(^source-repository-package\|^ *location:\|^ *tag:\)' - | sed 's|^source-repository-package|-|g' | \
yq eval -P -j \
> tmp/repositories.json

Expand Down
183 changes: 183 additions & 0 deletions .github/workflows/haskell-wasm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
name: Haskell CI (WASM)

on:
merge_group:
pull_request:
push:
# we need this to populate cache for `master` branch to make it available to the child branches, see
# https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows#restrictions-for-accessing-a-cache
branches:
- master
# GH caches are removed when not accessed within 7 days - this schedule runs the job every 6 days making
# sure that we always have some caches on master
# schedule:
# - cron: '0 0 */6 * *'

jobs:
build:
runs-on: ${{ matrix.sys.os }}

strategy:
fail-fast: false
matrix:
sys:
- { os: ubuntu-latest, shell: bash }
# - { os: macos-latest, shell: bash }

defaults:
run:
shell: ${{ matrix.sys.shell }}

env:
# Modify this value to "invalidate" the cabal cache.
CABAL_CACHE_VERSION: "2025-05-29"

concurrency:
group: >
wasm
a+${{ github.event_name }}
b+${{ github.workflow_ref }}
c+${{ github.job }}
f+${{ matrix.sys.os }}
g+${{ (startsWith(github.ref, 'refs/heads/gh-readonly-queue/') && github.run_id) || github.event.pull_request.number || github.ref }}
cancel-in-progress: true

steps:
- name: Concurrency group
run: >
echo
wasm
a+${{ github.event_name }}
b+${{ github.workflow_ref }}
c+${{ github.job }}
f+${{ matrix.sys.os }}
g+${{ (startsWith(github.ref, 'refs/heads/gh-readonly-queue/') && github.run_id) || github.event.pull_request.number || github.ref }}

- uses: actions/checkout@v4

- uses: cachix/install-nix-action@v30
with:
nix_path: nixpkgs=channel:nixos-unstable
extra_nix_config: |
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ=
substituters = https://cache.iog.io/ https://cache.nixos.org/

- uses: rrbutani/use-nix-shell-action@v1
with:
devShell: .#wasm

- name: Cabal update
run: |
wasm32-wasi-cabal update

# A dry run `build all` operation does *NOT* downlaod anything, it just looks at the package
# indices to generate an install plan.
- name: Build dry run
run: |
wasm32-wasi-cabal build cardano-wasm --dry-run

# From the install plan we generate a dependency list.
- name: Record dependencies
id: record-deps
run: |
cat dist-newstyle/cache/plan.json | jq -r '."install-plan"[] | select(.style != "local") | .id' | sort | uniq > dependencies.txt

- name: Store month number as environment variable used in cache version
run: |
cat <<EOF >> $GITHUB_ENV
MONTHNUM=$(date -u '+%m')
GHC=$(ghc --numeric-version)
STORE=$(wasm32-wasi-cabal path --store | tail -n 1)
EOF

# Cache is disabled because GHA default builders are not able to build all dependencies
# because they lack RAM, so having the cache expire would break the CI check.
# For this reason, we are providing a build of the dependencies instead in
# the "Restore cached deps" step, and we make the check not required.
# When we are able to make this CI check self-sufficient, we should reenable the
# caching and remove the manual restoring of cached deps.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that nix shell isn't caching those dependencies?

Copy link
Contributor

@carbolymer carbolymer Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is missing a lot https://ci.iog.io/build/8189984#tabs-buildsteps

I think we should try bugging devx for help. Maybe building cardano-wasm with nix could populate IOG cache. But I guess it may be hard to do so with haskell.nix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the nix shell is only caching the build tools. I don't know how to build it for wasm using haskell.nix, but that would probably solve the issue. There is an effort towards achieving that, but I don't think they got there yet. I can ask devx more directly, and see what they say


# From the dependency list we restore the cached dependencies.
# We use the hash of `dependencies.txt` as part of the cache key because that will be stable
# until the `index-state` values in the `cabal.project` file changes.
# - name: Restore cached dependencies
# uses: actions/cache/restore@v4
# id: cache
# with:
# path: |
# ${{ env.STORE }}
# dist-newstyle
# key:
# wasm-cache-${{ env.CABAL_CACHE_VERSION }}-${{ runner.os }}-${{ env.GHC }}-${{ hashFiles('cardano-wasm/dependencies.txt') }}
# restore-keys: |
# wasm-cache-${{ env.CABAL_CACHE_VERSION }}-${{ runner.os }}-${{ env.GHC }}-

- name: Restore cached deps
run: |
wget "https://agrius.feralhosting.com/palas/wasm-cache/4c200033737be4736cd2a363d64c49a385937d5ea57d8e52773f65d08bbd1342.tar.bz2"
tar -jxf 4c200033737be4736cd2a363d64c49a385937d5ea57d8e52773f65d08bbd1342.tar.bz2
rm -fr ~/.ghc-wasm/.cabal/store/
mv store ~/.ghc-wasm/.cabal/

# Now we install the dependencies. If the cache was found and restored in the previous step,
# this should be a no-op, but if the cache key was not found we need to build stuff so we can
# cache it for the next step.
- name: Install dependencies
run: |
wasm32-wasi-cabal build cardano-wasm --only-dependencies --no-semaphore -j1 --ghc-options="-j1"

# Always store the cabal cache.
# - name: Cache Cabal store
# uses: actions/cache/save@v4
# if: always()
# with:
# path: |
# ${{ env.STORE }}
# dist-newstyle
# key:
# ${{ steps.cache.outputs.cache-primary-key }}

# Now we build.
- name: Build all
run: |
wasm32-wasi-cabal build cardano-wasm --no-semaphore -j1 --ghc-options="-j1"

# - name: Run tests
# env:
# TMPDIR: ${{ runner.temp }}
# TMP: ${{ runner.temp }}
# KEEP_WORKSPACE: 1
# run: cabal test all --enable-tests --test-show-details=direct

# Uncomment the following back in for debugging. Remember to launch a `pwsh` from
# the tmux session to debug `pwsh` issues. And be reminded that the `/msys2` and
# `/msys2/mingw64` paths are not in PATH by default for the workflow, but tmate
# will put them in.
# You may also want to run
#
# $env:PATH=("C:\Program Files\PowerShell\7;{0}" -f $env:ORIGINAL_PATH)
#
# to restore the original path. Do note that some test might need msys2
# and will silently fail if msys2 is not in path. See the "Run tests" step.
#
# - name: Setup tmate session
# if: ${{ failure() }}
# uses: mxschmitt/action-tmate@v3
# with:
# limit-access-to-actor: true

wasm-builds-complete:
needs: [build]
if: ${{ always() }}
runs-on: ubuntu-latest
steps:
- name: Check if any previous job failed
run: |
if [[ "${{ needs.build.result }}" == "failure" ]]; then
# this ignores skipped dependencies
echo 'Required jobs failed to build.'
exit 1
else
echo 'Build complete'
fi

2 changes: 1 addition & 1 deletion .github/workflows/hls.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
test-hls-works:
env:
# Modify this value to "invalidate" the cache.
HLS_CACHE_VERSION: "2024-07-24"
HLS_CACHE_VERSION: "2025-06-11"

runs-on: ubuntu-latest
timeout-minutes: 60
Expand Down
11 changes: 7 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ dist/
result*
/launch-*
stack.yaml.lock
*.hi
*.o
Main

/.cache
/db
Expand All @@ -40,12 +43,12 @@ supervisord.pid
tags
/config
/data
./*.skey
./*.vkey
./*.cert
/*.skey
/*.vkey
/*.cert

# For now require that users generate their own hie.yaml
hie.yaml
/hie.yaml

# Ignore Visual Studio code configuration
.vscode/tasks.json
Expand Down
92 changes: 92 additions & 0 deletions cabal.project
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ index-state:
packages:
cardano-api
cardano-api-gen
cardano-wasm

extra-packages: Cabal, process

Expand Down Expand Up @@ -59,3 +60,94 @@ if impl (ghc >= 9.12)
-- https://github.com/kapralVV/Unique/issues/11
, Unique:hashable

-- WASM compilation specific

if arch(wasm32)
source-repository-package
type: git
location: https://github.com/amesgen/plutus.git
tag: dc1edea4458d6fb794b245a26c730620265645f3
subdir:
plutus-core
plutus-ledger-api
plutus-tx
--sha256: sha256-QBtLmoS54b5QMAKIDOJIM6lmRC+1leBpuGKaFc7QQos=

package plutus-core
flags: +do-not-build-plutus-exec

source-repository-package
type: git
location: https://github.com/haskell-wasm/hs-memory.git
tag: a198a76c584dc2cfdcde6b431968de92a5fed65e
--sha256: sha256-LRC3L+J921+/moZS7F17vCfM/4usYy/eMR+w/mXsjeA=

source-repository-package
type: git
location: https://github.com/palas/ouroboros-network.git
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be able to get our changes easily merged for IOE libraries.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a very nice patch: IntersectMBO/ouroboros-network@ef3e306
But we can try, maybe we can make a better patch

tag: ef3e30603e4e45dac336a085114ee22b7aa8c9ed
subdir:
ouroboros-network
ouroboros-network-framework
--sha256: sha256-+IdAmWJqzRy+erKONywtk+5YLrm63q942nZavoEA4E4=

source-repository-package
type: git
location: https://github.com/palas/criterion.git
tag: dd160d2b5f051e918e72fe1957d77905682b8d6c
subdir:
criterion-measurement
--sha256: sha256-wzEwOUTeFL0C3QnS25/3X1ue2tUuedrLqtT0h1JZW6c=

source-repository-package
type: git
location: https://github.com/palas/haskell-lmdb-mock.git
tag: c8d61e6eee03ee271e7768c0576110da885aec48
--sha256: sha256-+gB1MmM6qRApz1p7tFsdvKoAWDrYB4a+bJ9Djm6ieYI=

source-repository-package
type: git
location: https://github.com/palas/double-conversion.git
tag: b2030245727ee56de76507fe305e3741f6ce3260
--sha256: sha256-kzwHHQzHPfPnIDtnSDAom7YGSzWjr0113x0zsfI/Tb0=

source-repository-package
type: git
location: https://github.com/amesgen/cborg
tag: 2dff24d241d9940c5a7f5e817fcf4c1aa4a8d4bf
subdir: cborg
--sha256: sha256-yuz1apKQ0EB9LtJkc/I1EEtB4oZnURMvCckvdFbT6qM=

source-repository-package
type: git
location: https://github.com/Jimbo4350/foundation.git
tag: b3cb78484fe6f6ce1dfcef59e72ceccc530e86ac
subdir:
basement
foundation
--sha256: sha256-QKKHl/XocxGD7bwAoGe7VaIg9o8x4dA20j3sJOgiTBw=

source-repository-package
type: git
location: https://github.com/palas/mempack.git
tag: 0211addbbbf51011e5348d3696566eb12ccbef07
--sha256: sha256-iLc+foF2AM3vG6deuZ51+faI6buMkubMP75md51hMe8=

source-repository-package
type: git
location: https://github.com/haskell-wasm/network
tag: ab92e48e9fdf3abe214f85fdbe5301c1280e14e9
--sha256: sha256-U+ln/gbXoQZpNjZHydNa0FG/9GdJFgL1+T3+7KTzDWo=

package cardano-crypto-praos
flags: -external-libsodium-vrf

package atomic-counter
flags: +no-cmm

constraints: time installed
allow-newer: time

package crypton
ghc-options: -optc-DARGON2_NO_THREADS

3 changes: 2 additions & 1 deletion cardano-api/cardano-api.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ common project-config
-Wunused-packages

common maybe-unix
if !os(windows)
if !(os(windows)|| arch(wasm32))
build-depends: unix

common maybe-Win32
Expand Down Expand Up @@ -236,6 +236,7 @@ library
Cardano.Api.IO.Internal.Base
Cardano.Api.IO.Internal.Compat
Cardano.Api.IO.Internal.Compat.Posix
Cardano.Api.IO.Internal.Compat.Wasm
Cardano.Api.IO.Internal.Compat.Win32
Cardano.Api.Internal.Orphans
Cardano.Api.Internal.Orphans.Misc
Expand Down
1 change: 1 addition & 0 deletions cardano-api/src/Cardano/Api/IO/Internal/Compat.hs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ where
import Cardano.Api.Error
import Cardano.Api.IO.Internal.Base
import Cardano.Api.IO.Internal.Compat.Posix
import Cardano.Api.IO.Internal.Compat.Wasm
import Cardano.Api.IO.Internal.Compat.Win32

import Control.Monad.Except (ExceptT)
Expand Down
2 changes: 1 addition & 1 deletion cardano-api/src/Cardano/Api/IO/Internal/Compat/Posix.hs
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{-# LANGUAGE CPP #-}
{-# LANGUAGE ScopedTypeVariables #-}

#if !defined(mingw32_HOST_OS)
#if !defined(mingw32_HOST_OS) && !defined(wasm32_HOST_ARCH)
#define UNIX
#endif

Expand Down
Loading
Loading