Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support torch 2.2.2 in docker images #1578

Merged
merged 3 commits into from
Apr 3, 2024
Merged

support torch 2.2.2 in docker images #1578

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1bd8a31
support torch 2.2.2 in docker image
csukuangfj Apr 1, 2024
4b1e612
small fixes
csukuangfj Apr 3, 2024
7c96aad
update doc
csukuangfj Apr 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions .github/scripts/docker/generate_build_matrix.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,13 @@ def get_torchaudio_version(torch_version):
def get_matrix():
k2_version = "1.24.4.dev20240223"
kaldifeat_version = "1.25.4.dev20240223"
version = "20240318"
version = "20240401"
python_version = ["3.8", "3.9", "3.10", "3.11", "3.12"]
torch_version = ["1.13.0", "1.13.1", "2.0.0", "2.0.1", "2.1.0", "2.1.1", "2.1.2"]
torch_version += ["2.2.0", "2.2.1"]
torch_version = []
torch_version += ["1.13.0", "1.13.1"]
torch_version += ["2.0.0", "2.0.1"]
torch_version += ["2.1.0", "2.1.1", "2.1.2"]
torch_version += ["2.2.0", "2.2.1", "2.2.2"]

matrix = []
for p in python_version:
Expand All @@ -62,10 +65,17 @@ def get_matrix():
if version_gt(p, "3.11") and not version_gt(t, "2.1"):
continue

k2_version_2 = k2_version
kaldifeat_version_2 = kaldifeat_version

if t == "2.2.2":
k2_version_2 = "1.24.4.dev20240328"
kaldifeat_version_2 = "1.25.4.dev20240329"

matrix.append(
{
"k2-version": k2_version,
"kaldifeat-version": kaldifeat_version,
"k2-version": k2_version_2,
"kaldifeat-version": kaldifeat_version_2,
"version": version,
"python-version": p,
"torch-version": t,
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build-docker-image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest]
image: ["torch2.2.1-cuda12.1", "torch2.2.1-cuda11.8", "torch2.2.0-cuda12.1", "torch2.2.0-cuda11.8", "torch2.1.0-cuda12.1", "torch2.1.0-cuda11.8", "torch2.0.0-cuda11.7", "torch1.13.0-cuda11.6", "torch1.12.1-cuda11.3", "torch1.9.0-cuda10.2"]
image: ["torch2.2.2-cuda12.1", "torch2.2.2-cuda11.8", "torch2.2.1-cuda12.1", "torch2.2.1-cuda11.8", "torch2.2.0-cuda12.1", "torch2.2.0-cuda11.8", "torch2.1.0-cuda12.1", "torch2.1.0-cuda11.8", "torch2.0.0-cuda11.7", "torch1.13.0-cuda11.6", "torch1.12.1-cuda11.3", "torch1.9.0-cuda10.2"]

steps:
# refer to https://github.com/actions/checkout
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/run-docker-image.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest]
image: ["torch2.2.1-cuda12.1", "torch2.2.1-cuda11.8", "torch2.2.0-cuda12.1", "torch2.2.0-cuda11.8", "torch2.1.0-cuda12.1", "torch2.1.0-cuda11.8", "torch2.0.0-cuda11.7", "torch1.13.0-cuda11.6", "torch1.12.1-cuda11.3", "torch1.9.0-cuda10.2"]
image: ["torch2.2.2-cuda12.1", "torch2.2.2-cuda11.8", "torch2.2.1-cuda12.1", "torch2.2.1-cuda11.8", "torch2.2.0-cuda12.1", "torch2.2.0-cuda11.8", "torch2.1.0-cuda12.1", "torch2.1.0-cuda11.8", "torch2.0.0-cuda11.7", "torch1.13.0-cuda11.6", "torch1.12.1-cuda11.3", "torch1.9.0-cuda10.2"]
steps:
# refer to https://github.com/actions/checkout
- uses: actions/checkout@v2
Expand Down
1 change: 1 addition & 0 deletions docker/torch2.0.0-cuda11.7.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM pytorch/pytorch:2.0.0-cuda11.7-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

Expand Down
1 change: 1 addition & 0 deletions docker/torch2.1.0-cuda11.8.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM pytorch/pytorch:2.1.0-cuda11.8-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

Expand Down
1 change: 1 addition & 0 deletions docker/torch2.1.0-cuda12.1.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM pytorch/pytorch:2.1.0-cuda12.1-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

Expand Down
1 change: 1 addition & 0 deletions docker/torch2.2.0-cuda11.8.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM pytorch/pytorch:2.2.0-cuda11.8-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

Expand Down
1 change: 1 addition & 0 deletions docker/torch2.2.0-cuda12.1.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM pytorch/pytorch:2.2.0-cuda12.1-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

Expand Down
1 change: 1 addition & 0 deletions docker/torch2.2.1-cuda11.8.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM pytorch/pytorch:2.2.1-cuda11.8-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

Expand Down
1 change: 1 addition & 0 deletions docker/torch2.2.1-cuda12.1.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
FROM pytorch/pytorch:2.2.1-cuda12.1-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

Expand Down
71 changes: 71 additions & 0 deletions docker/torch2.2.2-cuda11.8.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
FROM pytorch/pytorch:2.2.2-cuda11.8-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

ARG DEBIAN_FRONTEND=noninteractive

# python 3.10
ARG K2_VERSION="1.24.4.dev20240328+cuda11.8.torch2.2.2"
ARG KALDIFEAT_VERSION="1.25.4.dev20240329+cuda11.8.torch2.2.2"
ARG TORCHAUDIO_VERSION="2.2.2+cu118"

LABEL authors="Fangjun Kuang <[email protected]>"
LABEL k2_version=${K2_VERSION}
LABEL kaldifeat_version=${KALDIFEAT_VERSION}
LABEL github_repo="https://github.com/k2-fsa/icefall"

RUN apt-get update && \
apt-get install -y --no-install-recommends \
curl \
vim \
libssl-dev \
autoconf \
automake \
bzip2 \
ca-certificates \
ffmpeg \
g++ \
gfortran \
git \
libtool \
make \
patch \
sox \
subversion \
unzip \
valgrind \
wget \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*

# Install dependencies
RUN pip install --no-cache-dir \
torchaudio==${TORCHAUDIO_VERSION} -f https://download.pytorch.org/whl/torch_stable.html \
k2==${K2_VERSION} -f https://k2-fsa.github.io/k2/cuda.html \
git+https://github.com/lhotse-speech/lhotse \
kaldifeat==${KALDIFEAT_VERSION} -f https://csukuangfj.github.io/kaldifeat/cuda.html \
kaldi_native_io \
kaldialign \
kaldifst \
kaldilm \
sentencepiece>=0.1.96 \
tensorboard \
typeguard \
dill \
onnx \
onnxruntime \
onnxmltools \
multi_quantization \
typeguard \
numpy \
pytest \
graphviz

RUN git clone https://github.com/k2-fsa/icefall /workspace/icefall && \
cd /workspace/icefall && \
pip install --no-cache-dir -r requirements.txt

ENV PYTHONPATH /workspace/icefall:$PYTHONPATH

WORKDIR /workspace/icefall
71 changes: 71 additions & 0 deletions docker/torch2.2.2-cuda12.1.dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
FROM pytorch/pytorch:2.2.2-cuda12.1-cudnn8-devel
# python 3.10

ENV LC_ALL C.UTF-8

ARG DEBIAN_FRONTEND=noninteractive

# python 3.10
ARG K2_VERSION="1.24.4.dev20240328+cuda12.1.torch2.2.2"
ARG KALDIFEAT_VERSION="1.25.4.dev20240329+cuda12.1.torch2.2.2"
ARG TORCHAUDIO_VERSION="2.2.2+cu121"

LABEL authors="Fangjun Kuang <[email protected]>"
LABEL k2_version=${K2_VERSION}
LABEL kaldifeat_version=${KALDIFEAT_VERSION}
LABEL github_repo="https://github.com/k2-fsa/icefall"

RUN apt-get update && \
apt-get install -y --no-install-recommends \
curl \
vim \
libssl-dev \
autoconf \
automake \
bzip2 \
ca-certificates \
ffmpeg \
g++ \
gfortran \
git \
libtool \
make \
patch \
sox \
subversion \
unzip \
valgrind \
wget \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*

# Install dependencies
RUN pip install --no-cache-dir \
torchaudio==${TORCHAUDIO_VERSION} -f https://download.pytorch.org/whl/torch_stable.html \
k2==${K2_VERSION} -f https://k2-fsa.github.io/k2/cuda.html \
git+https://github.com/lhotse-speech/lhotse \
kaldifeat==${KALDIFEAT_VERSION} -f https://csukuangfj.github.io/kaldifeat/cuda.html \
kaldi_native_io \
kaldialign \
kaldifst \
kaldilm \
sentencepiece>=0.1.96 \
tensorboard \
typeguard \
dill \
onnx \
onnxruntime \
onnxmltools \
multi_quantization \
typeguard \
numpy \
pytest \
graphviz

RUN git clone https://github.com/k2-fsa/icefall /workspace/icefall && \
cd /workspace/icefall && \
pip install --no-cache-dir -r requirements.txt

ENV PYTHONPATH /workspace/icefall:$PYTHONPATH

WORKDIR /workspace/icefall
2 changes: 2 additions & 0 deletions docs/source/docker/intro.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ which will give you something like below:

.. code-block:: bash
"torch2.2.2-cuda12.1"
"torch2.2.2-cuda11.8"
"torch2.2.1-cuda12.1"
"torch2.2.1-cuda11.8"
"torch2.2.0-cuda12.1"
Expand Down
Loading