ModuleNotFoundError: No module named 'transformer_engine'

[lulzcoder]

Why, when I enter this command python ./test/test_evo2.py --model_name evo2_7b I get this error:

Traceback (most recent call last):
File "C:\Users\User\evo2\test\test_evo2.py", line 9, in
from evo2 import Evo2
File "C:\ProgramData\anaconda3\envs\evo2\Lib\site-packages\evo2-0.1.0-py3.12.egg\evo2_init_.py", line 1, in
File "C:\ProgramData\anaconda3\envs\evo2\Lib\site-packages\evo2-0.1.0-py3.12.egg\evo2\models.py", line 12, in
File "C:\Users\User\evo2\vortex\vortex\model\model.py", line 14, in
from vortex.model.layers import (
File "C:\Users\User\evo2\vortex\vortex\model\layers.py", line 10, in
from transformer_engine.pytorch import Linear
ModuleNotFoundError: No module named 'transformer_engine'

[nicegiving]

I didn't run into this error. Here's how I set up Evo2, it might be helpful for you:

Install cuda12.4:

$ wget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda_12.4.0_550.54.14_linux.run
$ sudo sh cuda_12.4.0_550.54.14_linux.run

Install cudnn:
Firstly, download cudnn-linux-x86_64-8.9.7.29_cuda12-archive.tar.xz from https://developer.nvidia.com/rdp/cudnn-archive (need to login), then execute the following commands:

$ xz -dk cudnn-linux-x86_64-8.9.7.29_cuda12-archive.tar.xz

$ tar -xvf cudnn-linux-x86_64-8.9.7.29_cuda12-archive.tar
$ cd cudnn-linux-x86_64-8.9.7.29_cuda12-archive/
$ sudo cp include/* /usr/local/cuda/include/
$ sudo cp lib/libcudnn* /usr/local/cuda/lib64/ 
$ sudo chmod a+r /usr/local/cuda/include/cudnn.h 
$ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*

Set environment variables:
Add this to ~/.bashrc

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

then:
$ source ~/.bashrc

Create conda env for Evo2:

$ conda create -n evo2 python=3.11

Install latest torch , cause I found this in vortex's requirements.txt: "torch>=2.6.0"

$ pip3 install torch torchvision torchaudio

Install Evo2:

$ git clone --recurse-submodules [email protected]:ArcInstitute/evo2.git
$ cd evo2
$ pip install .

It takes a little while, but finally success.

If got the error: "ModuleNotFoundError: No module named 'vortex.model', install vortex manually:

$ cd vortex 
$ pip install .

Hope it helpful for you!

[quito418]

$ pip install .

Thank you for sharing, can you also share what GPU model you are using?

[yz103]

I didn't run into this error. Here's how I set up Evo2, it might be helpful for you:

Install cuda12.4:

$ wget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda_12.4.0_550.54.14_linux.run
$ sudo sh cuda_12.4.0_550.54.14_linux.run

Install cudnn: Firstly, download cudnn-linux-x86_64-8.9.7.29_cuda12-archive.tar.xz from https://developer.nvidia.com/rdp/cudnn-archive (need to login), then execute the following commands:

$ xz -dk cudnn-linux-x86_64-8.9.7.29_cuda12-archive.tar.xz

$ tar -xvf cudnn-linux-x86_64-8.9.7.29_cuda12-archive.tar
$ cd cudnn-linux-x86_64-8.9.7.29_cuda12-archive/
$ sudo cp include/* /usr/local/cuda/include/
$ sudo cp lib/libcudnn* /usr/local/cuda/lib64/ 
$ sudo chmod a+r /usr/local/cuda/include/cudnn.h 
$ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*

Set environment variables: Add this to ~/.bashrc

export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

then: $ source ~/.bashrc

Create conda env for Evo2:

$ conda create -n evo2 python=3.11

Install latest torch , cause I found this in vortex's requirements.txt: "torch>=2.6.0"

$ pip3 install torch torchvision torchaudio

Install Evo2:

$ git clone --recurse-submodules [email protected]:ArcInstitute/evo2.git
$ cd evo2
$ pip install .

It takes a little while, but finally success.

If got the error: "ModuleNotFoundError: No module named 'vortex.model', install vortex manually:

$ cd vortex 
$ pip install .

Hope it helpful for you!

that works for me , especially the transformer engineer could not installed successfully for me due to cudnn install. After following your instructions to install cudnn everything works out. Thanks so much

[zzzwenao]

I meet the same problem. I spent several days and find out that the module transfomer_engine == 1.13.0 which is necessary requires glibc >= 2.28 on Linux. So if you are on Linux, you should check your transfomer_engine to see if it download transfomer_engine == 0.0.0 automatically. Every version of transfomer_engine needs glibc >= 2.28, except for transfomer_engine == 0.0.0

[zzzwenao]

[victornemeth]

I had similar issues and decided to make a docker container, if you have docker you can do this:

git clone -b add-dockerfile https://github.com/victornemeth/evo2.git
cd evo2
docker build -t evo2 
docker run -it --rm --gpus all -v ./huggingface:/root/.cache/huggingface evo2 bash

Than you should enter the interactive container and you can test if evo2 works with:

python3 ./test/test_evo2.py --model_name evo2_7b

However, I encountered another error "RuntimeError: CUDA error: CUDA-capable device(s) is/are busy or unavailable",
But that could be solved by putting CUDA_VISIBLE_DEVICES=0 in front of my command like this:

CUDA_VISIBLE_DEVICES=0 python3 ./test/test_evo2.py --model_name evo2_7b

But I think this last error was specific for my system configuration, and thus not necessary for everyone.