Can't run the demo

[KramPiotr]

When I want to run any of the demos (the classic and the real time) my code gets stuck at the below point:

Do you know what's the reason?

Also, I believe documentation should include more precise instructions into how the VITA_ckpt directory look like.
I assumed that I can I will just download whatever is in Hugging Face but I am still not sure whether that is 100% correct as this isn't obvious for beginners.

[lxysl]

This issue seems to be unclear as to the cause, I recommend you check that the versions of all packages are correct, and ensure that the files in vllm_tools are copied out according to the instructions in the readme.

Regarding the content in VITA_ckpt, you can directly download all files from https://huggingface.co/VITA-MLLM/VITA-1.5/tree/main. However, this is not all the files required for running. You have two options.

1. Follow the instructions in the readme to download InternViT and AudioEnc, and manually modify the "mm_audio_encoder" and "mm_vision_tower" in VITA-1.5/config.json to point to the corresponding paths.
2. Without manually downloading and modifying the directory, simply specify --model_path as the path to VITA-1.5 folder and run, allowing transformers to automatically download InternViT and AudioEnc, which will be automatically downloaded to the user's .cache directory.