mlc_llm/serve/engine.py", line 101, in _convert_model_info assert isinstance(chat_config.conv_template, Conversation) AssertionError

[omkar806]

I am getting this error . python -m mlc_llm.serve.server --model dist/phi-2-q4f16_1-MLC --model-lib-path dist/prebuilt/phi-2/phi-2-q4f16_1-metal.so --device "metal" --host 127.0.0.1 --port 8000 I am running this command and installing the libraries python3 -m pip install --pre -U -f https://mlc.ai/wheels mlc-llm-nightly mlc-ai-nightly using this command.

[anibohara2000]

This is because the format of conv_template in mlc-chat-config.json is now changed, introduced in PR #1965 . You need to re-generate your mlc-chat-config.json using the python -m mlc_llm gen_config ... command

[omkar806]

Hey also I wanted to deploy this test api on hugging face spaces . I tried to do it using python client.But there was an error of device_type = none. Umm so as I was hosting on a docker. So I am not sure is this the right way to host the mlc compiled models .I have also raised a bug it's very latest.It would be helpful if I can get a advice on this.