[Bug] Android app does not take input; 'user 'role' is not defined' error

[afsara-ben]

I installed the mlc app by compiling from scratch according to the documentation. Now after I have downloaded the llama model, when I enter the chat UI, it does not let me send the input in most cases (notice the muted arrow sign in image). A few times it did allow me to send a message but I got the error User 'role' is not defined. I am unsure why would this be happening.

Complete build output

(android3.10) Afsaras-MacBook-Pro:MLCChat afsarabenazir$ mlc_llm package
[2025-02-04 16:53:32] INFO package.py:327: MLC LLM HOME: "/Users/afsarabenazir/Downloads/mlc-llm"
[2025-02-04 16:53:32] INFO package.py:28: Clean up all directories under "dist/bundle"
[2025-02-04 16:53:32] INFO jit.py:43: MLC_JIT_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2025-02-04 16:53:32] INFO download_cache.py:227: Downloading model from HuggingFace: HF://mlc-ai/Phi-3.5-mini-instruct-q4f16_0-MLC
[2025-02-04 16:53:32] INFO download_cache.py:29: MLC_DOWNLOAD_CACHE_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2025-02-04 16:53:32] INFO download_cache.py:166: Weights already downloaded: /Users/afsarabenazir/.cache/mlc_llm/model_weights/hf/mlc-ai/Phi-3.5-mini-instruct-q4f16_0-MLC
[2025-02-04 16:53:32] INFO package.py:81: Model lib is not specified for model "Phi-3.5-mini-instruct-q4f16_0-MLC". Now jit compile the model library.
[2025-02-04 16:53:32] INFO jit.py:158: Using cached model lib: /Users/afsarabenazir/.cache/mlc_llm/model_lib/35a0c13d99e88b056340407deeef9863.tar
[2025-02-04 16:53:32] INFO download_cache.py:227: Downloading model from HuggingFace: HF://mlc-ai/Qwen2.5-1.5B-Instruct-q4f16_1-MLC
[2025-02-04 16:53:32] INFO download_cache.py:29: MLC_DOWNLOAD_CACHE_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2025-02-04 16:53:32] INFO download_cache.py:166: Weights already downloaded: /Users/afsarabenazir/.cache/mlc_llm/model_weights/hf/mlc-ai/Qwen2.5-1.5B-Instruct-q4f16_1-MLC
[2025-02-04 16:53:32] INFO package.py:81: Model lib is not specified for model "Qwen2.5-1.5B-Instruct-q4f16_1-MLC". Now jit compile the model library.
[2025-02-04 16:53:32] INFO jit.py:158: Using cached model lib: /Users/afsarabenazir/.cache/mlc_llm/model_lib/32b77923e5ce45a6001ac73424864101.tar
[2025-02-04 16:53:32] INFO download_cache.py:227: Downloading model from HuggingFace: HF://mlc-ai/gemma-2-2b-it-q4f16_1-MLC
[2025-02-04 16:53:32] INFO download_cache.py:29: MLC_DOWNLOAD_CACHE_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2025-02-04 16:53:32] INFO download_cache.py:166: Weights already downloaded: /Users/afsarabenazir/.cache/mlc_llm/model_weights/hf/mlc-ai/gemma-2-2b-it-q4f16_1-MLC
[2025-02-04 16:53:32] INFO package.py:81: Model lib is not specified for model "gemma-2-2b-it-q4f16_1-MLC". Now jit compile the model library.
[2025-02-04 16:53:32] INFO jit.py:158: Using cached model lib: /Users/afsarabenazir/.cache/mlc_llm/model_lib/52ab370f0b0aefe65cfaa623106eeac4.tar
[2025-02-04 16:53:32] INFO package.py:129: Bundle weight for gemma-2-2b-it-q4f16_1-MLC, copy into dist/bundle/gemma-2-2b-it-q4f16_1-MLC
[2025-02-04 16:53:33] INFO download_cache.py:227: Downloading model from HuggingFace: HF://mlc-ai/Llama-3.2-3B-Instruct-q4f16_0-MLC
[2025-02-04 16:53:33] INFO download_cache.py:29: MLC_DOWNLOAD_CACHE_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2025-02-04 16:53:33] INFO download_cache.py:166: Weights already downloaded: /Users/afsarabenazir/.cache/mlc_llm/model_weights/hf/mlc-ai/Llama-3.2-3B-Instruct-q4f16_0-MLC
[2025-02-04 16:53:33] INFO package.py:81: Model lib is not specified for model "Llama-3.2-3B-Instruct-q4f16_0-MLC". Now jit compile the model library.
[2025-02-04 16:53:33] INFO jit.py:158: Using cached model lib: /Users/afsarabenazir/.cache/mlc_llm/model_lib/cd60010df00473b5fc18700bec5764e1.tar
[2025-02-04 16:53:33] INFO download_cache.py:227: Downloading model from HuggingFace: HF://mlc-ai/Mistral-7B-Instruct-v0.3-q4f16_1-MLC
[2025-02-04 16:53:33] INFO download_cache.py:29: MLC_DOWNLOAD_CACHE_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2025-02-04 16:53:33] INFO download_cache.py:166: Weights already downloaded: /Users/afsarabenazir/.cache/mlc_llm/model_weights/hf/mlc-ai/Mistral-7B-Instruct-v0.3-q4f16_1-MLC
[2025-02-04 16:53:33] INFO package.py:81: Model lib is not specified for model "Mistral-7B-Instruct-v0.3-q4f16_1-MLC". Now jit compile the model library.
[2025-02-04 16:53:33] INFO jit.py:158: Using cached model lib: /Users/afsarabenazir/.cache/mlc_llm/model_lib/d8cb02f5ad0cd0a4e2a8a46a85e1ff2e.tar
[2025-02-04 16:53:33] INFO package.py:154: Dump the app config below to "dist/bundle/mlc-app-config.json":
{
  "model_list": [
    {
      "model_id": "Phi-3.5-mini-instruct-q4f16_0-MLC",
      "model_lib": "phi3_q4f16_0_5fe42298399a05eb2a1878fdc1c8c115",
      "model_url": "https://huggingface.co/mlc-ai/Phi-3.5-mini-instruct-q4f16_0-MLC",
      "estimated_vram_bytes": 4250586449
    },
    {
      "model_id": "Qwen2.5-1.5B-Instruct-q4f16_1-MLC",
      "model_lib": "qwen2_q4f16_1_2e221f430380225c03990ad24c3d030e",
      "model_url": "https://huggingface.co/mlc-ai/Qwen2.5-1.5B-Instruct-q4f16_1-MLC",
      "estimated_vram_bytes": 3980990464
    },
    {
      "model_id": "gemma-2-2b-it-q4f16_1-MLC",
      "model_lib": "gemma2_q4f16_1_5cc7dbd3ae3d1040984d9720b2d7b7d4",
      "model_url": "https://huggingface.co/mlc-ai/gemma-2-2b-it-q4f16_1-MLC",
      "estimated_vram_bytes": 3000000000
    },
    {
      "model_id": "Llama-3.2-3B-Instruct-q4f16_0-MLC",
      "model_lib": "llama_q4f16_0_4f9f463aaf4b0ede5dc8c431d8a8fb3b",
      "model_url": "https://huggingface.co/mlc-ai/Llama-3.2-3B-Instruct-q4f16_0-MLC",
      "estimated_vram_bytes": 4679979417
    },
    {
      "model_id": "Mistral-7B-Instruct-v0.3-q4f16_1-MLC",
      "model_lib": "mistral_q4f16_1_c2cba77a6def4dd52f7e20b5d8576ab5",
      "model_url": "https://huggingface.co/mlc-ai/Mistral-7B-Instruct-v0.3-q4f16_1-MLC",
      "estimated_vram_bytes": 4115131883
    }
  ]
}
[2025-02-04 16:53:35] INFO package.py:211: Creating lib from ['/Users/afsarabenazir/.cache/mlc_llm/model_lib/35a0c13d99e88b056340407deeef9863.tar', '/Users/afsarabenazir/.cache/mlc_llm/model_lib/32b77923e5ce45a6001ac73424864101.tar', '/Users/afsarabenazir/.cache/mlc_llm/model_lib/52ab370f0b0aefe65cfaa623106eeac4.tar', '/Users/afsarabenazir/.cache/mlc_llm/model_lib/cd60010df00473b5fc18700bec5764e1.tar', '/Users/afsarabenazir/.cache/mlc_llm/model_lib/d8cb02f5ad0cd0a4e2a8a46a85e1ff2e.tar']
[2025-02-04 16:53:35] INFO package.py:212: Validating the library dist/lib/libmodel_android.a
[2025-02-04 16:53:35] INFO package.py:213: List of available model libs packaged: ['phi3_q4f16_0_5fe42298399a05eb2a1878fdc1c8c115', 'qwen2_q4f16_1_2e221f430380225c03990ad24c3d030e', 'gemma2_q4f16_1_5cc7dbd3ae3d1040984d9720b2d7b7d4', 'llama_q4f16_0_4f9f463aaf4b0ede5dc8c431d8a8fb3b', 'mistral_q4f16_1_c2cba77a6def4dd52f7e20b5d8576ab5'], if we have '-' in the model_lib string, it will be turned into '_'
[2025-02-04 16:53:35] INFO package.py:256: Validation pass
[2025-02-04 16:53:35] INFO package.py:270: Moving "dist/lib/libmodel_android.a" to "build/lib/libmodel_android.a"
[2025-02-04 16:53:35] INFO package.py:274: Building mlc4j
info: component 'rust-std' for target 'aarch64-linux-android' is up to date
[2025-02-04 16:53:36] INFO prepare_libs.py:91: Entering "/Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build" for MLC LLM and tvm4j build.
[2025-02-04 16:53:36] INFO prepare_libs.py:95: Set TVM_SOURCE_DIR to "/Users/afsarabenazir/Downloads/mlc-llm/3rdparty/tvm/"
[2025-02-04 16:53:36] INFO prepare_libs.py:23: Running cmake
CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android.toolchain.cmake:35 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/3.31.4/CMakeSystem.cmake:6 (include)
  CMakeLists.txt:3 (project)


CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android-legacy.toolchain.cmake:35 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android.toolchain.cmake:55 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/3.31.4/CMakeSystem.cmake:6 (include)
  CMakeLists.txt:3 (project)


CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/flags.cmake:18 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/Platform/Android-Clang.cmake:23 (include)
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/Platform/Android-Clang-C.cmake:1 (include)
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/CMakeCInformation.cmake:48 (include)
  CMakeLists.txt:3 (project)


-- Hide private symbols
-- TVM_SOURCE_DIR: /Users/afsarabenazir/Downloads/mlc-llm/3rdparty/tvm/
-- Hide private symbols...
-- Forbidding undefined symbols in shared library, using -Wl,--no-undefined on platform Android
CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android.toolchain.cmake:35 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/3.31.4/CMakeSystem.cmake:6 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/CMakeFiles/CMakeTmp/CMakeLists.txt:4 (project)


CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android-legacy.toolchain.cmake:35 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android.toolchain.cmake:55 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/3.31.4/CMakeSystem.cmake:6 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/CMakeFiles/CMakeTmp/CMakeLists.txt:4 (project)


CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/flags.cmake:18 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/Platform/Android-Clang.cmake:23 (include)
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/Platform/Android-Clang-CXX.cmake:1 (include)
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/CMakeCXXInformation.cmake:48 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/CMakeFiles/CMakeTmp/CMakeLists.txt:4 (project)


CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android.toolchain.cmake:35 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/3.31.4/CMakeSystem.cmake:6 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/CMakeFiles/CMakeTmp/CMakeLists.txt:4 (project)


CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android-legacy.toolchain.cmake:35 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/android.toolchain.cmake:55 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/3.31.4/CMakeSystem.cmake:6 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/CMakeFiles/CMakeTmp/CMakeLists.txt:4 (project)


CMake Deprecation Warning at /Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/build/cmake/flags.cmake:18 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.
Call Stack (most recent call first):
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/Platform/Android-Clang.cmake:23 (include)
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/Platform/Android-Clang-CXX.cmake:1 (include)
  /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/cmake/data/share/cmake-3.31/Modules/CMakeCXXInformation.cmake:48 (include)
  /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/CMakeFiles/CMakeFiles/CMakeTmp/CMakeLists.txt:4 (project)


-- Building for Android
-- Didn't find the path to CCACHE, disabling ccache
-- Enabled runtime search for OpenCL library location
-- Couldn't build OpenCL-Gtests
-- Android Vulkan_INCLUDE_DIRS=/Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/sources/third_party/vulkan/src/include
-- Skip finding SPIRV in Android, make sure you only build tvm runtime.
-- Build with contrib.random
-- Build with contrib.sort
-- Build with contrib.hybriddump
-- Git found: /usr/bin/git
-- Found TVM_GIT_COMMIT_HASH=7ed4584952546fa5d54366b72a6862f919c18daa
-- Found TVM_GIT_COMMIT_TIME=2024-12-15 09:56:40 -0500
-- Building with TVM Map...
-- Build with thread support...
-- CLANG_VERSION 19.0
-- Setting enhanced clang warning flags
-- Added "-fuse-ld=lld" to linker flags -static-libstdc++ -Wl,--build-id=sha1 -Wl,--no-rosegment -Wl,--no-undefined-version -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments 
-- Build without FlashInfer
-- system-nameAndroid
CMake Deprecation Warning at /Users/afsarabenazir/Downloads/mlc-llm/3rdparty/tokenizers-cpp/msgpack/CMakeLists.txt:1 (CMAKE_MINIMUM_REQUIRED):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.


-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
CMake Warning (dev) at /Users/afsarabenazir/Downloads/mlc-llm/3rdparty/tokenizers-cpp/CMakeLists.txt:101 (add_custom_command):
  The following keywords are not supported when using
  add_custom_command(OUTPUT): POST_BUILD.

  Policy CMP0175 is not set: add_custom_command() rejects invalid arguments.
  Run "cmake --help-policy CMP0175" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Deprecation Warning at /Users/afsarabenazir/Downloads/mlc-llm/3rdparty/tokenizers-cpp/sentencepiece/CMakeLists.txt:15 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.


-- VERSION: 0.2.00
CMake Warning at /Users/afsarabenazir/Downloads/mlc-llm/CMakeLists.txt:109 (message):
  Cannot find libflash_attn.  The model must not have been built with
  --use-flash-attn-mqa option.


-- TVM_SOURCE_DIR: /Users/afsarabenazir/Downloads/mlc-llm/android/mlc4j/../../3rdparty/tvm
-- JNI_INCLUDE_DIRS=/Users/afsarabenazir/Library/Android/sdk/ndk/28.0.12916984/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include
-- Configuring done (7.0s)
-- Generating done (0.5s)
-- Build files have been written to: /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build
[2025-02-04 16:53:46] INFO prepare_libs.py:55: Running cmake build
[  0%] Built target tokenizers_c
[  6%] Built target tvm4j_core
[  6%] Built target tvm_libinfo_objs
[  6%] Built target tokenizer_cpp_objs
[ 31%] Built target sentencepiece-static
[ 31%] Built target tokenizers_cpp
[ 31%] Building CXX object mlc_llm/CMakeFiles/mlc_llm_objs.dir/cpp/serve/sampler/cpu_sampler.cc.o
[ 68%] Built target tvm_runtime_objs
[ 68%] Built target tvm_runtime
[100%] Built target mlc_llm_objs
[100%] Linking CXX static library libmlc_llm.a
[100%] Built target mlc_llm_static
[100%] Building CXX object CMakeFiles/tvm4j_runtime_packed.dir/Users/afsarabenazir/Downloads/mlc-llm/3rdparty/tvm/jvm/native/src/main/native/org_apache_tvm_native_c_api.cc.o
[100%] Linking CXX shared library libtvm4j_runtime_packed.so
[100%] Built target tvm4j_runtime_packed
[2025-02-04 16:54:30] INFO prepare_libs.py:70: Running cmake install
Install the project...
-- Install configuration: "Release"
-- Up-to-date: /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/output/tvm4j_core.jar
-- Installing: /Users/afsarabenazir/Downloads/mlc-llm/android/MLCChat/build/output/arm64-v8a/libtvm4j_runtime_packed.so
[2025-02-04 16:54:30] INFO package.py:279: Clean up all directories under "dist/lib/mlc4j"
[2025-02-04 16:54:31] INFO package.py:286: Copying "/Users/afsarabenazir/Downloads/mlc-llm/android/mlc4j/src" to "dist/lib/mlc4j/src"
[2025-02-04 16:54:31] INFO package.py:291: Copying "/Users/afsarabenazir/Downloads/mlc-llm/android/mlc4j/build.gradle" to "dist/lib/mlc4j/build.gradle"
[2025-02-04 16:54:31] INFO package.py:296: Copying "build/output" to "dist/lib/mlc4j/output"
[2025-02-04 16:54:31] INFO package.py:302: Moving "dist/bundle/mlc-app-config.json" to "dist/lib/mlc4j/src/main/assets/mlc-app-config.json"
[2025-02-04 16:54:31] INFO package.py:367: All finished.

Environment

• Platform: Android AArch64
• OS: Android 10
• Device Xiaomi MI A2
• CPU: Qualcomm Snapdragon 660
• GPU: Adreno (TM) 512
• Kernel: Linux 4.4.192-perf+ (aarch64)
• How you installed MLC-LLM : source
• How you installed TVM-Unity: as per the guideline, not from source
• Python 3.10
• TVM Unity Hash Tag (python -c "import tvm; print('\n'.join(f'{k}: {v}' for k, v in tvm.support.libinfo().items()))", applicable if you compile models):

USE_NVTX: OFF
USE_GTEST: AUTO
SUMMARIZE: OFF
TVM_DEBUG_WITH_ABI_CHANGE: OFF
USE_IOS_RPC: OFF
USE_MSC: OFF
CUDA_VERSION: NOT-FOUND
USE_LIBBACKTRACE: AUTO
DLPACK_PATH: 3rdparty/dlpack/include
USE_TENSORRT_CODEGEN: OFF
USE_OPENCL_EXTN_QCOM: NOT-FOUND
USE_TARGET_ONNX: OFF
USE_AOT_EXECUTOR: ON
BUILD_DUMMY_LIBTVM: OFF
USE_CUDNN: OFF
USE_TENSORRT_RUNTIME: OFF
USE_ARM_COMPUTE_LIB_GRAPH_EXECUTOR: OFF
USE_THRUST: OFF
USE_CCACHE: AUTO
USE_ARM_COMPUTE_LIB: OFF
USE_CPP_RTVM: 
USE_OPENCL_GTEST: /path/to/opencl/gtest
TVM_LOG_BEFORE_THROW: OFF
USE_MKL: OFF
USE_PT_TVMDSOOP: OFF
MLIR_VERSION: NOT-FOUND
USE_CLML: OFF
USE_STACKVM_RUNTIME: OFF
USE_GRAPH_EXECUTOR_CUDA_GRAPH: OFF
ROCM_PATH: /opt/rocm
USE_DNNL: OFF
USE_MSCCL: OFF
USE_NNAPI_RUNTIME: OFF
USE_VITIS_AI: OFF
USE_MLIR: OFF
USE_RCCL: OFF
USE_LLVM: llvm-config --link-static
USE_VERILATOR: OFF
USE_TF_TVMDSOOP: OFF
USE_THREADS: ON
USE_MSVC_MT: OFF
BACKTRACE_ON_SEGFAULT: OFF
USE_GRAPH_EXECUTOR: ON
USE_NCCL: OFF
USE_ROCBLAS: OFF
GIT_COMMIT_HASH: f24d32cba56d5af60a70f4e8d6668d30553a5efc
USE_VULKAN: OFF
USE_RUST_EXT: OFF
USE_CUTLASS: OFF
USE_CPP_RPC: OFF
USE_HEXAGON: OFF
USE_CUSTOM_LOGGING: OFF
USE_UMA: OFF
USE_FALLBACK_STL_MAP: OFF
USE_SORT: ON
USE_RTTI: ON
GIT_COMMIT_TIME: 2025-01-27 09:33:35 -0500
USE_HIPBLAS: OFF
USE_HEXAGON_SDK: /path/to/sdk
USE_BLAS: none
USE_LIBTORCH: OFF
USE_RANDOM: ON
USE_CUDA: OFF
USE_COREML: OFF
USE_AMX: OFF
BUILD_STATIC_RUNTIME: OFF
USE_KHRONOS_SPIRV: OFF
USE_CLML_GRAPH_EXECUTOR: OFF
USE_TFLITE: OFF
USE_HEXAGON_GTEST: /path/to/hexagon/gtest
PICOJSON_PATH: 3rdparty/picojson
USE_OPENCL_ENABLE_HOST_PTR: OFF
INSTALL_DEV: OFF
USE_PROFILER: ON
USE_NNPACK: OFF
LLVM_VERSION: 15.0.7
USE_MRVL: OFF
USE_OPENCL: OFF
COMPILER_RT_PATH: 3rdparty/compiler-rt
USE_NNAPI_CODEGEN: OFF
RANG_PATH: 3rdparty/rang/include
USE_SPIRV_KHR_INTEGER_DOT_PRODUCT: OFF
USE_OPENMP: OFF
USE_BNNS: OFF
USE_FLASHINFER: 
USE_CUBLAS: OFF
USE_METAL: ON
USE_HEXAGON_EXTERNAL_LIBS: OFF
USE_ALTERNATIVE_LINKER: AUTO
USE_BYODT_POSIT: OFF
USE_NVSHMEM: OFF
USE_HEXAGON_RPC: OFF
DMLC_PATH: 3rdparty/dmlc-core/include
INDEX_DEFAULT_I64: ON
USE_RELAY_DEBUG: OFF
USE_RPC: ON
USE_TENSORFLOW_PATH: none
TVM_CLML_VERSION: 
USE_MIOPEN: OFF
USE_ROCM: OFF
USE_PAPI: OFF
USE_CURAND: OFF
TVM_CXX_COMPILER_PATH: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
HIDE_PRIVATE_SYMBOLS: ON

[afsara-ben]

@philippgille

[Mawriyo]

Hey Afsara,
Quick sanity check, Do the other models allow you to run? If not, I suspect there's something else going on.
Building is always tricky, but I believe you're fairly close. If the others do work, I think you may need to just redownload llama. I think something may have been corrupted in the MLC_CONFIG. Given the model on the repo hasnt changed in 4 months, I feel as if others would report this as well but this is one developer to another...
To redownload try MLC_DOWNLOAD_CACHE_POLICY=REDO before running the mlc package command.

[MasterJH5574]

Hi @afsara-ben thank you for reporting this. I just tried to build the app from fresh with the Llama 3.2 3B model, and the app works well. The specific mlc-package-config.json is as follows:

{
    "device": "android",
    "model_list": [
        {
            "model": "HF://mlc-ai/Llama-3.2-3B-Instruct-q4f16_0-MLC",
            "estimated_vram_bytes": 4679979417,
            "model_id": "Llama-3.2-3B-Instruct-q4f16_0-MLC"
        }
    ]
}

Since we are not able to reproduce the error of User 'role' is not defined, would you mind sharing a bit more about how to reproduce the error? Also, it would be very helpful if you can share your mlc-package-config.json.