translategemma-4b-it-4bit error

[iamckun]

Error: {"error":{"message":"Chat template error: User role must provide content as an iterable with exactly one item. That item must be a mapping(type:'text' | 'image', source_lang_code:string, target_lang_code:string, text:string | none, image:string | none).","type":"invalid_request_error","param":null,"code":null}}

[jundot]

Hi, thanks for the report.

This isn't really a bug. TranslateGemma-4b-it is a specialized translation-only model from Google, and its chat template requires a custom content structure with source_lang_code / target_lang_code fields instead of plain text. Omlx's /v1/chat/completions endpoint doesn't support that structure, so the template errors out.

This model isn't usable through omlx's chat api directly. To run it, please apply the chat template on the client side and send the resulting prompt to /v1/completions instead. That path takes a raw prompt string and bypasses omlx's chat template logic entirely, so translation should work fine.

Closing this one out.

[nightguarder]

Hi,
I would like to reopen this issue, since I want to use TranslateGemma + OMLX as well. Unfortunately client-side fix caused an TypeError error for me after submitting the translation request to v1/completions

[nightguarder]

Actually opened a PR to address this. Thanks for looking into it.
#1141