How to use template, after adding llama_chat_apply_template?

[Folko-Ven]

I've been struggling with template for a long time, and now I've discovered that in the last commits 11b12de what I've been waiting for. But how to use it? Do I need to specify any parameters? Or is everything automatically loaded from the model?

[ibehnam]

bumping this because the server docs don't mention this feature.

[Nick-infinity]

Any help on this?

[Folko-Ven]

Nope.

[Jeximo]

Do I need to specify any parameters? Or is everything automatically loaded from the model?

Hi, Server readme shows command line option --chat-template JINJA_TEMPLATE. By default, llama_chat_apply_template() uses the template from a models metadata, tokenizer.chat_template.

Here's supported templates for --chat-template. For example,

./server -m ... --chat-template chatml