Skip to content

normalizers: support Replace and Prepend (fixes gemma-4-31B tokenizer load)#38

Open
itayh-atero wants to merge 4 commits into
crusoecloud:mainfrom
itayh-atero:itayh/prepend-replace-normalizers
Open

normalizers: support Replace and Prepend (fixes gemma-4-31B tokenizer load)#38
itayh-atero wants to merge 4 commits into
crusoecloud:mainfrom
itayh-atero:itayh/prepend-replace-normalizers

python: release the GIL during encode / encode_batch

8c5e1d3
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs