All issues

Search results

Nightly Miri failure: undefined behavior detected

bugSomething isn't workingSomething isn't working

Status: Open.

#390 In zspacelabs/wordchipper;

· github-actions opened on Jun 1, 2026

Add support for orphaned token / degenerate BPE vocabularies.

bugSomething isn't workingSomething isn't working

enhancementNew feature or requestNew feature or request

Status: Open.

#386 In zspacelabs/wordchipper;

· crutcher opened on May 11, 2026

Python: encode_single_token(bytes) fails on non-UTF-8 byte tokens

python

Status: Open.

#378 In zspacelabs/wordchipper;

· antimora opened on Apr 3, 2026

Python: HF tokenizers compat nice-to-haves (from_file, save, add_tokens, training)

python

Status: Open.

#377 In zspacelabs/wordchipper;

· antimora opened on Apr 3, 2026

Python: tiktoken compat nice-to-haves (encode_with_unstable, decode_with_offsets, Encoding constructor)

python

Status: Open.

#376 In zspacelabs/wordchipper;

· antimora opened on Apr 3, 2026

Python: numpy array support for tiktoken compat encode/decode

python

Status: Open.

#375 In zspacelabs/wordchipper;

· antimora opened on Apr 3, 2026

Python: implement is_pretokenized parameter in HF tokenizers compat

python

Status: Open.

#374 In zspacelabs/wordchipper;

· antimora opened on Apr 3, 2026

Avoid per-token Vec<u8> clone in ids_to_tokens and id_to_token

performance

Status: Open.

#372 In zspacelabs/wordchipper;

· antimora opened on Apr 2, 2026

Missing test for get_regex_accelerator negative path

enhancementNew feature or requestNew feature or request

testing

Status: Open.

#257 In zspacelabs/wordchipper;

· antimora opened on Feb 28, 2026

Missing next_span termination and edge case tests

enhancementNew feature or requestNew feature or request

testing

Status: Open.

#256 In zspacelabs/wordchipper;

· antimora opened on Feb 28, 2026

Missing SpanEncoderSelector strum roundtrip test

enhancementNew feature or requestNew feature or request

testing

Status: Open.

#255 In zspacelabs/wordchipper;

· antimora opened on Feb 28, 2026

Default SpanLexer::next_span returns None with no warning

bugSomething isn't workingSomething isn't working

safety

Status: Open.

#254 In zspacelabs/wordchipper;

· antimora opened on Feb 28, 2026