Missing next_span termination and edge case tests

[antimora]

Summary

The next_span tests for all three logos lexers only test the first two spans of "hello world". There are no tests for the termination condition (None on empty/exhausted input), which is critical for the default for_each_word loop.

Details

All three logos lexer test_span_lexer tests follow the same pattern:

let text = "hello world";
assert_eq!(lexer.next_span(text, 0), Some((0, 5)));
assert_eq!(lexer.next_span(&text[5..], 5), Some((5, 11)));

Missing cases:

• next_span("", 0) should return None
• next_span on fully consumed text should return None
• RegexWrapper::next_span has no direct unit test at all (only indirect coverage)

Suggested additions

assert_eq!(lexer.next_span("", 0), None);
assert_eq!(lexer.next_span("   ", 0), None); // whitespace-only (no Word spans)

Files

• crates/wordchipper/src/spanners/span_lexers/logos/cl100k.rs:131-144
• crates/wordchipper/src/spanners/span_lexers/logos/o200k.rs:145-158
• crates/wordchipper/src/spanners/span_lexers/logos/r50k.rs:129-142
• crates/wordchipper/src/support/regex/regex_wrapper.rs (no direct next_span test)