feat(cli): add --stats flag for token comparison #41
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adds
--statsflag to the CLI for displaying token count statistics when encoding JSON to TOON. Uses the existingcompare_formats()utility fromutils.pyto show a comparison table with token counts and savings percentage.Type of Change
Related Issues
Closes #
Changes Made
--statsargument to CLI argparse configurationcompare_formats()function from utils.pytest_cli.py--statsflagSPEC Compliance
Testing
Test Output
799 passed, 13 skipped in 3.85s
TOTAL 1123 76 93.23%
Coverage : 93.23%
Code Quality
ruff check src/toon_format tests- no issuesruff format src/toon_format tests- code formattedmypy src/toon_format- no critical errorspytest tests/ -vChecklist
Performance Impact
Breaking Changes
Screenshots / Examples
Output:
Additional Context
This is my first contribution to toon-python. The feature leverages existing token counting utilities from
utils.pythat were already part of the public API but not exposed in the CLI. No new dependencies required - uses existingtiktokenfrom thebenchmarkdependency group.Checklist for Reviewers