Skip to content

Replace Levenshtein with polyeven#122

Merged
ibolmo merged 3 commits intobraintrustdata:mainfrom
zpahuja:zubin/replace_levenshtein_with_autoevals
Mar 10, 2025
Merged

Replace Levenshtein with polyeven#122
ibolmo merged 3 commits intobraintrustdata:mainfrom
zpahuja:zubin/replace_levenshtein_with_autoevals

Conversation

@zpahuja
Copy link
Contributor

@zpahuja zpahuja commented Mar 10, 2025

Polyeven's implementation of levenshtein distance is faster. It has no dependencies

with open(os.path.join(dir_name, "README.md"), "r", encoding="utf-8") as f:
long_description = f.read()

install_requires = ["chevron", "levenshtein", "pyyaml", "braintrust_core", "jsonschema"]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

let's do a bump autoevals/py/autoevals/version.py. I'll take care of the publishing after merging.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh and I think it should be polyleven?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed typo and bumped version. Thanks!

Copy link
Collaborator

@ibolmo ibolmo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally. Waiting on another pipeline (outside of this PR) to finish, and I can then merge.

@ibolmo
Copy link
Collaborator

ibolmo commented Mar 10, 2025

Ok we're good to merge.

@ibolmo ibolmo merged commit 4607b69 into braintrustdata:main Mar 10, 2025
1 of 7 checks passed
@github-actions
Copy link

github-actions bot commented Mar 10, 2025

Braintrust eval report

Autoevals (main-1741646581)

Score Average Improvements Regressions
NumericDiff 60.9% (-14.000000000000002pp) 4 🟢 39 🔴
Start 1741646581.14s - -
End 1741646584.41s - -
Duration 3.27s (+1.77s) - 100 🔴
Llm_duration 2.2s - -
Prompt_tokens 277.98tok (-1.26tok) 48 🟢 -
Completion_tokens 19.05tok (-1.45tok) 29 🟢 14 🔴
Total_tokens 297.03tok (-2.7tok) 77 🟢 12 🔴
Estimated_cost 0$ - -

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants