-
Notifications
You must be signed in to change notification settings - Fork 12
Run example unit tests on CI #65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Paul Zabelin <[email protected]>
Signed-off-by: Paul Zabelin <[email protected]>
Signed-off-by: Paul Zabelin <[email protected]>
ci: exclude test_compute_alignment from unit tests
Signed-off-by: Paul Zabelin <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request updates test assertions and CI workflows to improve test reliability and logging clarity. Key changes include updates to embedding and cosine similarity tolerances in tests, additions to the GitHub Actions workflow to run tests on examples, and replacing print statements with logger calls in test helpers.
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| examples/team_recommender/tests/example_1_text_response/test_good_fit_for_project.py | Updated embedding test assertions and renamed variables for fixture consistency |
| .github/workflows/python-tests.yml | Added new steps to run unit tests and all tests for example 1 |
| examples/team_recommender/tests/example_1_text_response/test_compute_cosine_similarity.py | Adjusted tolerance levels and added an xfail marker for a known snapshot issue |
| examples/team_recommender/tests/helpers.py | Replaced print statements with logger.info for improved logging |
| tests/test_statistical_analysis.py | Reformatted test assertions for clarity |
| examples/team_recommender/tests/example_3_loop/test_allocations_loop.py | Improved multi-line assertion formatting |
| examples/team_recommender/tests/example_1_text_response/cosine_similarity.py | Added a trailing blank line for formatting consistency |
| examples/team_recommender/tests/example_1_text_response/openai_embeddings.py | Introduced type hints in the function signature |
Comments suppressed due to low confidence (2)
examples/team_recommender/tests/example_1_text_response/test_good_fit_for_project.py:116
- [nitpick] Avoid using the magic number 3072 directly; consider defining a constant or adding a comment explaining its significance to improve code clarity.
assert len(embedding_object["embedding"]) == 3072
examples/team_recommender/tests/example_1_text_response/test_compute_cosine_similarity.py:97
- [nitpick] Document the rationale behind updating the tolerance thresholds and the expected count (from 900 to 952) to clarify the intended behavior of the test.
outside_tolerance_count = np.sum(np.abs(diff_val) >= 0.1)
Run example unit tests on CI
This pull request includes several changes to the test suite and minor code improvements. The most important changes include adding new test cases, updating existing test cases to handle edge cases, and improving logging for better debugging.
Test Suite Enhancements:
.github/workflows/python-tests.yml: Added steps to run unit tests for examples and specific tests for example 1.examples/team_recommender/tests/example_1_text_response/test_compute_cosine_similarity.py:test_reproducing_the_same_text_embeddingas expected to fail due to snapshot mismatch.test_cosine_similarity_generated_responsesto use a relative tolerance in the assertion.test_embedding_equivalenceto better reflect expected values.examples/team_recommender/tests/example_1_text_response/test_good_fit_for_project.py: Updatedtest_llm_will_hallucinate_given_no_datato include an assertion for embedding length and improve stability. [1] [2]Code Improvements:
examples/team_recommender/tests/example_1_text_response/openai_embeddings.py: Added type annotations to thecreate_embedding_objectfunction.examples/team_recommender/tests/helpers.py: Replacedprintstatements withlogger.infofor better logging and debugging. [1] [2]examples/team_recommender/tests/example_3_loop/test_allocations_loop.py: Reformatted a long assertion message for better readability.Minor Changes:
examples/team_recommender/tests/helpers.py: Imported theloggingmodule.tests/test_statistical_analysis.py: Reformatted parameterized test cases for consistency. [1] [2]