Skip to content

Conversation

@carl
Copy link
Contributor

@carl carl commented Mar 26, 2025

Run example unit tests on CI

This pull request includes several changes to the test suite and minor code improvements. The most important changes include adding new test cases, updating existing test cases to handle edge cases, and improving logging for better debugging.

Test Suite Enhancements:

  • .github/workflows/python-tests.yml: Added steps to run unit tests for examples and specific tests for example 1.

  • examples/team_recommender/tests/example_1_text_response/test_compute_cosine_similarity.py:

    • Marked test_reproducing_the_same_text_embedding as expected to fail due to snapshot mismatch.
    • Updated test_cosine_similarity_generated_responses to use a relative tolerance in the assertion.
    • Adjusted tolerance levels in test_embedding_equivalence to better reflect expected values.
  • examples/team_recommender/tests/example_1_text_response/test_good_fit_for_project.py: Updated test_llm_will_hallucinate_given_no_data to include an assertion for embedding length and improve stability. [1] [2]

Code Improvements:

Minor Changes:

@paulz paulz requested review from Copilot and paulz March 26, 2025 19:27
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request updates test assertions and CI workflows to improve test reliability and logging clarity. Key changes include updates to embedding and cosine similarity tolerances in tests, additions to the GitHub Actions workflow to run tests on examples, and replacing print statements with logger calls in test helpers.

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.

Show a summary per file
File Description
examples/team_recommender/tests/example_1_text_response/test_good_fit_for_project.py Updated embedding test assertions and renamed variables for fixture consistency
.github/workflows/python-tests.yml Added new steps to run unit tests and all tests for example 1
examples/team_recommender/tests/example_1_text_response/test_compute_cosine_similarity.py Adjusted tolerance levels and added an xfail marker for a known snapshot issue
examples/team_recommender/tests/helpers.py Replaced print statements with logger.info for improved logging
tests/test_statistical_analysis.py Reformatted test assertions for clarity
examples/team_recommender/tests/example_3_loop/test_allocations_loop.py Improved multi-line assertion formatting
examples/team_recommender/tests/example_1_text_response/cosine_similarity.py Added a trailing blank line for formatting consistency
examples/team_recommender/tests/example_1_text_response/openai_embeddings.py Introduced type hints in the function signature
Comments suppressed due to low confidence (2)

examples/team_recommender/tests/example_1_text_response/test_good_fit_for_project.py:116

  • [nitpick] Avoid using the magic number 3072 directly; consider defining a constant or adding a comment explaining its significance to improve code clarity.
assert len(embedding_object["embedding"]) == 3072

examples/team_recommender/tests/example_1_text_response/test_compute_cosine_similarity.py:97

  • [nitpick] Document the rationale behind updating the tolerance thresholds and the expected count (from 900 to 952) to clarify the intended behavior of the test.
outside_tolerance_count = np.sum(np.abs(diff_val) >= 0.1)

@paulz paulz changed the title Paulz/run tests on ci Run example unit tests on CI Mar 26, 2025
@paulz paulz marked this pull request as ready for review March 26, 2025 19:28
@paulz paulz merged commit b032668 into thisisartium:main Mar 26, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants