Skip to content

Support Document Models with dicts in LLMRater#295

Merged
IsmailMehdi merged 4 commits intoGoogleCloudPlatform:mainfrom
pl04351820:llmrater
Mar 31, 2026
Merged

Support Document Models with dicts in LLMRater#295
IsmailMehdi merged 4 commits intoGoogleCloudPlatform:mainfrom
pl04351820:llmrater

Conversation

@pl04351820
Copy link
Copy Markdown
Contributor

This PR fixes a TypeError in llmrater.py that occurred when processing document models (nested lists/dicts).

Changes:

  • util.make_hashable: Fixed to recursively hash list items so nested dictionaries are properly converted into hashable frozensets.
  • setmatcher.py: Cleaned up the redundant inline _make_hashable from PR Generalize SetMatcher to support Document Model #288 to use util.make_hashable. Also migrated multiset equality to use collections.Counter instead of sorted(), which is unstable for frozenset objects in Python 3.
  • llmrater_test.py: Added a new unit test to verify that take_n_uniques safely handles nested document structures.

Sichen Liu added 2 commits March 30, 2026 07:51
Fix util.make_hashable to recursively unwrap nested objects in lists, preventing TypeErrors when caching. Cleaned up duplication in SetMatcher and adopted collections.Counter for robust multiset matching.
Added tests for standard flat dictionaries, limits enforcement, and empty datasets.
@pl04351820 pl04351820 requested a review from IsmailMehdi as a code owner March 30, 2026 14:57
Sichen Liu added 2 commits March 30, 2026 08:20
@IsmailMehdi
Copy link
Copy Markdown
Collaborator

/gcbrun

@IsmailMehdi IsmailMehdi merged commit cc50287 into GoogleCloudPlatform:main Mar 31, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants