Skip to content

Conversation

@niemyjski
Copy link
Member

Introduces improved nested field support and related tests.

This commit addresses the need for testing nested fields. It includes a new test file, updates configurations, and adds utilities for generating test data with nested properties. A nested field is also added to the default search fields.

Introduces nested field support and related tests.

This commit addresses the need for testing nested fields.
It includes a new test file, updates configurations,
and adds utilities for generating test data with nested properties.
A nested field is also added to the default search fields.
@niemyjski niemyjski self-assigned this Jul 18, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces improved nested field support for Elasticsearch repositories with comprehensive test coverage. The changes enable testing and functionality for nested field queries, aggregations, and default search configuration.

  • Adds a new comprehensive test file for nested field functionality
  • Updates the Employee model generator to support peer review data
  • Configures default search fields to include nested field properties

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
Employee.cs Adds peerReviews parameter to the Generate method for creating test data with nested fields
EmployeeIndex.cs Configures default search fields to include employee ID and nested peer review reviewer ID
NestedFieldTests.cs New comprehensive test file covering nested field queries, aggregations, and serialization scenarios
AggregationQueryTests.cs Removes duplicate nested aggregation test and unused import
Comments suppressed due to low confidence (1)

tests/Foundatio.Repositories.Elasticsearch.Tests/AggregationQueryTests.cs:114

  • This line overwrites the first employee's ID with "employee2", which should likely be setting the second employee's ID instead. This should be employees[1].Id = "employee2";
            Age = 30,

Adds tests to verify filtering of nested aggregations using include and exclude.

This ensures that aggregations can be filtered based on specific values within nested fields, providing more granular control over the aggregation results.

Also improves readability in existing tests.

var nestedPeerReviewsAgg = result.Aggregations["nested_peerReviews"] as SingleBucketAggregate;
Assert.NotNull(nestedPeerReviewsAgg);
Assert.NotEmpty(nestedPeerReviewsAgg.Aggregations);

var termsRatingAgg = nestedReviewRatingAgg.Aggregations["terms_rating"] as BucketAggregate;
Assert.NotNull(termsRatingAgg);
Assert.Equal(2, termsRatingAgg.Items.Count);

var userFilteredAgg = nestedReviewRatingFilteredAgg.Aggregations["user_" + employees[0].Id] as SingleBucketAggregate;
Assert.NotNull(userFilteredAgg);
Assert.Single(userFilteredAgg.Aggregations.Terms("terms_rating").Buckets);
var nestedPeerReviewsAgg = result.Aggregations["nested_peerReviews"] as SingleBucketAggregate;
Assert.NotNull(nestedPeerReviewsAgg);

var reviewerTermsAgg = nestedPeerReviewsAgg.Aggregations.Terms<string>("terms_peerReviews.reviewerEmployeeId");
var nestedPeerReviewsAggWithInclude = resultWithInclude.Aggregations["nested_peerReviews"] as SingleBucketAggregate;
Assert.NotNull(nestedPeerReviewsAggWithInclude);

var reviewerTermsAggWithInclude = nestedPeerReviewsAggWithInclude.Aggregations.Terms<string>("terms_peerReviews.reviewerEmployeeId");
var nestedPeerReviewsAggWithExclude = resultWithExclude.Aggregations["nested_peerReviews"] as SingleBucketAggregate;
Assert.NotNull(nestedPeerReviewsAggWithExclude);

var reviewerTermsAggWithExclude = nestedPeerReviewsAggWithExclude.Aggregations.Terms<string>("terms_peerReviews.reviewerEmployeeId");
var nestedPeerReviewsAgg = result.Aggregations["nested_peerReviews"] as SingleBucketAggregate;
Assert.NotNull(nestedPeerReviewsAgg);

var ratingTermsAgg = nestedPeerReviewsAgg.Aggregations.Terms<int>("terms_peerReviews.rating");
var roundTrippedNestedAgg = roundTripped.Aggregations["nested_peerReviews"] as SingleBucketAggregate;
Assert.NotNull(roundTrippedNestedAgg);

var roundTrippedRatingTermsAgg = roundTrippedNestedAgg.Aggregations.Terms<int>("terms_peerReviews.rating");
roundTrippedNestedAgg = roundTripped.Aggregations["nested_peerReviews"] as SingleBucketAggregate;
Assert.NotNull(roundTrippedNestedAgg);

roundTrippedRatingTermsAgg = roundTrippedNestedAgg.Aggregations.Terms<int>("terms_peerReviews.rating");
var nestedReviewRatingAgg = result["nested_reviewRating"] as SingleBucketAggregate;
Assert.NotNull(nestedReviewRatingAgg);

var termsRatingAgg = nestedReviewRatingAgg.Aggregations["terms_rating"] as BucketAggregate;
@scharde-encora
Copy link

scharde-encora commented Dec 5, 2025

@niemyjski
Issue is, exists dont work on Nested array type, Here is step by step scenario, where exists not working.

  1. Model to have List/set
    public ISet<Tracking> Trackings { get; set; } = new HashSet<Tracking>();

  2. This prop should be Nested type in Index

        return map
            .Dynamic(false)
            .Properties(p => p
                .SetupDefaults()
                .Keyword(f => f.Name(o => o.Id))
                .Nested<Tracking>(f => f
                        .Dynamic(false)
                        .Name(i => i.Trackings )
                        .Properties(mdp => mdp
                            .Keyword(k => k.Name(i => i.Id))
                            .Keyword(k => k.Name(i => i.OriginTags))
                        )
                )
  1. Now exists wonts work
_exists_:trackings
_exists_:trackings.id

Addresses issues with deserializing nested aggregations in Elasticsearch queries.

This change ensures that nested aggregations, particularly those involving terms aggregations on nested fields, are correctly serialized and deserialized. It corrects the format of aggregations expressions in tests and handles the deserialization of SingleBucketAggregates correctly.
roundTrippedNestedAgg = roundTripped.Aggregations["nested_peerReviews"] as SingleBucketAggregate;
Assert.NotNull(roundTrippedNestedAgg);

roundTrippedRatingTermsAgg = roundTrippedNestedAgg.Aggregations.Terms<int>("terms_peerReviews.rating");
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants