Add subquery support for EmbeddedModelArrayField

[WaVEV]

Subquery and annotation handling.

[Jibola]

If I understand this code correctly:

1. We rename a field (The array) to a tmp_name
   a. If there's a subquery, it will produce another array.
2. We unwind (the array)
3. We take the value and add it back to a set to group all items into one
4. We only pick the first item in the set? Or is this returning an array of arrays?

[WaVEV]

Yes, the field we're querying on the RHS consists of multiple arrays.
Maybe an example will make this clearer. This example uses ArrayField, but it's analogous to EMAF:
Given the following documents.

[{'_id': ObjectId('684a4ddbc6db6230d7591003'),
  'field': [1],
  'field_nested': None,
  'order': 1},
 {'_id': ObjectId('684a4ddbc6db6230d7591004'),
  'field': [2],
  'field_nested': None,
  'order': 2},
 {'_id': ObjectId('684a4ddbc6db6230d7591005'),
  'field': [2, 3],
  'field_nested': None,
  'order': 3},
 {'_id': ObjectId('684a4ddbc6db6230d7591006'),
  'field': [20, 30, 40],
  'field_nested': None,
  'order': 4},
 {'_id': ObjectId('684a4ddbc6db6230d7591007'),
  'field': None,
  'field_nested': None,
  'order': 5}]

Suppose we want to apply an $overlap query using a subquery. For example:

qs = NullableIntegerArrayModel.objects.filter(order__lt=3)
        self.assertCountEqual(
            NullableIntegerArrayModel.objects.filter(
                field__overlap=qs.values_list("field"),
            ),
            self.objs[:3],
        )

To do this, we need to concatenate all the values from the RHS subquery. That means using an $unwind followed by a $group.

The $group stage collects values into an array using $addToSet. Since we use _id: null, this results in a single grouped array. However, because we're aggregating arrays from multiple documents, the result is a list of lists.

This pipeline was originally copied or adapted from ArrayField, because the structure of EmbeddedModelArrayField on the RHS behaves similarly to ArrayField.

[Jibola]

Suggested change

	"group": [
	"group": [

Let's name it something else. Since group is an existing predicate, it could get confusing. How about array_agg_query_mixin. Just be as explicit as that

[WaVEV]

🤔 IMO naming with some with meaning in the context of the pipeline is better. I am thinking in something like gathered_values, or concatenated_values.

[Jibola]

Is this a marked limitation?

[timgraham]

The query is really quite obscure, so I don't think it needs to be documented, if that's what you meant.

[WaVEV]

🤔 , yes. it is obscure. Maybe the behave isn't expected nor documented. It tries to show that an annotated embeddedmodelArrayField should be handled very similar to an Arrayfield.
BTW, I think the annotation should be reworked in the searchVector. So, we could return to this if we have time and we want to have this kind of functionality.

changes addressed