Skip to content

.Net: [MEVD] Test cleanup pack#13307

Merged
roji merged 3 commits into
microsoft:mainfrom
roji:TestCleanup
Oct 29, 2025
Merged

.Net: [MEVD] Test cleanup pack#13307
roji merged 3 commits into
microsoft:mainfrom
roji:TestCleanup

Conversation

@roji

@roji roji commented Oct 28, 2025

Copy link
Copy Markdown
Member

This is an (almost) entirely test-only cleanup (I plan to do a bit more later).

  • Remove BasicQueryTests, rename BasicFilterTests to FilterTests and move out to the root
    • BasicFilterTests and BasicQueryTests both run the same filter scenarios, the former via SearchAsync and the latter via filtering GetAsync. Although in theory, some filter query construct could work in one but not in the other, in practice no such example exists in the 12 providers we have (all databases use the same filter language/engine for both APIs. So we had needless duplication for all the various test exceptions/customizations, which had to be overridden in exactly the same way in both test suites.
    • At least for now, decided to remove BasicQueryTests as it adds maintenance burden without any real coverage; coverage (included missing coverage) specifically for filtered GetAsync was added separately (e.g. we weren't testing ordering, skip/take with filtered get).
    • If we do want to add back support for all query constructs (I'd want to have a good reason to do so), I'd do that within FilterTests: just like each scenario already exercises both static and dynamic models, it could run GetAsync as well.
  • Renamed CollectionConformanceTests to CollecitonManagementTests and moved it out to the root.
    • In general, this PR removes "Conformance" from the test type names.
  • Big cleanup/refactor of all the so-called CRUD tests.
    • We now have a ModelTests folder with a test suite for each different model; BasicModelTests covers Get/Upsert/Delete (and their batching variants) for the basic scenario (strongly-typed model with data properties and a single vector property). Alongside that we have DynamicModelTests, NoDataModelTests, NoVectorModelTests (I'll be added MultipleVectorsModelTests).
    • These tests no longer use random GUIDs for the collection name (difficult to debug, the collection never gets cleaned up if the test crashes/is stopped in the middle), but rather has fixed names per test suite.
    • The collection is created once per test suite, but cleared and reseeded between each test run, providing full isolation (previously we had no isolation, and documented each record in the seed data as belonging to this scenario or that - which is brittle).
    • Many other small improvements and cleanup.

Part of #12507

@roji roji requested a review from a team as a code owner October 28, 2025 21:34
@markwallace-microsoft markwallace-microsoft added the .NET Issue or Pull requests regarding .NET code label Oct 28, 2025
@github-actions github-actions Bot changed the title [MEVD] Test cleanup pack .Net: [MEVD] Test cleanup pack Oct 28, 2025
@westey-m

Copy link
Copy Markdown
Contributor

If we do want to add back support for all query constructs (I'd want to have a good reason to do so), I'd do that within FilterTests: just like each scenario already exercises both static and dynamic models, it could run GetAsync as well.

@roji, it might be worth considering covering both when we publish the tests as a nuget for external implementations to verify conformance, although I agree it's an edge for a DB to require different mapping code for Search and Get

@roji

roji commented Oct 29, 2025

Copy link
Copy Markdown
Member Author

it might be worth considering covering both when we publish the tests as a nuget for external implementations to verify conformance, although I agree it's an edge for a DB to require different mapping code for Search and Get

Yeah. We can add this, but I'd just like to first at least see a case or two before we do, is all - in the 12 providers we have, we haven't had such a case (and it's really weird to imagine a case that would).

@roji roji added this pull request to the merge queue Oct 29, 2025
Merged via the queue into microsoft:main with commit cc43169 Oct 29, 2025
22 of 25 checks passed
@roji roji deleted the TestCleanup branch October 29, 2025 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.NET Issue or Pull requests regarding .NET code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants