Database/Registry integration test

[kurtisc]

Closes RV-859

What

Integration test for Durable Storage

Why

We want to:

1. Expose the public parts of the Database and Registry APIs
2. Check that everything works as expected

How

• Move TestableTmpDir to a new utils module so that external crates (e.g. the integration test, the benchmarks) can use it.
• Add a new integration test

Manually Testing

make all

In the root directory::

cargo test

or in durable-storage/tests/integration/

cargo test

But not in durable-storage/!

Tasks for the Author

• Link all Linear issues related to this MR using magic words (e.g. part of, relates to, closes).
• Eliminate dead code and other spurious artefacts introduced in your changes.
• Document new public functions, methods and types.
• Make sure the documentation for updated functions, methods, and types is correct.
• Add tests for bugs that have been fixed.
• Explain changes to regression test captures when applicable.
• Write commit messages in agreement with our guidelines.
• Self-review your changes to ensure they are high-quality.
• Complete all of the above before assigning this MR to reviewers.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (kurtis/durable-storage/testable-tmp-dir-util-merge@22feb1f). Learn more about missing BASE report.

Additional details and impacted files

@@                                  Coverage Diff                                  @@
##             kurtis/durable-storage/testable-tmp-dir-util-merge     #781   +/-   ##
=====================================================================================
  Coverage                                                      ?   91.06%           
=====================================================================================
  Files                                                         ?      110           
  Lines                                                         ?    20831           
  Branches                                                      ?    20831           
=====================================================================================
  Hits                                                          ?    18969           
  Misses                                                        ?     1489           
  Partials                                                      ?      373           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Benchmark results for revision b3189af:

Metric	Duration	TPS
Mean	1.517304362s	26.363
Worst	1.525648177s	26.218
Best	1.509170389s	26.505
Standard Deviation	±4.72137ms	±0.082

Full results

Run	Transfers	Duration	TPS
1	40	1.524598427s	26.236
2	40	1.516323701s	26.380
3	40	1.520148428s	26.313
4	40	1.525648177s	26.218
5	40	1.518133021s	26.348
6	40	1.509170389s	26.505
7	40	1.523289016s	26.259
8	40	1.51426328s	26.415
9	40	1.524550514s	26.237
10	40	1.513452249s	26.430
11	40	1.511163451s	26.470
12	40	1.518338743s	26.345
13	40	1.515528116s	26.393
14	40	1.511679416s	26.461
15	40	1.511000607s	26.473
16	40	1.516275279s	26.380
17	40	1.520133394s	26.313
18	40	1.520277795s	26.311
19	40	1.515629399s	26.392
20	40	1.516483832s	26.377

Compare the results above with those for the default branch.

[victor-dumitrescu]

Seems low given the database supports much larger values?

[kurtisc]

This is also to try to balance performance of the test but I will look into something that tests some large values.

[victor-dumitrescu]

Suggested change

// Auto formatting doesn't work within the proptest macro

[kurtisc]

This is intentional, to explain why I'm using a standalone function outside of the proptest macro.

[victor-dumitrescu]

Oh, I'd maybe add that to the comment then, otherwise it looked to me like just stating a fact so I didn't understand why it was there. Feel free to resolve either way.

[vapourismo]

What is the reason for making the integration tests a separate crate? What prevents them from existing as integration tests in the octez-riscv-durable-storage crate?

[kurtisc]

Oops, I didn't know that was a thing. Thanks, done.

[vapourismo]

Would this be suitable for the octez-riscv-test-utils crate in /test-utils?

[kurtisc]

I'll make this a standalone PR now

[kurtisc]

Draft until parent branches are merged #803 #804