Storage: Add extend_json Tag for Enhanced JSON Field Handling in bulk_update Operations

[rabbull]

In #6587, it was determined that when performing bulk_update operations on JSON fields like extra, the existing JSON should be extended rather than overwritten. To address this, this PR introduces a new tag, extend_json, that controls how bulk_update handles JSON fields. By default, extend_json is set to False, ensuring backward compatibility. When explicitly enabled, bulk_update merges the provided JSON with the existing JSON field, following to the RFC 7386 - JSON Merge Patch standard. This feature supports both SQLite and PostgreSQL backend.

Note:

1. This implementation currently assumes that only attributes and extra are JSON fields of interest, and few additional JSON fields will be introduced. As a result, automatic detection of JSON fields is not implemented.
2. The bulk_update logic for the SQLite DOS backend currently resides in PsqlDosBackend. A dirty workaround has to be added in PsqlDosBackend to address the differences between SQLite and PostgreSQL. This is also stated in a comment block in the code.

[unkcpz]

If some test hang up, it might benefit from #6674.

[codecov]

Codecov Report

Attention: Patch coverage is 88.70968% with 7 lines in your changes missing coverage. Please review.

Project coverage is 76.67%. Comparing base (f43a510) to head (eb3f17e).

Files with missing lines	Patch %	Lines
src/aiida/storage/psql_dos/utils.py	37.50%	5 Missing ⚠️
src/aiida/storage/psql_dos/backend.py	96.78%	1 Missing ⚠️
src/aiida/storage/sqlite_temp/backend.py	95.24%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6659      +/-   ##
==========================================
- Coverage   78.09%   76.67%   -1.41%     
==========================================
  Files         564      564              
  Lines       42544    42591      +47     
==========================================
- Hits        33219    32653     -566     
- Misses       9325     9938     +613     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[unkcpz]

It seems the pytest hangs somewhere again. If we know which one is hang and can reproduce locally, I think I know how to debug it if it is the RMQ problem.

[rabbull]

It seems the pytest hangs somewhere again. If we know which one is hang and can reproduce locally, I think I know how to debug it if it is the RMQ problem.

The failure seems related to my changes. I doubled the timeout limit and re-ran the CI pipelines to get a full output in which the exact failed tests are expected to show up.