DPL: add support for decompressing directly to shared memory#13763
DPL: add support for decompressing directly to shared memory#13763ktf merged 3 commits intoAliceO2Group:devfrom
Conversation
|
REQUEST FOR PRODUCTION RELEASES: This will add The following labels are available |
|
First step towards being able to read and write RNTuple and TTree seamleassly. |
|
Tested on hyperloop. Not OK. One leak fixed, one more to go. |
|
Error while checking build/O2/fullCI for 8c955dd at 2024-12-14 04:53: Full log here. |
|
Error while checking build/O2/fullCI for 84ca389 at 2025-01-01 00:35: Full log here. |
|
Error while checking build/O2/fullCI for 70b4151 at 2025-01-29 23:28: Full log here. |
|
Error while checking build/O2/fullCI_slc9 for 70b4151 at 2025-01-28 00:44: Full log here. |
1519b94 to
05f2abc
Compare
|
Error while checking build/O2/fullCI_slc9 for 1fe4d36 at 2025-02-03 22:41: Full log here. |
|
Error while checking build/O2/fullCI for 1fe4d36 at 2025-02-04 07:17: Full log here. |
78d334e to
e9b282d
Compare
|
Error while checking build/O2/fullCI for e9b282d at 2025-02-11 22:07: Full log here. |
|
Error while checking build/O2/fullCI_slc9 for e9b282d at 2025-02-05 10:26: Full log here. |
This PR postpones the read operations which would usually populate an intermediate RecordBatch and it performs them directly on its subsequent shared memory serialization. Doing so avoids having the intermediate representation allocate most of the memory. For the moment this is only done for TTree. RNtuple support will come in a subsequent PR.
|
Tested on hyperloop, including using a workflows from a few usual suspects and actually submitting on the Grid. Merging to get it in tomorrow tag. |
This PR postpones the read operations which would usually populate an intermediate
RecordBatch and it performs them directly on its subsequent shared memory
serialization. Doing so avoids having the intermediate representation allocate most
of the memory.
For the moment this is only done for TTree. RNtuple support will come in a subsequent PR.