You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Testing the coffea202X skimming capabilities and benchmarking performance:
81
+
* Testing the skimming workflow (cortado) developed last quarter, now running at realistic R2+R3 scale, with unskimmed inputs from SMP-24-015 (200/fb) as the test case
82
+
* Encountered challenges when attempting to process datasets in parallel, working with TaskVine developers on handling this, using their new "dynamic data reduction" implementation with cortado
83
+
* Benchmarking wall time and Hz/core with varying numbers of CPU cores (at UF T2)
84
+
* Able to achieve processing of full 13 TB inputs in a few hours with a few thousand cores (at UF)
85
+
* Physics validation of output skim underway, debugging in progress
86
+
* Explore the acceleration of the columnar processing via GPU offloading:
87
+
* Working on implementing and testing 8 ADL Benchmark Queries on GPU
88
+
* Four Queries are successfully implemented and show agreement with CPU outputs
89
+
* Four Queries have uncovered bugs in the cuda backends, reported on the relevant repositories, with debugging is in progress
90
+
* Performance comparisons of the GPU implemented Queries show performance improvements (over CPU) of up to ~800x
91
+
92
+
<br>
93
+
<b>2025 Q1 </b>
94
+
<br>
95
+
96
+
* Progress on testing the coffea202X skimming capabilities and benchmark performance:
97
+
* Set up a coffea 202X-based skimming workflow (cortado)
98
+
* Tested single file and single dataset (~4M events) runs locally and with the TaskVine scheduler, and documenting performance
99
+
* Working towards realistic R2+R3 scale tests: Transferred a realistic R2+R3 200/fb scale set of unskimmed samples to the UF T2 (~13.5 TB, ~12B events, ~400 datasets of data and MC, corresponding to the set used in the SMP-24-015 analysis)
100
+
101
+
<br>
102
+
<b>2024 Q4 </b>
103
+
<br>
104
+
105
+
* Wrapped up project, prepared the project summary document, documented the code and setup
106
+
* Worked with SONIC team to help with the handoff of the project, helping to get another member of the team set up and successfully run the SONIC+LST workflow
107
+
* Transitioning into columnar R&D project (benchmark the performance of the step of end-user data analysis, and explore the acceleration of columnar processing with GPUs)
108
+
* Starting with the exploration of the the performance and scaling of the skimming step with coffea 202X
109
+
* Progress: successfully set up and ran a 202X skimmer on one file
0 commit comments