[CELEBORN-1233] Add unit test to verify data correctness #2638
Conversation
jiang13021
force-pushed
the
add_ut_for_CELEBORN-1233
branch
from
1c4fee0 to
789f32c
Compare
|
ping @waitinfuture @RexXiong Please take a look. Thanks. |
see comments at #2609 |
|
This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
Hi, we found that without CELEBORN-1233's PR: #2235 , if a file is deleted, the worker won't throw an exception and this could lead to data correctness issues. There is no unit test for CELEBORN-1233, so I have written one. However, it is not a unit test for #2609. |
|
This PR is stale because it has been open 20 days with no activity. Remove stale label or comment or this will be closed in 10 days. |
|
This issue was closed because it has been staled for 10 days with no activity. |
What changes were proposed in this pull request?
Add a unit test for #2235
Why are the changes needed?
ditto
Does this PR introduce any user-facing change?
No
How was this patch tested?
org.apache.celeborn.tests.spark.CelebornWorkerRestartSuite