Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-51030][CORE][TESTS] Add a check before Utils.deleteRecursively(tempDir) to ensure tempDir won't be cleaned up by the ShutdownHook in afterEach of LocalRootDirsTest #49723

Closed
wants to merge 10 commits into from
Closed

[SPARK-51030][CORE][TESTS] Add a check before Utils.deleteRecursively(tempDir) to ensure tempDir won't be cleaned up by the ShutdownHook in afterEach of LocalRootDirsTest #49723

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
47c59c4
init
LuciferYang Jan 29, 2025
9b93245
test maven on macos
LuciferYang Jan 29, 2025
0f49680
use hasShutdownDeleteDir
LuciferYang Jan 29, 2025
7de1c89
double check
LuciferYang Jan 29, 2025
4ddf191
Merge branch 'apache:master' into LocalRootDirsTest-hasRootAsShutdown…
LuciferYang Jan 29, 2025
2929287
Merge branch 'apache:master' into LocalRootDirsTest-hasRootAsShutdown…
LuciferYang Jan 29, 2025
5740ba5
Merge branch 'apache:master' into LocalRootDirsTest-hasRootAsShutdown…
LuciferYang Jan 30, 2025
8b0d86e
revert macos test
LuciferYang Jan 30, 2025
d7f738a
Merge branch 'apache:master' into LocalRootDirsTest-hasRootAsShutdown…
LuciferYang Jan 30, 2025
2221f4b
add comments
LuciferYang Jan 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions core/src/test/scala/org/apache/spark/LocalRootDirsTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ package org.apache.spark
import java.io.File
import java.util.UUID

import org.apache.spark.util.Utils
import org.apache.spark.util.{ShutdownHookManager, Utils}


trait LocalRootDirsTest extends SparkFunSuite with LocalSparkContext {
Expand All @@ -46,7 +46,13 @@ trait LocalRootDirsTest extends SparkFunSuite with LocalSparkContext {

override def afterEach(): Unit = {
try {
Utils.deleteRecursively(tempDir)
// SPARK-51030: Only perform manual cleanup of the `tempDir` if it has
// not been registered for cleanup via a shutdown hook, to avoid potential
// IOException due to race conditions during multithreaded cleanup.
if (!ShutdownHookManager.hasShutdownDeleteDir(tempDir) &&
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spark/core/src/test/scala/org/apache/spark/LocalRootDirsTest.scala

Line 42 in 42898c6

tempDir = Utils.createTempDir(namePrefix = "local")

In the beforeEach method, tempDir = Utils.createTempDir(namePrefix = "local") invokes ShutdownHookManager.registerShutdownDeleteDir to register tempDir for deletion via a shutdown hook. Therefore, if tempDir remains unchanged during the test case, manual cleanup is unnecessary.

However, given that tempDir is defined as protected var tempDir, it is possible for it to be altered within the test case. Hence, additional checks are implemented before manually cleaning up tempDir:

  1. tempDir has not been registered for cleanup via a shutdown hook.
  2. tempDir is not a subpath of any path that has been registered for cleanup via a shutdown hook.

This approach helps to minimize the risk of a race condition where multiple threads attempt to clean up the tempDir concurrently. It seems that this issue is more prevalent on macOS, so let's conduct several more rounds of testing to ensure its effectiveness.

!ShutdownHookManager.hasRootAsShutdownDeleteDir(tempDir)) {
Utils.deleteRecursively(tempDir)
}
Utils.clearLocalRootDirs()
} finally {
super.afterEach()
Expand Down