Skip to content

Commit

Permalink
[SPARK-51209][CORE] Improve getCurrentUserName to handle Java 24+
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to improve `Utils.getCurrentUserName` to handle Java 24+

### Why are the changes needed?

To handle [JEP 486: Permanently Disable the Security Manager](https://openjdk.org/jeps/486) from Java 24+

### Does this PR introduce _any_ user-facing change?

Previously, a user can provide the environment variable `SPARK_USER` as a workaround.
This PR provides additional automated workaround by providing a fallback user name, `spark`.

### How was this patch tested?

Without Java 24, it's difficult to make a unit test.

Compiled with Java 21 and manual run on Java 24-ea.

**BEFORE**
```
$ bin/spark-shell
...
25/02/13 14:27:39 ERROR Main: Failed to initialize Spark session.
java.lang.UnsupportedOperationException: getSubject is not supported
	at java.base/javax.security.auth.Subject.getSubject(Subject.java:277)
	at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588)
	at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2446)
```

**AFTER (Warning messages are omitted)**
```
$ bin/spark-shell
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 4.1.0-SNAPSHOT
      /_/

Using Scala version 2.13.16 (OpenJDK 64-Bit Server VM, Java 24-ea)
Type in expressions to have them evaluated.
Type :help for more information.

Spark context Web UI available at http://localhost:4040
Spark context available as 'sc' (master = local[*], app id = local-1739485527653).
Spark session available as 'spark'.

scala>
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #49943 from dongjoon-hyun/SPARK-51209.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
dongjoon-hyun committed Feb 14, 2025
1 parent e397207 commit ffd58bc
Showing 1 changed file with 9 additions and 3 deletions.
12 changes: 9 additions & 3 deletions core/src/main/scala/org/apache/spark/util/Utils.scala
Original file line number Diff line number Diff line change
Expand Up @@ -2439,11 +2439,17 @@ private[spark] object Utils

/**
* Returns the current user name. This is the currently logged in user, unless that's been
* overridden by the `SPARK_USER` environment variable.
* overridden by the `SPARK_USER` environment variable. In case of exceptions, returns 'spark'.
*/
def getCurrentUserName(): String = {
Option(System.getenv("SPARK_USER"))
.getOrElse(UserGroupInformation.getCurrentUser().getShortUserName())
try {
Option(System.getenv("SPARK_USER"))
.getOrElse(UserGroupInformation.getCurrentUser().getShortUserName())
} catch {
// JEP 486: Permanently Disable the Security Manager
case e: UnsupportedOperationException if e.getMessage().contains("getSubject") =>
"spark"
}
}

val EMPTY_USER_GROUPS = Set.empty[String]
Expand Down

0 comments on commit ffd58bc

Please sign in to comment.