Skip to content

Commit

Permalink
[SPARK-29124][CORE] Use MurmurHash3 bytesHash(data, seed) instead o…
Browse files Browse the repository at this point in the history
…f `bytesHash(data)`

This PR changes `bytesHash(data)` API invocation with the underlaying `byteHash(data, arraySeed)` invocation.
```scala
def bytesHash(data: Array[Byte]): Int = bytesHash(data, arraySeed)
```

The original API is changed between Scala versions by the following commit. From Scala 2.12.9, the semantic of the function is changed. If we use the underlying form, we are safe during Scala version migration.
- scala/scala@846ee2b#diff-ac889f851e109fc4387cd738d52ce177

No.

This is a kind of refactoring.

Pass the Jenkins with the existing tests.

Closes apache#25821 from dongjoon-hyun/SPARK-SCALA-HASH.

Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: HyukjinKwon <[email protected]>
  • Loading branch information
dongjoon-hyun authored and rshkv committed Jul 16, 2020
1 parent b3ad44e commit 6d4d6c3
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ private[spark] object XORShiftRandom {
/** Hash seeds to have 0/1 bits throughout. */
private[random] def hashSeed(seed: Long): Long = {
val bytes = ByteBuffer.allocate(java.lang.Long.SIZE).putLong(seed).array()
val lowBits = MurmurHash3.bytesHash(bytes)
val lowBits = MurmurHash3.bytesHash(bytes, MurmurHash3.arraySeed)
val highBits = MurmurHash3.bytesHash(bytes, lowBits)
(highBits.toLong << 32) | (lowBits.toLong & 0xFFFFFFFFL)
}
Expand Down

0 comments on commit 6d4d6c3

Please sign in to comment.