Skip to content

[CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files #13360

[CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files

[CELEBORN-1319] Optimize skew partition logic for Reduce Mode to avoid sorting shuffle files #13360

Triggered via pull request February 17, 2025 22:18
Status Failure
Total duration 5m 12s
Artifacts 3

maven.yml

on: pull_request
Matrix: mr
Matrix: service
Matrix: spark2
Matrix: spark3
Fit to window
Zoom out
Zoom in

Annotations

34 errors and 31 warnings
spark3 (8, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
flink (8, 1.16)
Process completed with exit code 1.
mr (8)
Process completed with exit code 1.
flink (11, 1.19)
Process completed with exit code 1.
flink (11, 1.20)
Process completed with exit code 1.
spark3 (11, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
mr (11)
Process completed with exit code 1.
spark3 (8, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
flink (8, 1.19)
Process completed with exit code 1.
flink (8, 1.18)
Process completed with exit code 1.
spark3 (17, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
flink (8, 1.20)
Process completed with exit code 1.
flink (11, 1.17)
Process completed with exit code 1.
spark3 (17, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
Process completed with exit code 1.
spark2 (8, 2.4)
Process completed with exit code 1.
flink (11, 1.18)
Process completed with exit code 1.
spark3 (8, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
Process completed with exit code 1.
flink (8, 1.17)
Process completed with exit code 1.
spark3 (8, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
flink (11, 1.16)
Process completed with exit code 1.
spark3 (8, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (17, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (17, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (8, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
Process completed with exit code 1.
spark3 (11, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
Process completed with exit code 1.
service (11)
Process completed with exit code 1.
service (17)
Process completed with exit code 1.
service (8)
Process completed with exit code 1.
spark3 (8, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (8, 1.16)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
mr (8)
No files were found with the provided path: **/target/test/ **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (11, 1.19)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (11, 1.20)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
mr (11)
No files were found with the provided path: **/target/test/ **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (8, 1.19)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (8, 1.18)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (17, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (8, 1.20)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (11, 1.17)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (17, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark2 (8, 2.4)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (11, 1.18)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (8, 1.17)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
flink (11, 1.16)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (17, 3.5, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.3, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.0, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (17, 3.4, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (8, 3.1, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.2, org.apache.spark.shuffle.sort.io.LocalDiskShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.
spark3 (11, 3.5, org.apache.spark.shuffle.celeborn.CelebornShuffleDataIO)
No files were found with the provided path: **/target/test-reports/** **/target/unit-tests.log. No artifacts will be uploaded.

Artifacts

Produced during runtime
Name Size
service-java-11-unit-test-log Expired
27.5 KB
service-java-17-unit-test-log Expired
27.3 KB
service-java-8-unit-test-log Expired
27.6 KB