Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[#2388] improvement(test): mr, tez, spark integration test use random port #2397

Merged
merged 1 commit into from
Mar 17, 2025
Merged

[#2388] improvement(test): mr, tez, spark integration test use random port #2397

merged 1 commit into from
Mar 17, 2025

Conversation

summaryzb
Copy link
Contributor

@summaryzb summaryzb commented Mar 12, 2025 •
edited
Loading

What changes were proposed in this pull request?

  • Integration test of mr, tez, spark use random port
  • Re-enable AccessClusterTest

Why are the changes needed?

Fix: #2388

Does this PR introduce any user-facing change?

No.

How was this patch tested?

UT

@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 12, 2025 •
edited
Loading

Test Results

 3 011 files  +32   3 011 suites  +32   6h 43m 59s ⏱️ + 15m 41s
 1 177 tests + 1   1 176 ✅ + 3   1 💤  -  1  0 ❌ ±0 
14 914 runs  +79  14 899 ✅ +95  15 💤  - 15  0 ❌ ±0 

Results for commit eb2df4d. ± Comparison against base commit b7969b4.

This pull request removes 3 and adds 4 tests. Note that renamed tests count towards both. 
org.apache.uniffle.test.AccessClusterTest ‑ org.apache.uniffle.test.AccessClusterTest
org.apache.uniffle.test.AutoAccessTest ‑ test
org.apache.uniffle.test.CoordinatorAssignmentTest ‑ org.apache.uniffle.test.CoordinatorAssignmentTest

org.apache.uniffle.test.AccessClusterTest ‑ testUsingCustomExtraProperties
org.apache.uniffle.test.AccessClusterTest ‑ test{File}
org.apache.uniffle.test.AutoAccessTest ‑ test{File}
org.apache.uniffle.test.GetReaderTest ‑ test{File}

♻️ This comment has been updated with latest results.

@summaryzb
Copy link
Contributor Author

@zuston @jerqi PTAL

@LuciferYang LuciferYang marked this pull request as draft March 14, 2025 04:00
@summaryzb summaryzb marked this pull request as ready for review March 14, 2025 11:17
@jerqi jerqi requested a review from LuciferYang March 17, 2025 03:22
LuciferYang
LuciferYang reviewed Mar 17, 2025
View reviewed changes
integration-test/spark-common/src/test/java/org/apache/uniffle/test/RSSStageResubmitTest.java
createCoordinatorServer(coordinatorConf);
ShuffleServerConf grpcShuffleServerConf = getShuffleServerConf(ServerType.GRPC);
createMockedShuffleServer(grpcShuffleServerConf);
storeCoordinatorConf(coordinatorConf);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

storeCoordinatorConf
storeMockShuffleServerConf
storeMockShuffleServerConf
startServersWithRandomPorts

looks ok, but is the code above a pattern? Should there be a method defined like startServersWithRandomPorts with type parameters CoordinatorConf and List<ShuffleServerConf>? This pattern seems to have some repetition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are quite a amount of cases，that the coordinator and shuffleServer assemble configuration in different place, it's better use fine-grained method to control

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got ~

@summaryzb summaryzb requested a review from LuciferYang March 17, 2025 08:24
LuciferYang
LuciferYang approved these changes Mar 17, 2025
View reviewed changes
Copy link
Contributor

@LuciferYang LuciferYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM

@jerqi Could you take another look?

@LuciferYang LuciferYang merged commit 3874f15 into apache:master Mar 17, 2025
41 checks passed
@LuciferYang
Copy link
Contributor

Merged into master. Thanks @summaryzb and @jerqi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LuciferYang LuciferYang LuciferYang approved these changes

@jerqi jerqi jerqi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Subtask] mr, tez, spark integration test use random port
3 participants
@summaryzb @LuciferYang @jerqi