Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix failed Spark SQL tests due to shuffle enabled #231

Open
7 tasks
viirya opened this issue Mar 26, 2024 · 0 comments
Open
7 tasks

Fix failed Spark SQL tests due to shuffle enabled #231

viirya opened this issue Mar 26, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@viirya
Copy link
Member

viirya commented Mar 26, 2024

Describe the bug

#210 enables Comet shuffle in Spark SQL tests. However, there are a few tests failed due to that. We need to investigate the cause and fix the tests.

  • DataFrameSuite.scala: SPARK-35886: PromotePrecision should be subexpr replaced
  • FlatMapGroupsWithStateDistributionSuite.scala: SPARK-38204: flatMapGroupsWithState should require StatefulOpClusteredDistribution
  • FlatMapGroupsWithStateDistributionSuite.scala: SPARK-38204: flatMapGroupsWithState should require ClusteredDistribution from children if the query starts from checkpoint in 3.2.x - without initial state
  • FlatMapGroupsWithStateDistributionSuite.scala: SPARK-38204: flatMapGroupsWithState should require ClusteredDistribution from children if the query starts from checkpoint in prior to 3.2
  • FlatMapGroupsWithStateWithInitialStateSuite.scala: flatMapGroupsWithState - initial state - initial state has flatMapGroupsWithState
  • FlatMapGroupsWithStateSuite.scala
  • hive/AggregationQuerySuite.scala: single distinct multiple columns set

Steps to reproduce

No response

Expected behavior

No response

Additional context

No response

@viirya viirya added the bug Something isn't working label Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant