Skip to content

Commit 54cde81

Browse files
xianzhe-databricksHyukjinKwon
authored andcommitted
[SPARK-54481][SPARK-54495][SPARK-54483][TESTS][PYTHON][CONNECT] Reenable a few spark connect tests
### What changes were proposed in this pull request? * This PR re-enables 3 tests, which only fail during the cross-version scenario: Spark Connect 4.0 client <-> 4.2 Spark Server * We fixed these tests on the 4.0 branch separately in another PR #53250 * ‼️ This PR needs to be backported to `branch-4.1`. How can I make it possible? HyukjinKwon ### Why are the changes needed? To regain test coverage ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? ### Was this patch authored or co-authored using generative AI tooling? no Closes #53235 from xianzhe-databricks/enable-tests. Authored-by: xianzhe-databricks <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent aecd932 commit 54cde81

File tree

3 files changed

+0
-10
lines changed

3 files changed

+0
-10
lines changed

python/pyspark/pandas/tests/test_typedef.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -311,9 +311,6 @@ def f() -> ps.Series[pdf.a.dtype]:
311311

312312
self.assertRaisesRegex(TypeError, "object.*not understood", try_infer_return_type)
313313

314-
@unittest.skipIf(
315-
os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1", "SPARK-54495: To be reenabled"
316-
)
317314
def test_as_spark_type_pandas_on_spark_dtype(self):
318315
type_mapper = {
319316
# binary

python/pyspark/sql/tests/pandas/test_pandas_cogrouped_map.py

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@
1818
import unittest
1919
import logging
2020
from typing import cast
21-
import os
2221

2322
from pyspark.sql import functions as sf
2423
from pyspark.sql.functions import pandas_udf, udf
@@ -244,9 +243,6 @@ def merge_pandas(lft, rgt):
244243

245244
self._test_merge_empty(fn=merge_pandas)
246245

247-
@unittest.skipIf(
248-
os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1", "SPARK-54481: To be reenabled"
249-
)
250246
def test_apply_in_pandas_returning_incompatible_type(self):
251247
with self.quiet():
252248
self.check_apply_in_pandas_returning_incompatible_type()

python/pyspark/sql/tests/pandas/test_pandas_map.py

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -273,9 +273,6 @@ def test_dataframes_with_more_columns(self):
273273
actual = df.repartition(1).mapInPandas(f, "id long, value long").collect()
274274
self.assertEqual(actual, expected)
275275

276-
@unittest.skipIf(
277-
os.environ.get("SPARK_SKIP_CONNECT_COMPAT_TESTS") == "1", "SPARK-54483: To be reenabled"
278-
)
279276
def test_dataframes_with_incompatible_types(self):
280277
with self.quiet():
281278
self.check_dataframes_with_incompatible_types()

0 commit comments

Comments
 (0)