Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended explain info missing fallback reason #1313

Open
andygrove opened this issue Jan 20, 2025 · 0 comments
Open

Extended explain info missing fallback reason #1313

andygrove opened this issue Jan 20, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@andygrove
Copy link
Member

Describe the bug

I added a new test in #1308 and it fails on Spark 4.0 with:

2025-01-20T02:01:13.8889203Z - array_remove - fallback for unsupported type struct *** FAILED *** (172 milliseconds)
2025-01-20T02:01:13.8891189Z   Extended explain diff did not contain [data type not supported: ArrayType(StructType(StructField(_1,BooleanType,true),StructField(_2,ByteType,true)),false)]. Diff: metColumnarToRow
2025-01-20T02:01:13.8893645Z      +- CometScan parquet [_1#562725,_2#562726] Batched: true, DataFilters: [], Format: CometParquet, Location: InMemoryFileIndex(1 paths)[file:/__w/datafusion-comet/datafusion-comet/spark/target/tmp/spark-f50..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_1:boolean,_2:tinyint>

The fallback reason is not getting reported in the extended explain info. For now I added a parameters so that I can disable the extended info check.

Steps to reproduce

Modify the test added in #1308 to enable the extended info check.

Expected behavior

No response

Additional context

No response

@andygrove andygrove added the bug Something isn't working label Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant