You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When converting a Parquet type to an Arrow type, if the original type is int96 and the option convertInt96ToArrowTimestamp is set to false, the resulting Arrow type defaults to Binary. However, it might be more appropriate to use FixedSizeBinary instead.
Component(s)
No response
The text was updated successfully, but these errors were encountered:
I agree that it would be more appropriate to return a FixedSizeBinary, let's ask @wgtmac, since he's the expert on Arrow. Just to make sure that there is no historical reason to return Binary instead.
I agree that FixedSizeBinary is more appropriate than Binary. However, I would argue that it is invalid to use INT96 for non-timestamp type. So I think it is better to ignore convertInt96ToArrowTimestamp and directly return Timestamp.
Describe the enhancement requested
When converting a Parquet type to an Arrow type, if the original type is int96 and the option convertInt96ToArrowTimestamp is set to false, the resulting Arrow type defaults to Binary. However, it might be more appropriate to use FixedSizeBinary instead.
Component(s)
No response
The text was updated successfully, but these errors were encountered: