Skip to content

Nullability of extension DTypes #3064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Tracked by #2989
gatesn opened this issue Apr 22, 2025 · 0 comments · May be fixed by #3123
Open
Tracked by #2989

Nullability of extension DTypes #3064

gatesn opened this issue Apr 22, 2025 · 0 comments · May be fixed by #3123
Labels
wire-break Includes a break to the serialized IPC or file format
Milestone

Comments

@gatesn
Copy link
Contributor

gatesn commented Apr 22, 2025

Currently we infer the nullability of a DType::Extension as the nullability of the extension's storage DType.

Take as an example the extension dtype, complex64({i: f32, j: f32}). The problem is a nullable complex64 would have to be a different type from a non-nullable complex64, i.e. complex64({i: f32, j: f32}?).

The proposal is to lift nullability of extensions into the DType enum, and force a non-nullable storage dtype. This non-nullable constraint could be lifted later... but for now I think it makes sense.

@gatesn gatesn added this to the 2025.05 milestone Apr 22, 2025
@gatesn gatesn added the wire-break Includes a break to the serialized IPC or file format label Apr 22, 2025
@onursatici onursatici linked a pull request Apr 25, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wire-break Includes a break to the serialized IPC or file format
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant