-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Feature request
It would be great if we could support fixed-list-array input, maybe casting them into "normal" list-arrays. Example of a failure
import nested_pandas as npd
npd.read_parquet(
'https://huggingface.co/datasets/MultimodalUniverse/desi/resolve/main/edr_sv3/train-00003-of-00007.parquet?download=true'
)
...
File ~/projects/lincc-frameworks/nested-pandas/src/nested_pandas/series/utils.py:78, in validate_struct_list_array_for_equal_lengths(array)
76 continue
77 # compare offsets from the first list array with the current one
---> 78 if not first_list_array.offsets.equals(list_array.offsets):
79 raise ValueError("Offsets of all ListArrays must be the same")
AttributeError: 'pyarrow.lib.FixedSizeListArray' object has no attribute 'offsets'
Before submitting
Please check the following:
- I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
- I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
- If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request