Skip to content

Support FixedSizeListArray inputs #276

@hombit

Description

@hombit

Feature request

It would be great if we could support fixed-list-array input, maybe casting them into "normal" list-arrays. Example of a failure

import nested_pandas as npd

npd.read_parquet(
    'https://huggingface.co/datasets/MultimodalUniverse/desi/resolve/main/edr_sv3/train-00003-of-00007.parquet?download=true'
)
...
File ~/projects/lincc-frameworks/nested-pandas/src/nested_pandas/series/utils.py:78, in validate_struct_list_array_for_equal_lengths(array)
     76     continue
     77 # compare offsets from the first list array with the current one
---> 78 if not first_list_array.offsets.equals(list_array.offsets):
     79     raise ValueError("Offsets of all ListArrays must be the same")

AttributeError: 'pyarrow.lib.FixedSizeListArray' object has no attribute 'offsets'

Before submitting
Please check the following:

  • I have described the purpose of the suggested change, specifying what I need the enhancement to accomplish, i.e. what problem it solves.
  • I have included any relevant links, screenshots, environment information, and data relevant to implementing the requested feature, as well as pseudocode for how I want to access the new functionality.
  • If I have ideas for how the new feature could be implemented, I have provided explanations and/or pseudocode and/or task lists for the steps.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions