Skip to content

WEB: add note to PDEP-10 about delayed timeline for requiring pyarrow #61706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jorisvandenbossche
Copy link
Member

(will add context later)

Copy link
Member

@datapythonista datapythonista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I find it quite disturbing the double standards on how the new governance is being applied.

If you want to ammend PDEP-10, please follow our governance process, I don't think it's ok to open this as if it was a web change.

@jorisvandenbossche
Copy link
Member Author

jorisvandenbossche commented Jun 25, 2025

@datapythonista sorry for opening this PR without the proper context, I should just have waited with opening it until I had written that up.

This is the PR that I should have done a year ago after PDEP-14 got voted on (#58551, voting issue at #59160). With that PDEP, and at the time with the intent of releasing pandas 3.0 already somewhere last year, the plan was to add this new default string dtype that would use pyarrow, but only if installed and otherwise still fallback on the object-dtype based implementation of the string dtype.
Essentially that PDEP detailed the plan for a string dtype mentioned in PDEP-10, while also delaying the rest of PDEP-10 until after pandas 3.0 (i.e. not yet making pyarrow a required dependency for pandas 3.0).

I think that if I would have done this PR directly after the vote on PDEP-14, it would have been clear in that context and that such an amendment to a PDEP would not necessarily require a vote or a separate PDEP (since the amendment is reflecting that a subsequent PDEP changed/superseded a part of a previous PDEP).

However, given the current context with the revived discussions about defaulting to pyarrow dtypes and requiring pyarrow and the vote on the PDEP about rejecting PDEP-10 (and with this PR being done long after the PDEP-14 acceptance), I entirely understand that this PR is now a lot more controversial (all the more reason that I should have only opened this PR with proper context .. again apologies about that).

To conclude, with this PR I mostly want to illustrate what I personally would do on the short term for pandas 3.0, as I also mentioned in the PDEP-15 (reject PDEP-10) discussion two days ago at #58623 (comment).
(and apologies for not having done this PR much sooner .. I should have handled the public communication after PDEP-14's acceptance better)

EDIT: and with the above, I hope that I clarified that it is not my intent to apply double standards

@datapythonista
Copy link
Member

Now that we finally could vote on PDEP-15 and know what the team wants, and since it seems like most people is happy to consider PDEP-10 officially rejected, I'm ok with this.

I still think it's very poor team working to fully ignore the governance procedure when it's convenient, and use it as if we were lawyers in other cases (not that you personally did the latter).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants