Skip to content

fix(rust): Make casting to date timezone-aware #22888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from

Conversation

florian-klein
Copy link
Contributor

Context

This fixes #22864

DateTime stores the timestamp in unix time, which leads to inconsistencies when timezones are used.
This for example leads to an incorrect cast to date, which previously acted as if no timezone was present at all.

I used chrono to implement the parsing - let me know if there's a more idiomatic approach to this that doesn't add to much boilerplate to polars.

@github-actions github-actions bot added fix Bug fix rust Related to Rust Polars labels May 22, 2025
@MarcoGorelli
Copy link
Collaborator

thanks for your pr! just left a couple of comments

Copy link

codecov bot commented May 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.62%. Comparing base (5888845) to head (1f876d7).
Report is 33 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #22888      +/-   ##
==========================================
- Coverage   81.01%   80.62%   -0.40%     
==========================================
  Files        1671     1677       +6     
  Lines      236927   222321   -14606     
  Branches     2792     2801       +9     
==========================================
- Hits       191949   179241   -12708     
+ Misses      44309    42412    -1897     
+ Partials      669      668       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@florian-klein florian-klein marked this pull request as ready for review May 23, 2025 07:51
Comment on lines +97 to +108
let from_tz = tz.to_chrono()?;
let ambiguous = StringChunked::from_iter(std::iter::once("raise"));
self.phys.apply_values(|timestamp| {
let ndt = timestamp_to_datetime(timestamp);
let res = convert_to_naive_local(
&from_tz,
&Tz::UTC,
ndt,
Ambiguous::from_str(ambiguous.get(0).unwrap()).unwrap(),
NonExistent::Raise,
)
.unwrap();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for updating

i think there's a function that you can use to do this whole part (replace_time_zone(..., None))?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep you're right, that's also where I got the logic from.

However, replace_time_zone is in polar-ops, which depends on polars-core.

Then datetime.rs is in polars-core, so we'd have a circular dependency when we'd use something from polars-ops in polars-core.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fix Bug fix rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unexpected behaviour when casting time-zone aware datetime to date
2 participants