Description
Pandas version checks
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
df = pd.DataFrame([{'start': pd.Timestamp('2025-01-01 10:00:00'), 'end':pd.Timestamp('2025-01-01 10:00:15.12345678')},
{'start': pd.Timestamp('2025-01-01 10:00:30.999999'), 'end':pd.Timestamp('2025-01-01 10:00:45')}])
df['pause_duration'] = (df['start'].shift(-1) - df['end']).apply(lambda x: pd.NA if pd.isna(x) else x.total_seconds())
df['pause_duration'].round(1)
Issue Description
In pandas 2.2.3 rounding of Nan values just silently failed (values did not get rounded) while the same code causes a TypeError in 2.3.0
Sample data preparation:
import pandas as pd
df = pd.DataFrame([{'start': pd.Timestamp('2025-01-01 10:00:00'), 'end':pd.Timestamp('2025-01-01 10:00:15.12345678')},
{'start': pd.Timestamp('2025-01-01 10:00:30.999999'), 'end':pd.Timestamp('2025-01-01 10:00:45')}])
df['pause_duration'] = (df['start'].shift(-1) - df['end']).apply(lambda x: pd.NA if pd.isna(x) else x.total_seconds())
df['pause_duration']
Version: 2.2.3
Out[4]:
0 15.876542
1 <NA>
Name: pause_duration, dtype: object```
Round fails for 2.2.3:
```python
df['pause_duration'].round(1)
Out[5]:
0 15.876542
1 <NA>
In 2.3.0 this causes a TypeError instead:
import pandas as pd
print('Version:', pd.__version__)
df = pd.DataFrame([{'start': pd.Timestamp('2025-01-01 10:00:00'), 'end':pd.Timestamp('2025-01-01 10:00:15.12345678')},
{'start': pd.Timestamp('2025-01-01 10:00:30.999999'), 'end':pd.Timestamp('2025-01-01 10:00:45')}])
df['pause_duration'] = (df['start'].shift(-1) - df['end']).apply(lambda x: pd.NA if pd.isna(x) else x.total_seconds())
df['pause_duration']
Version: 2.3.0
Out[14]:
0 15.876542
1 <NA>
Name: pause_duration, dtype: object
Round causes a TypeError:
>>> df['pause_duration'].round(1)
Traceback (most recent call last):
File "C:\Users\Schleehauf\PycharmProjects\viodata\viotools\.venv\Lib\site-packages\IPython\core\interactiveshell.py", line 3672, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-15-0e359609e34a>", line 1, in <module>
df['pause_duration'].round(1)
File "C:\Users\Schleehauf\PycharmProjects\viodata\viotools\.venv\Lib\site-packages\pandas\core\series.py", line 2818, in round
TypeError: Expected numeric dtype, got object instead.
For both versions, type conversion (and rounding) only works with pyarrow:
df['pause_duration'].astype('float[pyarrow]').round(1)
Out[20]:
0 15.9
1 <NA>
Name: pause_duration, dtype: float[pyarrow]
And fails with TypeError:
df['pause_duration'].astype(float).round(1)
Traceback (most recent call last):
...
TypeError: float() argument must be a string or a real number, not 'NAType'
Expected Behavior
- Do not throw an exception but warn instead
- When subtracting Timestamps the datatype shoulde be timedelta and not object even when there are NaT values
- an timedelta-NaN that has a total_seconds()-method returning float-nan such that
df['pause_duration'].apply(lambda x: x.total_seconds())
Traceback (most recent call last):
...
AttributeError: 'float' object has no attribute 'total_seconds'
will just work in the future and yields the same result as df['pause_duration'].astype('float[pyarrow]').round(1)
Installed Versions
pd.show_versions()
INSTALLED VERSIONS
commit : 2cc3762
python : 3.12.10
python-bits : 64
OS : Windows
OS-release : 11
Version : 10.0.26100
machine : AMD64
processor : Intel64 Family 6 Model 186 Stepping 2, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : de_DE.cp1252
pandas : 2.3.0
numpy : 2.2.6
pytz : 2025.2
dateutil : 2.9.0.post0
pip : None
Cython : None
sphinx : None
IPython : 9.3.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.13.4
blosc : None
bottleneck : 1.5.0
dataframe-api-compat : None
fastparquet : None
fsspec : 2025.5.1
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : 3.1.6
lxml.etree : 5.4.0
matplotlib : 3.10.3
numba : 0.61.2
numexpr : 2.11.0
odfpy : None
openpyxl : 3.1.5
pandas_gbq : None
psycopg2 : None
pymysql : None
pyarrow : 20.0.0
pyreadstat : None
pytest : 8.4.1
python-calamine : None
pyxlsb : 1.0.10
s3fs : 2025.5.1
scipy : None
sqlalchemy : None
tables : None
tabulate : 0.9.0
xarray : None
xlrd : 2.0.2
xlsxwriter : 3.2.5
zstandard : 0.23.0
tzdata : 2025.2
qtpy : None
pyqt5 : None