Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IO issues for 'io.network_from_csv' #2959

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

hpc-dave
Copy link
Contributor

@hpc-dave hpc-dave commented Feb 4, 2025

Here, two issues are addressed:

  1. A DtypeWarning appears when using 'io.network_from_csv', that mixed types were encountered. Since specifying the dtype is not generally possible, the flag low_memory=False has to be set to avoid this warning by pandas.
  2. For the columns with boolean values, pandas keeps the general Object type (dtype('O')), which can lead to issues later on, especially when trying to use the values directly. Especially nasty to debug when used in combination with topotools. This issue is fixed by adding an additional conversion step at the end of the function 'io.network_from_csv'

Note:

  • pytest is currently failing due to updates in scipy 1.15,*, see discussions here and here

Here, two issues are addressed:
1) A DtypeWarning appears when using 'io.network_from_csv', that mixed types were encountered. Since specifying the dtype is not generally possible, the flag low_memory=False has to be set to avoid this warning by pandas.
2) For the columns with boolean values, pandas keeps the general Object type (dtype('O')), which can lead to issues later on, especially when trying to use the values directly. Especially nasty to debug when used in combination with topotools. This issue is fixed by adding an additional conversion step at the end of the function 'io.network_from_csv'
Copy link

codecov bot commented Feb 5, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.7%. Comparing base (18d004e) to head (df66574).
Report is 8 commits behind head on dev.

Additional details and impacted files
@@          Coverage Diff          @@
##             dev   #2959   +/-   ##
=====================================
  Coverage   89.7%   89.7%           
=====================================
  Files        148     148           
  Lines       8662    8665    +3     
=====================================
+ Hits        7774    7777    +3     
  Misses       888     888           

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant