Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove pytorch dependency for parsing .pt #79

Open
1 of 3 tasks
geoffwoollard opened this issue Aug 12, 2024 · 4 comments
Open
1 of 3 tasks

remove pytorch dependency for parsing .pt #79

geoffwoollard opened this issue Aug 12, 2024 · 4 comments
Assignees
Labels
enhancement New feature or request high priority

Comments

@geoffwoollard
Copy link
Collaborator

geoffwoollard commented Aug 12, 2024

External programs that need access to the volumes in .pt (for external map to map distances), would need to have pytorch as a dependency.

There are a few options

  • leave as is, and require pytorch to be installed to parse these files
  • replace with .npz format, or .pkl
  • see if .pt can be read it without pytorch (e.g. read in as pickle or some other seralized reader). This may require saving the .pt file in a special way (serlize it with some encoding?)
@geoffwoollard geoffwoollard added the enhancement New feature or request label Aug 12, 2024
@geoffwoollard
Copy link
Collaborator Author

@DSilva27 what do you think?

@geoffwoollard
Copy link
Collaborator Author

geoffwoollard commented Aug 12, 2024

@DavidHerreros to look into writing a dataloader (without pytorch as dependency) for these, which will help external map to map distances.

It can be installed as a external and related repo (without pytorch as a dependency)

@geoffwoollard
Copy link
Collaborator Author

Note that the ground truth maps are a flat file of the array. They are in OSF as a .pt and .npy format

In the map to map config file, you need to use the .pt format. But this could be extended.

  • data/dataset_2_ground_truth/maps_gt_flat.pt

@DSilva27
Copy link
Collaborator

If you really think this is necessary, you could simply do np.save with allow_pickle=True. We are saving dictionaries, so this should work. The only issue is that when you load you have to do np.load with allow_pickle=True, and then do a np.ndarray.tolist() transform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request high priority
Projects
None yet
Development

No branches or pull requests

2 participants