Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deep copy in copy_td #127

Open
bagibence opened this issue Jan 7, 2022 · 1 comment
Open

Deep copy in copy_td #127

bagibence opened this issue Jan 7, 2022 · 1 comment

Comments

@bagibence
Copy link
Collaborator

I don't think df.copy() does what we want in the @copy_td decorator because it doesn't make an actual copy of array fields.
We will probably have to make recursive deep copy with something like pd.DataFrame(data=copy.deepcopy(df.values), index=df.index, columns=df.columns).
Unfortunately, it will be definitely slower and memory intensive, so we might have to think about those consequences later.

import copy
dfa = df.copy(deep=True)
dfb = deepcopy(df)
dfc = pd.DataFrame(data=copy.deepcopy(df.values), index=df.index, columns=df.columns)

dfa.PMd_spikes[0][0, 0] = -1 # changes the value in df
dfb.PMd_spikes[1][0, 0] = -1 # changes the value in df
dfc.PMd_spikes[2][0, 0] = -1 # leaves the value in df intact
@raeedcho
Copy link
Contributor

raeedcho commented Jan 7, 2022

I agree that we should change @copy_td to do a full deep copy, just so there aren't any surprises later on. Regarding the memory usage and slowness, I suppose we could re-evaluate places where we use the copy_td decorator to ensure that we really need to use it, but it seems like it should be used most of the time, if we're going for non-destructive manipulations as intended.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants