Skip to content
This repository was archived by the owner on Mar 8, 2020. It is now read-only.

Node::ValueAt may be leaking memory #113

Closed
ncordon opened this issue Aug 14, 2019 · 2 comments
Closed

Node::ValueAt may be leaking memory #113

ncordon opened this issue Aug 14, 2019 · 2 comments
Assignees
Labels
Milestone

Comments

@ncordon
Copy link
Member

ncordon commented Aug 14, 2019

After a glimpse to the methods in Node (C++ side), I have the (untested of course) hypothesis that ValueAt may be leaking memory.

  • ValueAt creates new global ref for and object obj and calls lookupOrCreate(obj)
  • lookupOrCreate tries to find a Node for obj (if it previously existed, we created a new reference to it, and we are only deallocating one). If no Node existed, calls the constructor of Node, which creates a new reference to obj, but we do not release the original one at any point D:

Proposed solution is end ValueAt method with:

return lookupOrCreate(val);

instead of

return lookupOrCreate(env->NewGlobalRef(val));
@ncordon ncordon self-assigned this Aug 14, 2019
bzz added a commit to bzz/client-scala that referenced this issue Aug 15, 2019
Original impl used a reference as a key in native map,
but in JNI object references are neither constant nor unique,
and thus were violating the contract on hashCode.

This allowed to get rid of global reference in 2 places,
but there are 2 more (clearly identified) cases where
we still rely on side-effects of having a glabal reference
and thus leak some memory

Signed-off-by: Alexander Bezzubov <[email protected]>
@bzz bzz mentioned this issue Aug 15, 2019
5 tasks
bzz added a commit to bzz/client-scala that referenced this issue Aug 16, 2019
Original impl used a reference as a key in native map,
but in JNI object references are neither constant nor unique,
and thus were violating the contract on hashCode.

This allowed to get rid of global reference in 2 places,
but there are 2 more (clearly identified) cases where
we still rely on side-effects of having a glabal reference
and thus leak some memory

Signed-off-by: Alexander Bezzubov <[email protected]>
@bzz bzz modified the milestones: v2.1.0, v2.0.1 Aug 16, 2019
bzz added a commit to bzz/client-scala that referenced this issue Aug 16, 2019
Original impl used a reference as a key in a native map,
but in JNI object references are neither constant nor unique,
and thus were violating the contract on hashCode for a map key.

This should allow to get rid of global reference in interface lookup,
but there are 2 more (clearly identified) cases where
we still rely on side-effects of having a glabal reference
and thus leak some memory

Signed-off-by: Alexander Bezzubov <[email protected]>
bzz added a commit to bzz/client-scala that referenced this issue Aug 16, 2019
Original impl used a reference as a key in a native map,
but in JNI object references are neither constant nor unique,
and thus were violating the contract on hashCode for a map key.

This should allow to get rid of global reference in interface lookup,
but there are 2 more (clearly identified) cases where
we still rely on side-effects of having a glabal reference
and thus leak some memory

Signed-off-by: Alexander Bezzubov <[email protected]>
@bzz
Copy link
Contributor

bzz commented Aug 16, 2019

This is a good observation, but merely implementing the suggestion will result in test failures.

I have submitted a patch for what seems to be one of the causes of this at #116 (cann't use refs as keys for the map really) but it is still a partial solution - if global ref is removed there, it will fail the PreOrder managed iterator test :/

I suggest also adding to the scope of this issue identifying the reason of why does Interface::toJ has to do getJNIEnv()->NewGlobalRef()?

It does not seem to need one for the same reasons, doesn't it? But removing it causes BblfshClientLoadTest to fail with java.lang.AbstractMethodError: org.bblfsh.client.v2.JNode.size()I on Context.encode.

I hope that new materials from #115 on JNI debugging at CONTRIBUTING.m will help to find an answer.

IMO it makes sense to start with this one and when fixed 🔪 a release, so added it to v2.0.1 milestone.

bzz added a commit that referenced this issue Aug 16, 2019
Original impl used a reference as a key in a native map,
but in JNI object references are neither constant nor unique,
and thus were violating the contract on hashCode for a map key.

This should allow to get rid of global reference in interface lookup,
but there are 2 more (clearly identified) cases where
we still rely on side-effects of having a glabal reference
and thus leak some memory

Signed-off-by: Alexander Bezzubov <[email protected]>
@bzz bzz added bug and removed question labels Aug 16, 2019
ncordon added a commit that referenced this issue Aug 19, 2019
ncordon added a commit that referenced this issue Aug 19, 2019
ncordon added a commit that referenced this issue Aug 20, 2019
ncordon pushed a commit that referenced this issue Aug 20, 2019
Original impl used a reference as a key in a native map,
but in JNI object references are neither constant nor unique,
and thus were violating the contract on hashCode for a map key.

This should allow to get rid of global reference in interface lookup,
but there are 2 more (clearly identified) cases where
we still rely on side-effects of having a glabal reference
and thus leak some memory

Signed-off-by: Alexander Bezzubov <[email protected]>
ncordon added a commit that referenced this issue Aug 20, 2019
ncordon added a commit that referenced this issue Aug 20, 2019
ncordon pushed a commit that referenced this issue Aug 26, 2019
Original impl used a reference as a key in a native map,
but in JNI object references are neither constant nor unique,
and thus were violating the contract on hashCode for a map key.

This should allow to get rid of global reference in interface lookup,
but there are 2 more (clearly identified) cases where
we still rely on side-effects of having a glabal reference
and thus leak some memory

Signed-off-by: Alexander Bezzubov <[email protected]>
ncordon added a commit that referenced this issue Aug 26, 2019
ncordon added a commit that referenced this issue Aug 26, 2019
ncordon pushed a commit that referenced this issue Aug 28, 2019
Original impl used a reference as a key in a native map,
but in JNI object references are neither constant nor unique,
and thus were violating the contract on hashCode for a map key.

This should allow to get rid of global reference in interface lookup,
but there are 2 more (clearly identified) cases where
we still rely on side-effects of having a glabal reference
and thus leak some memory

Signed-off-by: Alexander Bezzubov <[email protected]>
ncordon added a commit that referenced this issue Aug 28, 2019
ncordon added a commit that referenced this issue Aug 28, 2019
@ncordon
Copy link
Member Author

ncordon commented Aug 29, 2019

Closed by #116

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants