-
Notifications
You must be signed in to change notification settings - Fork 38
Data queue memory leak itoolkit 0.1.6 + idb-connector #230
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
There are some big overhauls going on, though we plan to be 99% compatible with v0.1 in v1.0. Those APIs will be deprecated once #151 lands, but they will stick around for the whole v1 lifetime. Please open a feature request if you want keyed data queue support. As to this issue, looks like were not closing the statement object we create. Compare v1.0.6 with master |
Also of note, the new DATA_QUEUE_* SQL services have support for keyed data queues and allow you to read/write. |
Deleting the statement object doesn't free the underlying resources, causing memory leaks. Instead, we need to close both the connection and the statement to free these resources. We can let the runtime delete the objects as well. Fixes #230
Thanks for the quick response. Tested your changes above and it looks much better now. Job temporary storage use is down at 50MB after 1000 send and receives rather than in the 300's and increasing. Will raise a feature request for keyed data queues tomorrow. |
Deleting the statement object doesn't free the underlying resources, causing memory leaks. Instead, we need to close both the connection and the statement to free these resources. We can let the runtime delete the objects as well. Fixes #230
We're having issues in production similar to this issue raised in idb-connector a while ago.
We're seeing the temporary storage use on the node job increase rather fast when reading and writing to data queues. This doesn't happen when using the REST transport so I thought this may be an issue with idb-connector. On further testing, I've found the issue doesn't seem to occur with the newer alpha1 version of itoolkit in npm repos using the Toolkit class to read/write data queues so decided to raise it here instead.
Are you still supporting issues with the older version of itoolkit? If not, are there plans to implement reading/writing keyed data queues in the newer version soon™ (it looks like Toolkit dataqueue stuff is being canned)?
Thanks.
The text was updated successfully, but these errors were encountered: