Memory leak on backup restore with xquery triggers

[line-o]

Describe the bug

I was able to reproduce the memory leak using the original dataset provided by @dariok. With enough resources, about 10 GB of RAM, the process did finish successfully. After restoring the entire dataset of 35 GB, containing 26874 XML files (including .html, .xsl, .svg), almost 5 GB of RAM retained.

There is ~59000 instances of TriggerStatePerThread$TriggerState. Since I can see that there is two instances for each restored item that is ~29500 CREATE_DOCUMENT triggers - one for BEFORE and for the AFTER phase.

I was then wondering whether I was missing +3000 documents until I discovered that CREATE_COLLECTION was also invoked:

The original `collection.xconf` defines an update trigger as such: ``` ```

With the collection.xconf as it is in the issue description (without any triggers configured) the outcome was very different. RAM usage stayed within reasonable bounds (<2 GB) and all memory was freed afterwards (see screenshot).

My analysis indicates that TriggerStates are held in memory indefinitely and are the cause of the leak. This is especially interesting as the events that were triggered are not the ones that are defined and thus one would not expect them to be fired at all.

References

refs #4890
refs #5295

Expected behavior

XQueryTriggerStates to be removed from memory once the trigger is done.

To Reproduce

If the above isn't working, please tell us the exact steps you took when you encountered the problem:

full-backup-restore-with-triggers.zip

1. start a clean instance of exist-db v6.2.0 with a profiler attached
2. restore the backup in the attached zip
3. the backup finishes
4. in the Profiler you should be able to find XqueryTriggerState instances that retain in memory

Screenshots

see above

Context

• Build: eXist-6.2.0
• Java: 1.8.0_352
• OS: Mac OS X 14.5

Additional context

• How is eXist-db installed? built from source
• Any custom changes in e.g. conf.xml? none

[dizzzz]

images are not visible?

[adamretter]

images are not visible?

@line-o Could we get an update please?

[line-o]

@adamretter Are you waiting for the screenshots? They are the same ones as in the other issue. I was expecting your PR, you were so eager to publish.
I really did not expect these to be important.

[adamretter]

They are the same ones as in the other issue

@line-o Good to hear - but we had no idea that was the case as you hadn't replied! I have now fixed the links to the images for you so that the issue makes more sense (cc @dizzzz).