Sisyphus tasks are getting stuck

[benoit74]

Yesterday I cancelled 1 task which was running since multiple days, but in fact this morning I realize that all tasks are stuck:

To be monitored in the coming days

[benoit74]

Again.

This time I went into the container and send SIGTERM to the crawler. Result is ... interesting (see last logs):

• https://farm.zimit.kiwix.org/pipeline/4bb5b36a-933a-4383-8e12-cc732688fb5d
• https://farm.zimit.kiwix.org/pipeline/37d89a48-f952-40e8-9736-d3f25c0dc08c
• https://farm.zimit.kiwix.org/pipeline/29eeae21-fde5-4e00-929d-44248fc74b85 (not interesting for this one)

For two tasks out of three, we have lots of Error getting cookies messages after the SIGTERM is received and hence while the crawler is exiting. Looks like these calls are blocking the exit.

[benoit74]

I've opened webrecorder/browsertrix-crawler#769

[benoit74]

Tasks continue to get stuck, will release latest browsertrix crawler this morning, let's see.

It also got stuck due to insufficient disk space as perceived by the manager, looks like OS is a bit bigger now. I decreased zimfarm size from 200G to 180G.