While using karlsend v2.2.1 SDK in a mining pool proxy, we observed significant memory pressure and high allocation rates (~150MB per share submission), leadng to gigabytes of total allocations even with a low number of miners.
Profiling with pprof shows that the majority of allocations originate from pow.getContext and pow.buildLightCache:
4617MB 98.12% 98.12% 4689MB 99.65% github.com/karlsen-network/karlsend/v2/domain/consensus/utils/pow.getContext
72MB 1.53% 99.65% 72MB 1.53% github.com/karlsen-network/karlsend/v2/domain/consensus/utils/pow.buildLightCache
0 0% 99.65% 4689MB 99.65% ...proxy.(*ProxyServer).handleSubmitRPC.func1
0 0% 99.65% 4689MB 99.65% ...proxy.(*ProxyServer).processShare
0 0% 99.65% 4689MB 99.65% github.com/karlsen-network/karlsend/v2/domain/consensus/utils/pow.NewState
Cause: pow.NewState is called for every submitted share to verify PoW. It seems pow.getContext recreates the light cache/dataset context every time instead of caching it globally or within a session, even when the block height/epoch hasn't changed.
Requested Change: Implement internal caching for the light cache context within the pow package based on the epoch, similar to how it's handled in other PoW implementations (e.g., ethash).
While using karlsend v2.2.1 SDK in a mining pool proxy, we observed significant memory pressure and high allocation rates (~150MB per share submission), leadng to gigabytes of total allocations even with a low number of miners.
Profiling with pprof shows that the majority of allocations originate from pow.getContext and pow.buildLightCache:
4617MB 98.12% 98.12% 4689MB 99.65% github.com/karlsen-network/karlsend/v2/domain/consensus/utils/pow.getContext
72MB 1.53% 99.65% 72MB 1.53% github.com/karlsen-network/karlsend/v2/domain/consensus/utils/pow.buildLightCache
0 0% 99.65% 4689MB 99.65% ...proxy.(*ProxyServer).handleSubmitRPC.func1
0 0% 99.65% 4689MB 99.65% ...proxy.(*ProxyServer).processShare
0 0% 99.65% 4689MB 99.65% github.com/karlsen-network/karlsend/v2/domain/consensus/utils/pow.NewState
Cause: pow.NewState is called for every submitted share to verify PoW. It seems pow.getContext recreates the light cache/dataset context every time instead of caching it globally or within a session, even when the block height/epoch hasn't changed.
Requested Change: Implement internal caching for the light cache context within the pow package based on the epoch, similar to how it's handled in other PoW implementations (e.g., ethash).