eckit::linalg::sparse::LinearAlgebraTorch backend #165
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a sparse lienar algebra backend to allow for GPU-based matrix multiplications (and other operations), which translates into a significant performance increase for interpolations, in the right conditions (right environment, advanced use of mir.)
It makes use of a deployed version of PyTorch (findable by CMake), specifically its lower level component "Torch", which is part of the same package (this is how it is released to the public.) I've exposed all possible hardware configuration options, contemporary. But obviously, the better development is to improve the whole workflow to avoid copies to/from the CPU/GPU, so this develolpment is purelly a stepping stone -- it has already allowed me to run both mars-client (C) and pgen on GPUs, and of course mir. It would be great to follow this up with a publication. Possibly, this could be configurable in earthkit-regrid for maximum marketing :-)
I've held back this development for several months, but I couldn't find a definite response on when to post it -- here it is.