opt: Replace COO with CSR sparse tensors for bfloat16/float16 support & faster speedups [IN PROGRESS]

[wz-ml]

Description

WIP for further optimization experimentation. Stuff that works:

• CSR format
  • Supports bfloat16 & float16 on Ampere
  • Speedups are significant on Ampere
  • Supports bfloat16 & float16 on Hopper
  • Speedups are significant on Hopper
• Gather-reduce implementation (experiment)

Type of change

• New feature (non-breaking change which adds functionality)

Checklist:

• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing tests pass locally with my changes
• I have not rewritten tests relating to key interfaces which would affect backward compatibility

You have tested formatting, typing and tests

• I have run make check-ci to check format and linting. (you can run make format to format code if needed.)