Propagate FA3 build fix: rename `#include`-d `.cu` files to `.cuh` to avoid double-compilation

[moon-bot-app]

Tracking issue to propagate the build optimization landed in Flash Attention 3 (#999) to other kernel sources in this org when relevant.

Background

FA3 (and some other kernels) have a pattern where .cu files are #include-d inside other .cu files. Because the CMake rule compiles every .cu file, the included files end up compiled twice — once directly, once through the file that includes them. This inflates both compilation time and binary size.

The core issue is that #include semantics are "literally paste the file contents here," but that breaks down once build rules are applied to files by extension.

The fix

Rename any .cu file that is only meant to be #include-d into another translation unit to .cuh, so it's treated as a header/dependency of the kernel component rather than a separately-compiled source file.

Impact (FA3, PR #999)

• Substantially faster compilation.
• Binary size reduction, e.g. 821 MiB → 550 MiB for CUDA 13.

We could apply the same to sgl-flash-attn3 and vllm-flash-attn3.

Ask

Audit other kernels in kernels-community for the same #include-d-.cu pattern and apply the equivalent rename where relevant.

---

Requested by Sayak Paul - Slack thread - Agent trace