op: render_pipeline,alpha_to_coverage

[shrekshao]

[kainino0x]

Note about another test we should add here:

Reviewing the PR reminded me of a very old thought, which is that we should have a test which doesn't have a 0th color attachment, and see what alpha-to-coverage does in that case. I actually have no idea what the behavior is supposed to be across APIs, but this would be a good thing to make sure actually behaves the same across APIs (and across implementations, depending on whether we pack sparse color attachments). Could be any of:

• coverage is always 100%
• coverage is always 0%
• it uses the first non-null attachment
• it's an error

Vulkan seems to say coverage is always 100%:

All alpha values in this section refer only to the alpha component of the fragment shader output that has a Location and Index decoration of zero (see the Fragment Output Interface section). If that shader output has an integer or unsigned integer type, then these operations are skipped.

See also gpuweb/gpuweb#1250

[shrekshao]

alpha_to_coverage with target[0] being empty test code is at #2393

Looks like on Win D3D12 and Linux Vulkan, the alphaToCoverage mask still uses shader output to target[0].alpha value. (test passes)

On Mac Metal, it looks like when target[0] is empty, it always uses alpha = 1 for the alphaToCoverage mask

[kainino0x]

Great find, thanks for testing it! I didn't even think about the fact we could have a fragment output that isn't used by an attachment.

Looks like on Win D3D12 and Linux Vulkan, the alphaToCoverage mask still uses shader output to target[0].alpha value

I had a lot of thoughts about this so have formatted it as a checklist of things to investigate/do:

• What happens on D3D12 or Vulkan if there is no fragment output at location(0)? undefined value, 1, 0, error?
  Vulkan spec sounds like it would be effectively 1 like I mentioned before.
  We should also test multiple GPUs in case there's any undefined behavior here.
  IMO most likely we should just add validation that there's a location(0) output (with an alpha component) if using alphaToCoverage.

• Similarly, what happens on Metal if there's an attachment 0 but it doesn't have an alpha channel (is an R or RG format)?

• We should bring findings to the gpuweb group, ideally with a proposal on what the spec should say and how to polyfill it across backends. Options:

  • 1: If allowed I think we would have to do the Metal behavior (use 1.0 if there's no attachment, regardless of what the shader outputs). In order to polyfill this behavior we would need to inject code to overwrite the fragment location 0 output to 1 if there's no corresponding attachment in the pipeline. Seems overly complex.
    • It's possible the Metal behavior is actually unintentional, a bug caused by optimizing away writes to the unused attachment 0. Worth asking.
  • 2: Just validate you must have an attachment 0 to use alphaToCoverage. This requirement doesn't strictly make sense but would prevent users from being able to hit the inconsistent behavior. And we could lift it in the future since Vulkan/D3D12 don't require it.

---

• A tangential thought: The Vulkan spec also has something called alphaToOne, which, IIUC, replaces the alpha channel of output 0 with 1.0 after generating the alphaToCoverage mask. We might already test this, but we should double check that this is not enabled by default on D3D12/Metal. Actually, the D3D11 functional spec is clear about this, so that's good:

  The fact that .a from output o0 is read and converted to coverage when AlphaToCoverageEnable is true does not change the .a value going to the Blender at RenderTarget 0 (if a RenderTarget happens to be set there).

• And another interesting tidbit from the D3D11 functional spec, worth mentioning even though we can't actually test NaNs. Let's leave a "POSTV1" TODO to add this when we add NaN support.

  An alpha value of NaN results in a no coverage (zero) mask.