WebGPURenderer: Unlit materials should not be tone mapped

[WestLangley]

Description

WebGLRenderer:

MeshNormalMaterial inherits the .toneMapped property as true, but the material omits the tone mapping shader chunks, so the material is never tone mapped.

This is the correct behavior. We never want the material to be tone mapped.

WebGPURenderer:

MeshNormalMaterial responds to tone mapping. It should not. This needs to be fixed.

Note: There are other problems with MeshNormalMaterial that cause the material to be washed out when rendered with WebGPURenderer. That will be addressed in a separate PR after this issue is resolved.

Reproduction steps

Code

// code goes here

Live example

• jsfiddle-latest-release
• jsfiddle-dev

Screenshots

No response

Version

r167dev

Device

No response

Browser

No response

OS

No response

[sunag]

This is not a bug, .material.toneMapped=false and material.colorSpaced=false are not supported by WebGPURenderer. #27880 #28779 (comment) #27238 (comment)

It is still possible to use MeshNormalMaterial in RenderTarget without toneMapping, where it would have more useful.

[Mugen87]

Maybe it's important to highlight once more that WebGPURenderer does not support inline tone mapping and color space conversion. Hence, it is not possible to control which materials should be tone mapped or not. I think this is a good change since it solves #23019 and makes the tone mapping and color space step more consistent.

So WebGPURenderer always handles tone mapping and color space conversion as a separate render pass and thus processing the entire image.

[Mugen87]

The main purpose for MeshNormalMaterial is to save normals in a render target for further processing. The fact that it can be used as a "normal" material for rendering to screen is a nice side effect but not its main purpose. So retaining the visual look of MeshNormalMaterial is negligible, imo.

[WestLangley]

WebGLRenderer on the left; WebGPURenderer on the right. No tone mapping applied.

IMHO, I do not think this is acceptable. By convention, the packed normal, when interpreted as a color, is assumed to be in sRGB color space. WebGPURenderer should not be applying the sRGB transfer function to an sRGB color.

[WestLangley]

Maybe it's important to highlight once more that WebGPURenderer does not support inline tone mapping and color space conversion. Hence, it is not possible to control which materials should be tone mapped or not.

Hmm... I am inclined to think that is too restrictive. If all materials are tone mapped, I think we have to have a way to disable it.

Think of overlay text or a heads-up display. That would not normally be tone mapped.

Nor should something like AxesHelper -- and MeshLineMaterial. Also MeshBasicMaterial in certain cases.

Tone mapping can cause significant hue shifts. So can exposure control.

[sunag]

Think of overlay text or a heads-up display. That would not normally be tone mapped.

HUD is normally used in another render-pass, so it won't be affected.

[WestLangley]

What about the other issues I mentioned?

[Mugen87]

The toneMapped property was coupled to inline tone mapping. There is no way to support this property when tone mapping is applied as a pass.

@WestLangley I don't think we can achieve what you are asking for without changing some fundamentals in the renderer. I do not vote for doing this since tone mapping and color space conversion is now more correctly implemented than in WebGLRenderer.

[Mugen87]

There was always a difference when tone mapping was applied inline and when used in FX workflows via OutputPass. This difference is now finally resolved in WebGPURenderer.

[sunag]

Since AxesHelper overlaps all other objects, this could be solved with a render-pass as well.

I believe that other cases could be solved with MRT properties applied to the material and PostProcessing, such as:

material.mrtNode = mrt( { overlay: directionToColor( normalWorld ) } );

This is being developed initially here #28833

if we look at the projects from the three.js home page, I think 99% will make better use of the new system, normals for visual purposes is more related to our example.

[WestLangley]

WebGPURenderer removes the ability for users to set material.toneMapped to false, a feature which has been used by WebGLRenderer internally -- particularly for objects that do not respond to lights.

Another example use case would be text labels.

Historical context: a related post from @bhouston.

[Mugen87]

Sorry, but I think this goes in the wrong direction. The toneMapped property is controversial right from the beginning since you can solve the mentioned issues with other approaches as well. Separate, non-tone mapped render passes or optimized MRT passes should address stuff like helper objects, text labels or HUD.

If tone mapping is configured for a render output, it should affect the entire image and not specific objects. That is also true for color space conversion. Issues like #24362, #23019 or clear color related tone mapping and sRGB issues stem from the fact the engine did not implement this step correctly.

I think it's best if Material.toneMapped is deprecated as soon as WebGLRenderer is deprecated in the future.

[bhouston]

Normal, depth and velocity materials (and similar utility materials) are intended to be rendered into linear space only - never sRGB and also never tone mapped. This is because they need to be encoded consistently so that they can be decoded consistently. Thus toneMapping should never be supported on them.

[mrdoob]

Should we add more logic in the renderers to automatically render objects that shouldn't be tone mapped in a different pass?

That way we won't need to deprecate material.toneMapped... 🤔

[donmccurdy]

Should we add more logic in the renderers to automatically render objects that shouldn't be tone mapped in a different pass?

I think that could be a good option.

I'm also wondering if we may want to bring back the option of rendering directly to the drawing buffer (with tone mapping and color space transforms in the fragment shader), simply for performance reasons. If so that would be another option here.

[WestLangley]

I'm also wondering if we may want to bring back the option of rendering directly to the drawing buffer (with tone mapping and color space transforms in the fragment shader).

Yes. I support that. I think that should be the default -- just as it is in WebGLRenderer. Granted, blending is not performed in linear color space in that case, but I think 99% of users do not care about those artifacts. Users who do care, can use post-processing. This would also allow material.toneMapped to be supported again, allowing for greater flexibility.

[bhouston]

It is 2x faster or better than using post-processing in my original tests - in part because it can use RGBA8 buffers rather than multiple FP buffers: #8250

[sunag]

Just wanted to clarify some pros and cons and why material.toneMapped is deprecated in WebGPURenderer.

Pros

• Pipeline optimized for refraction, we can have access to what has already been rendered in linear-color-space, so for materials with transmission or refractions we do not need to render opaque materials twice, this has proven to be much more performant for PBR.
• Transparent materials respect tone mapping and color conversion since they not are applied in overlay with opaque materials as it was inline.
• It is important that the changes are more aligned with future generations of hardware than with hardware that is limited to RGBA8 buffers. In this scenario, PostProcessing is something used in any project, like bloom for example and there will certainly not be the load mentioned above.
• Improve capacity of sharing the same GPU program when using RenderTargets.

Cons

• material.toneMapped will not be available per material.
• PostProcessing is mandatory if using tone-mapping or color-space conversion.

Alternative

• Implement a renderer.outputNode where we could control the output similar to what we did in PostProcessing.outputNode. XRManager: Fix tone mapping and output color space. #30387 (comment)

Just wanted to make a note of this. MeshNormalMaterial, MeshDepthMaterial, MeshDistanceMaterial are meant to be used in calculations in PostProcessing or during the beauty pass as uniform, if the user is mixing them with the beauty pass with others kind of materials for aesthetic purposes, they are certainly creating a project for fun and not serious math. That's why our recommendation is to use MRT or RenderTarget for this purpose.

[WestLangley]

Thank you, @sunag, for your comments.

I use MeshNormalMaterial for debugging normals. Rendering the correct colors is important in this case.

Regarding tone mapping, only lit materials need to be tone mapped -- not the entire scene. You wouldn't typically tone map a solid scene background color, scene clear-color, a heads-up display, LineBasicMaterial, or MeshBasicMaterial (unless it was reflecting an HDR environment). This is why material.toneMapped is important.

[donmccurdy]

For me the overriding concern is that color management and tone mapping must be cheap for mobile and WebXR devices with WebGPURenderer. When an application's performance budget can afford lighting, tone mapping should be near-zero extra cost. If that's not possible with post-processing (I don't have hardware to test this; do we feel #30387 has solved it?) then I think we really need to find a solution for the performance, and materials used for debugging like MeshNormalMaterial can be considered downstream of that problem.

That said... I don't feel that per-material opt-out for tone mapping is the only solution here at all, we don't have to take the same approach as WebGLRenderer if that's a concern.

Something like renderer.outputNode would be interesting. An API to move some objects (as opposed to materials?) to a different render pass, and to disable tone mapping on that pass, could work. That would be helpful for rendering UI, not just these debugging materials.

[WestLangley]

How about if we inline tone mapping in WebGPURenderer -- like we do for WebGLRenderer -- and apply the color space transform in OutputPass?

This allows material.toneMapped to be supported. Also, tone mapping does not change the linear color space, so blending is still correctly implemented with linear colors.

This way, we are able to tone map only the materials that respond to light -- not the entire scene.

[sunag]

Regarding tone mapping, only lit materials need to be tone mapped -- not the entire scene. You wouldn't typically tone map a solid scene background color, scene clear-color, a heads-up display, LineBasicMaterial, or MeshBasicMaterial (unless it was reflecting an HDR environment). This is why material.toneMapped is important.

I don't think all users are looking for this in the same way #26208 (comment), #26857 (comment). I created an example showing how to use fog and background without tone-mapping in a scene with tone-mapping using TSL https://threejs.org/examples/?#webgpu_custom_fog_background. It is recommended that HUDs be made in another render-pass, even to enable advanced effects such as blurring or blending like overlay. Inline tone-mapping does not solve what I mentioned above #28831 (comment) and TSL already supports blending background and fog without tone-mapping, the only strong reason I see is about skipping post-processing, which seems to me to be far from the reality of the current graphics generation but if we are going to add it as an alternative like renderer.outputNode then we can add this support for material.toneMapped.