feat: Gaussians LOD #189

HectorArellanoDev · 2025-05-04T12:51:14Z

Adding the gaussians splats rendering with LOD to Potree, has the new features:

loading and decoding the gaussian splatting information from Potree files.
implementing rendering with harmonics, enabling to include them or not when loading the files.
load the splats using a hierarchy, which enables LOD loading and rendering.
adaptive splat size rendering depending on the LOD and the camera distance.
new picking method for splats measurement rendering the IDs of the splats.

…ng will be done as points or splats

… to the GPU

…he LOD

turien-pix4d

The .DS_Store files cannot be committed to this repo. The rest is good tho 👍

example/main.ts

example/viewer.ts

turien-pix4d · 2025-05-05T06:40:28Z

src/loading2/decoder.ts

+			buffer = await response.arrayBuffer();
+		}
+
+        const pointAttributes = node.octreeGeometry.pointAttributes;


There are a lot of formatting issue in this repo, probably because there is no CI to check that automatically. I'll just make a PR where I run prettier once this one is merged

turien-pix4d · 2025-05-05T06:51:44Z

src/workers/SortWorker.ts

+
+        */
+
+        const buffer = toUint8Array('AGFzbQEAAAAADwhkeWxpbmsuMAEEAAAAAAEPAmAAAGAIf39/f39/f38AAg8BA2VudgZtZW1vcnkCAAADAwIAAQcjAhFfX3dhc21fY2FsbF9jdG9ycwAAC3NvcnRJbmRleGVzAAEKhgMCAwABC/8CAgN/A30gBwRAIAQqAighCyAEKgIYIQwgBCoCCCENQfj///8HIQlBiICAgHghBANAIAIgCkECdCIIaiALIAEgACAIaigCAEEEdGoiCCoCCJQgDSAIKgIAlCAMIAgqAgSUkpJDAACARZT8ACIINgIAIAkgCCAIIAlKGyEJIAQgCCAEIAhKGyEEIApBAWoiCiAHRw0ACyAGQQFrsyAEsiAJspOVIQtBACEEA0AgAiAEQQJ0aiIBIAsgASgCACAJa7KU/AAiATYCACADIAFBAnRqIgEgASgCAEEBajYCACAEQQFqIgQgB0cNAAsLIAZBAk8EQCADKAIAIQlBASEEA0AgAyAEQQJ0aiIBIAEoAgAgCWoiCTYCACAEQQFqIgQgBkcNAAsLIAdBAEoEQCAHIQQDQCAFIAcgAyACIARBAWsiAUECdCIGaigCAEECdGoiCSgCACIIa0ECdGogACAGaigCADYCACAJIAhBAWs2AgAgBEEBSyEGIAEhBCAGDQALCws=')


Probably that it would be easier to maintain to let webpack inject the wasm in the generated code when building the library. I found an example here where it intanciate the Webasembly on its own: https://github.com/ballercat/minimal-webpack5-wasm-demo/blob/main/webpack.config.js, and a simpler one here where webpack inject it as a string. I both case, the main benefit is that you don't have to copy pat code

yeah exactly. Or remove the wasm2js as a dependency since today it's not used during runtime, nor in the build process.

example/.DS_Store

src/workers/compile_wasm_no_simd_non_shared.sh

remove the .DS_Store file that went through the PR

remove the .DS_Store that was committed by mistake

example/main.ts

pjurczynski · 2025-05-05T09:30:35Z

example/main.ts

-        version: 'v1'
-    }, {
        file: 'metadata.json',
        url: 'https://test-pix4d-cloud-eu-central-1.s3.eu-central-1.amazonaws.com/lion_takanawa_converted/',


this doesn't look right. The v1 should still be in potree v1.
When I launch example, I cannot load the first model. In the console I get this error:

main.ts:91 TypeError: Failed to fetch at xhrRequest (potree.ts:82:1) at OctreeLoader.fetchMetadata (octree-loader.ts:264:1) at OctreeLoader.load (octree-loader.ts:251:1) at Potree.loadOctree [as loadGeometry] (load-octree.ts:5:1) eval @ main.ts:91 Promise.catch setupPointCloud @ main.ts:91 eval @ main.ts:119

in general I see a few issues with the example. I got another error when I moved the second slider

main.ts:61 Uncaught TypeError: Cannot set properties of undefined (setting 'value') at HTMLInputElement.eval (main.ts:61:1) eval @ main.ts:61

Could we add some labels to the sliders to understand what they're doing? It would also be great to have both models described somehow so that we know what we're loading in the ui.

pjurczynski · 2025-05-05T09:35:55Z

scripts/compile_wasm_script.sh

@@ -0,0 +1 @@
+em++ -std=c++11 sorter_no_simd.cpp -Os -s WASM=1 -s SIDE_MODULE=2 -o sorter_no_simd_non_shared.wasm -s IMPORTED_MEMORY=1


please add a target shell notation to the script:
https://www.shellcheck.net/wiki/SC2148

pjurczynski · 2025-05-05T09:40:05Z

scripts/compile_wasm_script.sh

redundant name "_script".

nitpick: I'm wondering if it wouldn't be better to put it in under "scripts" section in the package.json file.
Should this be a part of the building process? Or under what circumstances this should be helpful again?

renamed, it shouldn´t be too helpful since I do not think people will compile the sorter again, but I left it there if anyone wants to test how to compile the C sorter.

pjurczynski · 2025-05-05T09:44:51Z

src/loading2/load-octree.ts

 	url: string,
 	getUrl: GetUrlFn,
 	xhrRequest: XhrRequest,
+	loadHarmonics: boolean = false


You are using 2 different names for the feature. Should it be loadGaussians or loadHarmonics?

Fixed, it was an issue with the loaders that is fixed now, it only has to take into account if it load the harmonics (for the V2 version)

pjurczynski

Example shoudl work before we merge anything.

I would also like to take a look at testing. What can we effectively test here and how?

turien-pix4d · 2025-05-05T15:05:01Z

What can we effectively test here and how?

Setting up test is not super straightforward as we have no CI for this repo. So either we set up one, we agree on manually running UTs, or we run them in pre-commit hook.

turien-pix4d · 2025-05-06T07:10:31Z

src/loading2/gltf-splats-decoder.worker.js

+				rotations[4 * j + 3] = tempRotation.w;
+			}
+
+			attributeBuffers["orientation"] = { buffer: bufferRotations, attribute: "orientation" };


Some attributes are local static string and some others are defined in the enum PointAttibute. It would be better to have them all in the enum

pjurczynski · 2025-05-07T15:32:20Z

What can we effectively test here and how?

Setting up test is not super straightforward as we have no CI for this repo. So either we set up one, we agree on manually running UTs, or we run them in pre-commit hook.

I can setup a CI to run tests against this repo.

…ussians-lod

Hector added 23 commits April 8, 2025 14:31

start integration

c7fab8f

integrating

dc481e4

testing mobile version

95651ae

setting up a way to define if the harmonics should be loaded

f6b0583

setup a way to check if it is IOS, define a way to see if the renderi…

8c1368d

…ng will be done as points or splats

testing memory, defining parameters

e03f2e9

remove flickering

d70ba47

using a defer function to check the next frame so the data is updated…

5ad1005

… to the GPU

cleaning

77c0d40

refactoring the delay function

5bc9c9a

remove splats scenes

fe641f4

setup the decoding worker for the splats to work without Three

cd3afe1

updates for the shader to test harmonics

e07bd85

changes from Thibault requests

9c7144b

inserting the splats into the octree from Potree

2e789b2

include the splats mesh into the octree

54a0382

testing Thibault requests

bd69f09

testing

b1b4df8

solving the loading issue for the new decoder refactoring

a640039

level of detail for the splats size implementation

c87bc09

adding the sorter as base64 and implement the splats scale based on t…

f7b382a

…he LOD

modify the splats to be adaptive to the LOD and the camera distance

fdb835e

yarn lock as master

e2cdc7a

HectorArellanoDev requested review from pjurczynski and turien-pix4d May 4, 2025 12:53

HectorArellanoDev self-assigned this May 4, 2025

turien-pix4d requested changes May 5, 2025

View reviewed changes

Hector and others added 2 commits May 5, 2025 10:01

updates from Thibault

57f4973

Delete example/.DS_Store

5e9aebd

remove the .DS_Store file that went through the PR

HectorArellanoDev requested a review from turien-pix4d May 5, 2025 08:10

Delete .DS_Store

b63c50c

remove the .DS_Store that was committed by mistake

turien-pix4d approved these changes May 5, 2025

View reviewed changes

pjurczynski reviewed May 5, 2025

View reviewed changes

example/main.ts Outdated Show resolved Hide resolved

pjurczynski reviewed May 5, 2025

View reviewed changes

pjurczynski requested changes May 5, 2025

View reviewed changes

turien-pix4d reviewed May 6, 2025

View reviewed changes

Hector added 2 commits May 12, 2025 20:09

fix the example to load V1, V2 and splats

4dcc8cc

fix issue with DS_Store

2fbe25a

HectorArellanoDev requested a review from pjurczynski May 12, 2025 18:13

HectorArellanoDev and others added 4 commits May 12, 2025 20:13

Delete .DS_Store

733af0b

setup the depth mode for the point cloud material

0911020

modify the example to remove ths splats

8d0c444

Merge branch 'gaussians-lod' of github.com:pnext/three-loader into ga…

aa0f516

…ussians-lod

pjurczynski changed the title ~~Gaussians LOD~~ feat: Gaussians LOD Jun 4, 2025

pjurczynski approved these changes Jun 4, 2025

View reviewed changes

HectorArellanoDev merged commit 9d3f79b into master Jun 4, 2025

HectorArellanoDev deleted the gaussians-lod branch June 4, 2025 13:14

HectorArellanoDev restored the gaussians-lod branch June 17, 2025 08:25


		*/

		const buffer = toUint8Array('AGFzbQEAAAAADwhkeWxpbmsuMAEEAAAAAAEPAmAAAGAIf39/f39/f38AAg8BA2VudgZtZW1vcnkCAAADAwIAAQcjAhFfX3dhc21fY2FsbF9jdG9ycwAAC3NvcnRJbmRleGVzAAEKhgMCAwABC/8CAgN/A30gBwRAIAQqAighCyAEKgIYIQwgBCoCCCENQfj///8HIQlBiICAgHghBANAIAIgCkECdCIIaiALIAEgACAIaigCAEEEdGoiCCoCCJQgDSAIKgIAlCAMIAgqAgSUkpJDAACARZT8ACIINgIAIAkgCCAIIAlKGyEJIAQgCCAEIAhKGyEEIApBAWoiCiAHRw0ACyAGQQFrsyAEsiAJspOVIQtBACEEA0AgAiAEQQJ0aiIBIAsgASgCACAJa7KU/AAiATYCACADIAFBAnRqIgEgASgCAEEBajYCACAEQQFqIgQgB0cNAAsLIAZBAk8EQCADKAIAIQlBASEEA0AgAyAEQQJ0aiIBIAEoAgAgCWoiCTYCACAEQQFqIgQgBkcNAAsLIAdBAEoEQCAHIQQDQCAFIAcgAyACIARBAWsiAUECdCIGaigCAEECdGoiCSgCACIIa0ECdGogACAGaigCADYCACAJIAhBAWs2AgAgBEEBSyEGIAEhBCAGDQALCws=')

		@@ -0,0 +1 @@
		em++ -std=c++11 sorter_no_simd.cpp -Os -s WASM=1 -s SIDE_MODULE=2 -o sorter_no_simd_non_shared.wasm -s IMPORTED_MEMORY=1 No newline at end of file

feat: Gaussians LOD #189

feat: Gaussians LOD #189

Uh oh!

Conversation

HectorArellanoDev commented May 4, 2025

Uh oh!

turien-pix4d left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pjurczynski left a comment

Choose a reason for hiding this comment

Uh oh!

turien-pix4d commented May 5, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pjurczynski commented May 7, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

turien-pix4d left a comment •

edited

Loading