Better compression

[utagawal]

According to https://makina-corpus.com/sig-webmapping/optimisation-tuiles-mnt-rgb-ombrage-dynamique-mapbox-gl-maplibre-gl
it could be possible to save up to 40% size by using Webp format.

They also have a way to reduce tha altitude precision by zoom level and would generate up to 80% space savings.

See implementation here : https://www.data.gouv.fr/fr/datasets/r/7ce01e48-fb89-4440-8098-78d7f758829a

Any chance that could be tested / implemented if this would be the case ?

[acalcutt]

Based on my recent testing in this thread ConservationMetrics/mapgl-tile-renderer#44 (comment) , I would say the size difference is probably true. In the rendered tiles there webp was much smaller. The only thing that concerns me is webp compatibility. Now that most browsers are based on chrome and IE is unsupported, its probably not as much of and issue, but I feel like even firefox support is pretty recent ( though looking it seems most bowsers after 2020 support it now )

As for the adjusting precision, my last attempt of that I did not like. It worked ok for hillshade, but I wasn't happy at all with how it looked for terrain. It is possible their process looks better, but I'm not sure I have the time to work on it right now.

[utagawal]

Thanks. When you'll have the bandwidth : The second link gives the "for loop" to change the precision. Le dimanche 3 mars 2024 à 18:06:44 UTC+1, Andrew Calcutt ***@***.***> a écrit : Based on my recent testing in this thread ConservationMetrics/mapgl-tile-renderer#44 (comment) , I would say the size difference is probably true. In the rendered tiles there webp was much smaller. The only thing that concerns me is webp compatibility. Now that most browsers are based on chrome and IE is unsupported, its probably not as much of and issue, but I feel like even firefox support is pretty recent ( though looking it seems most bowsers after 2020 support it now ) As for the adjusting precision, my last attempt of that I did not like. It worked ok for hillshade, but I wasn't happy at all with how it looked for terrain. It is possible their process looks better, but I'm not sure I have the time to work on it right now. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[acalcutt]

I looked at that, but it is very similar to what i tried before that did not work well for terrain
https://github.com/acalcutt/jaxa_AW3D30_to_MBTiles/blob/smaller_terrainrgb/create_terrainrgb.sh

[acalcutt]

Since I have the JAXA geotiffs downloaded already and I've been wanting to test the compact feature I added to https://github.com/acalcutt/rio-rgbify/tree/compact , I made a run with webp z0-Z12 that you can download by torrent here
https://techidiots.net/downloads/open-base-map-torrents/mbtiles/jaxa_terrainrgb_z0-12_webp-torrent/view

I am making a new png version right now so I can compare the actual space saving, since the old file I made was not deduplicated, which should help a lot for oceans where the jaxa dataset has no data

[utagawal]

Great news @acalcutt this is an impressive size improvement !

That could then revise the feasibility of #3 (comment) ?

[acalcutt]

I finished making the new png version of z0-Z12 https://www.techidiots.net/downloads/open-base-map-torrents/mbtiles/jaxa_terrainrgb0-12-torrent/view

The new file made with my compact feature looks like it only saved about 23GB, since the new file was 404GB vs the old png one I made that was 427GB. The WEBP one with the compact feature is only 263GB

[utagawal]

New Webp terrain layer is in production on our map server. Thanks to you we saved a lot of disc space and bandwidth.
Have a look at https://maps.UtagawaVTT.com

[acalcutt]

Your map is looking great :-)

[utagawal]

In case you want to try another way (?) have a look here to see how they implement better compression : https://github.com/SilvioDallAcqua/terrain-rgb/tree/main?tab=readme-ov-file

using this command :

rio rgbify -b -10000 -i 0.1 --max-z 5 --min-z 5 --format webp -j 16 --round-digits 11 swissaltiregio_3857.tif swissaltiregio_05.mbtiles
rio rgbify -b -10000 -i 0.1 --max-z 6 --min-z 6 --format webp -j 16 --round-digits 10 swissaltiregio_3857.tif swissaltiregio_06.mbtiles
rio rgbify -b -10000 -i 0.1 --max-z 7 --min-z 7 --format webp -j 16 --round-digits 9 swissaltiregio_3857.tif swissaltiregio_07.mbtiles
rio rgbify -b -10000 -i 0.1 --max-z 8 --min-z 8 --format webp -j 16 --round-digits 8 swissaltiregio_3857.tif swissaltiregio_08.mbtiles
rio rgbify -b -10000 -i 0.1 --max-z 9 --min-z 9 --format webp -j 16 --round-digits 7 swissaltiregio_3857.tif swissaltiregio_09.mbtiles
rio rgbify -b -10000 -i 0.1 --max-z 10 --min-z 10 --format webp -j 16 --round-digits 6 swissaltiregio_3857.tif swissaltiregio_10.mbtiles
rio rgbify -b -10000 -i 0.1 --max-z 11 --min-z 11 --format webp -j 16 --round-digits 5 swissaltiregio_3857.tif swissaltiregio_11.mbtiles
rio rgbify -b -10000 -i 0.1 --max-z 12 --min-z 12 --format webp -j 16 --round-digits 4 swissaltiregio_3857.tif swissaltiregio_12.mbtiles
rio rgbify -b -10000 -i 0.1 --max-z 13 --min-z 13 --format webp -j 16 --round-digits 3 swissaltiregio_3857.tif swissaltiregio_13.mbtiles