Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scaling factor calculation #11

Open
dshean opened this issue Jan 5, 2025 · 0 comments
Open

Scaling factor calculation #11

dshean opened this issue Jan 5, 2025 · 0 comments
Labels

Comments

@dshean
Copy link
Member

dshean commented Jan 5, 2025

In https://github.com/uw-cryo/DeepDEM/blob/main/notebooks/0d_Calculating_Scale_Factor.ipynb

    std_values = sum(map(return_sample_std, sampler), [])
            
    lower_percentile, upper_percentile = np.percentile(std_values,  5), np.percentile(std_values,  95)
    std_values = np.ma.masked_where((std_values < lower_percentile) & (std_values > upper_percentile), std_values)

The & logic doesn't make sense to me. I'm not sure this is actually doing anything. This should be or or should be specified for range between lower_percentile and upper_percentile.

            with rasterio.open(dsm_file) as ds:
                img = ds.read(1, window=rasterio.windows.from_bounds(minx, miny, maxx, maxy, transform=ds.transform)).flatten()
                img = np.ma.masked_where(img == ds.nodata, img)
                std_values.append(np.std(img))

See previously issue about opening directly as masked array using rasterio, rather than a separate step.
I haven't tested, but if img is a masked array, then you should be able to directly do img.std() here to computed std of unmasked values. I don't know if np.std will honor the masked array values, or if you need to explicitly use np.ma.std - and behavior may be different for numpy 1.X vs 2.X. I noticed environment.yml has numpy pinned at 1.26.4 (https://github.com/uw-cryo/DeepDEM/blob/main/environment.yml#L11), not entirely sure why.

@dshean dshean added the priority label Jan 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant