Suggestion: Adversarial Training to Improve Robustness in Coastline Extraction

[iamkb28]

Hi @fwitmer @rawann31,

I just read through the current progress on the project, and I thought of an idea that could help improve the model’s performance even further. If it’s already been considered, that’s great, but I wanted to suggest adversarial training.

While the pipeline for coastline extraction using NDWI with sliding windows and the re-training of the DeepWaterMap algorithm is a solid approach, satellite imagery (even with 3-5m resolution) can still be affected by factors like atmospheric interference, sensor noise, and lighting variations. These factors can lead to inaccuracies in coastline detection.

By training the model on adversarial images, we could improve its robustness to these distortions and ensure better generalization, especially in more complex or noisy imagery. If this hasn’t been implemented yet, it might be worth exploring.

Looking forward to hearing your thoughts on this!

[Shashank-248]

Hi @fwitmer @rawann31 @iamkb28 ,

Adversarial training to make the model more robust to noise and distortions , is an excellent idea. While adversarial training is a powerful method for improving generalization, I wanted to propose an additional idea to complement the existing pipeline:

Integrating Digital Elevation Models (DEMs) for terrain-based refinement and Short-Wave Infrared (SWIR) data for spectral enhancements.

Incorporating DEM data into the pipeline can help refine segmentation by addressing challenges in steep terrains or cliffs. For instance, areas above a certain elevation can be excluded as water bodies, reducing misclassifications.
DEM integration is computationally light and feasible for localized areas like Deering.
SWIR data can enhance the model’s ability to distinguish between water and land under challenging conditions like shadows, haze, or mixed pixels.
Given my current hardware setup, I’d like to start with integrating DEM into the existing NDWI and Otsu-based pipeline. Once this is in place, SWIR integration can be explored as a future enhancement to address spectral complexities.

I believe this combination of DEM and SWIR with adversarial training could significantly improve the model’s overall robustness and accuracy. Would love to hear your feedback on this direction!

Looking forward to collaborating on these exciting ideas!

[iamkb28]

Thanks @Shashank-248 for the suggestions! DEM and SWIR integration sound great for improving accuracy. I agree that combining them with adversarial training can make the model more robust. Looking forward to collaborating!

[rawann31]

@iamkb28 @Shashank-248

Thank you so much for your recommendations. I really appreciate them.
You are right, some Planet images, despite their high resolution, suffer from issues like lighting, shadows, and sensor noise. Integrating SWIR data is a great idea, but it comes with challenges. Since Planet does not provide SWIR data, we would need to explore other satellites that do. This could be tricky due to calibration requirements and differences in spatial resolution. However, if we find a suitable satellite and calibrate the data properly, it would be a valuable addition. I also recommend reading papers that discuss challenges in coastline extraction to identify other potential problems we may have overlooked.

If you have any other ideas or tested anything, please share it. I’m eager to hear from you!

[Shashank-248]

Hi @fwitmer @rawann31,

Thank you for your feedback. I understand the challenges associated with integrating SWIR data due to the lack of availability from Planet satellites and the calibration requirements with other sources( such as Sentinel-2 and Found in a thread/guide that platforms like Google Earth Engine allow remote processing of SWIR data. However much research and lookout is needed). I will look into relevant papers to identify additional challenges in coastline extraction and explore other potential solutions. In the meantime, I will proceed with integrating DEM into the existing NDWI and Otsu-based pipeline as a first step.

Looking forward to sharing more updates and collaborating further!

Best regards,
~ Shashank

[iamkb28]

Hi @fwitmer @rawann31,

Thank you for your thoughtful suggestions and insights! I'll definitely explore platforms like Google Earth Engine for SWIR data and review research papers to better understand the challenges in coastline extraction. Additionally, I'll focus on researching adversarial training techniques to explore how they can improve the model's robustness and accuracy. If I come across any promising findings or ideas, I'll be sure to share them with the team.

Looking forward to learning and contributing meaningfully!

Regards,
Khushi Bansal

[iamkb28]

Hi @fwitmer @rawann31 @Shashank-248,

I came across a research paper that proposes an innovative approach to coastline extraction, which could significantly enhance the current model without the need for adversarial training. By integrating these methods, we can potentially reduce computational resource usage and improve the overall accuracy of the coastline extraction process. The key steps from the paper are as follows:

1. Image Extraction
2. Histogram Matching: This step normalizes image intensity across different batches, addressing spectral variations in the PlanetLabs imagery.
3. Gaussian Blur Filter: This helps selectively reduce noise while preserving important coastal details.
4. Locally Adaptive Thresholding: Instead of using a fixed NDWI threshold, adaptive methods are employed to handle complex regions, such as water shadows and steep cliffs.
5. Canny Edge Detection: This enhances feature detection, improving the accuracy of coastline vectorization and providing cleaner outputs.

In the research, the authors extract images from a variety of satellite sources, including Envisat, ERS, Landsat, IKONOS, DMC, ALOS, SPOT, Kompsat, Proba, IRS, and SCISAT.

The image below serves as a reference of how this technique works:

Figure: Coastline extraction example using the proposed steps (source: [An Algorithm for Coastline Extraction
from Satellite Imagery, Vukadinov, Dejan, Raka Jovanovic, and M. I. L. A. N. Tuba, 2017]).

I believe this approach can significantly improve the performance and complement the existing plans with NDWI-based labeling and SWIR/DEM integration.

If this approach seems promising to you, I'd like to begin testing it on my local system to evaluate its effectiveness and compatibility with the current model. Please share your thoughts or any feedback on this approach.

Looking forward to hearing from you!

[rawann31]

https://github.com/rawann31/CoastlineExtraction/tree/master/DigitizedCoastlines

The available files are:

• HiResFinalCoastline_gt – Ground truth shapefile.

• PlanetCoastline_gt – Ground truth shapefile (more accurate than HiResFinalCoastline_gt, but either can be used).

• WestChukchi_exposed_STepr_rates – Transect lines used to calculate RMSE by measuring the difference between points that intersect our ground truth line and the generated coastline.

I get this image from a previous , I hope this image make things clearer. Black lines are transect lines.
Paper

[rawann31]

We use these files in the RMSE code.
For more details, you can check this file: rmse.py.

[iamkb28]

Thank you for the update and for uploading the ground truth coastlines!

[iamkb28]

Hi @rawann31!

I have emailed the GSoC proposal for your review. Please take a look when possible so I can make any necessary changes and submit it as soon as possible. Let me know if there’s any feedback.

[iamkb28]

Hi @rawann31!

I have re-shared my proposal. Please review it. Thanks.

[Shashank-248]

Hey @iamkb28 ,
Your proposed techniques are promising and would improve current model's accuracy conceptually. But on further lookin these multiple preprocessing steps increase computational load which offset the benefits gained. Further techniques like adaptive thresholding and edge detection require careful parameter tuning any improper tuning leads to suboptimal results as missed features and over-segmentation which leads to reduced accuracy. These techniques also are highly data dependent on the input satellite imagery. I will further look into these techniques for any improvements contrast to my explaining.

Hi @rawann31 @fwitmer ,
I am planning to incorperate Modified Normalized Difference Water Index along with NDWI for refined extractions in complex regions such as urban regions. Also training Support Vector Machine or Random Forest classifier using these spectral indices along with spatial feature such as DEM as input features. This would add robustness to the classification of water and land on water and non water pixels. I am proposing this hybrid approach to handle edge cases and improve the generalization.

Future planning about integration of additional SWIR bands for better differentiation between water, vegetation, and built-up areas.

Share your thoughts about this direction!

[iamkb28]

Hi @Shashank-248 ,

Thank you so much for your valuable feedback and thoughtful suggestions on my approach. I really appreciate the time and insights you've shared.

I found your approach with Modified NDWI and spatial features quite interesting. One aspect I'm particularly curious about is Alaska's dynamic coastline. Given the region's dynamic nature—frequent tidal fluctuations, glacial calving, and sediment movement—how do you envision the model adapting to these changes? Would it require frequent retraining, or do you suggest any adaptive modeling techniques to maintain accuracy over time?

[prachi-kedar]

Hello, @iamkb28 , @Shashank-248

I have really like the idea of modifying NDWI with spatial features. To address the issue of improving accuracy for dynamic nature of Alaskan Coastline we need to perform model retraining. Retraining model from scratch is not suggestive as it will take more time and computation. However , we can opt for transfer learning or can perform fine tunning. We can choose between two model retraining strategy such as Periodic retraining or Triggered Retraining. However , most robust strategy will be triggered retraining when significant changes in coastline behavior are detected. To detect those significant changes we can use time series analysis of satellite imagery. We can identify period where frequent changes has happened and predict the same of probability of happening the same events to perform model retraining. Second approach will be performing continual learning strategy such as based on reply and regularization we can used Average Gradient Episodic Memory ensures that the average loss on the old tasks does not decrease when learning the current one or architectural strategy such as Progressive Neural Network which uses transfer learning to combine the previous knowledge with the current one

[prachi-kedar]

Hello mentor @rawann31 ,

I have submitted my proposal & it would be great if I could get some feedback on my proposal so I could make necessary changes if required.

Thanks,
Prachi

[iamkb28]

Hi @rawann31,

I have submitted my proposal on the GSoC portal and I’d really appreciate any further feedback..

Regards,
Khushi Bansal