Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature maps from intermediate layer #318

Closed
patriksabol opened this issue Aug 9, 2024 · 2 comments
Closed

Feature maps from intermediate layer #318

patriksabol opened this issue Aug 9, 2024 · 2 comments

Comments

@patriksabol
Copy link

Hello,

Are there any best practices or guidelines for selecting feature maps from the intermediate layers (out of the 12 available)? Specifically, what do these feature maps represent, and in what scenarios should they be used?

For example, in downstream training with NAIP (RGBN) data, feature maps from layers 3, 5, 7, and 11 were utilized. For RGB data, would layers 3, 5, and 7 be appropriate? Why not layers 2, 6, or 8, for instance?

Cheers.

@srmsoumya
Copy link
Collaborator

@patriksabol There isn't a specific meaning assigned to each layer, but typically, earlier layers tend to capture simpler features like lines, edges, and basic shapes, while later layers identify more complex structures like field boundaries and road networks. When working on segmentation-style modeling, it's beneficial to select a balanced mix of both simple and complex features. You might consider using layers like 1, 3, 7, and 9 or 2, 4, 6, and 8. For a deeper understanding, I recommend checking out this paper: https://arxiv.org/pdf/2212.06727.

@brunosan
Copy link
Member

brunosan commented Dec 6, 2024

Since this tickets was opened, @srmsoumya reports that we have since simplified the decoder, we take the last layer output at scale of 32 x 32 and then upscale it using convolution blocks in the decoder. See https://github.com/Clay-foundation/model/blob/main/finetune/segment/factory.py and documentation for v1.5.

@brunosan brunosan closed this as completed Dec 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants