Spectral-to-Spectral Loss Code

[AI-Tianlong]

您好，非常出色的工作。在查看您源代码的时候，在models_mae_spectral.py中，我并没有看到关于Spectral-to-Spectral的 LOSS 是如何实现的，仅看到了 loss1+loss3，应该是关于Token-to-Token的损失，并且看到一个未使用的计算光谱角的calculate_metrics_per_pixel函数，但是并没有在LOSS中看到使用，想请教一下Spectral-to-Spectral Loss 是如何实现的呢？

Hello, excellent work. When reviewing your source code in models_mae_spectral.py, I did not see how the Spectral-to-Spectral LOSS is implemented. I only saw loss1 + loss3, which seems to be related to Token-to-Token loss. I also noticed an unused calculate_metrics_per_pixel function for calculating the spectral angle, but it is not used in the LOSS. Could you please explain how the Spectral-to-Spectral Loss is implemented?

IEEE_TPAMI_SpectralGPT/models_mae_spectral.py

Lines 453 to 465 in b42a079

	loss1 = (pred - target1) ** 2 # pred: 2,1024,192
	loss1 = loss1.mean(dim=-1) # [N, L], mean loss per patch #2,1024
	mask = mask.view(loss1.shape)
	loss1 = (loss1 * mask).sum() / mask.sum()
	loss3 = (pred_spatial - target_spatial) ** 2 # 2,4,192
	loss3 = loss3.mean(dim=-1)
	mask3 = torch.ones([N, h * w], device=loss3.device)
	loss3 = (loss3 * mask3).sum() / mask3.sum()
	loss = loss1 + loss3 # mean loss on removed patches
	return loss

[moonboy12138]

你好，loss1是token-to-token的实现，loss3是Spectral-to-Spectral的实现，这与论文中的公式（3）是保持一致的。calculate_metrics_per_pixel函数是我们在后续研究中所使用的函数，与本工作没有关系~后续研究表面该函数计算loss是也可以使得模型很好地学习光谱表征。

Hello, loss1 is the implementation of token-to-token, and loss3 is the implementation of Spectral-to-Spectral, which is consistent with formula (3) in the paper. The calculate_metrics_per_pixel function is used in our subsequent research and is not related to this work. Subsequent research shows that this function can also enable the model to effectively learn spectral representation when calculating the loss.

[AI-Tianlong]

感谢回复！真的很棒的工作！