Question on [shortcut] shape calculation

[jerry73204]

I read the actual implementation of [shortcut]. Look like it supports multilayer shortcut in the configuration, but I found something alarming to me.

If I understanding it correctly. The [shortcut] section has a from option, which are extra input layer indexes separated by comma apart from the previous layer (code). It never checks if one of the indexes happens to be the previous layer, nor checks if any of them are duplicated.

[shortcut] performs addition on from layers plus the previous layer. I expect the shapes (h, w, c) are equal among input layers. During the parsing phase, it checks the heights and widths, but does not panics but simply print error message if channels mismatch (code). In the training phase, it checks all of h, w, c matches only if there is single from layer (code), it also does not check channels in shape calculation (code). I find nowhere the about channels on multi-layer from case.

It looks weird to me. If the parsing code prints channels mismatch as an error, why channels are never verified against with each other? Maybe for some reason or I'm missing something. Interestingly, the example config files in cfg directory always shortcuts up to single layer. Hence, the multi-layer code are never touched in runtime.

Please point out if anything is missing here, or we can file a bug. Thanks.

[lsd1994]

Maybe not a bug, you can look here #4662 for more explanations.

[jerry73204]

Look it allows distinct shapes among input layers. I guess the they follow the rules. Please tell me if it is correct or not.

• The output shape is equal to that of immediate previous layer.
• When addition is performed on each from layer, we take the minimum number of channels among previous layer and current from layer, and add them to output up to the minimum number of channels.

So let's assume

• immediate previous layer has 5 channels
• two input layers in from option, respectively has 3 and 8 channels.

Is it true that for the first from layer, 3 channels are added to the output, while the latter from layer takes 5 channels?

[lsd1994]

I'm not sure, @AlexeyAB can you explain this?

[jerry73204]

After reading the actual implementation, I can confirm my guess is somewhat correct. Let me describe the working flow so others can see why.

The output shape is direct copy of shape of immediate previous layer. It ensures every from layer has the same heights and widths (code). The computation implementation can be found at forward_shortcut_layer(), which calls shortcut_multilayer_cpu() when running on CPU. The actual computation is located at the region.

The region consists of two levels of loops. The outer computes per shortcut output component, while the inner aggregates corresponding components from each from layer. It checks whether the output component index exceeds the corresponding component index of that from layer. So it's true that only up to a number of channels of values are taken. A minor detail is that values are normalized before adding to the output.

Hope this helps who want to understand about [shortcut].

[Fetulhak]

@jerry73204 @lsd1994 @eagledot I need a little bit example. if the immediate previous feature map is (26,26,256) and the from feature map is ( 26,26, 128) is it possible [shortcut ] layer to perform its operation? I was thinking [shortcut] only done if the two tesnors have exact shape for all c,h,w.