How to understand SENet

How to understand SENet, I believe that many inexperienced people are at a loss about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

SENet is the champion model of ImageNet 2017 (ImageNet final). Similar to the emergence of ResNet, it reduces the error rate of the previous model to a great extent (see appendix), and has low complexity, new parameters and less computation. Here are some of the wonders of SENet.

The full name of SENet is Squeeze-and-ExcitationNetworks, and it can be translated into compressed and energized network in Chinese. It mainly consists of two parts:

1. Squeeze part. That is, the compressed part, the dimension of the original feature map is H*W*C, where H is the height (Height), W is the width (width) and C is the number of channels (channel). What Squeeze does is to compress H*W*C to 1mm 1cm C, which is equivalent to compressing HandW into one dimension. In practice, it is generally implemented in global average pooling. After HendW is compressed into one dimension, this parameter obtains the global view of HallowW, and the receptive area is wider.

2. Excitation part. After getting the representation of Squeeze, add a FC full connection layer (Fully Connected), predict the importance of each channel, get the importance of different channel, and then act on the corresponding channel of the previous feature map, and then carry on the follow-up operation.

As you can see, SENet and ResNet are very similar, but do more than ResNet. ResNet only adds a skip connection, while SENet adds processing between adjacent layers, which makes the exchange of information between channel possible and further improves the accuracy of the network.

SENet can be inserted into any network at will, and the improvement effect is quite significant. The result given in this paper is that there is a reduction of error in the range of 0.4% to 1.8%.

The training curve is also beautiful, and the bottom orange is the result of SENet:

Appendix:

ImageNet classification Top5 error rate:

2014 GoogLeNet 6.67%

2015 ResNet 3.57%

2016 ~ 2.99%

2017 SENet 2.25%

After reading the above, have you mastered how to understand SENet? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!