Shulou(Shulou.com)06/02 Report--

This article mainly explains "tensorflow view gradient mode is what," interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let Xiaobian take you to learn "tensorflow view gradient mode is what"!

1. Why View Gradient

For beginners, the network often does not converge, and the loss is very strange (that is, it does not converge), so it is suspected that it is a gradient problem in reverse propagation.

(1) The number (absolute value) after the derivative is smaller and smaller (approaching 0), which is the disappearance of the gradient.

(2) The number (absolute value) after the derivative is getting larger and larger (especially large and divergent), which is gradient explosion.

So, when the loss is abnormal, you can see if the gradient is exploding or disappearing. If the gradient explodes, the W in the network will also be very large. Manual control (make it smaller when initializing, etc. There must be other methods, but I don't know. The great god who knows can also tell me a little ~~). If the gradient disappears, you can try using resnet,densenet and the like.

2. How to view gradients in tensorflow

tf.gradients(y,x) where y is the derivative of x (dy/dx),x and y must be related oh ~

IF IF

At this point, I believe everyone has a deeper understanding of "tensorflow view gradient mode", may wish to actually operate it! Here is the website, more related content can enter the relevant channels for inquiry, pay attention to us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.