What is pytorch Cross Entropy loss function

This article mainly explains "what is the pytorch cross-entropy loss function". Interested friends may wish to take a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what is the pytorch cross-entropy loss function"?

Catalogue

1. Derivation of Cross Entropy loss function

two。 Intuitive understanding of Cross Entropy loss function

3. Other forms of Cross Entropy loss function

Speaking of the cross-entropy loss function "Cross Entropy Loss", its formula immediately comes to mind:

We are already very familiar with this cross-entropy function, and in most cases we just use it directly. But where did it come from? Why can it represent the difference between the real sample label and the predicted probability? Are there any other variants of the cross-entropy function above?

1. Derivation of Cross Entropy loss function

We know that in the binary classification problem model, such as logical regression "Logistic Regression", neural network "Neural Network" and so on, the label of the real sample is [0jue 1], which represents negative class and positive class respectively. At the end of the model, a Sigmoid function is usually used to output a probability value, which reflects the possibility of being predicted as a positive class: the greater the probability, the greater the probability.

The expression and graph of the Sigmoid function are as follows:

Where s is the output of the upper layer of the model, and the Sigmoid function has the following characteristics: when s = 0, g (s) = 0.510 s > > 0, g ≈ 1m s