How to realize softmax in pytorch

This article introduces the relevant knowledge of "how to achieve softmax in pytorch". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Catalogue

Initialize model parameters

Re-examine the implementation of softmax

Optimization algorithm

The classification model can also be implemented more easily through the advanced API of the deep learning framework. Let's continue with the Fashion-MNIST dataset and keep the batch size at 256.

Import torchfrom torch import nnfrom d2l import torch as d2lbatch_size = 256train_iter, test_iter = d2l.load_data_fashion_mnist (batch_size) initialize model parameters

Since the output layer of sofrmax regression is a full connection layer, in order to implement our model, we only need to add a full connection layer with 10 outputs to Sequential. Again, Sequential is not necessary here, but we may form this habit. Because when implementing a deep model, Sequential will be everywhere. We still randomly initialize the weights with mean 0 and standard deviation 0.01.

# PyTorch does not implicitly adjust the shape of the input. Therefore, we define a flattening layer (flatten) in front of the linear layer to adjust the shape of the network input net = nn.Sequential (nn.Flatten (), nn.Linear (784,10)) def init_weights (m): if type (m) = = nn.Linear: nn.init.normal_ (m.weight, std=0.01) net.apply (init_weights) re-examine the implementation of softmax

In the previous example, we calculated the output of the model and then fed this output into the cross-entropy loss. Mathematically speaking, this is a perfectly reasonable thing. However, from a computational point of view, the index may cause problems of numerical stability, including overflows and underspills.

We also want to keep the traditional softmax function in case we need to evaluate the probability of passing the model output. However, instead of transferring the softmax probability to the loss function, we transfer the unnormalized prediction in the cross-entropy loss function, and calculate the softmax and its logarithm at the same time.

Loss = nn.CrossEntropyLoss () optimization algorithm

Here, we use a small batch random gradient descent with a learning rate of 0.1 as the optimization algorithm. This is the same as our linear regression example, which illustrates the universality of the optimizer.

Trainer = torch.optim.SGD (net.parameters (), lr=0.1)

This is the end of the content of "how to implement softmax in pytorch". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!