Class Constructor

• It's good practice to create a model in a Python class, inheriting from the torch.nn.Module. In the constructor, always call the parent (Module) constructor first, as it has useful attributes and methods.
• Use the constructor to define the layers, which can be for instance linear (traditional artificial neural network, with weights and biases), convolutional or recurrent. See available options here: https://pytorch.org/docs/stable/nn.html#linear-layers
• In here, dropout can also be defined, specifying the probability associated to it. Learn more about it here: https://pytorch.org/docs/stable/nn.html#dropout-layers

Feedforward

• A forward method is defined, receiving the input and applying the network's layers to it.
• The layers are connected in this method, usually using activation functions such as ReLU in hidden layers and softmax in the output layer. All of these activation functions tend to be used from the torch.nn.functional class. See available activation functions here: https://pytorch.org/docs/stable/nn.html#non-linear-activation-functions
• If dropout is being used, one must specify in which layers it is being applied. However, it always has to be specified in the constructor first. Don't ever use dropout in the final layer!

Example

from torch import nn, optim
import torch.nn.functional as F

class Classifier(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(784, 256)
        self.fc2 = nn.Linear(256, 128)
        self.fc3 = nn.Linear(128, 64)
        self.fc4 = nn.Linear(64, 10)

        # Dropout module with 0.2 drop probability
        self.dropout = nn.Dropout(p=0.2)

    def forward(self, x):
        # Make sure input tensor is flattened
        x = x.view(x.shape[0], -1)

        # Hidden layers with dropout
        x = self.dropout(F.relu(self.fc1(x)))
        x = self.dropout(F.relu(self.fc2(x)))
        x = self.dropout(F.relu(self.fc3(x)))

        # Output so no dropout here
        x = F.log_softmax(self.fc4(x), dim=1)

        return x