Dropout

Dropout randomly zeroes elements of its input tensor with probability p during training, then scales the remaining values by 1/(1-p) to keep the expected magnitude constant. This regularisation technique prevents co-adaptation of neurons and reduces overfitting. During evaluation (after calling model.set_eval_mode()), the layer becomes a pass-through with no masking applied.

import simplegrad as sg
import simplegrad.nn as nn

model = nn.Sequential(
    nn.Linear(128, 64),
    nn.ReLU(),
    nn.Dropout(p=0.3),
    nn.Linear(64, 10),
)
out = model(sg.ones((8, 128)))

Dropout

Bases: Module

Apply dropout regularization during training.

During training, randomly zeroes elements of the input tensor with probability p. Disabled automatically in evaluation mode.

Parameters:

• p (float, default: 0.5 ) –

  Probability of zeroing each element. Must be in [0, 1). Defaults to 0.5.

Raises:

• ValueError –

  If p is not in [0, 1).

Attributes

Attribute	Type	Description
.p	float	Probability of zeroing each element during training.

Methods

Method	Description
.forward()	Apply dropout mask during training; pass through during evaluation.

Inherits all methods from Module: .parameters(), .submodules(), .to_device(), .summary(), .set_train_mode(), .set_eval_mode().