Declaration of the WeightNorm layer class. More...
Public Member Functions | |
| WeightNorm (LayerTypes< CustomLayers... > layer=LayerTypes< CustomLayers... >()) | |
| Create the WeightNorm layer object. More... | |
| ~WeightNorm () | |
| Destructor to release allocated memory. More... | |
template < typename eT > | |
| void | Backward (const arma::Mat< eT > &input, const arma::Mat< eT > &gy, arma::Mat< eT > &g) |
| Backward pass through the layer. More... | |
| OutputDataType const & | Delta () const |
| Get the delta. More... | |
| OutputDataType & | Delta () |
| Modify the delta. More... | |
template < typename eT > | |
| void | Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output) |
| Forward pass of the WeightNorm layer. More... | |
template < typename eT > | |
| void | Gradient (const arma::Mat< eT > &input, const arma::Mat< eT > &error, arma::Mat< eT > &gradient) |
| Calculate the gradient using the output delta, input activations and the weights of the wrapped layer. More... | |
| OutputDataType const & | Gradient () const |
| Get the gradient. More... | |
| OutputDataType & | Gradient () |
| Modify the gradient. More... | |
| LayerTypes< CustomLayers... > const & | Layer () |
| Get the wrapped layer. More... | |
| OutputDataType const & | OutputParameter () const |
| Get the output parameter. More... | |
| OutputDataType & | OutputParameter () |
| Modify the output parameter. More... | |
| OutputDataType const & | Parameters () const |
| Get the parameters. More... | |
| OutputDataType & | Parameters () |
| Modify the parameters. More... | |
| void | Reset () |
| Reset the layer parameters. More... | |
template < typename Archive > | |
| void | serialize (Archive &ar, const uint32_t) |
| Serialize the layer. More... | |
Detailed Description
template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat, typename... CustomLayers>
class mlpack::ann::WeightNorm< InputDataType, OutputDataType, CustomLayers >
Declaration of the WeightNorm layer class.
The layer reparameterizes the weight vectors in a neural network, decoupling the length of those weight vectors from their direction. This reparameterization does not introduce any dependencies between the examples in a mini-batch.
This class will be a wrapper around existing layers. It will just modify the calculation and updation of weights of the layer.
For more information, refer to the following paper,
- Template Parameters
-
InputDataType Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube). OutputDataType Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube). CustomLayers Additional custom layers that can be added.
Definition at line 217 of file layer_types.hpp.
Constructor & Destructor Documentation
◆ WeightNorm()
| WeightNorm | ( | LayerTypes< CustomLayers... > | layer = LayerTypes< CustomLayers... >() | ) |
Create the WeightNorm layer object.
- Parameters
-
layer The layer whose weights are needed to be normalized.
◆ ~WeightNorm()
| ~WeightNorm | ( | ) |
Destructor to release allocated memory.
Member Function Documentation
◆ Backward()
| void Backward | ( | const arma::Mat< eT > & | input, |
| const arma::Mat< eT > & | gy, | ||
| arma::Mat< eT > & | g | ||
| ) |
Backward pass through the layer.
This function calls the Backward() function of the wrapped layer.
- Parameters
-
input The input activations. gy The backpropagated error. g The calculated gradient.
◆ Delta() [1/2]
|
inline |
Get the delta.
Definition at line 118 of file weight_norm.hpp.
◆ Delta() [2/2]
|
inline |
Modify the delta.
Definition at line 120 of file weight_norm.hpp.
◆ Forward()
| void Forward | ( | const arma::Mat< eT > & | input, |
| arma::Mat< eT > & | output | ||
| ) |
Forward pass of the WeightNorm layer.
Calculates the weights of the wrapped layer from the parameter vector v and the scalar parameter g. It then calulates the output of the wrapped layer from the calculated weights.
- Parameters
-
input Input data for the layer. output Resulting output activations.
◆ Gradient() [1/3]
| void Gradient | ( | const arma::Mat< eT > & | input, |
| const arma::Mat< eT > & | error, | ||
| arma::Mat< eT > & | gradient | ||
| ) |
Calculate the gradient using the output delta, input activations and the weights of the wrapped layer.
- Parameters
-
input The input activations. error The calculated error. gradient The calculated gradient.
◆ Gradient() [2/3]
|
inline |
Get the gradient.
Definition at line 123 of file weight_norm.hpp.
◆ Gradient() [3/3]
|
inline |
Modify the gradient.
Definition at line 125 of file weight_norm.hpp.
◆ Layer()
|
inline |
Get the wrapped layer.
Definition at line 138 of file weight_norm.hpp.
References WeightNorm< InputDataType, OutputDataType, CustomLayers >::serialize().
◆ OutputParameter() [1/2]
|
inline |
Get the output parameter.
Definition at line 128 of file weight_norm.hpp.
◆ OutputParameter() [2/2]
|
inline |
Modify the output parameter.
Definition at line 130 of file weight_norm.hpp.
◆ Parameters() [1/2]
|
inline |
Get the parameters.
Definition at line 133 of file weight_norm.hpp.
◆ Parameters() [2/2]
|
inline |
Modify the parameters.
Definition at line 135 of file weight_norm.hpp.
◆ Reset()
| void Reset | ( | ) |
Reset the layer parameters.
◆ serialize()
| void serialize | ( | Archive & | ar, |
| const uint32_t | |||
| ) |
Serialize the layer.
Referenced by WeightNorm< InputDataType, OutputDataType, CustomLayers >::Layer().
The documentation for this class was generated from the following files:
- /home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/layer_types.hpp
- /home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/weight_norm.hpp