Declaration of the VirtualBatchNorm layer class. More...

Public Member Functions
	VirtualBatchNorm ()
	Create the VirtualBatchNorm object. More...

template < typename eT >
	VirtualBatchNorm (const arma::Mat< eT > &referenceBatch, const size_t size, const double eps=1e-8)
	Create the VirtualBatchNorm layer object for a specified number of input units. More...

template < typename eT >
void	Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Backward pass through the layer. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

double	Epsilon () const
	Get the epsilon value. More...

template < typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Forward pass of the Virtual Batch Normalization layer. More...

template < typename eT >
void	Gradient (const arma::Mat< eT > &, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
	Calculate the gradient using the output delta and the input activations. More...

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

size_t	InSize () const
	Get the number of input units. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

void	Reset ()
	Reset the layer parameters. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

Detailed Description

template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >

Declaration of the VirtualBatchNorm layer class.

Instead of using the batch statistics for normalizing on a mini-batch, it uses a reference subset of the data for calculating the normalization statistics.

For more information, refer to the following paper,

@article{Goodfellow2016,
  author  = {Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung,
             Alec Radford, Xi Chen},
  title   = {Improved Techniques for Training GANs},
  year    = {2016},
  url     = {https://arxiv.org/abs/1606.03498},
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 119 of file layer_types.hpp.

Constructor & Destructor Documentation

◆ VirtualBatchNorm() [1/2]

VirtualBatchNorm ( )

Create the VirtualBatchNorm object.

◆ VirtualBatchNorm() [2/2]

VirtualBatchNorm	(	const arma::Mat< eT > &	referenceBatch,
		const size_t	size,
		const double	eps = `1e-8`
	)

Create the VirtualBatchNorm layer object for a specified number of input units.

Parameters

referenceBatch	The data from which the normalization statistics are computed.
size	The number of input units / channels.
eps	The epsilon added to variance to ensure numerical stability.

Member Function Documentation

◆ Backward()

void Backward	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Backward pass through the layer.

Parameters

*	(input) The input activations.
gy	The backpropagated error.
g	The calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 116 of file virtual_batch_norm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 118 of file virtual_batch_norm.hpp.

◆ Epsilon()

double Epsilon ( ) const

inline

Get the epsilon value.

Definition at line 129 of file virtual_batch_norm.hpp.

References VirtualBatchNorm< InputDataType, OutputDataType >::serialize().

◆ Forward()

void Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Forward pass of the Virtual Batch Normalization layer.

Transforms the input data into zero mean and unit variance, scales the data by a factor gamma and shifts it by beta.

Parameters

input	Input data for the layer.
output	Resulting output activations.

◆ Gradient() [1/3]

void Gradient	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	error,
		arma::Mat< eT > &	gradient
	)

Calculate the gradient using the output delta and the input activations.

Parameters

*	(input) The input activations.
error	The calculated error.
gradient	The calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 121 of file virtual_batch_norm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 123 of file virtual_batch_norm.hpp.

◆ InSize()

size_t InSize ( ) const

inline

Get the number of input units.

Definition at line 126 of file virtual_batch_norm.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 111 of file virtual_batch_norm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 113 of file virtual_batch_norm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 106 of file virtual_batch_norm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 108 of file virtual_batch_norm.hpp.

◆ Reset()

void Reset ( )

Reset the layer parameters.

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by VirtualBatchNorm< InputDataType, OutputDataType >::Epsilon().

The documentation for this class was generated from the following files:

/home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/layer_types.hpp
/home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/virtual_batch_norm.hpp

Public Member Functions

Detailed Description

template<typenameInputDataType=arma::mat,typenameOutputDataType=arma::mat> class mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >

Constructor & Destructor Documentation

◆ VirtualBatchNorm() [1/2]

◆ VirtualBatchNorm() [2/2]

Member Function Documentation

◆ Backward()

◆ Delta() [1/2]

◆ Delta() [2/2]

◆ Epsilon()

◆ Forward()

◆ Gradient() [1/3]

◆ Gradient() [2/3]

◆ Gradient() [3/3]

◆ InSize()

◆ OutputParameter() [1/2]

◆ OutputParameter() [2/2]

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Reset()

◆ serialize()

template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::VirtualBatchNorm< InputDataType, OutputDataType >