Declaration of the Group Normalization class. More...

Public Member Functions
	GroupNorm ()
	Create the GroupNorm object. More...

	GroupNorm (const size_t groupCount, const size_t size, const double eps=1e-8)
	Create the GroupNorm object for a specified number of input units. More...

template < typename eT >
void	Backward (const arma::Mat< eT > &input, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Backward pass through the layer. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

double	Epsilon () const
	Get the value of epsilon. More...

template < typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Forward pass of Group Normalization. More...

template < typename eT >
void	Gradient (const arma::Mat< eT > &input, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
	Calculate the gradient using the output delta and the input activations. More...

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

size_t	GroupCount () const
	Get the group count. More...

size_t	InputShape () const
	Get the shape of the input. More...

size_t	InSize () const
	Get the number of input units. More...

OutputDataType	Mean ()
	Get the mean across single training data. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

void	Reset ()
	Reset the layer parameters. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

OutputDataType	Variance ()
	Get the variance across single training data. More...

Detailed Description

template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::GroupNorm< InputDataType, OutputDataType >

Declaration of the Group Normalization class.

The group transforms the input data into zero mean and unit variance and then scales and shifts the data by parameters, gamma and beta respectively over a single training data. These parameters are learnt by the network. Group Normalization is different from Group Normalization in the way that normalization is done for individual training cases, and the mean and standard deviations are computed across the layer dimensions divided into groups, as opposed to across the group.

For more information, refer to the following papers,

@article{wu2018group,
  author    = {Wu, Yuxin and He, Kaiming},
  title     = {Group normalization},
  year      = {2018},
  url       = {https://arxiv.org/abs/1803.08494}
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 50 of file group_norm.hpp.

Constructor & Destructor Documentation

◆ GroupNorm() [1/2]

GroupNorm ( )

Create the GroupNorm object.

◆ GroupNorm() [2/2]

GroupNorm	(	const size_t	groupCount,
		const size_t	size,
		const double	eps = `1e-8`
	)

Create the GroupNorm object for a specified number of input units.

Parameters

size	The number of input units.
eps	The epsilon added to variance to ensure numerical stability.

Member Function Documentation

◆ Backward()

void Backward	(	const arma::Mat< eT > &	input,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Backward pass through the layer.

Parameters

input	The input activations.
gy	The backpropagated error.
g	The calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 115 of file group_norm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 117 of file group_norm.hpp.

◆ Epsilon()

double Epsilon ( ) const

inline

Get the value of epsilon.

Definition at line 134 of file group_norm.hpp.

◆ Forward()

void Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Forward pass of Group Normalization.

Transforms the input data into zero mean and unit variance, scales the data by a factor gamma and shifts it by beta.

Parameters

input	Input data for the layer.
output	Resulting output activations.

◆ Gradient() [1/3]

void Gradient	(	const arma::Mat< eT > &	input,
		const arma::Mat< eT > &	error,
		arma::Mat< eT > &	gradient
	)

Calculate the gradient using the output delta and the input activations.

Parameters

input	The input activations.
error	The calculated error.
gradient	The calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 120 of file group_norm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 122 of file group_norm.hpp.

◆ GroupCount()

size_t GroupCount ( ) const

inline

Get the group count.

Definition at line 143 of file group_norm.hpp.

References GroupNorm< InputDataType, OutputDataType >::serialize().

◆ InputShape()

size_t InputShape ( ) const

inline

Get the shape of the input.

Definition at line 137 of file group_norm.hpp.

◆ InSize()

size_t InSize ( ) const

inline

Get the number of input units.

Definition at line 131 of file group_norm.hpp.

◆ Mean()

OutputDataType Mean ( )

inline

Get the mean across single training data.

Definition at line 125 of file group_norm.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 110 of file group_norm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 112 of file group_norm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 105 of file group_norm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 107 of file group_norm.hpp.

◆ Reset()

void Reset ( )

Reset the layer parameters.

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by GroupNorm< InputDataType, OutputDataType >::GroupCount().

◆ Variance()

OutputDataType Variance ( )

inline

Get the variance across single training data.

Definition at line 128 of file group_norm.hpp.

The documentation for this class was generated from the following file:

/home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/group_norm.hpp

Public Member Functions

Detailed Description

template<typenameInputDataType=arma::mat,typenameOutputDataType=arma::mat> class mlpack::ann::GroupNorm< InputDataType, OutputDataType >

Constructor & Destructor Documentation

◆ GroupNorm() [1/2]

◆ GroupNorm() [2/2]

Member Function Documentation

◆ Backward()

◆ Delta() [1/2]

◆ Delta() [2/2]

◆ Epsilon()

◆ Forward()

◆ Gradient() [1/3]

◆ Gradient() [2/3]

◆ Gradient() [3/3]

◆ GroupCount()

◆ InputShape()

◆ InSize()

◆ Mean()

◆ OutputParameter() [1/2]

◆ OutputParameter() [2/2]

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Reset()

◆ serialize()

◆ Variance()

template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::GroupNorm< InputDataType, OutputDataType >