FlexibleReLU< InputDataType, OutputDataType > Class Template Reference

The FlexibleReLU activation function, defined by. More...

Public Member Functions
	FlexibleReLU (const double alpha=0)
	Create the FlexibleReLU object using the specified parameters. More...
double const &	Alpha () const
	Get the parameter controlling the range of the relu function. More...
double &	Alpha ()
	Modify the parameter controlling the range of the relu function. More...
template < typename DataType >
void	Backward (const DataType &input, const DataType &gy, DataType &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. More...
OutputDataType const &	Delta () const
	Get the delta. More...
OutputDataType &	Delta ()
	Modify the delta. More...
template < typename InputType , typename OutputType >
void	Forward (const InputType &input, OutputType &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...
template < typename eT >
void	Gradient (const arma::Mat< eT > &input, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
	Calculate the gradient using the output delta and the input activation. More...
OutputDataType const &	Gradient () const
	Get the gradient. More...
OutputDataType &	Gradient ()
	Modify the gradient. More...
OutputDataType const &	OutputParameter () const
	Get the output parameter. More...
OutputDataType &	OutputParameter ()
	Modify the output parameter. More...
OutputDataType const &	Parameters () const
	Get the parameters. More...
OutputDataType &	Parameters ()
	Modify the parameters. More...
void	Reset ()
	Reset the layer parameter. More...
template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

Detailed Description

template

<

typename

InputDataType

=

arma::mat

,

typename

OutputDataType

=

arma::mat

>

class mlpack::ann::FlexibleReLU< InputDataType, OutputDataType >

The FlexibleReLU activation function, defined by.

For more information, read the following paper:

@article{Qiu2018,
 author  = {Suo Qiu, Xiangmin Xu and Bolun Cai},
 title   = {FReLU: Flexible Rectified Linear Units for Improving
            Convolutional Neural Networks}
 journal = {arxiv preprint},
 URL     = {https://arxiv.org/abs/1706.08098},
 year    = {2018}
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mar, arma::sp_mat or arma::cube)
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube)

Definition at line 59 of file flexible_relu.hpp.

Constructor & Destructor Documentation

FlexibleReLU()

FlexibleReLU

(

const double

alpha = 0

)

Create the FlexibleReLU object using the specified parameters.

The non zero parameter can be adjusted by specifying the parameter alpha which controls the range of the relu function. (Default alpha = 0) This parameter is trainable.

Parameters

alpha

Parameter for adjusting the range of the relu function.

Member Function Documentation

Alpha() [1/2]

double const& Alpha ( ) const

inline

Get the parameter controlling the range of the relu function.

Definition at line 134 of file flexible_relu.hpp.

Alpha() [2/2]

double& Alpha ( )

inline

Modify the parameter controlling the range of the relu function.

Definition at line 136 of file flexible_relu.hpp.

References FlexibleReLU< InputDataType, OutputDataType >::serialize().

Backward()

void Backward	(	const DataType &	input,
		const DataType &	gy,
		DataType &	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Delta() [1/2]

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 124 of file flexible_relu.hpp.

Delta() [2/2]

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 126 of file flexible_relu.hpp.

Forward()

void Forward	(	const InputType &	input,
		OutputType &	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Gradient() [1/3]

void Gradient	(	const arma::Mat< eT > &	input,
		const arma::Mat< eT > &	error,
		arma::Mat< eT > &	gradient
	)

Calculate the gradient using the output delta and the input activation.

Parameters

input	The input parameter used for calculating the gradient.
error	The calculated error.
gradient	The calculated gradient.

Gradient() [2/3]

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 129 of file flexible_relu.hpp.

Gradient() [3/3]

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 131 of file flexible_relu.hpp.

OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 119 of file flexible_relu.hpp.

OutputParameter() [2/2]

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 121 of file flexible_relu.hpp.

Parameters() [1/2]

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 114 of file flexible_relu.hpp.

Parameters() [2/2]

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 116 of file flexible_relu.hpp.

Reset()

void Reset

(

)

Reset the layer parameter.

serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by FlexibleReLU< InputDataType, OutputDataType >::Alpha().

---

The documentation for this class was generated from the following file:

• /home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/flexible_relu.hpp