Implementation of the Highway layer. More...

Public Member Functions
	Highway ()
	Create the Highway object. More...

	Highway (const size_t inSize, const bool model=true)
	Create the Highway object. More...

	~Highway ()
	Destroy the Highway object. More...

template<class LayerType , class... Args>
void	Add (Args... args)
	Add a new module to the model. More...

void	Add (LayerTypes< CustomLayers... > layer)
	Add a new module to the model. More...

template < typename eT >
void	Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Ordinary feed-backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

template < typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

template < typename eT >
void	Gradient (const arma::Mat< eT > &input, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
	Calculate the gradient using the output delta and the input activation. More...

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

InputDataType const &	InputParameter () const
	Get the input parameter. More...

InputDataType &	InputParameter ()
	Modify the input parameter. More...

size_t	InputShape () const
	Get the shape of the input. More...

size_t	InSize () const
	Get the number of input units. More...

std::vector< LayerTypes< CustomLayers... > > &	Model ()
	Return the modules of the model. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

void	Reset ()
	Reset the layer parameter. More...

template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat, typename... CustomLayers>
class mlpack::ann::Highway< InputDataType, OutputDataType, CustomLayers >

Implementation of the Highway layer.

The Highway class can vary its behavior between that of feed-forward fully connected network container and that of a layer which simply passes its inputs through depending on the transform gate. Note that the size of the input and output matrices of this class should be equal.

For more information, refer the following paper.

@article{Srivastava2015,
  author  = {Rupesh Kumar Srivastava, Klaus Greff, Jurgen Schmidhuber},
  title   = {Training Very Deep Networks},
  journal = {Advances in Neural Information Processing Systems},
  year    = {2015},
  url     = {https://arxiv.org/abs/1507.06228},
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 58 of file highway.hpp.

Constructor & Destructor Documentation

◆ Highway() [1/2]

Highway ( )

Create the Highway object.

◆ Highway() [2/2]

Highway	(	const size_t	inSize,
		const bool	model = `true`
	)

Create the Highway object.

Parameters

inSize	The number of input units.
model	Expose all the network modules.

◆ ~Highway()

~Highway ( )

Destroy the Highway object.

Member Function Documentation

◆ Add() [1/2]

void Add ( Args... args )

inline

Add a new module to the model.

Parameters

args	The layer parameter.

Definition at line 122 of file highway.hpp.

◆ Add() [2/2]

void Add ( LayerTypes< CustomLayers... > layer )

inline

Add a new module to the model.

Parameters

layer The Layer to be added to the model.

Definition at line 133 of file highway.hpp.

◆ Backward()

void Backward	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Ordinary feed-backward pass of a neural network, calculating the function f(x) by propagating x backwards through f.

Using the results from the feed-forward pass.

Parameters

*	(input) The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 166 of file highway.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 168 of file highway.hpp.

◆ Forward()

void Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Ordinary feed-forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

◆ Gradient() [1/3]

void Gradient	(	const arma::Mat< eT > &	input,
		const arma::Mat< eT > &	error,
		arma::Mat< eT > &	gradient
	)

Calculate the gradient using the output delta and the input activation.

Parameters

input	The input parameter used for calculating the gradient.
error	The calculated error.
gradient	The calculated gradient.

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 171 of file highway.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 173 of file highway.hpp.

◆ InputParameter() [1/2]

InputDataType const& InputParameter ( ) const

inline

Get the input parameter.

Definition at line 156 of file highway.hpp.

◆ InputParameter() [2/2]

InputDataType& InputParameter ( )

inline

Modify the input parameter.

Definition at line 158 of file highway.hpp.

◆ InputShape()

size_t InputShape ( ) const

inline

Get the shape of the input.

Definition at line 179 of file highway.hpp.

References Highway< InputDataType, OutputDataType, CustomLayers >::serialize().

◆ InSize()

size_t InSize ( ) const

inline

Get the number of input units.

Definition at line 176 of file highway.hpp.

◆ Model()

std::vector<LayerTypes<CustomLayers...> >& Model ( )

inline

Return the modules of the model.

Definition at line 140 of file highway.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 161 of file highway.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 163 of file highway.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 151 of file highway.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 153 of file highway.hpp.

◆ Reset()

void Reset ( )

Reset the layer parameter.

◆ serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by Highway< InputDataType, OutputDataType, CustomLayers >::InputShape().

The documentation for this class was generated from the following file:

/home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/highway.hpp

Public Member Functions

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat, typename... CustomLayers> class mlpack::ann::Highway< InputDataType, OutputDataType, CustomLayers >

Constructor & Destructor Documentation

◆ Highway() [1/2]

◆ Highway() [2/2]

◆ ~Highway()

Member Function Documentation

◆ Add() [1/2]

◆ Add() [2/2]

◆ Backward()

◆ Delta() [1/2]

◆ Delta() [2/2]

◆ Forward()

◆ Gradient() [1/3]

◆ Gradient() [2/3]

◆ Gradient() [3/3]

◆ InputParameter() [1/2]

◆ InputParameter() [2/2]

◆ InputShape()

◆ InSize()

◆ Model()

◆ OutputParameter() [1/2]

◆ OutputParameter() [2/2]

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Reset()

◆ serialize()

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat, typename... CustomLayers>
class mlpack::ann::Highway< InputDataType, OutputDataType, CustomLayers >