Lookup< InputDataType, OutputDataType > Class Template Reference

The Lookup class stores word embeddings and retrieves them using tokens. More...

Public Member Functions
	Lookup (const size_t vocabSize=0, const size_t embeddingSize=0)
	Create the Lookup object using the specified vocabulary and embedding size. More...
template < typename eT >
void	Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...
OutputDataType const &	Delta () const
	Get the delta. More...
OutputDataType &	Delta ()
	Modify the delta. More...
size_t	EmbeddingSize () const
	Get the length of each embedding vector. More...
template < typename eT >
void	Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...
template < typename eT >
void	Gradient (const arma::Mat< eT > &input, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
	Calculate the gradient using the output delta and the input activation. More...
OutputDataType const &	Gradient () const
	Get the gradient. More...
OutputDataType &	Gradient ()
	Modify the gradient. More...
OutputDataType const &	OutputParameter () const
	Get the output parameter. More...
OutputDataType &	OutputParameter ()
	Modify the output parameter. More...
OutputDataType const &	Parameters () const
	Get the parameters. More...
OutputDataType &	Parameters ()
	Modify the parameters. More...
template < typename Archive >
void	serialize (Archive &ar, const uint32_t)
	Serialize the layer. More...
size_t	VocabSize () const
	Get the size of the vocabulary. More...

Detailed Description

template

<

typename

InputDataType

=

arma::mat

,

typename

OutputDataType

=

arma::mat

>

class mlpack::ann::Lookup< InputDataType, OutputDataType >

The Lookup class stores word embeddings and retrieves them using tokens.

The Lookup layer is always the first layer of the network. The input to the Lookup class is a matrix of shape (sequenceLength, batchSize). The matrix consists of tokens which are used to lookup the table (i.e. weights) to find the embeddings of those tokens.

The input shape : (sequenceLength, batchSize). The output shape : (embeddingSize, sequenceLength, batchSize).

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 41 of file lookup.hpp.

Constructor & Destructor Documentation

Lookup()

Lookup	(	const size_t	vocabSize = 0,
		const size_t	embeddingSize = 0
	)

Create the Lookup object using the specified vocabulary and embedding size.

Parameters

vocabSize	The size of the vocabulary.
embeddingSize	The length of each embedding vector.

Member Function Documentation

Backward()

void Backward	(	const arma::Mat< eT > &	,
		const arma::Mat< eT > &	gy,
		arma::Mat< eT > &	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

*	(input) The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

Delta() [1/2]

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 99 of file lookup.hpp.

Delta() [2/2]

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 101 of file lookup.hpp.

EmbeddingSize()

size_t EmbeddingSize ( ) const

inline

Get the length of each embedding vector.

Definition at line 112 of file lookup.hpp.

References Lookup< InputDataType, OutputDataType >::serialize().

Forward()

void Forward	(	const arma::Mat< eT > &	input,
		arma::Mat< eT > &	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

Gradient() [1/3]

void Gradient	(	const arma::Mat< eT > &	input,
		const arma::Mat< eT > &	error,
		arma::Mat< eT > &	gradient
	)

Calculate the gradient using the output delta and the input activation.

Parameters

input	The input parameter used for calculating the gradient.
error	The calculated error.
gradient	The calculated gradient.

Gradient() [2/3]

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 104 of file lookup.hpp.

Gradient() [3/3]

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 106 of file lookup.hpp.

OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 94 of file lookup.hpp.

OutputParameter() [2/2]

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 96 of file lookup.hpp.

Parameters() [1/2]

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 89 of file lookup.hpp.

Parameters() [2/2]

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 91 of file lookup.hpp.

serialize()

void serialize	(	Archive &	ar,
		const uint32_t
	)

Serialize the layer.

Referenced by Lookup< InputDataType, OutputDataType >::EmbeddingSize().

VocabSize()

size_t VocabSize ( ) const

inline

Get the size of the vocabulary.

Definition at line 109 of file lookup.hpp.

---

The documentation for this class was generated from the following file:

• /home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/layer/lookup.hpp