mlpack::nn Namespace Reference

Classes
class	SparseAutoencoder
	A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network. More...
class	SparseAutoencoderFunction
	This is a class for the sparse autoencoder objective function. More...

Functions
void	MaximalInputs (const arma::mat &parameters, arma::mat &output)
	Given a parameters matrix from an autoencoder, maximize the hidden units of the parameters, storing the maximal inputs in the given output matrix. More...
void	NormalizeColByMax (const arma::mat &input, arma::mat &output)
	Normalize each column of the input matrix by its maximum value, if that maximum value is not zero. More...

Function Documentation

MaximalInputs()

void mlpack::nn::MaximalInputs	(	const arma::mat &	parameters,
		arma::mat &	output
	)

Given a parameters matrix from an autoencoder, maximize the hidden units of the parameters, storing the maximal inputs in the given output matrix.

Details can be found on the 'Visualizing a Trained Autoencoder' page of the Stanford UFLDL tutorial:

http://deeplearning.stanford.edu/wiki/index.php/Main_Page

This function is based on the implementation (display_network.m) from the "Exercise: Sparse Autoencoder" page of the UFLDL tutorial:

http://deeplearning.stanford.edu/wiki/index.php/Exercise:Sparse_Autoencoder

Example usage of this function can be seen below. Note that this function can work with the ColumnsToBlocks class in order to reshape the maximal inputs for visualization, as in the UFLDL tutorial. The code below demonstrates this.

arma::mat data; // Data matrix.
const size_t vSize = 64; // Size of visible layer, depends on the data.
const size_t hSize = 25; // Size of hidden layer, depends on requirements.

const size_t numBasis = 5; // Parameter required for L-BFGS algorithm.
const size_t numIterations = 100; // Maximum number of iterations.

// Use an instantiated optimizer for the training.
SparseAutoencoder<L_BFGS> encoder(data, vSize, hSize);

arma::mat maximalInput; // Store the features learned by sparse autoencoder
mlpack::nn::MaximalInputs(encoder.Parameters(), maximalInput);

arma::mat outputs;
const bool scale = true;

ColumnsToBlocks ctb(5,5);
arma::mat output;
ctb.Transform(maximalInput, output);
// Save the output as PGM, for visualization.
output.save(fileName, arma::pgm_binary);

Precondition

Layout of parameters

The layout of the parameters matrix should be same as following

//          vSize   1
//       |        |  |
//  hSize|   w1   |b1|
//       |________|__|
//       |        |  |
//  hSize|   w2'  |  |
//       |________|__|
//      1|   b2'  |  |

Also, the square root of vSize must be an integer (i.e. vSize must be a perfect square).

Parameters

parameters	The parameters of the autoencoder.
output	Matrix to store the maximal inputs in.

NormalizeColByMax()

void mlpack::nn::NormalizeColByMax	(	const arma::mat &	input,
		arma::mat &	output
	)

Normalize each column of the input matrix by its maximum value, if that maximum value is not zero.

Parameters

input	The input data to normalize.
output	A matrix to store the input data in after normalization.