An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom. More...
Public Member Functions | |
template < typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer > | |
| LocalCoordinateCoding (const arma::mat &data, const size_t atoms, const double lambda, const size_t maxIterations=0, const double tolerance=0.01, const DictionaryInitializer &initializer=DictionaryInitializer()) | |
| Set the parameters to LocalCoordinateCoding, and train the dictionary. More... | |
| LocalCoordinateCoding (const size_t atoms=0, const double lambda=0.0, const size_t maxIterations=0, const double tolerance=0.01) | |
| Set the parameters to LocalCoordinateCoding. More... | |
| size_t | Atoms () const |
| Get the number of atoms. More... | |
| size_t & | Atoms () |
| Modify the number of atoms. More... | |
| const arma::mat & | Dictionary () const |
| Accessor for dictionary. More... | |
| arma::mat & | Dictionary () |
| Mutator for dictionary. More... | |
| void | Encode (const arma::mat &data, arma::mat &codes) |
| Code each point via distance-weighted LARS. More... | |
| double | Lambda () const |
| Get the L1 regularization parameter. More... | |
| double & | Lambda () |
| Modify the L1 regularization parameter. More... | |
| size_t | MaxIterations () const |
| Get the maximum number of iterations. More... | |
| size_t & | MaxIterations () |
| Modify the maximum number of iterations. More... | |
| double | Objective (const arma::mat &data, const arma::mat &codes, const arma::uvec &adjacencies) const |
| Compute objective function given the list of adjacencies. More... | |
| void | OptimizeDictionary (const arma::mat &data, const arma::mat &codes, const arma::uvec &adjacencies) |
| Learn dictionary by solving linear system. More... | |
template < typename Archive > | |
| void | serialize (Archive &ar, const uint32_t) |
| Serialize the model. More... | |
| double | Tolerance () const |
| Get the objective tolerance. More... | |
| double & | Tolerance () |
| Modify the objective tolerance. More... | |
template < typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer > | |
| double | Train (const arma::mat &data, const DictionaryInitializer &initializer=DictionaryInitializer()) |
| Run local coordinate coding. More... | |
Detailed Description
An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom.
Let d be the number of dimensions in the original space, m the number of training points, and k the number of atoms in the dictionary (the dimension of the learned feature space). The training data X is a d-by-m matrix where each column is a point and each row is a dimension. The dictionary D is a d-by-k matrix, and the sparse codes matrix Z is a k-by-m matrix. This program seeks to minimize the objective: min_{D,Z} ||X - D Z||_{Fro}^2
- lambda sum_{i=1}^m sum_{j=1}^k dist(X_i,D_j)^2 Z_i^j where lambda > 0.
This problem is solved by an algorithm that alternates between a dictionary learning step and a sparse coding step. The dictionary learning step updates the dictionary D by solving a linear system (note that the objective is a positive definite quadratic program). The sparse coding step involves solving a large number of weighted l1-norm regularized linear regression problems problems; this can be done efficiently using LARS, an algorithm that can solve the LASSO (paper below).
The papers are listed below.
Constructor & Destructor Documentation
◆ LocalCoordinateCoding() [1/2]
| LocalCoordinateCoding | ( | const arma::mat & | data, |
| const size_t | atoms, | ||
| const double | lambda, | ||
| const size_t | maxIterations = 0, |
||
| const double | tolerance = 0.01, |
||
| const DictionaryInitializer & | initializer = DictionaryInitializer() |
||
| ) |
Set the parameters to LocalCoordinateCoding, and train the dictionary.
This constructor will also initialize the dictionary using the given DictionaryInitializer before training.
If you want to initialize the dictionary to a custom matrix, consider either writing your own DictionaryInitializer class (with void Initialize(const arma::mat& data, arma::mat& dictionary) function), or call the constructor that does not take a data matrix, then call Dictionary() to set the dictionary matrix to a matrix of your choosing, and then call Train() with sparse_coding::NothingInitializer (i.e. Train<sparse_coding::NothingInitializer>(data)).
- Parameters
-
data Data matrix. atoms Number of atoms in dictionary. lambda Regularization parameter for weighted l1-norm penalty. maxIterations Maximum number of iterations for training (0 runs until convergence). tolerance Tolerance for the objective function. initializer Intializer to use.
◆ LocalCoordinateCoding() [2/2]
| LocalCoordinateCoding | ( | const size_t | atoms = 0, |
| const double | lambda = 0.0, |
||
| const size_t | maxIterations = 0, |
||
| const double | tolerance = 0.01 |
||
| ) |
Set the parameters to LocalCoordinateCoding.
This constructor will not train the model, and a subsequent call to Train() will be required before the model can encode points with Encode(). The default values for atoms and lambda should be changed if you intend to train the model!
- Parameters
-
atoms Number of atoms in dictionary. lambda Regularization parameter for weighted l1-norm penalty. maxIterations Maximum number of iterations for training (0 runs until convergence). tolerance Tolerance for the objective function.
Member Function Documentation
◆ Atoms() [1/2]
◆ Atoms() [2/2]
◆ Dictionary() [1/2]
|
inline |
◆ Dictionary() [2/2]
◆ Encode()
| void Encode | ( | const arma::mat & | data, |
| arma::mat & | codes | ||
| ) |
Code each point via distance-weighted LARS.
- Parameters
-
data Matrix containing points to encode. codes Output matrix to store codes in.
◆ Lambda() [1/2]
|
inline |
◆ Lambda() [2/2]
|
inline |
◆ MaxIterations() [1/2]
|
inline |
◆ MaxIterations() [2/2]
|
inline |
◆ Objective()
| double Objective | ( | const arma::mat & | data, |
| const arma::mat & | codes, | ||
| const arma::uvec & | adjacencies | ||
| ) | const |
Compute objective function given the list of adjacencies.
- Parameters
-
data Matrix containing points to encode. codes Output matrix to store codes in. adjacencies Indices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms)
◆ OptimizeDictionary()
| void OptimizeDictionary | ( | const arma::mat & | data, |
| const arma::mat & | codes, | ||
| const arma::uvec & | adjacencies | ||
| ) |
Learn dictionary by solving linear system.
- Parameters
-
data Matrix containing points to encode. codes Output matrix to store codes in. adjacencies Indices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms)
◆ serialize()
| void serialize | ( | Archive & | ar, |
| const uint32_t | |||
| ) |
Serialize the model.
Referenced by LocalCoordinateCoding::Tolerance().
◆ Tolerance() [1/2]
|
inline |
◆ Tolerance() [2/2]
|
inline |
Modify the objective tolerance.
Definition at line 204 of file lcc.hpp.
References LocalCoordinateCoding::serialize().
◆ Train()
| double Train | ( | const arma::mat & | data, |
| const DictionaryInitializer & | initializer = DictionaryInitializer() |
||
| ) |
Run local coordinate coding.
- Parameters
-
data Data matrix. initializer Intializer to use.
- Returns
- The final objective value.
The documentation for this class was generated from the following file:
- /home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/local_coordinate_coding/lcc.hpp