The class HyperParameterTuner for the given MLAlgorithm utilizes the provided Optimizer to find the values of hyper-parameters that optimize the value of the given Metric. More...

Public Member Functions
template<typename... CVArgs>
	HyperParameterTuner (const CVArgs &...args)
	Create a HyperParameterTuner object by passing constructor arguments for the given cross-validation strategy (the CV class). More...

const MLAlgorithm &	BestModel () const
	Get the best model from the last run. More...

MLAlgorithm &	BestModel ()
	Modify the best model from the last run. More...

double	BestObjective () const
	Get the performance measurement of the best model from the last run. More...

double	MinDelta () const
	Get minimum increase of arguments for calculation of partial derivatives (by the definition) in gradient-based optimization. More...

double &	MinDelta ()
	Modify minimum increase of arguments for calculation of partial derivatives (by the definition) in gradient-based optimization. More...

template<typename... Args>
TupleOfHyperParameters< Args... >	Optimize (const Args &... args)
	Find the best hyper-parameters by using the given Optimizer. More...

OptimizerType &	Optimizer ()
	Access and modify the optimizer. More...

double	RelativeDelta () const
	Get relative increase of arguments for calculation of partial derivatives (by the definition) in gradient-based optimization. More...

double &	RelativeDelta ()
	Modify relative increase of arguments for calculation of partial derivatives (by the definition) in gradient-based optimization. More...

Detailed Description

template<typename MLAlgorithm, typename Metric, template< typename, typename, typename, typename, typename > class CV, typename OptimizerType = ens::GridSearch, typename MatType = arma::mat, typename PredictionsType = typename cv::MetaInfoExtractor<MLAlgorithm, MatType>::PredictionsType, typename WeightsType = typename cv::MetaInfoExtractor<MLAlgorithm, MatType, PredictionsType>::WeightsType>
class mlpack::hpt::HyperParameterTuner< MLAlgorithm, Metric, CV, OptimizerType, MatType, PredictionsType, WeightsType >

The class HyperParameterTuner for the given MLAlgorithm utilizes the provided Optimizer to find the values of hyper-parameters that optimize the value of the given Metric.

The value of the Metric is calculated by performing cross-validation with the provided cross-validation strategy.

To construct a HyperParameterTuner object you need to pass the same arguments as for construction of an object of the given CV class. For example, we can use the following code to try to find a good lambda value for LinearRegression.

// 100-point 5-dimensional random dataset.
arma::mat data = arma::randu<arma::mat>(5, 100);
// Noisy responses retrieved by a random linear transformation of data.
arma::rowvec responses = arma::randu<arma::rowvec>(5) * data +
    0.1 * arma::randn<arma::rowvec>(100);
// Using 80% of data for training and remaining 20% for assessing MSE.
double validationSize = 0.2;
HyperParameterTuner<LinearRegression, MSE, SimpleCV> hpt(validationSize,
    data, responses);
// Finding the best value for lambda from the values 0.0, 0.001, 0.01, 0.1,
// and 1.0.
arma::vec lambdas{0.0, 0.001, 0.01, 0.1, 1.0};
double bestLambda;
std::tie(bestLambda) = hpt.Optimize(lambdas);

When some hyper-parameters should not be optimized, you can specify values for them with the Fixed function as in the following example of finding good lambda1 and lambda2 values for LARS.

HyperParameterTuner<LARS, MSE, SimpleCV> hpt2(validationSize, data,
    responses);
bool transposeData = true;
bool useCholesky = false;
arma::vec lambda1Set{0.0, 0.001, 0.01, 0.1, 1.0};
arma::vec lambda2Set{0.0, 0.002, 0.02, 0.2, 2.0};
double bestLambda1, bestLambda2;
std::tie(bestLambda1, bestLambda2) = hpt2.Optimize(Fixed(transposeData),
    Fixed(useCholesky), lambda1Set, lambda2Set);

Template Parameters

MLAlgorithm	A machine learning algorithm.
Metric	A metric to assess the quality of a trained model.
CV	A cross-validation strategy used to assess a set of hyper-parameters.
OptimizerType	An optimization strategy (GridSearch and GradientDescent are supported).
MatType	The type of data.
PredictionsType	The type of predictions (should be passed when the predictions type is a template parameter in Train methods of the given MLAlgorithm; arma::Row<size_t> will be used otherwise).
WeightsType	The type of weights (should be passed when weighted learning is supported, and the weights type is a template parameter in Train methods of the given MLAlgorithm; arma::vec will be used otherwise).

Definition at line 96 of file hpt.hpp.

Constructor & Destructor Documentation

◆ HyperParameterTuner()

HyperParameterTuner ( const CVArgs &... args )

Create a HyperParameterTuner object by passing constructor arguments for the given cross-validation strategy (the CV class).

Parameters

args	Constructor arguments for the given cross-validation strategy (the CV class).

Member Function Documentation

◆ BestModel() [1/2]

const MLAlgorithm& BestModel ( ) const

inline

Get the best model from the last run.

Definition at line 184 of file hpt.hpp.

◆ BestModel() [2/2]

MLAlgorithm& BestModel ( )

inline

Modify the best model from the last run.

Definition at line 187 of file hpt.hpp.

◆ BestObjective()

double BestObjective ( ) const

inline

Get the performance measurement of the best model from the last run.

Definition at line 181 of file hpt.hpp.

◆ MinDelta() [1/2]

double MinDelta ( ) const

inline

Get minimum increase of arguments for calculation of partial derivatives (by the definition) in gradient-based optimization.

This value is going to be used when it is greater than the increase calculated with the rules described in the documentation for RelativeDelta().

The default value is 1e-10.

Definition at line 142 of file hpt.hpp.

◆ MinDelta() [2/2]

double& MinDelta ( )

inline

Modify minimum increase of arguments for calculation of partial derivatives (by the definition) in gradient-based optimization.

This value is going to be used when it is greater than the increase calculated with the rules described in the documentation for RelativeDelta().

The default value is 1e-10.

Definition at line 152 of file hpt.hpp.

References HyperParameterTuner< MLAlgorithm, Metric, CV, OptimizerType, MatType, PredictionsType, WeightsType >::Optimize().

◆ Optimize()

TupleOfHyperParameters<Args...> Optimize ( const Args &... args )

Find the best hyper-parameters by using the given Optimizer.

For each hyper-parameter one of the following should be passed as an argument.

A set of values to choose from (when using GridSearch as an optimizer). The set of values should be an STL-compatible container (it should provide begin() and end() methods returning iterators).
A starting value (when using any other optimizer than GridSearch).
A value fixed by using the function mlpack::hpt::Fixed. In this case the hyper-parameter will not be optimized.

All arguments should be passed in the same order as if the corresponding hyper-parameters would be passed into the Evaluate method of the given CV class (in the order as they appear in the constructor(s) of the given MLAlgorithm). Also, arguments for all required hyper-parameters (ones that don't have default values in the corresponding MLAlgorithm constructor) should be provided.

The method returns a tuple of values for hyper-parameters that haven't been fixed.

Parameters

args	Arguments corresponding to hyper-parameters (see the method description for more information).

Referenced by HyperParameterTuner< MLAlgorithm, Metric, CV, OptimizerType, MatType, PredictionsType, WeightsType >::MinDelta().

◆ Optimizer()

OptimizerType& Optimizer ( )

inline

Access and modify the optimizer.

Definition at line 110 of file hpt.hpp.

◆ RelativeDelta() [1/2]

double RelativeDelta ( ) const

inline

Get relative increase of arguments for calculation of partial derivatives (by the definition) in gradient-based optimization.

The exact increase for some particular argument is equal to the absolute value of the argument multiplied by the relative increase (see also the documentation for MinDelta()).

The default value is 0.01.

Definition at line 121 of file hpt.hpp.

◆ RelativeDelta() [2/2]

double& RelativeDelta ( )

inline

Modify relative increase of arguments for calculation of partial derivatives (by the definition) in gradient-based optimization.

The exact increase for some particular argument is equal to the absolute value of the argument multiplied by the relative increase (see also the documentation for MinDelta()).

The default value is 0.01.

Definition at line 132 of file hpt.hpp.

The documentation for this class was generated from the following file:

/home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/core/hpt/hpt.hpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ HyperParameterTuner()

Member Function Documentation

◆ BestModel() [1/2]

◆ BestModel() [2/2]

◆ BestObjective()

◆ MinDelta() [1/2]

◆ MinDelta() [2/2]

◆ Optimize()

◆ Optimizer()

◆ RelativeDelta() [1/2]

◆ RelativeDelta() [2/2]