GELUFunction Class Reference
The GELU function, defined by. More...
Static Public Member Functions | |
| static double | Deriv (const double y) |
| Computes the first derivative of the GELU function. More... | |
template < typename InputVecType , typename OutputVecType > | |
| static void | Deriv (const InputVecType &y, OutputVecType &x) |
| Computes the first derivatives of the GELU function. More... | |
| static double | Fn (const double x) |
| Computes the GELU function. More... | |
template < typename InputVecType , typename OutputVecType > | |
| static void | Fn (const InputVecType &x, OutputVecType &y) |
| Computes the GELU function. More... | |
Detailed Description
![\begin{eqnarray*} f(x) = 0.5 * x * {1 + tanh[(2/pi)^(1/2) * (x + 0.044715 * x^3)]} \\ f'(x) = 0.5 * tanh(0.0356774 * x^3) + 0.797885 * x) + (0.0535161x^3 + 0.398942 * x) * sech^2(0.0356774 * x^3+0.797885 * x) + 0.5\\ \end{eqnarray*}](form_85.png)
Member Function Documentation
◆ Deriv() [1/2]
|
inlinestatic |
Computes the first derivative of the GELU function.
- Parameters
-
y Input data.
- Returns
- f'(x)
Definition at line 65 of file gelu_function.hpp.
◆ Deriv() [2/2]
|
inlinestatic |
Computes the first derivatives of the GELU function.
- Parameters
-
y Input data. x The resulting derivatives.
Definition at line 80 of file gelu_function.hpp.
◆ Fn() [1/2]
|
inlinestatic |
Computes the GELU function.
- Parameters
-
x Input data.
- Returns
- f(x).
Definition at line 40 of file gelu_function.hpp.
References M_PI.
◆ Fn() [2/2]
|
inlinestatic |
Computes the GELU function.
- Parameters
-
x Input data. y The resulting output activation.
Definition at line 53 of file gelu_function.hpp.
References M_PI.
The documentation for this class was generated from the following file:
- /home/ryan/src/mlpack.org/_src/mlpack-git/src/mlpack/methods/ann/activation_functions/gelu_function.hpp