#include <optimize.h>
Inheritance diagram for _gd_momentum:
Public Member Functions | |
_gd_momentum (LM *lm, const Step &lr, const Step &m) | |
const Dir & | direction () |
Search direction at w. | |
Public Attributes | |
Step | momentum |
With momentum, the weight change is accumulated:
When the derivative is roughly constant, this speeds up the training by using an effectively larger learning rate
When the derivative somehow ``fluctuates'', the momentum stablizes the optimization to some exent by effectively decreasing the learning rate.
Definition at line 169 of file optimize.h.
|
Definition at line 172 of file optimize.h. |
|
Search direction at w.
Reimplemented from _gradient_descent. Definition at line 175 of file optimize.h. |
|
Definition at line 170 of file optimize.h. |