Hyperbolic Optimizer as a Dynamical System
Revista : Proceedings of the 41st International Conference on Machine Learning (PMLR)Volumen : 235
Páginas : 1243-1260
Tipo de publicación : ISI Ir a publicación
Abstract
During the last few years, the field of dynamical systems has been developing innovative tools to study the asymptotic behavior of different optimizers in the context of neural networks. In this work, we redefine an extensively studied optimizer, employing classical techniques from hyperbolic geometry. This new definition is linked to a non-linear differential equation as a continuous limit. Additionally, by utilizing Lyapunov stability concepts, we analyze the asymptotic behavior of its critical points.