Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training - Université Toulouse - Jean Jaurès
Article Dans Une Revue SIAM Journal on Optimization Année : 2023

Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training

Résumé

A class of multilevel algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the momentum-less AdaGrad method as a particular (single-level) instance. The choice of avoiding the evaluation of the objective function is intended to make the algorithms of the class less sensitive to noise, while the multilevel feature aims at reducing their computational cost. The evaluation complexity of these algorithms is analyzed and their behavior in the presence of noise is then illustrated in the context of training deep neural networks for supervised learning applications.
Fichier principal
Vignette du fichier
2302.07049v1.pdf (1.48 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04436393 , version 1 (14-10-2024)

Licence

Identifiants

Citer

Serge Gratton, Alena Kopaničáková, Philippe L. Toint. Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training. SIAM Journal on Optimization, 2023, 33 (4), pp.2772-2800. ⟨10.1137/23M1553455⟩. ⟨hal-04436393⟩
53 Consultations
0 Téléchargements

Altmetric

Partager

More