On the Invariance of the SELU Activation Function on Algorithm and Hyperparameter Selection in Neural Network Recommenders - Artificial Intelligence Applications and Innovations Access content directly
Conference Papers Year : 2019

On the Invariance of the SELU Activation Function on Algorithm and Hyperparameter Selection in Neural Network Recommenders

Flora Sakketou
  • Function : Author
  • PersonId : 1056781
Nicholas Ampazis
  • Function : Author
  • PersonId : 1056782

Abstract

In a number of recent studies the Scaled Exponential Linear Unit (SELU) activation function has been shown to automatically regularize network parameters and to make learning robust due to its self-normalizing properties. In this paper we explore the utilization of SELU in training different neural network architectures for recommender systems and validate that it indeed outperforms other activation functions for these types of problems. More interestingly however, we show that SELU also exhibits performance invariance with regards to the selection of the optimization algorithm and its corresponding hyperparameters. This is clearly demonstrated by a number of experiments which involve a number of activation functions and optimization algorithms for training different neural network architectures on standard recommender systems benchmark datasets.
Fichier principal
Vignette du fichier
483292_1_En_56_Chapter.pdf (657.68 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02331305 , version 1 (24-10-2019)

Licence

Attribution

Identifiers

Cite

Flora Sakketou, Nicholas Ampazis. On the Invariance of the SELU Activation Function on Algorithm and Hyperparameter Selection in Neural Network Recommenders. 15th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), May 2019, Hersonissos, Greece. pp.673-685, ⟨10.1007/978-3-030-19823-7_56⟩. ⟨hal-02331305⟩
195 View
83 Download

Altmetric

Share

Gmail Facebook X LinkedIn More