Stochastic validation for automated tuning of neural network’s hyper-parameters

Abstract

In this paper we describe a new method for automated tuning of hyper-parameters of supervised learning systems. It uses memory-based learning principles, follows certain ideas of experimental design and employs an alternative approach to resampling called stochastic validation. The described method allows not only for an efficient search through a decision space, but also for a corcurrent validation of the learning algorithm performance on a given data. Potential usefulness of the proposed approach is illustrated with the Fuzzy-ARTMAP neural network application to learning a qualitative positioning of an indoor mobile robot equipped with ultrasonic range sensors. Automatically selected neural network setpoints reach a comparable performance to those achieved by human experts in two-dimensional parameter optimization cases. Migration of the proposed method to higher-order optimization domains bears a big promise and requires further research.