An incremental approach to contribution-based feature selection

Abstract

This paper presents a novel feature selection approach based on an incremental neural network (NN) training approach. Instead of training input attributes in batch, this incremental approach trains input attributes one by one so that network performance keeps refined when each new attribute comes in. If an incoming attribute is consistent with previous attributes and relevant to output attributes, network performance will be improved, otherwise degraded. The contribution of an input attribute is evaluated through network performance evaluation. Attributes with little or no contribution will be discarded. In order to have fair feature selection, the individual discrimination ability of each attribute is evaluated before training by using a NN with only one attribute in the input layer. The attribute with the best discrimination ability will be introduced first, followed by those attributes with lower discrimination ability. Two feature-detection methods are discussed based on this incremental training approach. Unlike existing feature selection methods, the proposed feature selection methods are suitable not only for classification problems, but also for regression problems. Experimental results show that our methods work well on several benchmark problems and NN accuracy improved after feature selection.