Output Partitioning of Neural Networks

Abstract

Many constructive learning algorithms have been proposed to find an appropriate network structure for a classification problem automatically. Constructive learning algorithms have drawbacks especially when used for complex tasks and modular approaches have been devised to solve these drawbacks. At the same time, parallel training for neural networks with fixed configurations has also been proposed to accelerate the training process. A new approach that combines advantages of constructive learning and parallelism, output partitioning, is presented in this paper. Classification error is used to guide the proposed incremental-partitioning algorithm, which divides the original dataset into several smaller sub-datasets with distinct classes. Each sub-dataset is then handled in parallel, by a smaller constructively trained sub-network which uses the whole input vector and produces a portion of the final output vector where each class is represented by one unit. Three classification datasets are used to test the validity of this method, and results show that this method reduces the classification test error.