Architecture Selection in Neural Networks by Statistical and Machine Learning

One of the biggest problems in using artificial neural networks is to determine the best architecture. This is a crucial problem since there are no general rules to select the best architecture structure. Selection of the best architecture is to determine how many neurons should be used in the layers of a network. It is a well-known fact that using a proper architecture structure directly affect the performance of the method. Therefore, various approaches ranging from trial and error method to heuristic optimization algorithms have been suggested to solve this problem in the literature. Although there have been systematical approaches in the literature, trial and error method has been widely used in various applications to find a good architecture. This study propose a new architecture selection method based on statistical and machine learning. The proposed method utilizes regression analysis that is a supervised learning technique in machine learning. In this new architecture selection approach, it is aimed to combine statistical and machine learning to reach good architectures which has high performance. The proposed approach brings a new perspective since it is possible to perform statistical hypothesis tests and to statistically evaluate the obtained results when artificial neural networks are used. The best architecture structure can be statistically determined in the proposed approach. In addition to this, the proposed approach provides some important advantages. This is the first study using a statistical method to utilize statistical hypothesis tests in artificial neural networks. Using regression analysis is easy to use so applying the proposed method is also easy. And, the proposed approach saves time since the best architecture is determined by regression analysis. Furthermore, it is possible to make inference for architectures which is not examined. The proposed approach is applied to three real data sets to show the applicability of the approach. The obtained results show that the proposed method gives very satisfactory results for real data sets.