Entropy-Based Model Selection Using Monte Carlo Method

Abstract

In this investigation, we propose a new kind of simplification specialized for Multiple Regression Analysis (MRA) using Random sampling. We propose a novel approach to simplify MRA models for dimension reduction while preserving amount of information. After applying Principle Component Analysis (PCA) to explanatory variables of interests to simplify relationship among them, we reduce the variables (dimensions) quickly to avoid loss of entropy in an efficient manner. We show an experimental results to see the effectiveness of this approach. Our main idea comes from random sampling with the tight relationship between entropy and multiple correlation coefficients (MCC).