An Introduction of Prediction Models From the View of Integration Between Basic Models

The combination model is a widely used method way in machine learning to design the more sophisticated and well-performed model, which can be applied for the field of the recommendation system and so on. This paper is aims to explore the prediction model The normal prediction models have to deal with neither dense numerical features or sparse categorical features. Decision tree is a popular way to deal with dense data, doing both regression and classification. It has various famous expanding algorithms, such as GBDT, XGBoost, and so on, which is designed by different ways of combing the basic decision tree. GBDT and XGBoost are both good at Large-scale machine learning and perform well with dense numerical features. As for sparse categorical features. FM is no doubt the best basic algorithm designed for it. By combing FM with other models, Wide&Deep and DeepFM are better for the recommended system because they perform well and can deal with high-order feature intersection. Furthermore, we can not only just add models together, Deep&Cross and DeepGBM are examples which use NN (natural network) to approximate other models, combing the different parts to an integration. And NN can not only substitute the model is approximated, but also is suited for the online training. This paper is aims to explore the development of the prediction models and the future trend for its improvement.