A Survey of Research Progress and Theory Foundation in Large Model

Abstract

In recent years, with the rapid development of key elements and core technologies of artificial intelligence, large-scale pre-training model (large model) has achieved remarkable effects. As specific practice progresses of large model, it is useful to realize the universality and generalizability of artificial intelligence, and respond to the strategic goal of building a strong model framework. From the perspective of theory, this article explores the support points of large model in the theories of intrinsic subspace, effective model complexity, and low rank decomposition. We discuss the research findings, implications and limitations of model development, and puts forward relevant suggestions for the future trend.