A recent survey on controllable text generation: A causal perspective

Abstract

As an important subject of natural language generation, Controllable Text Generation (CTG) focuses on integrating additional constraints and controls while generating texts and has attracted a lot of attention. Existing controllable text generation approaches mainly capture the statistical association implied within training texts, but generated texts lack causality consideration. This paper intends to review recent CTG approaches from a causal perspective. Firstly, according to previous research on basic types of CTG models, it is discovered that their essence is to obtain the association, and then four kinds of challenges caused by absence of causality are introduced. Next, this paper reviews the improvements to address these challenges from four aspects, namely representation disentanglement, causal inference, knowledge enhancement and multi-aspect CTG respectively. Additionally, this paper inspects existing evaluations of CTG, especially evaluations for causality of CTG. Finally, this review discusses some future research directions for the causality improvement of CTG and makes a conclusion.