Self-Attention Mechanism in GANs for Molecule Generation

Abstract

In discrete sequence based Generative Adversarial Networks (GANs), it is important to both land the samples in the initial distribution and drive the generation towards desirable properties. However, in the case of longer molecules, the existing models seem to under-perform in producing new molecules. In this work, we propose the use of Self-Attention mechanism for Generative Adversarial Networks to allow long range dependencies. Self-Attention mechanism has produced improved rewards in novelty and promising results in generating molecules.