Online Adversarial Knowledge Distillation for Image Synthesis of Bridge Defect

Bridge defect detection is an essential task of its daily maintenance, which aims to protect people's life and property safety. However, for a variety of reasons, research institutions have been faced with the scarcity of anomaly samples. One solution is using generative adversarial network (GAN) to generate extra samples for data augmentation. In this paper, we draw on the idea from online knowledge distillation to improve the self-attention GAN, and propose a new framework called Online Knowledge Distillation -Self Attention Generative Adversarial Network (OKD-SAGAN). We introduce a new module called connector which has the same structure with discriminator to train multiple groups of SAGAN together. The role of the connector is to control the output distribution of the corresponding generator to be consistent with the surrounding generators in order to achieve the purpose of mutual learning. We have conducted experiments on the CODEBRIM dataset and in order to further illustrate the effectiveness of OKD structure, we also applied OKD on ACGAN for experiments. The results show that the performance of some generators has exceeded a single set of SAGAN and ACGAN. Compared with SAGAN, OKD-SAGAN ’s FID score decreases by 15.4% and the average FID score decreases by 5.5%. As for ACGAN, OKD-ACGAN ’s FID score decreases by 7.6% and the average FID score decreases by 3.8%, which proves the validity of OKD structure.