TransUNet for Cross-Domain Semantic Segmentation of Urban Scenery

Abstract

TransUNet is a hybrid architecture that combines a transformer-based encoder with a CNN-based UNet. Originally introduced for semantic segmentation of medical images, we show in our work that TransUNet can be successfully applied to urban scenery datasets commonly used for developing autonomous driving systems. We also explore the performance characteristics of training on multi-domain data from the real world and a simulator, and show that using simulated images to augment a live dataset can improve segmentation performance. Code will be made available at https://github.com/weiyuen.