A Review of Computer Vision for Railways

Modern railways continue to strive for remote and automated methods to improve the visual inspection procedures for their assets. In some cases, these inspections provide new information that could not previously be collected, while in other cases they help them improve upon the quality control, safety, time and costs associated with manual inspection. As such, computer vision continues to find applications for visually inspecting the track, earthworks, tunnels, overhead line equipment and rolling stock. Considering the recent pace of computer vision related developments, this paper seeks to review the state of the art of the field for railways. First, the hardware and data requirements are discussed, focusing on the unique challenges associated with operating optical equipment in a railway environment, such as contamination, power sources and lighting. This also discusses the most common mounting arrangements for camera hardware, including rolling-stock, satellites and way-side cameras. Next, image processing algorithms are discussed, comparing classical approaches and more modern artificial intelligence approaches, for example You Only Look Once (YOLO) and Region-Based Convolutional Neural Network (R-CNN). Then the most common applications for computer vision in the rail industry are analysed. First the track is studied considering computer vision analysis for the detection of different types of rail surface defects on plain line and turnouts, fastener defects, concrete track slab cracking and ballast particle characterisation. Next, the overhead line equipment is considered with applications related to detecting contact loss between pantograph and contact wire, stagger behaviour and defective catenary components. This is followed by discussion of other applications such as rail tunnelling subsidence, tunnel inspection, level crossings, trespass and on-track safety hazards. Finally, opportunities for future research are discussed such as hyperspectral imaging and generative AI, along with possible frontier technologies such as quantum computing.