User-Centered Development of a Pedestrian Assistance System Using End-to-End Learning

In this paper, we propose an algorithm developed to detect the curbstone and its surroundings. This work is a part of the user-centered development of an assistance system currently being developed to support the older pedestrians crossing the road. For the development of this algorithm, an end-to-end learning approach was chosen. The convolutional neural network was selected to process raw pixels from a mono camera and the network was trained on a dataset to detect the curb. The use of end-to-end learning with a convolutional neural network proved remarkably powerful in distinguishing the curbstone. In order to train the network, images of curb and their surroundings were essential. For this purpose, a new dataset was created where multiple requirements, for example, different approach angles to the curbstone, weather and light conditions etc, were considered. As this system is currently being developed for Berlin (Germany), an analysis was carried out to determine the types and frequencies of pavements in Berlin pathways. Based on this analysis and the requirements, a dataset was created which comprises the images of the pavements, for example, cobblestone, concrete slabs etc, in diverse sets of weather and light conditions. This dataset was developed using the videos taken at 10 frames per second from a mono camera. For the collection of dataset and for testing purposes, a prototype in the form of a walker was built which has sensors, Leddar and camera mounted on it. This paper gives an overview of the development of the algorithm and describes the procedures, such as district analysis of Berlin and data collection, needed to develop the algorithm.