Seeing Beyond Local Events: Recurrent Optical Flow Estimation With Hierarchical Motion Aggregation

Abstract

Current event-based optical flow estimation methods typically utilize at most two event streams as input, overlooking the role of temporal coherence present in continuous event streams for the current motion estimation. Moreover, existing simple motion propagation strategies are insufficient for propagating historical motion information effectively. To this end, we propose TREFlow, a recurrent event-based optical flow estimation framework with hierarchical motion aggregation. Our method aggregates rich motion features in a short-to-long-term manner. We introduce a Short-Term Motion Encoding (STME) module and a Long-Term Memory Aggregation (LTMA) module to capture dense motion features within the current temporal window and comprehensively incorporate historical motion prior knowledge, respectively, thereby enhancing and compensating the current motion representation. Our method outperforms other methods in optical flow inference on MVSEC and DSEC-Flow.