Attention-enhanced multi-branch spiking neural network for event stream super-resolution

Abstract

Traditional visual sensors capture images by sampling light at fixed intervals, producing a sequence of frames. In contrast, event vision sensors detect changes in light intensity asynchronously at the pixel level and generate discrete events with precise timing information, allowing to capture challenging scenes with high-speed object motion and extreme lighting conditions accurately. This paper introduces an attention-enhanced, polarity-aware multi-branch Spiking Neural Network (SNN) that directly super-resolves low-resolution event streams while preserving their temporal accuracy. The proposed architecture uses two parallel branches with novel spike-based spatial and temporal attention modules that give more importance to salient spatio-temporal structures, yet remaining fully asynchronous and hardware-friendly. Experimental results demonstrate that the proposed spike-based attention with multi-branch SNN significantly outperforms existing state-of-the-art methods across all evaluation metrics, while effectively maintaining the underlying spatio-temporal characteristics of the event stream.