Dual representation-based Light Field View Synthesis using deformable convolutional and Deep Residual Channel Attention networks

Abstract

Light Field (LF) cameras simultaneously capture both intensity values and directional information of light rays in a single exposure, providing a unique perspective for computational photography and 3D geometry perception. However, existing LF cameras are constrained by sensor resolution, limiting their ability to capture high spatial and angular resolutions simultaneously. To mitigate these issues, various learning-based methods have been proposed to increase the angular resolution of captured LFs, known as LF View Synthesis (LFVS). Many of these methods either neglect essential geometric cues or rely on neural networks with large receptive fields, which restrict their ability to accurately exploit LF structural characteristics. In response to these challenges, this paper introduces a dual representation-based LFVS method that employs deformable convolutional and Deep Residual Channel Attention (DRCA) networks. The proposed method includes two main modules: (i) Coarse Light Field View Synthesis (CLFVS) for initial LFVS, and (ii) Coarse-To-Fine Refinement (CFR) for final quality enhancement. The CLFVS module relies on deformable convolutions to adaptively extract LF features using two parallel networks: (i) Spatial Feature Extraction (SPFE) network using a depth-dependent LFVS approach, and (ii) Angular Feature Extraction (AFE) network using a non-depth-dependent LFVS approach. The CFR module refines the CLFVS output using a DRCA network, which employs dense residual connections between residual groups instead of conventional convolutional layers. The DRCA network employs Residual Channel Attention Blocks (RCABs) to model inter-channel dependencies, selectively enhancing meaningful features while suppressing irrelevant ones. The proposed method achieves state-of-the-art performance on synthetic and real-world LF benchmarks.