Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network

GA-CSPN: generative adversarial monocular depth estimation with second-order convolutional spatial propagation network

Journal of Electronic Imaging ◽

10.1117/1.jei.30.4.043019 ◽

2021 ◽

Vol 30 (04) ◽

Author(s):

Zhengyang Lu ◽

Ying Chen

Keyword(s):

Depth Estimation ◽

Second Order ◽

Spatial Propagation ◽

Monocular Depth ◽

Propagation Network

Download Full-text

Hyperspectral Image Classification With Spatial Consistence Using Fully Convolutional Spatial Propagation Network

IEEE Transactions on Geoscience and Remote Sensing ◽

10.1109/tgrs.2021.3049282 ◽

2021 ◽

pp. 1-13

Author(s):

Yenan Jiang ◽

Ying Li ◽

Shanrong Zou ◽

Haokui Zhang ◽

Yunpeng Bai

Keyword(s):

Image Classification ◽

Hyperspectral Image ◽

Hyperspectral Image Classification ◽

Spatial Propagation ◽

Propagation Network

Download Full-text

Learning Depth with Convolutional Spatial Propagation Network

IEEE Transactions on Pattern Analysis and Machine Intelligence ◽

10.1109/tpami.2019.2947374 ◽

2020 ◽

Vol 42 (10) ◽

pp. 2361-2379 ◽

Cited By ~ 4

Author(s):

Xinjing Cheng ◽

Peng Wang ◽

Ruigang Yang

Keyword(s):

Spatial Propagation ◽

Propagation Network

Download Full-text

CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i07.6635 ◽

2020 ◽

Vol 34 (07) ◽

pp. 10615-10622

Author(s):

Xinjing Cheng ◽

Peng Wang ◽

Chenye Guan ◽

Ruigang Yang

Keyword(s):

Color Image ◽

Depth Map ◽

Computational Resource ◽

Learning Context ◽

Structural Details ◽

Spatial Propagation ◽

Effectiveness And Efficiency ◽

Computational Budget ◽

Propagation Network ◽

Resource Aware

Depth Completion deals with the problem of converting a sparse depth map to a dense one, given the corresponding color image. Convolutional spatial propagation network (CSPN) is one of the state-of-the-art (SoTA) methods of depth completion, which recovers structural details of the scene. In this paper, we propose CSPN++, which further improves its effectiveness and efficiency by learning adaptive convolutional kernel sizes and the number of iterations for the propagation, thus the context and computational resource needed at each pixel could be dynamically assigned upon requests. Specifically, we formulate the learning of the two hyper-parameters as an architecture selection problem where various configurations of kernel sizes and numbers of iterations are first defined, and then a set of soft weighting parameters are trained to either properly assemble or select from the pre-defined configurations at each pixel. In our experiments, we find weighted assembling can lead to significant accuracy improvements, which we referred to as "context-aware CSPN", while weighted selection, "resource-aware CSPN" can reduce the computational resource significantly with similar or better accuracy. Besides, the resource needed for CSPN++ can be adjusted w.r.t. the computational budget automatically. Finally, to avoid the side effects of noise or inaccurate sparse depths, we embed a gated network inside CSPN++, which further improves the performance. We demonstrate the effectiveness of CSPN++ on the KITTI depth completion benchmark, where it significantly improves over CSPN and other SoTA methods 1.

Download Full-text