Unpaired Underwater Image Synthesis with a Disentangled Representation for Underwater Depth Map Prediction

As one of the key requirements for underwater exploration, underwater depth map estimation is of great importance in underwater vision research. Although significant progress has been achieved in the fields of image-to-image translation and depth map estimation, a gap between normal depth map estimation and underwater depth map estimation still remains. Additionally, it is a great challenge to build a mapping function that converts a single underwater image into an underwater depth map due to the lack of paired data. Moreover, the ever-changing underwater environment further intensifies the difficulty of finding an optimal mapping solution. To eliminate these bottlenecks, we developed a novel image-to-image framework for underwater image synthesis and depth map estimation in underwater conditions. For the problem of the lack of paired data, by translating hazy in-air images (with a depth map) into underwater images, we initially obtained a paired dataset of underwater images and corresponding depth maps. To enrich our synthesized underwater dataset, we further translated hazy in-air images into a series of continuously changing underwater images with a specified style. For the depth map estimation, we included a coarse-to-fine network to provide a precise depth map estimation result. We evaluated the efficiency of our framework for a real underwater RGB-D dataset. The experimental results show that our method can provide a diversity of underwater images and the best depth map estimation precision.

Download Full-text

The Synthesis of Unpaired Underwater Images for Monocular Underwater Depth Prediction

Frontiers in Marine Science ◽

10.3389/fmars.2021.690962 ◽

2021 ◽

Vol 8 ◽

Author(s):

Qi Zhao ◽

Ziqiang Zheng ◽

Huimin Zeng ◽

Zhibin Yu ◽

Haiyong Zheng ◽

...

Keyword(s):

Structural Information ◽

Depth Map ◽

Depth Estimation ◽

Image Synthesis ◽

Map Estimation ◽

Estimation Model ◽

Depth Prediction ◽

Underwater Image ◽

Underwater Vision ◽

Image Depth

Underwater depth prediction plays an important role in underwater vision research. Because of the complex underwater environment, it is extremely difficult and expensive to obtain underwater datasets with reliable depth annotation. Thus, underwater depth map estimation with a data-driven manner is still a challenging task. To tackle this problem, we propose an end-to-end system including two different modules for underwater image synthesis and underwater depth map estimation, respectively. The former module aims to translate the hazy in-air RGB-D images to multi-style realistic synthetic underwater images while retaining the objects and the structural information of the input images. Then we construct a semi-real RGB-D underwater dataset using the synthesized underwater images and the original corresponding depth maps. We conduct supervised learning to perform depth estimation through the pseudo paired underwater RGB-D images. Comprehensive experiments have demonstrated that the proposed method can generate multiple realistic underwater images with high fidelity, which can be applied to enhance the performance of monocular underwater image depth estimation. Furthermore, the trained depth estimation model can be applied to real underwater image depth map estimation. We will release our codes and experimental setting in https://github.com/ZHAOQIII/UW_depth.

Download Full-text

Unpaired Image- to- Image Translation using Cycle Generative Adversarial Networks

International Journal of Engineering and Advanced Technology - Regular Issue ◽

10.35940/ijeat.f1525.089620 ◽

2020 ◽

Vol 9 (6) ◽

pp. 380-385

Keyword(s):

Mapping Function ◽

Image Synthesis ◽

Input Image ◽

General Purpose ◽

Training Dataset ◽

Generative Adversarial Networks ◽

Inverse Mapping ◽

Style Transfer ◽

Adversarial Networks ◽

Image Translation

In this burgeoning age and society where people are tending towards learning the benefits adversarial network we hereby benefiting the society tend to extend our research towards adversarial networks as a general-purpose solution to image-to-image translation problems. Image to image translation comes under the peripheral class of computer sciences extending our branch in the field of neural networks. We apprentice Generative adversarial networks as an optimum solution for generating Image to image translation where our motive is to learn a mapping between an input image(X) and an output image(Y) using a set of predefined pairs[4]. But it is not necessary that the paired dataset is provided to for our use and hence adversarial methods comes into existence. Further, we advance a method that is able to convert and recapture an image from a domain X to another domain Y in the absence of paired datasets. Our objective is to learn a mapping function G: A —B such that the mapping is able to distinguish the images of G(A) within the distribution of B using an adversarial loss.[1] Because this mapping is high biased, we introduce an inverse mapping function F B—A and introduce a cycle consistency loss[7]. Furthermore we wish to extend our research with various domains and involve them with neural style transfer, semantic image synthesis. Our essential commitment is to show that on a wide assortment of issues, conditional GANs produce sensible outcomes. This paper hence calls for the attention to the purpose of converting image X to image Y and we commit to the transfer learning of training dataset and optimising our code.You can find the source code for the same here.

Download Full-text

SAR TO OPTICAL IMAGE SYNTHESIS FOR CLOUD REMOVAL WITH GENERATIVE ADVERSARIAL NETWORKS

ISPRS Annals of Photogrammetry Remote Sensing and Spatial Information Sciences ◽

10.5194/isprs-annals-iv-1-5-2018 ◽

2018 ◽

Vol IV-1 ◽

pp. 5-11 ◽

Cited By ~ 6

Author(s):

J. D. Bermudez ◽

P. N. Happ ◽

D. A. B. Oliveira ◽

R. Q. Feitosa

Keyword(s):

Mapping Function ◽

Image Synthesis ◽

Generative Adversarial Networks ◽

Atmospheric Conditions ◽

Sar Images ◽

Active Sensors ◽

Adversarial Networks ◽

Optical Images ◽

Image Translation ◽

Cloud Removal

<p><strong>Abstract.</strong> Optical imagery is often affected by the presence of clouds. Aiming to reduce their effects, different reconstruction techniques have been proposed in the last years. A common alternative is to extract data from active sensors, like Synthetic Aperture Radar (SAR), because they are almost independent on the atmospheric conditions and solar illumination. On the other hand, SAR images are more complex to interpret than optical images requiring particular handling. Recently, Conditional Generative Adversarial Networks (cGANs) have been widely used in different image generation tasks presenting state-of-the-art results. One application of cGANs is learning a nonlinear mapping function from two images of different domains. In this work, we combine the fact that SAR images are hardly affected by clouds with the ability of cGANS for image translation in order to map optical images from SAR ones so as to recover regions that are covered by clouds. Experimental results indicate that the proposed solution achieves better classification accuracy than SAR based classification.</p>

Download Full-text

Underwater Image Processing and Object Detection Based on Deep CNN Method

Journal of Sensors ◽

10.1155/2020/6707328 ◽

2020 ◽

Vol 2020 ◽

pp. 1-20

Author(s):

Fenglei Han ◽

Jingzheng Yao ◽

Haitao Zhu ◽

Chunhui Wang

Keyword(s):

Image Processing ◽

Deep Sea ◽

Autonomous Operation ◽

Underwater Image ◽

Underwater Vision ◽

Development And Utilization ◽

Deep Cnn ◽

Underwater Exploration ◽

Underwater Image Processing ◽

Intelligent Computer

Due to the importance of underwater exploration in the development and utilization of deep-sea resources, underwater autonomous operation is more and more important to avoid the dangerous high-pressure deep-sea environment. For underwater autonomous operation, the intelligent computer vision is the most important technology. In an underwater environment, weak illumination and low-quality image enhancement, as a preprocessing procedure, is necessary for underwater vision. In this paper, a combination of max-RGB method and shades of gray method is applied to achieve the enhancement of underwater vision, and then a CNN (Convolutional Neutral Network) method for solving the weakly illuminated problem for underwater images is proposed to train the mapping relationship to obtain the illumination map. After the image processing, a deep CNN method is proposed to perform the underwater detection and classification, according to the characteristics of underwater vision, two improved schemes are applied to modify the deep CNN structure. In the first scheme, a 1∗1 convolution kernel is used on the 26∗26 feature map, and then a downsampling layer is added to resize the output to equal 13∗13. In the second scheme, a downsampling layer is added firstly, and then the convolution layer is inserted in the network, the result is combined with the last output to achieve the detection. Through comparison with the Fast RCNN, Faster RCNN, and the original YOLO V3, scheme 2 is verified to be better in detecting underwater objects. The detection speed is about 50 FPS (Frames per Second), and mAP (mean Average Precision) is about 90%. The program is applied in an underwater robot; the real-time detection results show that the detection and classification are accurate and fast enough to assist the robot to achieve underwater working operation.

Download Full-text