A Novel Underwater Image Synthesis Method Based on a Pixel-Level Self-Supervised Training Strategy

Author(s):  
Zhiheng Wu ◽  
Zhengxing Wu ◽  
Yue Lu ◽  
Jian Wang ◽  
Junzhi Yu
2021 ◽  
Vol 8 ◽  
Author(s):  
Qi Zhao ◽  
Ziqiang Zheng ◽  
Huimin Zeng ◽  
Zhibin Yu ◽  
Haiyong Zheng ◽  
...  

Underwater depth prediction plays an important role in underwater vision research. Because of the complex underwater environment, it is extremely difficult and expensive to obtain underwater datasets with reliable depth annotation. Thus, underwater depth map estimation with a data-driven manner is still a challenging task. To tackle this problem, we propose an end-to-end system including two different modules for underwater image synthesis and underwater depth map estimation, respectively. The former module aims to translate the hazy in-air RGB-D images to multi-style realistic synthetic underwater images while retaining the objects and the structural information of the input images. Then we construct a semi-real RGB-D underwater dataset using the synthesized underwater images and the original corresponding depth maps. We conduct supervised learning to perform depth estimation through the pseudo paired underwater RGB-D images. Comprehensive experiments have demonstrated that the proposed method can generate multiple realistic underwater images with high fidelity, which can be applied to enhance the performance of monocular underwater image depth estimation. Furthermore, the trained depth estimation model can be applied to real underwater image depth map estimation. We will release our codes and experimental setting in https://github.com/ZHAOQIII/UW_depth.


2014 ◽  
Vol 63 ◽  
pp. 54-61 ◽  
Author(s):  
Young-Choon Kim ◽  
Tae-Wuk Bae ◽  
Hyuk-Ju Kwon ◽  
Byoung-Ik Kim ◽  
Sang-Ho Ahn

Science ◽  
2019 ◽  
Vol 364 (6439) ◽  
pp. eaav9436 ◽  
Author(s):  
Pouya Bashivan ◽  
Kohitij Kar ◽  
James J. DiCarlo

Particular deep artificial neural networks (ANNs) are today’s most accurate models of the primate brain’s ventral visual stream. Using an ANN-driven image synthesis method, we found that luminous power patterns (i.e., images) can be applied to primate retinae to predictably push the spiking activity of targeted V4 neural sites beyond naturally occurring levels. This method, although not yet perfect, achieves unprecedented independent control of the activity state of entire populations of V4 neural sites, even those with overlapping receptive fields. These results show how the knowledge embedded in today’s ANN models might be used to noninvasively set desired internal brain states at neuron-level resolution, and suggest that more accurate ANN models would produce even more accurate control.


2018 ◽  
Author(s):  
Gongbo Liang ◽  
Sajjad Fouladvand ◽  
Jie Zhang ◽  
Michael A. Brooks ◽  
Nathan Jacobs ◽  
...  

AbstractComputed tomography (CT) is a widely-used diag-reproducibility regarding radiomic features, such as intensity, nostic image modality routinely used for assessing anatomical tissue characteristics. However, non-standardized imaging pro-tocols are commonplace, which poses a fundamental challenge in large-scale cross-center CT image analysis. One approach to address the problem is to standardize CT images using generative adversarial network models (GAN). GAN learns the data distribution of training images and generate synthesized images under the same distribution. However, existing GAN models are not directly applicable to this task mainly due to the lack of constraints on the mode of data to generate. Furthermore, they treat every image equally, but in real applications, some images are more difficult to standardize than the others. All these may lead to the lack-of-detail problem in CT image synthesis. We present a new GAN model called GANai to mitigate the differences in radiomic features across CT images captured using non-standard imaging protocols. Given source images, GANai composes new images by specifying a high-level goal that the image features of the synthesized images should be similar to those of the standard images. GANai introduces an alternative improvement training strategy to alternatively and steadily improve model performance. The new training strategy enables a series of technical improvements, including phase-specific loss functions, phase-specific training data, and the adoption of ensemble learning, leading to better model performance. The experimental results show that GANai is significantly better than the existing state-of-the-art image synthesis algorithms on CT image standardization. Also, it significantly improves the efficiency and stability of GAN model training.


2019 ◽  
Author(s):  
Robin Liu ◽  
Lu Wang ◽  
Jim He ◽  
Wenfang Chen

AbstractThis paper introduces a detection-based framework to segment glomeruli from digital scanning image of light microscopic slide of renal biopsy specimens. The proposed method aims to better use the precise localization ability of Faster R-CNN and powerful segmentation ability of U-Net. We use a detector to localize the glomeruli from whole slide image to make the segmentation only focus on the most relevant area of the image. We explored the effectiveness of the network depth on its localization and segmentation ability in glomerular classification, and then propose to use the classification network with enhanced ability of localization and segmentation to construct and initialize a segmentation network. We also propose a weakly supervised training strategy to train the segmentation network by taking advantage of the unique morphology of the glomerulus. Both strong initialization and weakly supervised training are used to resolve the problem of insufficient and inaccurate data annotations and enhance the adaptability of the segmentation network. Experimental results demonstrate that the proposed framework is effective and robust.


2019 ◽  
Vol 2019 ◽  
pp. 1-17
Author(s):  
Tae Wuk Bae ◽  
Young Choon Kim ◽  
Sang Ho Ahn

Infrared (IR) target signatures and background scenes are mainly used for military research purposes such as reconnaissance and detection of enemy targets in modern IR imaging systems like IR search and track (IRST) system. For understanding and analyzing IR signatures and backgrounds in the IR imaging systems, an IR wavelength band (WB) conversion which transforms an arbitrary WB image to another WB is very important in the absence of equipment by WB. In addition, IR image synthesis of targets and backgrounds can provide a great deal of information in the IR target detection field. However, the WB conversion is actually a very challenging research due to lack of information on the absorptivity and transmittance of enormous components of an object or atmosphere. In addition, the radiation and reflectance characteristics of short-wave IR (SWIR)-WB are very different from those of long-wave IR (LWIR)-WB and middle-wave IR (MWIR)-WB. Therefore, the WB conversion in this paper is limited only to IR target signatures and monotonous backgrounds, which is commonly used for military purposes, at a long distance. This paper proposes an IR synthesis method for generating a synthesized IR image of three IR-WBs by synthesizing an IR target signature and a real background scene for an arbitrary IR-WB. In the proposed method, each temperature information is first estimated from an IR target signature and IR background image for an arbitrary IR-WB, and then a synthesized temperature image is generated by combining the respective temperature information estimated from the IR target signature and background scene. Finally, the synthesized temperature image is transformed into an IR radiance image of three IR-WBs. Through the proposed method, various IR synthesis experiments are performed for various IR target signature and background scenes.


Sign in / Sign up

Export Citation Format

Share Document