scholarly journals Image Inpainting Using Two-Stage Loss Function and Global and Local Markovian Discriminators

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6193
Author(s):  
Chen Li ◽  
Kai He ◽  
Kun Liu ◽  
Xitao Ma
Keyword(s):  
Loss Function ◽  
Signal To Noise Ratio ◽  
Image Inpainting ◽  
Structural Similarity ◽  
Two Stage ◽  
Generative Adversarial Network ◽  
Gaussian Kernels ◽  
Adversarial Network ◽  
Global And Local ◽  
Image Details

Image inpainting networks can produce visually reasonable results in the damaged regions. However, existing inpainting networks may fail to reconstruct the proper structures or tend to generate the results with color discrepancy. To solve this issue, this paper proposes an image inpainting approach using the proposed two-stage loss function. The loss function consists of different Gaussian kernels, which are utilized in different stages of network. The use of our two-stage loss function in coarse network helps to focus on the image structure, while the use of it in refinement network is helpful to restore the image details. Moreover, we proposed a global and local PatchGANs (GAN means generative adversarial network), named GL-PatchGANs, in which the global and local markovian discriminators were used to control the final results. This is beneficial to focus on the regions of interest (ROI) on different scales and tends to produce more realistic structural and textural details. We trained our network on three popular datasets on image inpainting separately, both Peak Signal to Noise ratio (PSNR) and Structural Similarity (SSIM) between our results, and ground truths on test images show that our network can achieve better performance compared with the recent works in most cases. Besides, the visual results on three datasets also show that our network can produce visual plausible results compared with the recent works.

scholarly journals Application of a Modified Generative Adversarial Network in the Superresolution Reconstruction of Ancient Murals

2020 ◽  
Vol 2020 ◽  
pp. 1-12
Author(s):  
Jianfang Cao ◽  
Zibang Zhang ◽  
Aidi Zhao
Keyword(s):  
High Resolution ◽  
Signal To Noise Ratio ◽  
Reconstruction Algorithm ◽  
Structural Similarity ◽  
Image Feature ◽  
Generative Adversarial Network ◽  
Resolution Image ◽  
High Resolution Image ◽  
Adversarial Network ◽  
Set Up

Considering the problems of low resolution and rough details in existing mural images, this paper proposes a superresolution reconstruction algorithm for enhancing artistic mural images, thereby optimizing mural images. The algorithm takes a generative adversarial network (GAN) as the framework. First, a convolutional neural network (CNN) is used to extract image feature information, and then, the features are mapped to the high-resolution image space of the same size as the original image. Finally, the reconstructed high-resolution image is output to complete the design of the generative network. Then, a CNN with deep and residual modules is used for image feature extraction to determine whether the output of the generative network is an authentic, high-resolution mural image. In detail, the depth of the network increases, the residual module is introduced, the batch standardization of the network convolution layer is deleted, and the subpixel convolution is used to realize upsampling. Additionally, a combination of multiple loss functions and staged construction of the network model is adopted to further optimize the mural image. A mural dataset is set up by the current team. Compared with several existing image superresolution algorithms, the peak signal-to-noise ratio (PSNR) of the proposed algorithm increases by an average of 1.2–3.3 dB and the structural similarity (SSIM) increases by 0.04 = 0.13; it is also superior to other algorithms in terms of subjective scoring. The proposed method in this study is effective in the superresolution reconstruction of mural images, which contributes to the further optimization of ancient mural images.

scholarly journals Wasserstein Generative Adversarial Network Based De-Blurring Using Perceptual Similarity

2019 ◽  
Vol 9 (11) ◽  
pp. 2358 ◽  
Author(s):  
Minsoo Hong ◽  
Yoonsik Choe
Keyword(s):  
Neural Network ◽  
Loss Function ◽  
Structural Similarity ◽  
Wasserstein Distance ◽  
Perceptual Similarity ◽  
Generative Adversarial Network ◽  
Edge Information ◽  
Adversarial Network ◽  
Blurred Images ◽  
Parameter Values

The de-blurring of blurred images is one of the most important image processing methods and it can be used for the preprocessing step in many multimedia and computer vision applications. Recently, de-blurring methods have been performed by neural network methods, such as the generative adversarial network (GAN), which is a powerful generative network. Among many different types of GAN, the proposed method is performed using the Wasserstein generative adversarial network with gradient penalty (WGANGP). Since edge information is the most important factor in an image, the style loss function is applied to represent the perceptual information of the edge in order to preserve small edge information and capture its perceptual similarity. As a result, the proposed method improves the similarity between sharp and blurred images by minimizing the Wasserstein distance, and it captures well the perceptual similarity using the style loss function, considering the correlation of features in the convolutional neural network (CNN). To confirm the performance of the proposed method, three experiments are conducted using two datasets: the GOPRO Large and Kohler dataset. The optimal solutions are found by changing the parameter values experimentally. Consequently, the experiments depict that the proposed method achieves 0.98 higher performance in structural similarity (SSIM) and outperforms other de-blurring methods in the case of both datasets.

scholarly journals QCBCT-NET for direct measurement of bone mineral density from quantitative cone-beam CT: a human skull phantom study

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Tae-Hoon Yong ◽  
Su Yang ◽  
Sang-Jeong Lee ◽  
Chansoo Park ◽  
Jo-Eun Kim ◽  
...  
Keyword(s):  
Signal To Noise Ratio ◽  
Cone Beam Ct ◽  
Structural Similarity ◽  
Training Data ◽  
Cone Beam ◽  
Absolute Difference ◽  
Mineral Density ◽  
Quantitative Ct ◽  
Generative Adversarial Network ◽  
Adversarial Network

AbstractThe purpose of this study was to directly and quantitatively measure BMD from Cone-beam CT (CBCT) images by enhancing the linearity and uniformity of the bone intensities based on a hybrid deep-learning model (QCBCT-NET) of combining the generative adversarial network (Cycle-GAN) and U-Net, and to compare the bone images enhanced by the QCBCT-NET with those by Cycle-GAN and U-Net. We used two phantoms of human skulls encased in acrylic, one for the training and validation datasets, and the other for the test dataset. We proposed the QCBCT-NET consisting of Cycle-GAN with residual blocks and a multi-channel U-Net using paired training data of quantitative CT (QCT) and CBCT images. The BMD images produced by QCBCT-NET significantly outperformed the images produced by the Cycle-GAN or the U-Net in mean absolute difference (MAD), peak signal to noise ratio (PSNR), normalized cross-correlation (NCC), structural similarity (SSIM), and linearity when compared to the original QCT image. The QCBCT-NET improved the contrast of the bone images by reflecting the original BMD distribution of the QCT image locally using the Cycle-GAN, and also spatial uniformity of the bone images by globally suppressing image artifacts and noise using the two-channel U-Net. The QCBCT-NET substantially enhanced the linearity, uniformity, and contrast as well as the anatomical and quantitative accuracy of the bone images, and demonstrated more accuracy than the Cycle-GAN and the U-Net for quantitatively measuring BMD in CBCT.

scholarly journals An Image Denoising Method Based on BM4D and GAN in 3D Shearlet Domain

2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Shengnan Zhang ◽  
Lei Wang ◽  
Chunhong Chang ◽  
Cong Liu ◽  
Longbo Zhang ◽  
...  
Keyword(s):  
Image Denoising ◽  
Signal To Noise Ratio ◽  
Evaluation Criteria ◽  
Structural Similarity ◽  
Block Matching ◽  
Generative Adversarial Network ◽  
Denoising Method ◽  
Edge Preserving ◽  
Block Matching Algorithm ◽  
Adversarial Network

To overcome the disadvantages of the traditional block-matching-based image denoising method, an image denoising method based on block matching with 4D filtering (BM4D) in the 3D shearlet transform domain and a generative adversarial network is proposed. Firstly, the contaminated images are decomposed to get the shearlet coefficients; then, an improved 3D block-matching algorithm is proposed in the hard threshold and wiener filtering stage to get the latent clean images; the final clean images can be obtained by training the latent clean images via a generative adversarial network (GAN).Taking the peak signal-to-noise ratio (PSNR), structural similarity (SSIM for short) of image, and edge-preserving index (EPI for short) as the evaluation criteria, experimental results demonstrate that the proposed method can not only effectively remove image noise in high noisy environment, but also effectively improve the visual effect of the images.

scholarly journals An Enhanced pix2pix Dehazing Network with Guided Filter Layer

2020 ◽  
Vol 10 (17) ◽  
pp. 5898
Author(s):  
Qirong Bu ◽  
Jie Luo ◽  
Kuan Ma ◽  
Hongwei Feng ◽  
Jun Feng
Keyword(s):  
Signal To Noise Ratio ◽  
Similarity Index ◽  
Structural Similarity ◽  
Comparison Method ◽  
Guided Filter ◽  
Generative Adversarial Network ◽  
Test Dataset ◽  
Adversarial Network ◽  
Filter Layer

In this paper, we propose an enhanced pix2pix dehazing network, which generates clear images without relying on a physical scattering model. This network is a generative adversarial network (GAN) which combines multiple guided filter layers. First, the input of hazy images is smoothed to obtain high-frequency features according to different smoothing kernels of the guided filter layer. Then, these features are embedded in higher dimensions of the network and connected with the output of the generator’s encoder. Finally, Visual Geometry Group (VGG) features are introduced to serve as a loss function to improve the quality of the texture information restoration and generate better hazy-free images. We conduct experiments on NYU-Depth, I-HAZE and O-HAZE datasets. The enhanced pix2pix dehazing network we propose produces increases of 1.22 dB in the Peak Signal-to-Noise Ratio (PSNR) and 0.01 in the Structural Similarity Index Metric (SSIM) compared with a second successful comparison method using the indoor test dataset. Extensive experiments demonstrate that the proposed method has good performance for image dehazing.

scholarly journals Low-Dose CT Image Denoising with Improving WGAN and Hybrid Loss Function

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Zhihua Li ◽  
Weili Shi ◽  
Qiwei Xing ◽  
Yu Miao ◽  
Wei He ◽  
...  
Keyword(s):  
Loss Function ◽  
Low Dose ◽  
Structural Similarity ◽  
Image Method ◽  
Ct Image ◽  
Generative Adversarial Network ◽  
Low Dose Ct ◽  
Adversarial Network ◽  
Quantitative Measurements ◽  
Better Than

The X-ray radiation from computed tomography (CT) brought us the potential risk. Simply decreasing the dose makes the CT images noisy and diagnostic performance compromised. Here, we develop a novel denoising low-dose CT image method. Our framework is based on an improved generative adversarial network coupling with the hybrid loss function, including the adversarial loss, perceptual loss, sharpness loss, and structural similarity loss. Among the loss function terms, perceptual loss and structural similarity loss are made use of to preserve textural details, and sharpness loss can make reconstruction images clear. The adversarial loss can sharp the boundary regions. The results of experiments show the proposed method can effectively remove noise and artifacts better than the state-of-the-art methods in the aspects of the visual effect, the quantitative measurements, and the texture details.

scholarly journals Artificial Intelligence-Assisted Fresco Restoration with Multiscale Line Drawing Generation

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Guanghui Song ◽  
Hai Wang
Keyword(s):  
Artificial Intelligence ◽  
Network Structure ◽  
Signal To Noise Ratio ◽  
Line Drawing ◽  
Structural Similarity ◽  
Block Matching ◽  
Colour Space ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Hsv Colour Space

In this article, we study the mural restoration work based on artificial intelligence-assisted multiscale trace generation. Firstly, we convert the fresco images to colour space to obtain the luminance and chromaticity component images; then we process each component image to enhance the edges of the exfoliated region using high and low hat operations; then we construct a multistructure morphological filter to smooth the noise of the image. Finally, the fused mask image is fused with the original mural to obtain the final calibration result. The fresco is converted to HSV colour space, and chromaticity, saturation, and luminance features are introduced; then the confidence term and data term are used to determine the priority of shedding boundary points; then a new block matching criterion is defined, and the best matching block is obtained to replace the block to be repaired based on the structural similarity between the block to be repaired and the matching block by global search; finally, the restoration result is converted to RGB colour space to obtain the final restoration result. An improved generative adversarial network structure is proposed to address the shortcomings of the existing network structure in mural defect restoration, and the effectiveness of the improved modules of the network is verified. Compared with the existing mural restoration algorithms on the test data experimentally verified, the peak signal-to-noise ratio (PSNR) score is improved by 4% and the structural similarity (SSIM) score is improved by 2%.

scholarly journals Super-Resolution Reconstruction of Underwater Image Based on Image Sequence Generative Adversarial Network

2020 ◽  
Vol 2020 ◽  
pp. 1-10
Author(s):  
Li Li ◽  
Zijia Fan ◽  
Mingyang Zhao ◽  
Xinlei Wang ◽  
Zhongyang Wang ◽  
...  
Keyword(s):  
Network Architecture ◽  
Signal To Noise Ratio ◽  
Super Resolution ◽  
Structural Similarity ◽  
Image Sequence ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Underwater Image ◽  
Dual Generator ◽  
The Stability

Since the underwater image is not clear and difficult to recognize, it is necessary to obtain a clear image with the super-resolution (SR) method to further study underwater images. The obtained images with conventional underwater image super-resolution methods lack detailed information, which results in errors in subsequent recognition and other processes. Therefore, we propose an image sequence generative adversarial network (ISGAN) method for super-resolution based on underwater image sequences collected by multifocus from the same angle, which can obtain more details and improve the resolution of the image. At the same time, a dual generator method is used in order to optimize the network architecture and improve the stability of the generator. The preprocessed images are, respectively, passed through the dual generator, one of which is used as the main generator to generate the SR image of sequence images, and the other is used as the auxiliary generator to prevent the training from crashing or generating redundant details. Experimental results show that the proposed method can be improved on both peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) compared to the traditional GAN method in underwater image SR.

scholarly journals Deblending galaxy superpositions with branched generative adversarial networks

2019 ◽  
Vol 485 (2) ◽  
pp. 2617-2627 ◽  
Author(s):  
David M Reiman ◽  
Brett E Göhre
Keyword(s):  
Signal To Noise Ratio ◽  
Ground Truth ◽  
Structural Similarity ◽  
Generative Models ◽  
Generative Adversarial Networks ◽  
Wide Field ◽  
Generative Adversarial Network ◽  
Galaxy Surveys ◽  
Adversarial Network ◽  
Natural Choice

Abstract Near-future large galaxy surveys will encounter blended galaxy images at a fraction of up to 50 per cent in the densest regions of the Universe. Current deblending techniques may segment the foreground galaxy while leaving missing pixel intensities in the background galaxy flux. The problem is compounded by the diffuse nature of galaxies in their outer regions, making segmentation significantly more difficult than in traditional object segmentation applications. We propose a novel branched generative adversarial network to deblend overlapping galaxies, where the two branches produce images of the two deblended galaxies. We show that generative models are a powerful engine for deblending given their innate ability to infill missing pixel values occluded by the superposition. We maintain high peak signal-to-noise ratio and structural similarity scores with respect to ground truth images upon deblending. Our model also predicts near-instantaneously, making it a natural choice for the immense quantities of data soon to be created by large surveys such as Large Synoptic Survey Telescope, Euclid, and Wide-Field Infrared Survey Telescope.

Sign in / Sign up

Export Citation Format

Share Document