scholarly journals Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Sensors ◽  
2021 ◽  
Vol 21 (15) ◽  
pp. 5163
Author(s):  
Yun-Hsuan Su ◽  
Wenfan Jiang ◽  
Digesh Chitrakar ◽  
Kevin Huang ◽  
Haonan Peng ◽  
...  
Keyword(s):  
Training Image ◽  
Synthetic Image ◽  
Generative Adversarial Network ◽  
Training Tool ◽  
Adversarial Network ◽  
Screening Endoscopy ◽  
Local Background ◽  
Synthetic Image Generation ◽  
Loss Component ◽  
The University

Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.

scholarly journals Synthesizing Depth Hand Images with GANs and Style Transfer for Hand Pose Estimation

Sensors ◽  
2019 ◽  
Vol 19 (13) ◽  
pp. 2919 ◽  
Author(s):  
Wangyong He ◽  
Zhongzhao Xie ◽  
Yongbo Li ◽  
Xinmei Wang ◽  
Wendi Cai
Keyword(s):  
Pose Estimation ◽  
Ground Truth ◽  
Training Image ◽  
Training Data ◽  
Generative Adversarial Network ◽  
Style Transfer ◽  
Visual Appearance ◽  
Hand Pose Estimation ◽  
Adversarial Network ◽  
Hand Pose

Hand pose estimation is a critical technology of computer vision and human-computer interaction. Deep-learning methods require a considerable amount of tagged data. Accordingly, numerous labeled training data are required. This paper aims to generate depth hand images. Given a ground-truth 3D hand pose, the developed method can generate depth hand images. To be specific, a ground truth can be 3D hand poses with the hand structure contained, while the synthesized image has an identical size to that of the training image and a similar visual appearance to the training set. The developed method, inspired by the progress in the generative adversarial network (GAN) and image-style transfer, helps model the latent statistical relationship between the ground-truth hand pose and the corresponding depth hand image. The images synthesized using the developed method are demonstrated to be feasible for enhancing performance. On public hand pose datasets (NYU, MSRA, ICVL), comprehensive experiments prove that the developed method outperforms the existing works.

scholarly journals Detecting and Measuring Defects in Wafer Die Using GAN and YOLOv3

2020 ◽  
Vol 10 (23) ◽  
pp. 8725
Author(s):  
Ssu-Han Chen ◽  
Chih-Hsiang Kang ◽  
Der-Baau Perng
Keyword(s):  
Training Image ◽  
Decimal Place ◽  
Average Precision ◽  
Generative Adversarial Network ◽  
The Real ◽  
Adversarial Network ◽  
Image Set ◽  
Bounding Boxes ◽  
Realistic Images ◽  
Quality Sorting

This research used deep learning methods to develop a set of algorithms to detect die particle defects. Generative adversarial network (GAN) generated natural and realistic images, which improved the ability of you only look once version 3 (YOLOv3) to detect die defects. Then defects were measured based on the bounding boxes predicted by YOLOv3, which potentially provided the criteria for die quality sorting. The pseudo defective images generated by GAN from the real defective images were used as the training image set. The results obtained after training with the combination of the real and pseudo defective images were 7.33% higher in testing average precision (AP) and more accurate by one decimal place in testing coordinate error than after training with the real images alone. The GAN can enhance the diversity of defects, which improves the versatility of YOLOv3 somewhat. In summary, the method of combining GAN and YOLOv3 employed in this study creates a feature-free algorithm that does not require a massive collection of defective samples and does not require additional annotation of pseudo defects. The proposed method is feasible and advantageous for cases that deal with various kinds of die patterns.

scholarly journals Synthetic image data augmentation for fibre layup inspection processes: Techniques to enhance the data set

2021 ◽  
Author(s):  
Sebastian Meister ◽  
Nantwin Möller ◽  
Jan Stüve ◽  
Roger M. Groves
Keyword(s):  
Deep Learning ◽  
Test Method ◽  
Process Efficiency ◽  
Synthetic Image ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Manufacturing Defects ◽  
Fiber Placement ◽  
Adversarial Network ◽  
Geometrical Transformation

AbstractIn the aerospace industry, the Automated Fiber Placement process is an established method for producing composite parts. Nowadays the required visual inspection, subsequent to this process, typically takes up to 50% of the total manufacturing time and the inspection quality strongly depends on the inspector. A Deep Learning based classification of manufacturing defects is a possibility to improve the process efficiency and accuracy. However, these techniques require several hundreds or thousands of training data samples. Acquiring this huge amount of data is difficult and time consuming in a real world manufacturing process. Thus, an approach for augmenting a smaller number of defect images for the training of a neural network classifier is presented. Five traditional methods and eight deep learning approaches are theoretically assessed according to the literature. The selected conditional Deep Convolutional Generative Adversarial Network and Geometrical Transformation techniques are investigated in detail, with regard to the diversity and realism of the synthetic images. Between 22 and 166 laser line scan sensor images per defect class from six common fiber placement inspection cases are utilised for tests. The GAN-Train GAN-Test method was applied for the validation. The studies demonstrated that a conditional Deep Convolutional Generative Adversarial Network combined with a previous Geometrical Transformation is well suited to generate a large realistic data set from less than 50 actual input images. The presented network architecture and the associated training weights can serve as a basis for applying the demonstrated approach to other fibre layup inspection images.

scholarly journals Pix2pix Conditional Generative Adversarial Network with MLP Loss Function for Cloud Removal in a Cropland Time Series

2021 ◽  
Vol 14 (1) ◽  
pp. 144
Author(s):  
Luiz E. Christovam ◽  
Milton H. Shimabukuro ◽  
Maria de Lourdes B. T. Galo ◽  
Eija Honkavaara
Keyword(s):  
Time Series ◽  
Loss Function ◽  
Visual Analysis ◽  
Spectral Response ◽  
Semantic Segmentation ◽  
Generative Models ◽  
Synthetic Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Crop Type

Clouds are one of the major limitations to crop monitoring using optical satellite images. Despite all efforts to provide decision-makers with high-quality agricultural statistics, there is still a lack of techniques to optimally process satellite image time series in the presence of clouds. In this regard, in this article it was proposed to add a Multi-Layer Perceptron loss function to the pix2pix conditional Generative Adversarial Network (cGAN) objective function. The aim was to enforce the generative model to learn how to deliver synthetic pixels whose values were proxies for the spectral response improving further crop type mapping. Furthermore, it was evaluated the generalization capacity of the generative models in producing pixels with plausible values for images not used in the training. To assess the performance of the proposed approach it was compared real images with synthetic images generated with the proposed approach as well as with the original pix2pix cGAN. The comparative analysis was performed through visual analysis, pixel values analysis, semantic segmentation and similarity metrics. In general, the proposed approach provided slightly better synthetic pixels than the original pix2pix cGAN, removing more noise than the original pix2pix algorithm as well as providing better crop type semantic segmentation; the semantic segmentation of the synthetic image generated with the proposed approach achieved an F1-score of 44.2%, while the real image achieved 44.7%. Regarding the generalization, the models trained utilizing different regions of the same image provided better pixels than models trained using other images in the time series. Besides this, the experiments also showed that the models trained using a pair of images selected every three months along the time series also provided acceptable results on images that do not have cloud-free areas.

scholarly journals Synthetic image augmentation with generative adversarial network for enhanced performance in protein classification

2020 ◽  
Vol 10 (3) ◽  
pp. 443-452
Author(s):  
Rohit Verma ◽  
Raj Mehrotra ◽  
Chinmay Rane ◽  
Ritu Tiwari ◽  
Arun Kumar Agariya
Keyword(s):  
Protein Classification ◽  
Synthetic Image ◽  
Generative Adversarial Network ◽  
Adversarial Network

scholarly journals Image Anonymization using Deep Convolutional Generative Adversarial Network

2021 ◽  
Vol 2089 (1) ◽  
pp. 012012
Author(s):  
K Nitalaksheswara Rao ◽  
P Jayasree ◽  
Ch.V.Murali Krishna ◽  
K Sai Prasanth ◽  
Ch Satyananda Reddy
Keyword(s):  
Deep Learning ◽  
Data Privacy ◽  
Input Image ◽  
Space Representation ◽  
Synthetic Image ◽  
Original Image ◽  
Generative Adversarial Network ◽  
Model Inversion ◽  
Adversarial Network ◽  
Recent Developments

Abstract Advancement in deep learning requires significantly huge amount of data for training purpose, where protection of individual data plays a key role in data privacy and publication. Recent developments in deep learning demonstarte a huge challenge for traditionally used approch for image anonymization, such as model inversion attack, where adversary repeatedly query the model, inorder to reconstrut the original image from the anonymized image. In order to apply more protection on image anonymization, an approach is presented here to convert the input (raw) image into a new synthetic image by applying optimized noise to the latent space representation (LSR) of the original image. The synthetic image is anonymized by adding well designed noise calculated over the gradient during the learning process, where the resultant image is both realistic and immune to model inversion attack. More presicely, we extend the approach proposed by T. Kim and J. Yang, 2019 by using Deep Convolutional Generative Adversarial Network (DCGAN) in order to make the approach more efficient. Our aim is to improve the efficiency of the model by changing the loss function to achieve optimal privacy in less time and computation. Finally, the proposed approach is demonstrated using a benchmark dataset. The experimental study presents that the proposed method can efficiently convert the input image into another synthetic image which is of high quality as well as immune to model inversion attack.

scholarly journals Optimization of Cardiac Magnetic Resonance Synthetic Image Based on Simulated Generative Adversarial Network

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Ying Fu ◽  
MinXue Gong ◽  
Guang Yang ◽  
JinRong Hu ◽  
Hong Wei ◽  
...  
Keyword(s):  
Cardiac Magnetic Resonance ◽  
Magnetic Resonance ◽  
Data Augmentation ◽  
Small Sample ◽  
Training Data ◽  
Synthetic Image ◽  
Generative Adversarial Network ◽  
Data Set ◽  
Adversarial Network ◽  
Convolution Model

The generative adversarial network (GAN) has advantage to fit data distribution, so it can achieve data augmentation by fitting the real distribution and synthesizing additional training data. In this way, the deep convolution model can also be well trained in the case of using a small sample medical image data set. However, some certain gaps still exist between synthetic images and real images. In order to further narrow those gaps, this paper proposed a method that applies SimGAN on cardiac magnetic resonance synthetic image optimization task. Meanwhile, the improved residual structure is used to deepen the network structure to improve the performance of the optimizer. Lastly, the experiments will show the good result of our data augmentation method based on GAN.

scholarly journals Unpaired medical image colorization using generative adversarial network

2021 ◽  
Author(s):  
Yihuai Liang ◽  
Dongho Lee ◽  
Yan Li ◽  
Byeong-Seok Shin
Keyword(s):  
Loss Function ◽  
Medical Image ◽  
Color Image ◽  
Medical Images ◽  
Diagnostic Errors ◽  
Training Image ◽  
Input Image ◽  
Generative Adversarial Network ◽  
Adversarial Network

AbstractWe consider medical image transformation problems where a grayscale image is transformed into a color image. The colorized medical image should have the same features as the input image because extra synthesized features can increase the possibility of diagnostic errors. In this paper, to secure colorized medical images and improve the quality of synthesized images, as well as to leverage unpaired training image data, a colorization network is proposed based on the cycle generative adversarial network (CycleGAN) model, combining a perceptual loss function and a total variation (TV) loss function. Visual comparisons and experimental indicators from the NRMSE, PSNR, and SSIM metrics are used to evaluate the performance of the proposed method. The experimental results show that GAN-based style conversion can be applied to colorization of medical images. As well, the introduction of perceptual loss and TV loss can improve the quality of images produced as a result of colorization better than the result generated by only using the CycleGAN model.

Sign in / Sign up

Export Citation Format

Share Document