Robust Denoising of Cryo-EM Images via β-GAN

Author(s):  
Hanlin Gu ◽  
Ilona Unarta ◽  
Xuhui Huang ◽  
Yuan Yao

Abstract The cryo-electron microscopy (Cryo-EM) becomes popular for macromolecular structure determination. However, the 2D images which Cryo-EM detects are of high noise and often mixed with multiple heterogeneous conformations and contamination, imposing a challenge for denoising. Traditional image denoising methods and simple Denoising Autoencoder can not remove Cryo-EM image noise well when the signal-noise-ratio (SNR) of images is meager and contamination distribution is complex. Thus it is desired to develop new effective denoising techniques to facilitate further research such as 3D reconstruction, 2D conformation classification, and so on. In this paper, we approach the robust denoising problem for Cryo-EM images by introducing a family of Generative Adversarial Networks (GAN), called β-GAN, which is able to achieve robust estimate of certain distributional parameters under Huber contamination model with statistical optimality. To address the challenge of robust denoising where the traditional image generative model might be contaminated by a small portion of unknown outliers, β-GANs are exploited to enhance the robustness of denoising Autoencoder. The method is evaluated by both a simulated dataset on the Thermus aquaticus RNA Polymerase (RNAP) and a real dataset on the Plasmodium falciparum 80S ribosome dataset (EMPIRE-10028), in terms of Mean Square Error (MSE), Peak Signal to Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM) and 3D Reconstruction as well. The results show that equipped with some designs of β-GANs and the robust ℓ1-Autoencoder, one can stabilize the training of GANs and achieve the state-of-the-art performance of robust denoising with low SNR data and against possible information contamination. Our proposed methodology thus provides an effective tool for robust denoising of Cryo-EM 2D images, which is helpful for 3D structure reconstruction.

2020 ◽  
Vol 25 (2) ◽  
pp. 86-97
Author(s):  
Sandy Suryo Prayogo ◽  
Tubagus Maulana Kusuma

DVB merupakan standar transmisi televisi digital yang paling banyak digunakan saat ini. Unsur terpenting dari suatu proses transmisi adalah kualitas gambar dari video yang diterima setelah melalui proses transimisi tersebut. Banyak faktor yang dapat mempengaruhi kualitas dari suatu gambar, salah satunya adalah struktur frame dari video. Pada tulisan ini dilakukan pengujian sensitifitas video MPEG-4 berdasarkan struktur frame pada transmisi DVB-T. Pengujian dilakukan menggunakan simulasi matlab dan simulink. Digunakan juga ffmpeg untuk menyediakan format dan pengaturan video akan disimulasikan. Variabel yang diubah dari video adalah bitrate dan juga group-of-pictures (GOP), sedangkan variabel yang diubah dari transmisi DVB-T adalah signal-to-noise-ratio (SNR) pada kanal AWGN di antara pengirim (Tx) dan penerima (Rx). Hasil yang diperoleh dari percobaan berupa kualitas rata-rata gambar pada video yang diukur menggunakan metode pengukuran structural-similarity-index (SSIM). Dilakukan juga pengukuran terhadap jumlah bit-error-rate BER pada bitstream DVB-T. Percobaan yang dilakukan dapat menunjukkan seberapa besar sensitifitas bitrate dan GOP dari video pada transmisi DVB-T dengan kesimpulan semakin besar bitrate maka akan semakin buruk nilai kualitas gambarnya, dan semakin kecil nilai GOP maka akan semakin baik nilai kualitasnya. Penilitian diharapkan dapat dikembangkan menggunakan deep learning untuk memperoleh frame struktur yang tepat di kondisi-kondisi tertentu dalam proses transmisi televisi digital.


Sensors ◽  
2021 ◽  
Vol 21 (16) ◽  
pp. 5540
Author(s):  
Nayeem Hasan ◽  
Md Saiful Islam ◽  
Wenyu Chen ◽  
Muhammad Ashad Kabir ◽  
Saad Al-Ahmadi

This paper proposes an encryption-based image watermarking scheme using a combination of second-level discrete wavelet transform (2DWT) and discrete cosine transform (DCT) with an auto extraction feature. The 2DWT has been selected based on the analysis of the trade-off between imperceptibility of the watermark and embedding capacity at various levels of decomposition. DCT operation is applied to the selected area to gather the image coefficients into a single vector using a zig-zig operation. We have utilized the same random bit sequence as the watermark and seed for the embedding zone coefficient. The quality of the reconstructed image was measured according to bit correction rate, peak signal-to-noise ratio (PSNR), and similarity index. Experimental results demonstrated that the proposed scheme is highly robust under different types of image-processing attacks. Several image attacks, e.g., JPEG compression, filtering, noise addition, cropping, sharpening, and bit-plane removal, were examined on watermarked images, and the results of our proposed method outstripped existing methods, especially in terms of the bit correction ratio (100%), which is a measure of bit restoration. The results were also highly satisfactory in terms of the quality of the reconstructed image, which demonstrated high imperceptibility in terms of peak signal-to-noise ratio (PSNR ≥ 40 dB) and structural similarity (SSIM ≥ 0.9) under different image attacks.


Author(s):  
Cuizhen Wang ◽  
Zhenxue Chen ◽  
Yan Wang ◽  
Zhifeng Wang

Three-dimensional reconstruction of teeth plays an important role in the operation of living dental implants. However, the tissue around teeth and the noise generated in the process of image acquisition bring a serious impact on the reconstruction results, which must be reduced or eliminated. Combined with the advantages of wavelet transform and bilateral filtering, this paper proposes an image denoising method based on the above methods. The method proposed in this paper not only removes the noise but also preserves the image edge details. The noise in high frequency subbands is denoised using a locally adaptive thresholding and the noise in low frequency subbands is filtered by the bilateral filtering. Peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) and 3D reconstruction using the iso-surface extraction method are used to evaluate the denoising effect. The experimental results show that the proposed method is better than the wavelet denoising and bilateral filtering, and the reconstruction results meet the requirements of clinical diagnosis.


2020 ◽  
Vol 14 ◽  
Author(s):  
Zhenmou Yuan ◽  
Mingfeng Jiang ◽  
Yaming Wang ◽  
Bo Wei ◽  
Yongming Li ◽  
...  

Research on undersampled magnetic resonance image (MRI) reconstruction can increase the speed of MRI imaging and reduce patient suffering. In this paper, an undersampled MRI reconstruction method based on Generative Adversarial Networks with the Self-Attention mechanism and the Relative Average discriminator (SARA-GAN) is proposed. In our SARA-GAN, the relative average discriminator theory is applied to make full use of the prior knowledge, in which half of the input data of the discriminator is true and half is fake. At the same time, a self-attention mechanism is incorporated into the high-layer of the generator to build long-range dependence of the image, which can overcome the problem of limited convolution kernel size. Besides, spectral normalization is employed to stabilize the training process. Compared with three widely used GAN-based MRI reconstruction methods, i.e., DAGAN, DAWGAN, and DAWGAN-GP, the proposed method can obtain a higher peak signal-to-noise ratio (PSNR) and structural similarity index measure(SSIM), and the details of the reconstructed image are more abundant and more realistic for further clinical scrutinization and diagnostic tasks.


2021 ◽  
Vol 9 (Suppl 3) ◽  
pp. A855-A856
Author(s):  
Abu Bakr Azam ◽  
Yu Qing Chang ◽  
Matthew Leong Tze Ker ◽  
Denise Goh ◽  
Jeffrey Chun Tatt Lim ◽  
...  

BackgroundExamining Hematoxylin & Eosin (H&E) images using brightfield microscopes is the gold standard of pathological diagnosis as it is an inexpensive method and provides basic information of tumors and other nuclei. Complementary to H&E-stained images, Immunohistochemical (IHC) images are crucial in identifying tumor subtypes and efficacy of treatment response. Other newer technologies such as Multiplex Immunofluorescence (mIF) in particular, identifies cells such as tumor infiltrating lymphocytes (TILs) which can be augmented via immunotherapy, an evolving form of cancer treatment. Immunotherapy helps in the manipulation of the host immune response and overcome limitations like the PD-1 (Programmed Cell Death-1) receptor induced restrictions on TIL production. If the same biopsy specimen is used for inspection, the higher order features in H&E images can be used to obtain information usually found in mIF images using Convolutional Neural Networks (CNNs), widely used in object detection and image segmentation tasks.MethodsAs shown in (figure 1), firstly, a novel optical flow-based image registration paradigm is prepared to co-register H&E and mIF image pairs, aided by adaptive color thresholding and automated color clustering. Secondly, generative adversarial networks (GANs) are adapted to predict TIL (CD3, CD45) regions. For this purpose, a unique dataset is ideated and used in which a given single channel mIF image, e.g., a CD3 channel mIF image is superimposed on the corresponding H&E image. Primarily, the Pix2Pix GAN model is used to predict CD3 and/or CD45 regions.ResultsThe intensity-based image registration workflow is fast and fully compatible with the given dataset, with an increase in evaluation metric scores after alignment (table 1). Furthermore, this study would be the first implementation of optical flow as the registration algorithm for pathological images. Next, the use of the special dataset not only reduces penalization during the training of the Pix2Pix model, but also helped in gaining repeatable results with high scores in metrics like structural similarity index measure and peak-signal to noise ratio, with minimal effects on location accuracy (table 2 and table 3).ConclusionsThis multi-modal pathological image transformation study could potentially reduce dependence on mIF and IHC images for TILs scoring, reducing the amount of tissue and cost needed for examination, as its information is derived directly from inexpensive H&E images automatically – ultimately develop into a pathologist-assisted tool for TILs scoring. This would be highly beneficial in facilities where resources are relatively limited.Ethics ApprovalThe Agency of Science, Technology and Research, Singapore, provided approval for the use of control tissue materials in this study IRB: 2020 112Abstract 818 Figure 1Proposed workflowAbstract 818 Table 1Image registration metricsAbstract 818 Table 2CD3 negative regions examplesAbstract 818 Table 3CD3 positive regions examples


2021 ◽  
Vol 38 (5) ◽  
pp. 1361-1368
Author(s):  
Fatih M. Senalp ◽  
Murat Ceylan

The thermal camera systems can be used in all kinds of applications that require the detection of heat change, but thermal imaging systems are highly costly systems. In recent years, developments in the field of deep learning have increased the success by obtaining quality results compared to traditional methods. In this paper, thermal images of neonates (healthy - unhealthy) obtained from a high-resolution thermal camera were used and these images were evaluated as high resolution (ground truth) images. Later, these thermal images were downscaled at 1/2, 1/4, 1/8 ratios, and three different datasets consisting of low-resolution images in different sizes were obtained. In this way, super-resolution applications have been carried out on the deep network model developed based on generative adversarial networks (GAN) by using three different datasets. The successful performance of the results was evaluated with PSNR (peak signal to noise ratio) and SSIM (structural similarity index measure). In addition, healthy - unhealthy classification application was carried out by means of a classifier network developed based on convolutional neural networks (CNN) to evaluate the super-resolution images obtained using different datasets. The obtained results show the importance of combining medical thermal imaging with super-resolution methods.


Thyroid ultrasonography is the most common and extremely useful, safe, and cost effective way to image the thyroid gland and its pathology. However, an inherent characteristic of Ultrasound (US) imaging is the presence of multiplicative speckle noise. Speckle noise reduces the ability of an observer to distinguish fine details, make diagnosis more difficult. It limits the effective implementation of image analysis steps such as edge detection, segmentation and classification. The main objective of this study is to compare the performance of various spatial and frequency domain filters so as to identify efficient and optimum filter for de-speckling Thyroid US images. The performance of these filters is evaluated using the image quality assessment parameters Signal to Noise Ratio (SNR), Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM), Mean Square Error (MSE) and Root Mean Square Error (RMSE) for different speckle variance. Experimental work revealed that kuan filter resulted in higher PSNR, SNR, SSIM and least MSE, RMSE values compared to other filters


2021 ◽  
Vol 12 ◽  
Author(s):  
Dercilio Junior Verly Lopes ◽  
Gustavo Fardin Monti ◽  
Greg W. Burgreen ◽  
Jordão Cabral Moulin ◽  
Gabrielly dos Santos Bobadilha ◽  
...  

Microscopic wood identification plays a critical role in many economically important areas in wood science. Historically, producing and curating relevant and representative microscopic cross-section images of wood species is limited to highly experienced and trained anatomists. This manuscript demonstrates the feasibility of generating synthetic microscopic cross-sections of hardwood species. We leveraged a publicly available dataset of 119 hardwood species to train a style-based generative adversarial network (GAN). The proposed GAN generated anatomically accurate cross-section images with remarkable fidelity to actual data. Quantitative metrics corroborated the capacity of the generative model in capturing complex wood structure by resulting in a Fréchet inception distance score of 17.38. Image diversity was calculated using the Structural Similarity Index Measure (SSIM). The SSIM results confirmed that the GAN approach can successfully synthesize diverse images. To confirm the usefulness and realism of the GAN generated images, eight professional wood anatomists in two experience levels participated in a visual Turing test and correctly identified fake and actual images at rates of 48.3 and 43.7%, respectively, with no statistical difference when compared to random guess. The generative model can synthesize realistic, diverse, and meaningful high-resolution microscope cross-section images that are virtually indistinguishable from real images. Furthermore, the framework presented may be suitable for improving current deep learning models, helping understand potential breeding between species, and may be used as an educational tool.


2021 ◽  
Vol 11 (8) ◽  
pp. 2153-2166
Author(s):  
Nurshafira Hazim Chan ◽  
Khairunnisa Hasikin ◽  
Nahrizul Adib Kadri ◽  
Mokhzaini Azizan ◽  
Muzammil B. Jusoh

Mammography has been known worldwide as the most common imaging modalities utilized for early detection of breast cancer. The mammographic images produced are in greyscale, however they often produced low contrast images, contain artefacts and noise, as well as non-uniform illumination. These limitations can be overcame in the pre-processing stage with the image enhancement process. Therefore, in this research we developed an optimized enhancement framework where the local contrast factor is manipulated to preserve details of the image. This method aims to improve the overall image visibility without altering histogram of the original image, which will affect the segmentation and classification processes. We performed dark background removal in the image histogram at early stage to increase the efficiency of new mean histogram calculation. Then, the histogram is separated into two partitions to allow histogram clipping process to be conducted individually for underexposed and overexposed areas. Consequently, the local contrast factor optimization is conducted to preserve the image details. The results from our proposed method are compared with other methods by the measurement of peak signal-to-noise ratio, structural similarity index, average contrast, and average entropy difference. The results portrayed that our proposed method yield better quality over the others with highest peak signal-to-noise ratio of 32.676. In addition, in terms of qualitative analysis, our proposed method depicted better lesion segmentation with smoother shape of the lesion.


Sign in / Sign up

Export Citation Format

Share Document