Data Denoising By Noise Invalidation

10.32920/ryerson.14644197.v1 ◽

2021 ◽

Author(s):

Nima Nikvand

Keyword(s):

Additive Noise ◽

Variance Estimation ◽

Additive White Gaussian Noise ◽

Visual Quality ◽

Noise Variance ◽

Soft Thresholding ◽

New Methods ◽

Shrinkage Methods ◽

Similarities And Differences ◽

Non Gaussian

In this thesis, the problem of data denoising is studied, and two new denoising approaches are proposed. Using statistical properties of the additive noise, the methods provide adaptive data-dependent soft thresholding techniques to remove the additive noise. The proposed methods, Point-wise Noise Invlaidating Soft Thresholding (PNIST) and Accumulative Noise Invalidation Soft Thresholding (ANIST), are based on Noise Invalidation. The invalidation exploits basic properties of the additive noise in order to remove the noise effects as much as possible. There are similarities and differences between ANIST and PNIST. While PNIST performs better in the case of additive white Gaussian noise, ANIST can be used with both Gaussian and non Gaussian additive noise. As part of a data denoising technique, a new noise variance estimation is also proposed. The thresholds proposed by NIST approaches are comparable to the shrinkage methods, and our simulation results promise that the new methods can outperform the existing approaches in various applications. We also explore the area of image denoising as one of the main applications of data denoising and extend the proposed approaches to two dimensional applications. Simulations show that the proposed methods outperform common shrinkage methods and are comparable to the famous BayesShrink method in terms of Mean Square Error and visual quality.

Download Full-text

Data Denoising In Analog And Digital Domains

10.32920/ryerson.14648886.v1 ◽

2021 ◽

Author(s):

Amroabadi S. Hashemi

Keyword(s):

Mean Square Error ◽

Variance Estimation ◽

Hyperspectral Image ◽

Dead Zone ◽

Estimation Method ◽

Noise Removal ◽

Mean Square ◽

Noise Variance ◽

Worst Case ◽

Soft Thresholding

In this thesis, we develop various methods for the purpose of data denoising. We propose a method for Mean Square Error (MSE) estimation in Soft Thresholding. The MSE estimator is based on Minimum Noiseless Data Length (MNDL). Our simulation results show that this MSE estimate is a valuable comparison measure for different soft thresholding methods. Two denoising methods are proposed for analog domain: Mean Square Error EstiMation (MSEEM) which minimizes the worst case MSE estimate, and Noise Invalidation Denoising (NIDe) method which is based on the newly prosposed idea of noise signature. While MSEEM shown to be the optimum denoising method for non-sparse signals, NIDe approach outperforms the other well known denoising methods in presence of colored noise. In digital domain we address two interesting problems: 1) simultaneous denoising and quantization method, 2) denoising a digital signal in digital domain. For problem one, we propose a new method that generalizes the idea of dead zone estimation to a multi-level noise removal. An example of this method is shown for hyperspectral image denoising and compression. A digital domain denoising approach pioneers in answering the second problem with only one prior knowledge on the desired signal, that it is digital. The method provides the optimum reconstruction levels in the MSE sense. One of the critical steps of denoising process is the noise variance estimation. As a part of this thesis, we propose a novel noise variance estimation method for BayesShrink that outperforms conventional MAD-based noise variance estimation. Although BayesShrink is one of the most efficient denoising methods, no analytical analysis is available for it. Here, we study Bayes estimators for General Gaussian Distribued (GGD) data and provide the theoretical justification for BayesShrink. This study enables us to generalize the BayesShrink threshold to Generalized BayesShrink which outperforms the BayesShrink itself.

Download Full-text

Data Denoising In Analog And Digital Domains

10.32920/ryerson.14648886 ◽

2021 ◽

Author(s):

Amroabadi S. Hashemi

Keyword(s):

Mean Square Error ◽

Variance Estimation ◽

Hyperspectral Image ◽

Dead Zone ◽

Estimation Method ◽

Noise Removal ◽

Mean Square ◽

Noise Variance ◽

Worst Case ◽

Soft Thresholding

In this thesis, we develop various methods for the purpose of data denoising. We propose a method for Mean Square Error (MSE) estimation in Soft Thresholding. The MSE estimator is based on Minimum Noiseless Data Length (MNDL). Our simulation results show that this MSE estimate is a valuable comparison measure for different soft thresholding methods. Two denoising methods are proposed for analog domain: Mean Square Error EstiMation (MSEEM) which minimizes the worst case MSE estimate, and Noise Invalidation Denoising (NIDe) method which is based on the newly prosposed idea of noise signature. While MSEEM shown to be the optimum denoising method for non-sparse signals, NIDe approach outperforms the other well known denoising methods in presence of colored noise. In digital domain we address two interesting problems: 1) simultaneous denoising and quantization method, 2) denoising a digital signal in digital domain. For problem one, we propose a new method that generalizes the idea of dead zone estimation to a multi-level noise removal. An example of this method is shown for hyperspectral image denoising and compression. A digital domain denoising approach pioneers in answering the second problem with only one prior knowledge on the desired signal, that it is digital. The method provides the optimum reconstruction levels in the MSE sense. One of the critical steps of denoising process is the noise variance estimation. As a part of this thesis, we propose a novel noise variance estimation method for BayesShrink that outperforms conventional MAD-based noise variance estimation. Although BayesShrink is one of the most efficient denoising methods, no analytical analysis is available for it. Here, we study Bayes estimators for General Gaussian Distribued (GGD) data and provide the theoretical justification for BayesShrink. This study enables us to generalize the BayesShrink threshold to Generalized BayesShrink which outperforms the BayesShrink itself.

Download Full-text

Використання візуальних метрик для аналізу стиснення з втратами зашумлених зображень

Aerospace technic and technology ◽

10.32620/aktt.2021.6.09 ◽

2021 ◽

pp. 83-91

Author(s):

Богдан Віталійович Коваленко ◽

Володимир Васильович Лукін

Keyword(s):

Image Quality ◽

Image Compression ◽

Gaussian Noise ◽

Additive White Gaussian Noise ◽

Visual Image ◽

Visual Quality ◽

Low Complexity ◽

Quality Metrics ◽

Operating Point ◽

Noise Variance

The subject of the article is to analyze the effectiveness of lossy image compression using a BPG encoder using visual metrics as a quality criterion. The aim is to confirm the existence of an operating point for images of varying complexity for visual quality metrics. The objectives of the paper are the following: to analyze for a set of images of varying complexity, where images are distorted by additive white Gaussian noise with different variance values, build and analyze dependencies for visual image quality metrics, provide recommendations on the choice of parameters for compression in the vicinity of the operating point. The methods used are the following: methods of mathematical statistics; methods of digital image processing. The following results were obtained. Dependencies of visual quality metrics for images of various degrees of complexity affected by noise with variance equal to 64, 100, and 196. It can be seen from the constructed dependence that a working point is present for images of medium and low complexity for both the PSNR-HVS-M and MS-SSIM metrics. Recommendations are given for choosing a parameter for compression based on the obtained dependencies. Conclusions. Scientific novelty of the obtained results is the following: for a new compression method using Better Portable Graphics (BPG), research has been conducted and the existence of an operating point for visual quality metrics has been proven, previously such studies were conducted only for the PSNR metric.The test images were distorted by additive white Gaussian noise and then compressed using the methods implemented in the BPG encoder. The images were compressed with different values of the Q parameter, which made it possible to estimate the image compression quality at different values of compression ratio. The resulting data made it possible to visualize the dependence of the visual image quality metric on the Q parameter. Based on the obtained dependencies, it can be concluded that the operating point is present both for the PSNR-HVS-M metric and for the MS-SSIM for images of medium and low complexity, it is also worth noting that, especially clearly, the operating point is noticeable at large noise variance values. As a recommendation, a formula is presented for calculating the value of the compression control parameter (for the case with the BPG encoder, it is the Q parameter) for images distorted by noise with variance varying within a wide range, on the assumption that the noise variance is a priori known or estimated with high accuracy.

Download Full-text