scholarly journals Newton-Type Optimal Thresholding Algorithms for Sparse Optimization Problems

Author(s):  
Nan Meng ◽  
Yun-Bin Zhao

AbstractSparse signals can be possibly reconstructed by an algorithm which merges a traditional nonlinear optimization method and a certain thresholding technique. Different from existing thresholding methods, a novel thresholding technique referred to as the optimal k-thresholding was recently proposed by Zhao (SIAM J Optim 30(1):31–55, 2020). This technique simultaneously performs the minimization of an error metric for the problem and thresholding of the iterates generated by the classic gradient method. In this paper, we propose the so-called Newton-type optimal k-thresholding (NTOT) algorithm which is motivated by the appreciable performance of both Newton-type methods and the optimal k-thresholding technique for signal recovery. The guaranteed performance (including convergence) of the proposed algorithms is shown in terms of suitable choices of the algorithmic parameters and the restricted isometry property (RIP) of the sensing matrix which has been widely used in the analysis of compressive sensing algorithms. The simulation results based on synthetic signals indicate that the proposed algorithms are stable and efficient for signal recovery.

2012 ◽  
Vol 2012 ◽  
pp. 1-22 ◽  
Author(s):  
Heping Song ◽  
Guoli Wang

The emerging theory of compressive sensing (CS) provides a new sparse signal processing paradigm for reconstructing sparse signals from the undersampled linear measurements. Recently, numerous algorithms have been developed to solve convex optimization problems for CS sparse signal recovery. However, in some certain circumstances, greedy algorithms exhibit superior performance than convex methods. This paper is a followup to the recent paper of Wang and Yin (2010), who refine BP reconstructions via iterative support detection (ISD). The heuristic idea of ISD was applied to greedy algorithms. We developed two approaches for accelerating the ECME iteration. The described algorithms, named ECME thresholding pursuits (EMTP), introduced two greedy strategies that each iteration detects a support setIby thresholding the result of the ECME iteration and estimates the reconstructed signal by solving a truncated least-squares problem on the support setI. Two effective support detection strategies are devised for the sparse signals with components having a fast decaying distribution of nonzero components. The experimental studies are presented to demonstrate that EMTP offers an appealing alternative to state-of-the-art algorithms for sparse signal recovery.


2011 ◽  
Vol 341-342 ◽  
pp. 629-633
Author(s):  
Madhuparna Chakraborty ◽  
Alaka Barik ◽  
Ravinder Nath ◽  
Victor Dutta

In this paper, we study a method for sparse signal recovery with the help of iteratively reweighted least square approach, which in many situations outperforms other reconstruction method mentioned in literature in a way that comparatively fewer measurements are needed for exact recovery. The algorithm given involves solving a sequence of weighted minimization for nonconvex problems where the weights for the next iteration are determined from the value of current solution. We present a number of experiments demonstrating the performance of the algorithm. The performance of the algorithm is studied via computer simulation for different number of measurements, and degree of sparsity. Also the simulation results show that improvement is achieved by incorporating regularization strategy.


2020 ◽  
Author(s):  
Xiaobin LIU ◽  
Feng Zhao ◽  
Xiaofeng Ai ◽  
Qihua Wu

Abstract The interrupted transmitting and receiving (ITR) can be used in anechoic chamber to solve the coupling between the transmitted and reflected signals. When the ITR periods are random, the fake peaks in high resolution range profile (HRRP) of ITR echo can be reduced. Then, by utilizing the piecewise sparse property of ITR echo, the HRRP is reconstructed based on compressive sensing (CS). However, the periods of ITR determine the restricted isometry property (RIP) condition and the HRRP reconstruction performance. In order to improve the HRRP reconstruction performance, the ITR periods sequence optimization method based on genetic algorithm (GA) is proposed in this paper. The correlation coefficient of the sensing matrix columns is minimized after optimization so that the RIP of sensing matrix can be well satisfied . Simulation results illustrate that the optimization method converges fast and the HRRP reconstruction performance is improved with the optimized ITR periods .


2021 ◽  
Author(s):  
Yixuan Wang ◽  
Faruk Alpak ◽  
Guohua Gao ◽  
Chaohui Chen ◽  
Jeroen Vink ◽  
...  

Abstract Although it is possible to apply traditional optimization algorithms to determine the Pareto front of a multi-objective optimization problem, the computational cost is extremely high, when the objective function evaluation requires solving a complex reservoir simulation problem and optimization cannot benefit from adjoint-based gradients. This paper proposes a novel workflow to solve bi-objective optimization problems using the distributed quasi-Newton (DQN) method, which is a well-parallelized and derivative-free optimization (DFO) method. Numerical tests confirm that the DQN method performs efficiently and robustly. The efficiency of the DQN optimizer stems from a distributed computing mechanism which effectively shares the available information discovered in prior iterations. Rather than performing multiple quasi-Newton optimization tasks in isolation, simulation results are shared among distinct DQN optimization tasks or threads. In this paper, the DQN method is applied to the optimization of a weighted average of two objectives, using different weighting factors for different optimization threads. In each iteration, the DQN optimizer generates an ensemble of search points (or simulation cases) in parallel and a set of non-dominated points is updated accordingly. Different DQN optimization threads, which use the same set of simulation results but different weighting factors in their objective functions, converge to different optima of the weighted average objective function. The non-dominated points found in the last iteration form a set of Pareto optimal solutions. Robustness as well as efficiency of the DQN optimizer originates from reliance on a large, shared set of intermediate search points. On the one hand, this set of searching points is (much) smaller than the combined sets needed if all optimizations with different weighting factors would be executed separately; on the other hand, the size of this set produces a high fault tolerance. Even if some simulations fail at a given iteration, DQN’s distributed-parallel information-sharing protocol is designed and implemented such that the optimization process can still proceed to the next iteration. The proposed DQN optimization method is first validated on synthetic examples with analytical objective functions. Then, it is tested on well location optimization problems, by maximizing the oil production and minimizing the water production. Furthermore, the proposed method is benchmarked against a bi-objective implementation of the MADS (Mesh Adaptive Direct Search) method, and the numerical results reinforce the auspicious computational attributes of DQN observed for the test problems. To the best of our knowledge, this is the first time that a well-parallelized and derivative-free DQN optimization method has been developed and tested on bi-objective optimization problems. The methodology proposed can help improve efficiency and robustness in solving complicated bi-objective optimization problems by taking advantage of model-based search optimization algorithms with an effective information-sharing mechanism.


2017 ◽  
Vol 7 (1) ◽  
pp. 83-104 ◽  
Author(s):  
R Baraniuk ◽  
S Foucart ◽  
D Needell ◽  
Y Plan ◽  
M Wootters

Abstract One-bit compressive sensing has extended the scope of sparse recovery by showing that sparse signals can be accurately reconstructed even when their linear measurements are subject to the extreme quantization scenario of binary samples—only the sign of each linear measurement is maintained. Existing results in one-bit compressive sensing rely on the assumption that the signals of interest are sparse in some fixed orthonormal basis. However, in most practical applications, signals are sparse with respect to an overcomplete dictionary, rather than a basis. There has already been a surge of activity to obtain recovery guarantees under such a generalized sparsity model in the classical compressive sensing setting. Here, we extend the one-bit framework to this important model, providing a unified theory of one-bit compressive sensing under dictionary sparsity. Specifically, we analyze several different algorithms—based on convex programming and on hard thresholding—and show that, under natural assumptions on the sensing matrix (satisfied by Gaussian matrices), these algorithms can efficiently recover analysis–dictionary-sparse signals in the one-bit model.


2020 ◽  
Author(s):  
Tongtong Xu ◽  
Zheng Xiang

Abstract In this work, concurrent modified constant modulus algorithm and decision-directed scheme with the Barzilai-Borwein method is proposed for blind equalization of wireless communications systems. The Barzilai-Borwein method, the two-step gradient method, is usually used to solve multidimensional unconstrained optimization problems. The proposed algorithm concurrently operates a modified constant modulus algorithm equalizer and a decision directed equalizer, it then adaptively adjusts the step size of the decision directed equalizer using Barzilai-Borwein method. Theoretical analysis is provided to illustrate that the proposed algorithm has a faster convergence speed, and better equalization performance than the original one. Simulation results support the new proposed technique.


2020 ◽  
Vol 37 (5) ◽  
pp. 723-732
Author(s):  
Shengjie Zhao ◽  
Jianchen Zhu ◽  
Di Wu

Compressive sensing (CS) is a novel paradigm to recover a sparse signal in compressed domain. In some overcomplete dictionaries, most practical signals are sparse rather than orthonormal. Signal space greedy method can derive the optimal or near-optimal projections, making it possible to identify a few most relevant dictionary atoms of an arbitrary signal. More practically, such projections can be processed by standard CS recovery algorithms. This paper proposes a signal space subspace pursuit (SSSP) method to compute spare signal representations with overcomplete dictionaries, whenever the sensing matrix satisfies the restricted isometry property adapted to dictionary (D-RIP). Specifically, theoretical guarantees were provided to recover the signals from their measurements with overwhelming probability, as long as the sensing matrix satisfies the D-RIP. In addition, a thorough analysis was performed to minimize the number of measurements required for such guarantees. Simulation results demonstrate the validity of our hypothetical theory, as well as the superiority of the proposed approach.


2013 ◽  
Vol 433-435 ◽  
pp. 617-620
Author(s):  
Fu Hua Fan

In order to obtain accurate Compressive Sensing UWB (CS-UWB) channel estimation coefficients, a method of constructing adaptive sensing matrix is proposed via incoherence criterion, making the greedy reconstruction algorithms obtain optimal atoms from redundancy dictionary. The modified OMP algorithm without a prior of sparsity and SNR information is also presented for convenient using. Simulation results show that much improvement on UWB channel estimation is obtained based on the adaptive sensing matrix using the proposed algorithm.


Sign in / Sign up

Export Citation Format

Share Document