Total variation (TV) regularization has received much attention in image restoration applications because of its advantages in denoising and preserving details. A common approach to address TV-based image restoration is to design a specific algorithm for solving typical cost function, which consists of conventional ℓ2 fidelity term and TV regularization. In this work, a novel objective function and an efficient algorithm are proposed. Firstly, a pseudoinverse transform-based fidelity term is imposed on TV regularization, and a closely-related optimization problem is established. Then, the split Bregman framework is used to decouple the complex inverse problem into subproblems to reduce computational complexity. Finally, numerical experiments show that the proposed method can obtain satisfactory restoration results with fewer iterations. Combined with the restoration effect and efficiency, this method is superior to the competitive algorithm. Significantly, the proposed method has the advantage of a simple solving structure, which can be easily extended to other image processing applications.