Reference-guided structure-aware deep sketch colorization for cartoons

AbstractDigital cartoon production requires extensive manual labor to colorize sketches with visually pleasant color composition and color shading. During colorization, the artist usually takes an existing cartoon image as color guidance, particularly when colorizing related characters or an animation sequence. Reference-guided colorization is more intuitive than colorization with other hints, such as color points or scribbles, or text-based hints. Unfortunately, reference-guided colorization is challenging since the style of the colorized image should match the style of the reference image in terms of both global color composition and local color shading. In this paper, we propose a novel learning-based framework which colorizes a sketch based on a color style feature extracted from a reference color image. Our framework contains a color style extractor to extract the color feature from a color image, a colorization network to generate multi-scale output images by combining a sketch and a color feature, and a multi-scale discriminator to improve the reality of the output image. Extensive qualitative and quantitative evaluations show that our method outperforms existing methods, providing both superior visual quality and style reference consistency in the task of reference-based colorization.

Download Full-text

On Verification of Blur and Sharpness Metrics for No-reference Image Visual Quality Assessment

2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP) ◽

10.1109/mmsp48831.2020.9287110 ◽

2020 ◽

Author(s):

Sheyda Ghanbaralizadeh Bahnemiri ◽

Mykola Ponomarenko ◽

Karen Egiazarian

Keyword(s):

Quality Assessment ◽

Visual Quality ◽

Reference Image ◽

Visual Quality Assessment ◽

Sharpness Metrics

Download Full-text

A Multi-Scale Feature Extraction-Based Normalized Attention Neural Network for Image Denoising

Electronics ◽

10.3390/electronics10030319 ◽

2021 ◽

Vol 10 (3) ◽

pp. 319

Author(s):

Yi Wang ◽

Xiao Song ◽

Guanghong Gong ◽

Ni Li

Keyword(s):

Neural Network ◽

Feature Extraction ◽

Image Denoising ◽

Color Image ◽

Rapid Development ◽

Similarity Index ◽

Structural Similarity ◽

Convolutional Network ◽

Scale Feature ◽

Multi Scale

Due to the rapid development of deep learning and artificial intelligence techniques, denoising via neural networks has drawn great attention due to their flexibility and excellent performances. However, for most convolutional network denoising methods, the convolution kernel is only one layer deep, and features of distinct scales are neglected. Moreover, in the convolution operation, all channels are treated equally; the relationships of channels are not considered. In this paper, we propose a multi-scale feature extraction-based normalized attention neural network (MFENANN) for image denoising. In MFENANN, we define a multi-scale feature extraction block to extract and combine features at distinct scales of the noisy image. In addition, we propose a normalized attention network (NAN) to learn the relationships between channels, which smooths the optimization landscape and speeds up the convergence process for training an attention model. Moreover, we introduce the NAN to convolutional network denoising, in which each channel gets gain; channels can play different roles in the subsequent convolution. To testify the effectiveness of the proposed MFENANN, we used both grayscale and color image sets whose noise levels ranged from 0 to 75 to do the experiments. The experimental results show that compared with some state-of-the-art denoising methods, the restored images of MFENANN have larger peak signal-to-noise ratios (PSNR) and structural similarity index measure (SSIM) values and get better overall appearance.

Download Full-text

Restoration method of sootiness mural images based on dark channel prior and Retinex by bilateral filter

Heritage Science ◽

10.1186/s40494-021-00504-5 ◽

2021 ◽

Vol 9 (1) ◽

Author(s):

Ning Cao ◽

Shuqiang Lyu ◽

Miaole Hou ◽

Wanfu Wang ◽

Zhenhua Gao ◽

...

Keyword(s):

Near Infrared ◽

Environmental Changes ◽

Color Image ◽

Visual Quality ◽

Bilateral Filter ◽

Dark Channel Prior ◽

Infrared Band ◽

Dark Channel ◽

Restoration Method ◽

Pseudo Color

AbstractEnvironmental changes and human activities can cause serious degradation of murals, where sootiness is one of the most common problems of ancient Chinese indoor murals. In order to improve the visual quality of the murals, a restoration method is proposed for sootiness murals based on dark channel prior and Retinex by bilateral filter using hyperspectral imaging technology. First, radiometric correction and denoising through band clipping and minimum noise fraction rotation forward and inverse transform were applied to the hyperspectral data of the sootiness mural to produce its denoised reflectance image. Second, a near-infrared band was selected from the reflectance image and combined with the green and blue visible bands to produce a pseudo color image for the subsequent sootiness removal processing. The near-infrared band is selected because it is better penetrating the sootiness layer to a certain extent comparing to other bands. Third, the sootiness covered on the pseudo color image was preliminarily removed by using the method of dark channel prior and by adjusting the brightness of the image. Finally, the Retinex by bilateral filter was performed on the image to get the final restored image, where the sootiness was removed. The results show that the images restored by the proposed method are superior in variance, average gradient, information entropy and gray scale contrast comparing to the results from the traditional methods of homomorphic filtering and Gaussian stretching. The results also show the highest score in comprehensive evaluation of edges, hue and structure; thus, the method proposed can support more potential studies or sootiness removal in real mural paintings with more detailed information. The method proposed shows strong evidence that it can effectively reduce the influence of sootiness on the moral images with more details that can reveal the original appearance of the mural and improve its visual quality.

Download Full-text

Combined full-reference image visual quality metrics

Electronic Imaging ◽

10.2352/issn.2470-1173.2016.15.ipas-180 ◽

2016 ◽

Vol 2016 (15) ◽

pp. 1-10 ◽

Cited By ~ 3

Author(s):

Oleg I Ieremeiev ◽

Vladimir V Lukin ◽

Nikolay N Ponomarenko ◽

Karen O Egiazarian ◽

Jaakko Astola

Keyword(s):

Visual Quality ◽

Quality Metrics ◽

Reference Image ◽

Full Reference

Download Full-text

A Multi-Model Stereo Similarity Function Based on Monogenic Signal Analysis in Poisson Scale Space

Mathematical Problems in Engineering ◽

10.1155/2011/202653 ◽

2011 ◽

Vol 2011 ◽

pp. 1-14 ◽

Cited By ~ 1

Author(s):

Jinjun Li ◽

Hong Zhao ◽

Chengying Shi ◽

Xiang Zhou

Keyword(s):

Color Image ◽

Image Features ◽

Image Feature ◽

Similarity Function ◽

Local Orientation ◽

Stereo Images ◽

Interest Points ◽

Monogenic Signal ◽

Local Color ◽

Instantaneous Phase

A stereo similarity function based on local multi-model monogenic image feature descriptors (LMFD) is proposed to match interest points and estimate disparity map for stereo images. Local multi-model monogenic image features include local orientation and instantaneous phase of the gray monogenic signal, local color phase of the color monogenic signal, and local mean colors in the multiscale color monogenic signal framework. The gray monogenic signal, which is the extension of analytic signal to gray level image using Dirac operator and Laplace equation, consists of local amplitude, local orientation, and instantaneous phase of 2D image signal. The color monogenic signal is the extension of monogenic signal to color image based on Clifford algebras. The local color phase can be estimated by computing geometric product between the color monogenic signal and a unit reference vector in RGB color space. Experiment results on the synthetic and natural stereo images show the performance of the proposed approach.

Download Full-text

Qualitative and quantitative evaluations of mathematical models for animal learning and behaviors

Japanese Journal of Animal Psychology ◽

10.2502/janip.66.2.4 ◽

2016 ◽

Vol 66 (2) ◽

pp. 109-119

Author(s):

KENTARO KATAHIRA

Keyword(s):

Mathematical Models ◽

Animal Learning ◽

Qualitative And Quantitative ◽

And Behaviors ◽

Quantitative Evaluations

Download Full-text

QUATERNION BASED COLOR IMAGE QUALITY ASSESSMENT INDEX

International Journal of Image and Graphics ◽

10.1142/s0219467811004111 ◽

2011 ◽

Vol 11 (02) ◽

pp. 195-206 ◽

Cited By ~ 1

Author(s):

YUQING WANG ◽

MING ZHU ◽

HAOCHEN PANG ◽

YONG WANG

Keyword(s):

Image Quality Assessment ◽

Color Image ◽

Structural Similarity ◽

Singular Value ◽

Reference Image ◽

Quaternion Matrix ◽

Color Information ◽

Assessment Index ◽

Local Variance ◽

Visual Characteristics

A quaternion model for describing color image is proposed in order to evaluate its quality. Local variance distribution of luminance layer is calculated. Color information is taken into account by using quaternion matrix. The description method is a combination of luminance layer and color information. The angle between the singular value feature vectors of the quaternion matrices corresponding to the reference image and the distorted image is used to measure the structural similarity of the two color images. When the reference image and distorted images are of unequal size it can also assess their quality. Results from experiments show that the proposed method is better consistent with the human visual characteristics than MSE, PSNR and MSSIM. The resized distorted images can also be assessed rationally by this method.

Download Full-text

Integrated Multi-Scalar Approach for 3D Cultural Heritage Acquisitions

Geospatial Intelligence ◽

10.4018/978-1-5225-8054-6.ch021 ◽

2019 ◽

pp. 443-468

Author(s):

Michele Russo ◽

Anna Maria Manferdini

Keyword(s):

Cultural Heritage ◽

Structure From Motion ◽

Gold Standard ◽

3D Models ◽

Qualitative And Quantitative ◽

Digital Representations ◽

Architectural Survey ◽

Quantitative Evaluations

This contribution presents the results of investigations on the reliability of techniques based on the Structure from Motion approach used for 3D digitizations of build heritage. In particular, we tested the performances of different SfM technologies within an architectural survey context and we developed a procedure with the purpose of easing the work of surveyors called to restore digital representations of artifacts at different scales of complexity. The restored 3D models were compared among each other and with a gold standard acquisition. These analysis led to qualitative and quantitative evaluations and to considerations on times and skills required by all tested technologies. In this work strengths and weaknesses are highlighted and the integration of different technologies is presented, as it represents the best solution in many and recurrent multi-scalar contexts.

Download Full-text