scholarly journals Demystifying Neural Style Transfer

2017 ◽  
Author(s):  
Yanghao Li ◽  
Naiyan Wang ◽  
Jiaying Liu ◽  
Xiaodi Hou
Keyword(s):  
Domain Adaptation ◽  
Second Order ◽  
Polynomial Kernel ◽  
Important Research ◽  
Feature Maps ◽  
Style Transfer ◽  
Maximum Mean Discrepancy ◽  
Research Fields ◽  
Gram Matrices ◽  
Order Polynomial

Neural Style Transfer has recently demonstrated very exciting results which catches eyes in both academia and industry. Despite the amazing results, the principle of neural style transfer, especially why the Gram matrices could represent style remains unclear. In this paper, we propose a novel interpretation of neural style transfer by treating it as a domain adaptation problem. Specifically, we theoretically show that matching the Gram matrices of feature maps is equivalent to minimize the Maximum Mean Discrepancy (MMD) with the second order polynomial kernel. Thus, we argue that the essence of neural style transfer is to match the feature distributions between the style images and the generated images. To further support our standpoint, we experiment with several other distribution alignment methods, and achieve appealing results. We believe this novel interpretation connects these two important research fields, and could enlighten future researches.

scholarly journals The Intra-Class and Inter-Class Relationships in Style Transfer

2018 ◽  
Vol 8 (9) ◽  
pp. 1681 ◽  
Author(s):  
Xin Cui ◽  
Meng Qi ◽  
Yi Niu ◽  
Bingxin Li
Keyword(s):  
Academic Research ◽  
Semantic Content ◽  
Careful Analysis ◽  
Perceptual Quality ◽  
Neural Activation ◽  
Feature Maps ◽  
Style Transfer ◽  
Gram Matrices ◽  
Lower Complexity

Neural style transfer, which has attracted great attention in both academic research and industrial engineering and demonstrated very exciting and remarkable results, is the technique of migrating the semantic content of one image to different artistic styles by using convolutional neural network (CNN). Recently, the Gram matrices used in the original and follow-up studies for style transfer were theoretically shown to be equivalent to minimizing a specific Maximum Mean Discrepancy (MMD). Since the Gram matrices are not a must for style transfer, how to design the proper process for aligning the neural activation between images to perform style transfer is an important problem. After careful analysis of some different algorithms for style loss construction, we discovered that some algorithms consider the relationships between different feature maps of a layer obtained from the CNN (inter-class relationships), while some do not (intra-class relationships). Surprisingly, the latter often show more details and finer strokes in the results. To further support our standpoint, we propose two new methods to perform style transfer: one takes inter-class relationships into account and the other does not, and conduct comparative experiments with existing methods. The experimental results verified our observation. Our proposed methods can achieve comparable perceptual quality yet with a lower complexity. We believe that our interpretation provides an effective design basis for designing style loss function for style transfer methods with different visual effects.

scholarly journals Adjoint-based exact Hessian computation

2021 ◽  
Author(s):  
Shin-ichi Ito ◽  
Takeru Matsuda ◽  
Yuto Miyatake
Keyword(s):  
Krylov Subspace ◽  
Hessian Matrix ◽  
Adjoint System ◽  
Coefficient Matrix ◽  
Scalar Function ◽  
Second Order ◽  
Subspace Method ◽  
Initial Value ◽  
Research Fields ◽  
Memory Efficiency

AbstractWe consider a scalar function depending on a numerical solution of an initial value problem, and its second-derivative (Hessian) matrix for the initial value. The need to extract the information of the Hessian or to solve a linear system having the Hessian as a coefficient matrix arises in many research fields such as optimization, Bayesian estimation, and uncertainty quantification. From the perspective of memory efficiency, these tasks often employ a Krylov subspace method that does not need to hold the Hessian matrix explicitly and only requires computing the multiplication of the Hessian and a given vector. One of the ways to obtain an approximation of such Hessian-vector multiplication is to integrate the so-called second-order adjoint system numerically. However, the error in the approximation could be significant even if the numerical integration to the second-order adjoint system is sufficiently accurate. This paper presents a novel algorithm that computes the intended Hessian-vector multiplication exactly and efficiently. For this aim, we give a new concise derivation of the second-order adjoint system and show that the intended multiplication can be computed exactly by applying a particular numerical method to the second-order adjoint system. In the discussion, symplectic partitioned Runge–Kutta methods play an essential role.

scholarly journals A Review of Human–Computer Interaction and Virtual Reality Research Fields in Cognitive InfoCommunications

2021 ◽  
Vol 11 (6) ◽  
pp. 2646
Author(s):  
Jozsef Katona
Keyword(s):  
Information And Communication Technology ◽  
Communication Technology ◽  
Cognitive Processes ◽  
Research Trends ◽  
Technological Innovations ◽  
Important Research ◽  
Cognitive Sciences ◽  
Digital Devices ◽  
Research Fields ◽  
Information And Communication

Cognitive infocommunications (CogInfoCom) is a young and evolving discipline that is at the crossroads of information and communication technology (ICT) and cognitive sciences with many promising results. The goal of the field is to provide insights into how human cognitive capabilities can be merged and extended with the cognitive capabilities of the digital devices surrounding us, with the goal of enabling more seamless interactions between humans and artificially cognitive agents. Results in the field have already led to the appearance of numerous CogInfoCom-based technological innovations. For example, the field has led to a better understanding of how humans can learn more effectively, and the development of new kinds of learning environment have followed accordingly. The goal of this paper is to summarize some of the most recent results in CogInfoCom and to introduce important research trends, developments and innovations that play a key role in understanding and supporting the merging of cognitive processes with ICT.

scholarly journals AN ALGORITHM FOR GENERATING NATURAL COLOR IMAGES FROM FALSE COLOR USING SPECTRAL TRANSFORMATION TECHNIQUE WITH HIGHER POLYNOMIAL ORDER

2018 ◽  
Vol XLII-5 ◽  
pp. 643-648
Author(s):  
S. Kala ◽  
A. Kumar ◽  
A. K. Joshi ◽  
V. M. Bothale ◽  
B. G. Krishna
Keyword(s):  
Second Order ◽  
Color Images ◽  
Water Bodies ◽  
Blue Band ◽  
Spectral Transformation ◽  
False Color ◽  
Natural Color ◽  
Polynomial Transformation ◽  
Order Polynomial ◽  
Color Composite

<p><strong>Abstract.</strong> Satellite imageries in True color composite or Natural Color composite (NCC) serves the best combination for visual interpretation. Red, Green and Infrared channels form false color composite which might not be as useful as NCC to a non-remote sensing professional. As blue band is affected by large atmospheric scattering, satellites like IRS-LISS IV, SPOT do not have blue band. To generate NCC from such satellite data blue band must be simulated. Existing algorithms of spectral transformation do not provide robust coefficients leading to wrong NCC colors especially in water bodies. To achieve more robust coefficients, we have proposed new algorithm to generate NCC for IRS-LISS IV data using second order polynomial regression technique. Second order polynomial transformation functions consider even minor variability present in the image as compared to 1st order so that the derived coefficients are adjustable to accommodate spatial and temporal variability while generating NCC. In this study, Sentinel-2 image was used for deriving coefficients with blue band as dependent and green, red and infrared as independent variables. Simulated Sentinel band showed high accuracy with correlation of 0.93 and 0.97 for two test sites. Using the same coefficients, blue band was simulated for LISS-IV which also showed good correlation of 0.90 with sentinel original blue band. On comparing LISS-IV simulated NCC with simulated NCC from other algorithms, it was observed that higher order polynomial transformation was able to achieve higher accuracy especially for water bodies where expected color is green. Thus, proposed algorithms can be used for transforming false color image to natural color images.</p>

Correlation Between Anterior–Posterior and Lateral Dimensions with the Effective and Water-Equivalent Diameters in Axial Images From Head Computed Tomography Examinations

2021 ◽  
Author(s):  
Winda Kusuma Dewi ◽  
Choirul Anam ◽  
Eko Hidayanto ◽  
Annisa Lidia Wati ◽  
Geoff Dougherty
Keyword(s):  
Computed Tomography ◽  
Linear Correlation ◽  
Second Order ◽  
Patient Dose ◽  
Equivalent Diameter ◽  
Effective Diameter ◽  
Head Ct ◽  
Order Polynomial ◽  
Head Computed Tomography ◽  
Anterior Posterior

Abstract The study aims to correlate the effective diameter (Deff) and water-equivalent diameter (Dw) parameters with anterior–posterior (AP), lateral (LAT) and AP + LAT dimensions in order to estimate the patient dose in head CT examinations. Seventy-four patient datasets from head CT examinations were retrospectively collected. The patient’s sizes were calculated from the middle slice using a software of IndoseCT. Dw and Deff were plotted as functions of AP, LAT and AP + LAT dimensions. The best trendline fit for LAT and AP functions was a second order polynomial, which resulted in R2 of 0.89 for Deff vs LAT, 0.88 for Dw vs LAT, 0.92 for Deff vs AP and 0.91 for Dw vs AP. A linear correlation was found for Deff vs AP + LAT, Dw vs AP + LAT and Dw vs Deff with R2 of 0.97, 0.96 and 0.98, respectively.

Sign in / Sign up

Export Citation Format

Share Document