Fine-Grained Cross-Modal Retrieval for Cultural Items with Focal Attention and Hierarchical Encodings

In this paper, we target the tasks of fine-grained image–text alignment and cross-modal retrieval in the cultural heritage domain as follows: (1) given an image fragment of an artwork, we retrieve the noun phrases that describe it; (2) given a noun phrase artifact attribute, we retrieve the corresponding image fragment it specifies. To this end, we propose a weakly supervised alignment model where the correspondence between the input training visual and textual fragments is not known but their corresponding units that refer to the same artwork are treated as a positive pair. The model exploits the latent alignment between fragments across modalities using attention mechanisms by first projecting them into a shared common semantic space; the model is then trained by increasing the image–text similarity of the positive pair in the common space. During this process, we encode the inputs of our model with hierarchical encodings and remove irrelevant fragments with different indicator functions. We also study techniques to augment the limited training data with synthetic relevant textual fragments and transformed image fragments. The model is later fine-tuned by a limited set of small-scale image–text fragment pairs. We rank the test image fragments and noun phrases by their intermodal similarity in the learned common space. Extensive experiments demonstrate that our proposed models outperform two state-of-the-art methods adapted to fine-grained cross-modal retrieval of cultural items for two benchmark datasets.

An asymptotic analysis for a generalized Cahn–Hilliard system with fractional operators

In the recent paper “Well-posedness and regularity for a generalized fractional Cahn–Hilliard system” (Colli et al. in Atti Accad Naz Lincei Rend Lincei Mat Appl 30:437–478, 2019), the same authors have studied viscous and nonviscous Cahn–Hilliard systems of two operator equations in which nonlinearities of double-well type, like regular or logarithmic potentials, as well as nonsmooth potentials with indicator functions, were admitted. The operators appearing in the system equations are fractional powers $$A^{2r}$$ A 2 r and $$B^{2\sigma }$$ B 2 σ (in the spectral sense) of general linear operators A and B, which are densely defined, unbounded, selfadjoint, and monotone in the Hilbert space $$L^2(\Omega )$$ L 2 ( Ω ) , for some bounded and smooth domain $$\Omega \subset {{\mathbb {R}}}^3$$ Ω ⊂ R 3 , and have compact resolvents. Existence, uniqueness, and regularity results have been proved in the quoted paper. Here, in the case of the viscous system, we analyze the asymptotic behavior of the solution as the parameter $$\sigma $$ σ appearing in the operator $$B^{2\sigma }$$ B 2 σ decreasingly tends to zero. We prove convergence to a phase relaxation problem at the limit, and we also investigate this limiting problem, in which an additional term containing the projection of the phase variable on the kernel of B appears.

Exact Computation of Maximum Rank Correlation Estimator

In this paper we provide a computation algorithm to get a global solution for the maximum rank correlation estimator using the mixed integer programming (MIP) approach. We construct a new constrained optimization problem by transforming all indicator functions into binary parameters to be estimated and show that it is equivalent to the original problem. We also consider an application of the best subset rank prediction and show that the original optimization problem can be reformulated as MIP. We derive the non-asymptotic bound for the tail probability of the predictive performance measure. We investigate the performance of the MIP algorithm by an empirical example and Monte Carlo simulations.

Identification of Disturbances for a Room Temperature Conditions Model Using a Neural Network

The use of IoT devices for building heating systems opens the possibility of collecting a large amount of various data about room temperature conditions. At the level of individual rooms, there are factors that can have a significant effect on the temperature conditions, the measurement of which involves difficulties. As a consequence, the models of room temperature conditions are identified incorrectly. In view of this circumstance, the consideration of unknown disturbances becomes of issue. A method to identify the building room temperature conditions is proposed that allows unknown disturbing inputs in dynamic systems to be taken into account. The unknown disturbance action time is described using indicator functions. The indicator function time characteristics are identified using neural LSTM networks by solving the problem of performing binary classification of whether the measured data sample time tags belong to unknown disturbances. The sequence in which unknown disturbances are taken into account in the model is found by sorting the evaluated degree to which the time tags belong to the onset of a certain unknown disturbance that is obtained by solving the binary classification problem. The application of the proposed approach is illustrated on the temperature conditions identification problem using test data with two samples of unknown disturbances with random action degree and time. The study results demonstrate correctness of the proposed approach, the use of which makes it possible to more accurately identify the static and dynamic parameters of system models under the effect of unknown disturbances.