Visual analytics tool for the interpretation of hidden states in recurrent neural networks

In this paper, we introduce a visual analytics approach aimed at helping machine learning experts analyze the hidden states of layers in recurrent neural networks. Our technique allows the user to interactively inspect how hidden states store and process information throughout the feeding of an input sequence into the network. The technique can help answer questions, such as which parts of the input data have a higher impact on the prediction and how the model correlates each hidden state configuration with a certain output. Our visual analytics approach comprises several components: First, our input visualization shows the input sequence and how it relates to the output (using color coding). In addition, hidden states are visualized through a nonlinear projection into a 2-D visualization space using t-distributed stochastic neighbor embedding to understand the shape of the space of the hidden states. Trajectories are also employed to show the details of the evolution of the hidden state configurations. Finally, a time-multi-class heatmap matrix visualizes the evolution of the expected predictions for multi-class classifiers, and a histogram indicates the distances between the hidden states within the original space. The different visualizations are shown simultaneously in multiple views and support brushing-and-linking to facilitate the analysis of the classifications and debugging for misclassified input sequences. To demonstrate the capability of our approach, we discuss two typical use cases for long short-term memory models applied to two widely used natural language processing datasets.

Knowledge discovery using clustering analysis of rainfall timeseries

<p>Rainfall time series analysis using clustering involves the identification of temporal patterns, with each data item representing an individual storm. This analysis results in clusters of data items that trend in a common way and can be utilized in stochastic simulation, water resources planning and the identification of future directions due to climate change. A comparative analysis is carried out of several methods that use intra versus inter-cluster distances, for the estimation of the relevant number of clusters using a big dataset of the described rainfall time series. Visualization using topographic maps that are produced via nonlinear projection techniques is applied, to validate the presence of both distance and density structures and to assist in the final determination of the numbers of clusters. This stands in contrast to empirical and not completely data-driven approaches of the literature, in which constrained clustering methods are employed with assumptions on the presence of four classes.</p>

An Image Reconstruction Model under Poisson Noise Using multiscale Compressed Sensing

For the influence of poisson noise images, in order to get rid of poisson noise, this paper put forward image reconstruction method by using multiscale compressed sensing. the algorithm can approximate the optimal sparse representation of the image edge details such as the characteristics of theShearlet domain based multi-scale compressed sensing method. The image is decomposed into the high-frequency subbands byShearlet, and the compressed sensing is applied into each subband to reconstruct the image. In this paper, A total variation of RL iterative algorithm constructed by nonlinear projection algorithm based on closed convex set is explored as the reconstruction method, which use derivation of the nonlinear projection instead of total variation. In mathematics, Shearlet has been proved to be a better tool for edge characterization than traditional wavelet. By using the nonlinear projection scheme to constrain the residual coefficients in the Shearlet domain, a better estimation can be obtained from the Shearlet representation. Numerical examples show that the denoising effect of these methods is very good, which is better than the correlation method based on Curvelet transform. In addition, the number of iterations required by our scheme is far less than that of our competitors.

System of Extended General Variational Inequalities for Relaxed Cocoercive Mappings in Hilbert Space

In this manuscript, we study a system of extended general variational inequalities (SEGVI) with several nonlinear operators, more precisely, six relaxed ( α , r ) -cocoercive mappings. Using the projection method, we show that a system of extended general variational inequalities is equivalent to the nonlinear projection equations. This alternative equivalent problem is used to consider the existence and convergence (or approximate solvability) of a solution of a system of extended general variational inequalities under suitable conditions.

Multi-Component Topology Optimization for Powder Bed Additive Manufacturing (MTO-A)

This paper presents a gradient-based multi-component topology optimization (MTO) method for structures assembled from components made by powder bed additive manufacturing. It is built upon our previous work on the continuously-relaxed MTO framework utilizing the concept of fractional component membership. The previous attempt on the integration of the relaxed MTO framework with additive manufacturing constraints, however, suffered from numerical instability for larger size problems, limiting its application to 2D low-resolution examples. To overcome this difficulty, this paper proposes an improved MTO formulation based on a design field regularization and a nonlinear projection of component membership variables, with a focus on powder bed additive manufacturing. For each component, constraints on the maximum allowable build volume (i.e., length, width, and height), the elimination of enclosed voids, and the minimum printable feature size are imposed during the simultaneous optimization of the overall base topology and component partitioning. The scalability of the new MTO formulation is demonstrated by a few 2D examples with much higher resolution than previously reported, and the first reported 3D example of MTO.

Deep Supervised Hashing with Nonlinear Projections

Hashing has attracted broad research interests in large scale image retrieval due to its high search speed and efficient storage. Recently, many deep hashing methods have been proposed to perform simultaneous nonlinear feature learning and hash projection learning, which have shown superior performance compared to hand-crafted feature based hashing methods. Nonlinear projection functions have shown their advantages over the linear ones due to their powerful generalization capabilities. To improve the performance of deep hashing methods by generalizing projection functions, we propose the idea of implementing a pure nonlinear deep hashing network architecture. By consolidating the above idea, this paper presents a Deep Supervised Hashing architecture with Nonlinear Projections (DSHNP). In particular, soft decision trees are adopted as the nonlinear projection functions, since they can generate differentiable nonlinear outputs and can be trained with deep neural networks in an end-to-end way. Moreover, to make the hash codes as independent as possible, we design two regularizers imposed on the parameter matrices of the leaves in the soft decision trees. Extensive evaluations on two benchmark image datasets show that the proposed DSHNP outperforms several state-of-the-art hashing methods.