Rich Structural Index for Stereoscopic Image Quality Assessment

The human visual system (HVS), affected by viewing distance when perceiving the stereo image information, is of great significance to study of stereoscopic image quality assessment. Many methods of stereoscopic image quality assessment do not have comprehensive consideration for human visual perception characteristics. In accordance with this, we propose a Rich Structural Index (RSI) for Stereoscopic Image objective Quality Assessment (SIQA) method based on multi-scale perception characteristics. To begin with, we put the stereo pair into the image pyramid based on Contrast Sensitivity Function (CSF) to obtain sensitive images of different resolution . Then, we obtain local Luminance and Structural Index (LSI) in a locally adaptive manner on gradient maps which consider the luminance masking and contrast masking. At the same time we use Singular Value Decomposition (SVD) to obtain the Sharpness and Intrinsic Structural Index (SISI) to effectively capture the changes introduced in the image (due to distortion). Meanwhile, considering the disparity edge structures, we use gradient cross-mapping algorithm to obtain Depth Texture Structural Index (DTSI). After that, we apply the standard deviation method for the above results to obtain contrast index of reference and distortion components. Finally, for the loss caused by the randomness of the parameters, we use Support Vector Machine Regression based on Genetic Algorithm (GA-SVR) training to obtain the final quality score. We conducted a comprehensive evaluation with state-of-the-art methods on four open databases. The experimental results show that the proposed method has stable performance and strong competitive advantage.

DuFeNet: Improve the Accuracy and Increase Shape Bias of Neural Network Models

In image classification field, existing work tends to modify the network structure to obtain higher accuracy or faster speed. However, some studies have found that the neural network usually has texture bias effect, which means that the neural network is more sensitive to the texture information than the shape information. Based on such phenomenon, we propose a new way to improve network performance by making full use of gradient information. The dual features network (DuFeNet) is proposed in this paper. In DuFeNet, one sub-network is used to learn the information of gradient features, and the other is a traditional neural network with texture bias. The structure of DuFeNet is easy to implement in the original neural network structure. The experimental results clearly show that DuFeNet can achieve better accuracy in image classification and detection. It can increase the shape bias of the network adapted to human visual perception. Besides, DuFeNet can be used without modifying the structure of the original network at lower additional parameters cost.

Identify Light-curve Signals with Deep Learning Based Object Detection Algorithm. I. Transit Detection

Deep learning techniques have been well explored in the transiting exoplanet field; however, previous work mainly focuses on classification and inspection. In this work, we develop a novel detection algorithm based on a well-proven object detection framework in the computer vision field. Through training the network on the light curves of the confirmed Kepler exoplanets, our model yields about 90% precision and recall for identifying transits with signal-to-noise ratio higher than 6 (set the confidence threshold to 0.6). Giving a slightly lower confidence threshold, recall can reach higher than 95%. We also transfer the trained model to the TESS data and obtain similar performance. The results of our algorithm match the intuition of the human visual perception and make it useful to find single-transiting candidates. Moreover, the parameters of the output bounding boxes can also help to find multiplanet systems. Our network and detection functions are implemented in the Deep-Transit toolkit, which is an open-source Python package hosted on Github and PyPI.

A Measurement of Visual Complexity for Heterogeneity in the Built Environment Based on Fractal Dimension and Its Application in Two Gardens

In this study, a fractal dimension-based method has been developed to compute the visual complexity of the heterogeneity in the built environment. The built environment is a very complex combination, structurally consisting of both natural and artificial elements. Its fractal dimension computation is often disturbed by the homogenous visual redundancy, which is textured but needs less attention to process, so that it leads to a pseudo-evaluation of visual complexity in the built environment. Based on human visual perception, the study developed a method: fractal dimension of heterogeneity in the built environment, which includes Potts segmentation and Canny edge detection as image preprocessing procedure and fractal dimension as computation procedure. This proposed method effectively extracts perceptually meaningful edge structures in the visual image and computes its visual complexity which is consistent with human visual characteristics. In addition, an evaluation system combining the proposed method and the traditional method has been established to classify and assess the visual complexity of the scenario more comprehensively. Two different gardens had been computed and analyzed to demonstrate that the proposed method and the evaluation system provide a robust and accurate way to measure the visual complexity in the built environment.

Objective Numerical Evaluation of Diffuse, Optically Reconstructed Images Using Structural Similarity Index

Diffuse optical tomography is emerging as a non-invasive optical modality used to evaluate tissue information by obtaining the optical properties’ distribution. Two procedures are performed to produce reconstructed absorption and reduced scattering images, which provide structural information that can be used to locate inclusions within tissues with the assistance of a known light intensity around the boundary. These methods are referred to as a forward problem and an inverse solution. Once the reconstructed image is obtained, a subjective measurement is used as the conventional way to assess the image. Hence, in this study, we developed an algorithm designed to numerically assess reconstructed images to identify inclusions using the structural similarity (SSIM) index. We compared four SSIM algorithms with 168 simulated reconstructed images involving the same inclusion position with different contrast ratios and inclusion sizes. A multiscale, improved SSIM containing a sharpness parameter (MS-ISSIM-S) was proposed to represent the potential evaluation compared with the human visible perception. The results indicated that the proposed MS-ISSIM-S is suitable for human visual perception by demonstrating a reduction of similarity score related to various contrasts with a similar size of inclusion; thus, this metric is promising for the objective numerical assessment of diffuse, optically reconstructed images.

Pre-Processing Filter Reflecting Human Visual Perception to Improve Saliency Detection Performance

Salient object detection is a method of finding an object within an image that a person determines to be important and is expected to focus on. Various features are used to compute the visual saliency, and in general, the color and luminance of the scene are widely used among the spatial features. However, humans perceive the same color and luminance differently depending on the influence of the surrounding environment. As the human visual system (HVS) operates through a very complex mechanism, both neurobiological and psychological aspects must be considered for the accurate detection of salient objects. To reflect this characteristic in the saliency detection process, we have proposed two pre-processing methods to apply to the input image. First, we applied a bilateral filter to improve the segmentation results by smoothing the image so that only the overall context of the image remains while preserving the important borders of the image. Second, although the amount of light is the same, it can be perceived with a difference in the brightness owing to the influence of the surrounding environment. Therefore, we applied oriented difference-of-Gaussians (ODOG) and locally normalized ODOG (LODOG) filters that adjust the input image by predicting the brightness as perceived by humans. Experiments on five public benchmark datasets for which ground truth exists show that our proposed method further improves the performance of previous state-of-the-art methods.

A method for presenting biomedical images based on a model of receptive fields of human visual perception

The paper is devoted to the development of a new method for presenting biomedical images based on local characteristics of the intensity of their shape. The proposed method of image processing is focused on images that have low indicators of the intensity of the recorded radiation, resolution, contrast and signal-to-noise ratio. The method is based on the principles of machine (Bayesian) learning and on samples of random photo reports. This paper presents the results of the method and its connection with modern approaches in the field of image processing.

Visual perception driven collage synthesis

A collage is a composite artwork made from the spatial layout of multiple pictures on a canvas, collected from the Internet or user photographs. Collages, usually made by skilled artists, involve a complex manual process, especially when searching for component pictures and adjusting their spatial layout to meet artistic requirements. In this paper, we present a visual perception driven method for automatically synthesizing visually pleasing collages. Unlike previous works, we focus on how to design a collage layout which not only provides easy access to the theme of the overall image, but also conforms to human visual perception. To achieve this goal, we formulate the generation of collages as a mapping problem: given a canvas image, first, compute a saliency map for it and a vector field for each sub-region of it. Second, using a divide-and-conquer strategy, generate a series of patch sets from the canvas image, where the salient map and the vector field are used to determine each patch’s size and direction respectively. Third, construct a Gestalt-based energy function to choose the most visually pleasing and orderly patch set as the final layout. Finally, using a semantic-color metric, map the picture set to the patch set to generate the final collage. Extensive experimental and user study results show that this method can generate visual pleasing collages.