Efficient Multi-modal Hashing with Online Query Adaption for Multimedia Retrieval

Multi-modal hashing supports efficient multimedia retrieval well. However, existing methods still suffer from two problems: (1) Fixed multi-modal fusion. They collaborate the multi-modal features with fixed weights for hash learning, which cannot adaptively capture the variations of online streaming multimedia contents. (2) Binary optimization challenge. To generate binary hash codes, existing methods adopt either two-step relaxed optimization that causes significant quantization errors or direct discrete optimization that consumes considerable computation and storage cost. To address these problems, we first propose a Supervised Multi-modal Hashing with Online Query-adaption method. A self-weighted fusion strategy is designed to adaptively preserve the multi-modal features into hash codes by exploiting their complementarity. Besides, the hash codes are efficiently learned with the supervision of pair-wise semantic labels to enhance their discriminative capability while avoiding the challenging symmetric similarity matrix factorization. Further, we propose an efficient Unsupervised Multi-modal Hashing with Online Query-adaption method with an adaptive multi-modal quantization strategy. The hash codes are directly learned without the reliance on the specific objective formulations. Finally, in both methods, we design a parameter-free online hashing module to adaptively capture query variations at the online retrieval stage. Experiments validate the superiority of our proposed methods.

Front vehicle detection based on improved fusion method for lidar and visual image

In order to ensure the detection accuracy, an improved adaptive weighted (IAW) method is proposed in this paper to fuse the data of images and lidar sensors for the vehicle object’s detection. Firstly, the IAW method is proposed in this paper and the first simulation is conducted. The unification of two sensors’ time and space should be completed at first. The traditional adaptive weighted average method (AWA) will amplify the noise in the fusion process, so the data filtered with Kalman Filter (KF) algorithm instead of with the AWA method. The proposed IAW method is compared with the AWA method and the Distributed Weighted fusion KF algorithm in the data fusion simulation to verify the superiority of the proposed algorithm. Secondly, the second simulation is conducted to verify the robustness and accuracy of the IAW algorithm. In the two experimental scenarios of sparse and dense vehicles, the vehicle detection based on image and lidar is completed, respectively. The detection data is correlated and merged through the IAW method, and the results show that the IAW method can correctly associate and fuse the data of the two sensors. Finally, the real vehicle test of object vehicle detection in different environments is carried out. The IAW method, the KF algorithm, and the Distributed Weighted fusion KF algorithm are used to complete the target vehicle detection in the real vehicle, respectively. The advantages of the two sensors can give full play, and the misdetection of the target objects can be reduced with proposed method. It has great potential in the application of object acquisition.

Infrared and Visible Image Fusion Based on Co-occurrence Analysis Shearlet Transform Context-Aware

This study based on co-occurrence analysis shearlet transform (CAST) effectively combines the latent low rank representation (LatLRR) and the regularization of zero-crossing counting in differences to fuse the heterogeneous images. First, the source images are decomposed by CAST method into base-layer and detail-layer sub-images. Secondly, for the base-layer components with larger-scale intensity variation, the LatLRR, is a valid method to extract the salient information from image sources, and can be applied to generate saliency map to implement the weighted fusion of base-layer images adaptively. Meanwhile, the regularization term of zero crossings in differences, which is a classic method of optimization, is designed as the regularization term to construct the fusion of detail-layer images. By this method, the gradient information concealed in the source images can be extracted as much as possible, then the fusion image owns more abundant edge information. Compared with other state-of-the-art algorithms on publicly available datasets, the quantitative and qualitative analysis of experimental results demonstrate that the proposed method outperformed in enhancing the contrast and achieving close fusion result.

Improving the Accuracy of Groundwater Storage Estimates Based on Groundwater Weighted Fusion Model

It is an effective measure to estimate groundwater storage anomalies (GWSA) by combining Gravity Recovery and Climate Experiment (GRACE) data and hydrological models. However, GWSA results based on a single hydrological model and GRACE data may have greater uncertainties, and it is difficult to verify in some regions where in situ groundwater-level measurements are limited. First, to solve this problem, a groundwater weighted fusion model (GWFM) is presented, based on the extended triple collocation (ETC) method. Second, the Shiyang River Basin (SYRB) is taken as an example, and in situ groundwater-level measurements are used to evaluate the performance of the GWFM. The comparison indicates that the correlation coefficient (CC) and Nash-Sutcliffe efficiency coefficient (NSE) are increased by 9–40% and 23–657%, respectively, relative to the original results. Moreover, the root mean squared error (RMSE) is reduced by 9–28%, which verifies the superiority of the GWFM. Third, the spatiotemporal distribution and influencing factors of GWSA in the Hexi Corridor (HC) are comprehensively analyzed during the period between 2003 and 2016. The results show that GWSA decline, with a trend of −2.37 ± 0.38 mm/yr from 2003 to 2010, and the downward trend after 2011 (−0.46 ± 1.35 mm/yr) slow down significantly compared to 2003–2010. The spatial distribution obtained by the GWFM is more reliable compared to the arithmetic average results, and GWFM-based GWSA fully retain the advantages of different models, especially in the southeastern part of the SYRB. Additionally, a simple index is used to evaluate the contributions of climatic factors and human factors to groundwater storage (GWS) in the HC and its different subregions. The index indicates that climate factors occupy a dominant position in the SLRB and SYRB, while human factors have a significant impact on GWS in the Heihe River Basin (HRB). This study can provide suggestions for the management and assessments of groundwater resources in some arid regions.

Weighted Direct Position Determination via the Dimension Reduction Method for Noncircular Signals

This work studies the direct position determination (DPD) of noncircular (NC) signals with multiple arrays. Existing DPD algorithms of NC sources ignore the impact of path propagation loss on the performance of the algorithms. In practice, the signal-to-noise ratios (SNRs) of different observation stations are often different and unstable when the NC signal of the same radiation target strikes different observation locations. Besides, NC features of the target signals are applied not only to extend the virtual array manifold but also to bring high-dimensional search. For the sake of addressing the above problems, this study develops a DPD method of NC sources for multiple arrays combing weighted subspace data fusion (SDF) and dimension reduction (RD) search. First, NC features of the target signals are applied to extend the virtual array manifold. Second, we assign a weight to balance the error and obtain higher location accuracy with better robustness. Then, the RD method is used to eliminate the high computational complexity caused by the NC phase search dimension. Finally, the weighted fusion cost function is constructed by using the eigenvalues of the received signal covariance matrixes. It is verified by simulation that the proposed algorithm can effectively improve the location performance, get better robustness, and distinguish more targets compared with two-step location technology and SDF technology. In addition, without losing the estimation performance, the proposed algorithm can significantly reduce the complexity caused by the NC phase search dimension.

A weighted fusion method with multi-system for improving measurement accuracy of structured light 3D profilometry

It is a challenge to improve the accuracy of 3D profile measurement based on binary coded structured light for complex surfaces. A new method of weighted fusion with multi-system is presented to reduce the measurement errors due to the stripe grayscale asymmetry, which is based on the analysis of stripe center deviation related to surface normal and the directions of incident and reflected rays. First, the stripe center deviation model is established according to the geometric relationship between the stripe center deviation, the incident and reflected angles at any measured point. The influence of each variable on stripe center deviation is analyzed, and three subsystems are formed by a binocular structured light framework to achieve multiple measurements based on the influence regularity. Then in order to improve the measurement accuracy, different weights are assigned to the measured point in different subsystems according to the stripe center deviation model and its relationship with measurement error, and the weighted data from different subsystems are fused. Experiments are carried out to validate the presented method, and the experimental results demonstrate that it effectively improves the measurement accuracy of complex surfaces and measurement accuracy is improved by about 27% compared with the conventional method.

Adaptive Feature Weighted Fusion Nested U-Net with Discrete Wavelet Transform for Change Detection of High-Resolution Remote Sensing Images

The characteristics of a wide variety of scales about objects and complex texture features of high-resolution remote sensing images make deep learning-based change detection methods the mainstream method. However, existing deep learning methods have problems with spatial information loss and insufficient feature representation, resulting in unsatisfactory effects of small objects detection and boundary positioning in high-resolution remote sensing images change detection. To address the problems, a network architecture based on 2-dimensional discrete wavelet transform and adaptive feature weighted fusion is proposed. The proposed network takes Siamese network and Nested U-Net as the backbone; 2-dimensional discrete wavelet transform is used to replace the pooling layer; and the inverse transform is used to replace the upsampling to realize image reconstruction, reduce the loss of spatial information, and fully retain the original image information. In this way, the proposed network can accurately detect changed objects of different scales and reconstruct change maps with clear boundaries. Furthermore, different feature fusion methods of different stages are proposed to fully integrate multi-scale and multi-level features and improve the comprehensive representation ability of features, so as to achieve a more refined change detection effect while reducing pseudo-changes. To verify the effectiveness and advancement of the proposed method, it is compared with seven state-of-the-art methods on two datasets of Lebedev and SenseTime from the three aspects of quantitative analysis, qualitative analysis, and efficiency analysis, and the effectiveness of proposed modules is validated by an ablation study. The results of quantitative analysis and efficiency analysis show that, under the premise of taking into account the operation efficiency, our method can improve the recall while ensuring the detection precision, and realize the improvement of the overall detection performance. Specifically, it shows an average improvement of 37.9% and 12.35% on recall, and 34.76% and 11.88% on F1 with the Lebedev and SenseTime datasets, respectively, compared to other methods. The qualitative analysis shows that our method has better performance on small objects detection and boundary positioning than other methods, and a more refined change map can be obtained.

A SAR Target Recognition Method via Combination of Multilevel Deep Features

For the problem of synthetic aperture radar (SAR) image target recognition, a method via combination of multilevel deep features is proposed. The residual network (ResNet) is used to learn the multilevel deep features of SAR images. Based on the similarity measure, the multilevel deep features are clustered and several feature sets are obtained. Then, each feature set is characterized and classified by the joint sparse representation (JSR), and the corresponding output result is obtained. Finally, the results of different feature sets are combined using the weighted fusion to obtain the target recognition results. The proposed method in this paper can effectively combine the advantages of ResNet and JSR in feature extraction and classification and improve the overall recognition performance. Experiments and analysis are carried out on the MSTAR dataset with rich samples. The results show that the proposed method can achieve superior performance for 10 types of target samples under the standard operating condition (SOC), noise interference, and occlusion conditions, which verifies its effectiveness.

Drug-Target Interaction Prediction Based on Multisource Information Weighted Fusion

Recently, in most existing studies, it is assumed that there are no interaction relationships between drugs and targets with unknown interactions. However, unknown interactions mean the relationships between drugs and targets have just not been confirmed. In this paper, samples for which the relationship between drugs and targets has not been determined are considered unlabeled. A weighted fusion method of multisource information is proposed to screen drug-target interactions. Firstly, some drug-target pairs which may have interactions are selected. Secondly, the selected drug-target pairs are added to the positive samples, which are regarded as known to have interaction relationships, and the original interaction relationship matrix is revised. Finally, the revised datasets are used to predict the interaction derived from the bipartite local model with neighbor-based interaction profile inferring (BLM-NII). Experiments demonstrate that the proposed method has greatly improved specificity, sensitivity, precision, and accuracy compared with the BLM-NII method. In addition, compared with several state-of-the-art methods, the area under the receiver operating characteristic curve (AUC) and the area under the precision-recall curve (AUPR) of the proposed method are excellent.