scholarly journals Gate Attentional Factorization Machines: An Efficient Neural Network Considering Both Accuracy and Speed

2021 ◽  
Vol 11 (20) ◽  
pp. 9546
Author(s):  
Huaidong Yu ◽  
Jian Yin ◽  
Yan Li

Nowadays, to deal with the increasing data of users and items and better mine the potential relationship between the data, the model used by the recommendation system has become more and more complex. In this case, how to ensure the prediction accuracy and operation speed of the recommendation system has become an urgent problem. Deep neural network is a good solution to the problem of accuracy, we can use more network layers, more advanced feature cross way to improve the utilization of data. However, when the accuracy is guaranteed, little attention is paid to the speed problem. We can only pursue better machine efficiency, and we do not pay enough attention to the speed efficiency of the model itself. Some models with advantages in speed, such as PNN, are slightly inferior in accuracy. In this paper, the Gate Attention Factorization Machine (GAFM) model based on the double factors of accuracy and speed is proposed, and the structure of gate is used to control the speed and accuracy. Extensive experiments have been conducted on data sets in various application scenarios, and the results show that the GAFM model is better than the existing factorization machines in both speed and accuracy.

GPS Solutions ◽  
2021 ◽  
Vol 25 (2) ◽  
Author(s):  
Bohua Huang ◽  
Zengxi Ji ◽  
Renjian Zhai ◽  
Changfu Xiao ◽  
Fan Yang ◽  
...  

AbstractIn a satellite navigation system, high-precision prediction of satellite clock bias directly determines the accuracy of navigation, positioning, and time synchronization and is the key to realizing autonomous navigation. To further improve satellite clock bias prediction accuracy, we establish a satellite clock bias prediction model by using long short-term memory (LSTM) that can accurately express the nonlinear characteristics of the navigation satellite clock bias. Outliers in the original clock bias should be preprocessed before using the clock bias for prediction. By analyzing the working principle of the traditional median absolute deviations method, the ambiguity of the mathematical model of that method was improved. Experimental results show that the improved method is better than the traditional method at detecting gross errors. The single difference sequence of the preprocessed satellite clock bias was taken as the research object. First, a quadratic polynomial model was fit to the trend term of the single difference sequence. Second, based on the LSTM neural network model and the basic principles of supervised learning, a supervised learning LSTM network model (SL-LSTM) was proposed that models cyclic and random terms. Finally, the prediction function of the satellite clock bias was realized by extrapolating the model by adding a trend term. We adopt the GPS precision satellite clock bias of International GNSS Service data forecast experiments and apply wavelet neural network (WNN), autoregressive integrated moving average (ARIMA), and quadratic polynomial (QP) models to compare their prediction effects. The average prediction RMSE for 3 h, 6 h, 12 h, 1 d, and 3 d based on the SL-LSTM improved by approximately −21.80, −1.85, 8.57, 2.27, and 40.79%, respectively, compared with the results of the WNN. The average prediction RMSE based on the SL-LSTM improved by approximately 38.23, 65.48, 80.22, 85.18, and 94.51% compared with the ARIMA results. The average prediction RMSE based on the SL-LSTM improved by approximately 82.37, 75.88, 67.24, 45.71, and 58.22% compared with the QP results. Compared with the WNN, the SL-LSTM method has no obvious advantages in the prediction accuracy and stability in short-term prediction but achieves a better long-term prediction accuracy and stability. With an increased prediction duration, the SL-LSTM method is clearly better than the other three methods in terms of the prediction accuracy and stability. The results indicated that the quality of satellite clock bias prediction by the SL-LSTM method is better than that of the above three methods and is more suitable for the middle- and long-term prediction of satellite clock bias.


Author(s):  
Yantao Yu ◽  
Zhen Wang ◽  
Bo Yuan

Factorization machines (FMs) are a class of general predictors working effectively with sparse data, which represents features using factorized parameters and weights. However, the accuracy of FMs can be adversely affected by the fixed representation trained for each feature, as the same feature is usually not equally predictive and useful in different instances. In fact, the inaccurate representation of features may even introduce noise and degrade the overall performance. In this work, we improve FMs by explicitly considering the impact of individual input upon the representation of features. We propose a novel model named \textit{Input-aware Factorization Machine} (IFM), which learns a unique input-aware factor for the same feature in different instances via a neural network. Comprehensive experiments on three real-world recommendation datasets are used to demonstrate the effectiveness and mechanism of IFM. Empirical results indicate that IFM is significantly better than the standard FM model and consistently outperforms four state-of-the-art deep learning based methods.


2019 ◽  
Author(s):  
James Stevenson ◽  
Leif D. Jacobson ◽  
Yutong Zhao ◽  
Chuanjie Wu ◽  
Jon Maple ◽  
...  

We have developed a neural network potential energy function for use in drug discovery, with chemical element support extended from 41% to 94% of druglike molecules based on ChEMBL. We expand on the work of Smith et al., with their highly accurate network for the elements H, C, N, O, creating a network for H, C, N, O, S, F, Cl, P. We focus particularly on the calculation of relative conformer energies, for which we show that our new potential energy function has an RMSE of 0.70 kcal/mol for prospective druglike molecule conformers, substantially better than the previous state of the art. The speed and accuracy of this model could greatly accelerate the parameterization of protein-ligand binding free energy calculations for novel druglike molecules.


2019 ◽  
Author(s):  
James Stevenson ◽  
Leif D. Jacobson ◽  
Yutong Zhao ◽  
Chuanjie Wu ◽  
Jon Maple ◽  
...  

We have developed a neural network potential energy function for use in drug discovery, with chemical element support extended from 41% to 94% of druglike molecules based on ChEMBL. We expand on the work of Smith et al., with their highly accurate network for the elements H, C, N, O, creating a network for H, C, N, O, S, F, Cl, P. We focus particularly on the calculation of relative conformer energies, for which we show that our new potential energy function has an RMSE of 0.70 kcal/mol for prospective druglike molecule conformers, substantially better than the previous state of the art. The speed and accuracy of this model could greatly accelerate the parameterization of protein-ligand binding free energy calculations for novel druglike molecules.


2010 ◽  
Vol 20-23 ◽  
pp. 82-87 ◽  
Author(s):  
Shan Ding ◽  
Yixin Yin ◽  
Wei Huang ◽  
Jie Dong ◽  
Xue Ming Ma

A temperature model of the electrical heating furnace which is commonly used in industry is built by means of Elman neural network combined with an improved particle swarm optimization (IPSO). The input is duty cycle, and IPSO is used to optimize the weights and threshold values of Elman neural network to improve convergence capability and generalization performance of Elman network. Results show that the method performs better than ordinary Elman network in convergence speed and accuracy.


1997 ◽  
Vol 08 (05n06) ◽  
pp. 581-599 ◽  
Author(s):  
Henrik Nielsen ◽  
Jacob Engelbrecht ◽  
Søren Brunak ◽  
Gunnar Von Heijne

We have developed a new method for the identification of signal peptides and their cleavage sites based on neural networks trained on separate sets of prokaryotic and eukaryotic sequences. The method performs significantly better than previous prediction schemes, and can easily be applied to genome-wide data sets. Discrimination between cleaved signal peptides and uncleaved N-terminal signal-anchor sequences is also possible, though with lower precision. Predictions can be made on a publicly available WWW server: .


2013 ◽  
Vol 427-429 ◽  
pp. 770-773
Author(s):  
Fan Yu ◽  
Ji Xin Li

This paper proposes a method to identify the singular sample with near infrared spectroscopy based on immune algorithm. T The immune algorithm and genetic algorithm for the same NIR spectral data sets are singular sample identification and comparison of the two methods. Remove the singular sample, immune algorithm is better than the genetic algorithm are used to PRESS of PLS model of water, fat, the protein increased by 25.8%, 32.1%, 21.7%.Experiments show that, artificial immune algorithm is not only suitable for near infrared spectra of singular sample, but also can improve the model prediction accuracy and robustness.


1999 ◽  
Vol 09 (04) ◽  
pp. 335-350 ◽  
Author(s):  
ROELOF K BROUWER

This paper describes a method for growing a recurrent neural network of fuzzy threshold units for the classification of feature vectors. Fuzzy networks seem natural for performing classification, since classification is concerned with set membership and objects generally belonging to sets of various degrees. A fuzzy unit in the architecture proposed here determines the degree to which the input vector lies in the fuzzy set associated with the fuzzy unit. This is in contrast to perceptrons that determine the correlation between input vector and a weighting vector. The resulting membership value, in the case of the fuzzy unit, is compared with a threshold, which is interpreted as a membership value. Training of a fuzzy unit is based on an algorithm for linear inequalities similar to Ho-Kashyap recording. These fuzzy threshold units are fully connected in a recurrent network. The network grows as it is trained. The advantages of the network and its training method are: (1) Allowing the network to grow to the required size which is generally much smaller than the size of the network which would be obtained otherwise, implying better generalization, smaller storage requirements and fewer calculations during classification; (2) The training time is extremely short; (3) Recurrent networks such as this one are generally readily implemented in hardware; (4) Classification accuracy obtained on several standard data sets is better than that obtained by the majority of other standard methods; and (5) The use of fuzzy logic is very intuitive since class membership is generally fuzzy.


2012 ◽  
Vol 468-471 ◽  
pp. 1714-1720 ◽  
Author(s):  
Li Meng ◽  
Li Jun Dong

The paper researches on the behavior of particles in the PSO, and improves the situation of easily falling into local optimum by the right combination of simulated annealing and PSO. In the paper, the author compared the original PSO and the improved SAPSO algorithms in neural network training. The empirical research shows that the improved algorithm performed better than the PSO algorithm in global search ability, and the prediction accuracy is greatly increased.


PLoS ONE ◽  
2020 ◽  
Vol 15 (12) ◽  
pp. e0240656
Author(s):  
Meng Wang

Recently, more personalized travel methods have emerged in the tourism industry, such as individual travel and self-guided travel. The service models of traditional tourism limit the diversity of service options and cannot fully meet the individual needs of tourists anymore. The aim is to integrate sparse tourism information on the Internet, thereby providing more convenient, faster, and more personalized tourism services. Based on the shortcomings of the traditional tourism recommendation system, a deep learning-based classification processing method of tourism product information is proposed. This method uses word embedding in the data preprocessing stage. The Convolutional Neural Network (CNN) is used to process review information of users and tourism service items. The Deep Neural Network (DNN) is used to process the necessary information of users and tourism service items. Also, factorization machine technology is used to learn the interaction between the extracted features to improve the prediction model. The results show that the proposed model can maintain an excellent precision of 64.2% when generating personalized recommendation lists for users. The sensitivity and accuracy of the recommendation list are better than other algorithms. By adding DNN, the word embedding method, and the factorization machine model, the precision is improved by 30%, 33.3%, and 40%, respectively. The model accuracy is the highest with 40 hidden factors, 100 convolutions, and a 100+50 combination hidden layer. Compared with traditional methods, the proposed algorithm can provide users with personalized travel products more accurately in personalized travel recommendations. The results have enriched and developed the theory of tourism service supply chain, providing a reference for constructing a personalized tourism service system.


Sign in / Sign up

Export Citation Format

Share Document