The Footprint of Factorization Models and Their Applications in Collaborative Filtering

2022 ◽  
Vol 40 (4) ◽  
pp. 1-32
Author(s):  
Jinze Wang ◽  
Yongli Ren ◽  
Jie Li ◽  
Ke Deng
Keyword(s):  
Collaborative Filtering ◽  
Gradient Descent ◽  
High Reliability ◽  
Stochastic Gradient Descent ◽  
Decision Making Process ◽  
Intermediate Data ◽  
Benchmark Datasets ◽  
Rich Information ◽  
Prediction Reliability

Factorization models have been successfully applied to the recommendation problems and have significant impact to both academia and industries in the field of Collaborative Filtering ( CF ). However, the intermediate data generated in factorization models’ decision making process (or training process , footprint ) have been overlooked even though they may provide rich information to further improve recommendations. In this article, we introduce the concept of Convergence Pattern, which records how ratings are learned step-by-step in factorization models in the field of CF. We show that the concept of Convergence Patternexists in both the model perspective (e.g., classical Matrix Factorization ( MF ) and deep-learning factorization) and the training (learning) perspective (e.g., stochastic gradient descent ( SGD ), alternating least squares ( ALS ), and Markov Chain Monte Carlo ( MCMC )). By utilizing the Convergence Pattern, we propose a prediction model to estimate the prediction reliability of missing ratings and then improve the quality of recommendations. Two applications have been investigated: (1) how to evaluate the reliability of predicted missing ratings and thus recommend those ratings with high reliability. (2) How to explore the estimated reliability to adjust the predicted ratings to further improve the predication accuracy. Extensive experiments have been conducted on several benchmark datasets on three recommendation tasks: decision-aware recommendation, rating predicted, and Top- N recommendation. The experiment results have verified the effectiveness of the proposed methods in various aspects.

scholarly journals Enhancement of Multilayer Perceptron Model Training Accuracy through the Optimization of Hyperparameters: A Case Study of the Quality Prediction of Injection Molded Parts

2021 ◽  
Author(s):  
Kun-Cheng Ke ◽  
Ming-Shyan Huang
Keyword(s):  
Gradient Descent ◽  
Training Model ◽  
Learning Rate ◽  
Stochastic Gradient Descent ◽  
Activation Functions ◽  
Molded Parts ◽  
Injection Molded ◽  
Testing Accuracy ◽  
Model Training

Abstract Injection molding has been broadly used in the mass production of plastic parts and must meet the requirements of efficiency and quality consistency. Machine learning can effectively predict the quality of injection molded part. However, the performance of machine learning models largely depends on the accuracy of the training. Hyperparameters such as activation functions, momentum, and learning rate are crucial to the accuracy and efficiency of model training. This research further analyzed the influence of hyperparameters on testing accuracy, explored the corresponding optimal learning rate, and provided the optimal training model for predicting the quality of injection molded parts. In this study, stochastic gradient descent (SGD) and stochastic gradient descent with momentum were used to optimize the artificial neural network model. Through optimization of these training model hyperparameters, the width testing accuracy of the injection product improved. The experimental results indicated that in the absence of momentum effects, all five activation functions can achieve more than 90% of the training accuracy with a learning rate of 0.1. Moreover, when optimized with the SGD, the learning rate of the Sigmoid activation function was 0.1, and the testing accuracy reached 95.8%. Although momentum had the least influence on accuracy, it affected the convergence speed of the Sigmoid function, which reduced the number of required learning iterations (82.4% reduction rate). Optimizing hyperparameter settings can improve the accuracy of model testing and markedly reduce training time.

Application and Need-Based Architecture Design of Deep Neural Networks

2020 ◽  
Vol 34 (13) ◽  
pp. 2052014 ◽  
Author(s):  
Soniya ◽  
Sandeep Paul ◽  
Lotika Singh
Keyword(s):  
Genetic Algorithm ◽  
Network Structure ◽  
Gradient Descent ◽  
Stochastic Gradient Descent ◽  
Number Of Layers ◽  
Effective Manner ◽  
Compact Genetic Algorithm ◽  
Benchmark Datasets ◽  
The Cost ◽  
Optimal Set

This paper applies a hybrid evolutionary approach to a convolutional neural network (CNN) and determines the number of layers and filters based on the application and user need. It integrates compact genetic algorithm with stochastic gradient descent (SGD) for simultaneously evolving structure and parameters of the CNN. It defines an effectual string representation for combining structure and parameters of the CNN. The compact genetic algorithm helps in the evolution of network structure by optimizing the number of convolutional layers and number of filters in each convolutional layer. At the same time, an optimal set of weight parameters of the network is obtained using the SGD law. This approach amalgamates exploration in network space by compact genetic algorithm and exploitation in weight space with SGD in an effective manner. The proposed approach also incorporates user-defined parameters in the cost function in an elegant manner which controls the network structure and hence the performance of the network based on the users need. The effectiveness of the proposed approach has been demonstrated on four benchmark datasets, namely MNIST, COIL-100, CIFAR-10 and CIFAR-100. The obtained results clearly demonstrate the potential of the proposed approach by evolving architectures based on the nature of the application and the need of the user.

scholarly journals Deep Learned Quantization-Based Codec for 3D Airborne LiDAR Point Cloud Images

2021 ◽  
Vol 8 ◽  
Author(s):  
A. Christoper Tamilmathi ◽  
P. L. Chithra
Keyword(s):  
Point Cloud ◽  
Gradient Descent ◽  
Mean Squared Error ◽  
Airborne Lidar ◽  
Stochastic Gradient Descent ◽  
Squared Error ◽  
Proposed Model ◽  
Optimization Function ◽  
Execution Speed

This paper introduces a novel deep learned quantization-based coding for 3D Airborne LiDAR (Light detection and ranging) point cloud (pcd) image (DLQCPCD). The raw pcd signals are sampled and transformed by applying the Nyquist signal sampling and Min-max signal transformation techniques, respectively for improving the efficiency of the training process. Then, the transformed signals are feed into the deep learned quantization module for compressing the data. To the best of our knowledge, this proposed DLQCPCD is the first deep learning-based model for 3D airborne LiDAR pcd compression. The functions of Mean Squared Error and Stochastic Gradient Descent optimization function enhance the quality of the decompressed image by 67.01 percent on average, compared to other functions. The model’s efficiency has been validated with established well-known compression techniques such as the 7-Zip, WinRAR, and tensor tucker decomposition algorithm on the three inconsistent airborne datasets. The experimental results show that the proposed model compresses every pcd image into constant 16 Number of Neurons of data and decompresses the image with approximately 160 dB of PSNR value, 174.46 s execution time with 0.6 s execution speed per instruction, and proved that it outperforms the other existing algorithms regarding space and time.

Improving the benefits of sewer condition deterioration modelling through information content analysis

2016 ◽  
Vol 74 (10) ◽  
pp. 2270-2279 ◽  
Author(s):  
M. M. Rokstad ◽  
R. M. Ugarelli
Keyword(s):  
Information Content ◽  
Failure Modes ◽  
Failure Mechanisms ◽  
Structural Condition ◽  
Decision Making Process ◽  
Structural Deterioration ◽  
Functional Defect ◽  
Data Heterogeneity ◽  
Prediction Reliability

As urban sewer infrastructures age, it becomes increasingly important to make effective decisions to maintain the structural condition of the sewers at an acceptable level. To support the decision-making process, the utility manager can apply sewer deterioration models. However, the quality of the decision support from such models is dependent on the accuracy and reliability of the predictions, and previous research has shown that sewer deterioration predictions can be unreliable. In this paper it is shown, by numerical experiment and analysis of information content, how the accuracy of sewer deterioration models is inhibited by data heterogeneity. The data heterogeneity arises when the condition class is used as a response variable, because the condition class is an aggregation of different failure modes, and contains information that does not describe structural deterioration. Based on these findings, the paper suggests changes to be implemented in the condition classification standard, which can mitigate heterogeneity and improve prediction reliability. The suggestions for improvement include distinguishing between structural and functional defect codes, defining new condition metrics better suited for deterioration modelling, and registration of detailed defect codes to allow distinction of different failure mechanisms.

scholarly journals pbSGD: Powered Stochastic Gradient Descent Methods for Accelerated Non-Convex Optimization

2020 ◽  
Author(s):  
Beitong Zhou ◽  
Jun Liu ◽  
Weigao Sun ◽  
Ruijuan Chen ◽  
Claire Tomlin ◽  
...  
Keyword(s):  
Gradient Descent ◽  
Gradient Methods ◽  
Rates Of Convergence ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Power Exponent ◽  
Additional Parameter ◽  
Nonconvex Functions ◽  
Rate Analysis ◽  
Benchmark Datasets

We propose a novel technique for improving the stochastic gradient descent (SGD) method to train deep networks, which we term pbSGD. The proposed pbSGD method simply raises the stochastic gradient to a certain power elementwise during iterations and introduces only one additional parameter, namely, the power exponent (when it equals to 1, pbSGD reduces to SGD). We further propose pbSGD with momentum, which we term pbSGDM. The main results of this paper present comprehensive experiments on popular deep learning models and benchmark datasets. Empirical results show that the proposed pbSGD and pbSGDM obtain faster initial training speed than adaptive gradient methods, comparable generalization ability with SGD, and improved robustness to hyper-parameter selection and vanishing gradients. pbSGD is essentially a gradient modifier via a nonlinear transformation. As such, it is orthogonal and complementary to other techniques for accelerating gradient-based optimization such as learning rate schedules. Finally, we show convergence rate analysis for both pbSGD and pbSGDM methods. The theoretical rates of convergence match the best known theoretical rates of convergence for SGD and SGDM methods on nonconvex functions.

scholarly journals Collaborative Filtering: Data Sparsity Challenges

2018 ◽  
Vol 6 (6) ◽  
Author(s):  
Er.Meenakshi . ◽  
Dr.Satpal .
Keyword(s):  
Collaborative Filtering ◽  
Recommendation System ◽  
Effective Means ◽  
Recommendation Systems ◽  
Decision Making Process ◽  
Huge Amount ◽  
Data Sparsity ◽  
Filtering Technique ◽  
The Cost

Today internet is a place where the huge amount of data is stored, there is need to sift, which create a problem for the internet user, so recommend system solve the problem. A recommendation system is a system that helps a user found the products and content by forecast the user’s rating of each item and showing them the items that they would rate highly. Recommendation systems are everywhere. With online shopping, customer has nearly infinite choices. No one has enough time to try every product for sale. Recommendation systems play an important role to solve the users search the products and content they care about. Recommendation system is a process of filtering the information that deal with information overloaded problems. Recommendation system is important for both user and service provider. It reduces the cost of transaction and selecting item in an online scenario it also improve the quality of decision making process. It is now an effective means for selling their product. So over emphasized of user is not good for recommendation system. To solve the problems of recommendation system like data sparsity we use one of best technique that is collaborative filtering technique.

Comparison of SVM, RF and SGD Methods for Determination of Programmer's Performance Classification Model in Social Media Activities

2020 ◽  
Vol 4 (2) ◽  
pp. 329-335
Author(s):  
Rusydi Umar ◽  
Imam Riadi ◽  
Purwono
Keyword(s):  
Social Media ◽  
Gradient Descent ◽  
Classification Model ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Svm Algorithm ◽  
Vector Machines ◽  
Performance Patterns ◽  
A Company

The failure of most startups in Indonesia is caused by team performance that is not solid and competent. Programmers are an integral profession in a startup team. The development of social media can be used as a strategic tool for recruiting the best programmer candidates in a company. This strategic tool is in the form of an automatic classification system of social media posting from prospective programmers. The classification results are expected to be able to predict the performance patterns of each candidate with a predicate of good or bad performance. The classification method with the best accuracy needs to be chosen in order to get an effective strategic tool so that a comparison of several methods is needed. This study compares classification methods including the Support Vector Machines (SVM) algorithm, Random Forest (RF) and Stochastic Gradient Descent (SGD). The classification results show the percentage of accuracy with k = 10 cross validation for the SVM algorithm reaches 81.3%, RF at 74.4%, and SGD at 80.1% so that the SVM method is chosen as a model of programmer performance classification on social media activities.

Sign in / Sign up

Export Citation Format

Share Document