scholarly journals Experimental Analysis of Hyperparameters for Deep Learning-Based Churn Prediction in the Banking Sector

Computation ◽  
2021 ◽  
Vol 9 (3) ◽  
pp. 34
Author(s):  
Edvaldo Domingos ◽  
Blessing Ojeme ◽  
Olawande Daramola
Keyword(s):  
Machine Learning ◽  
Experimental Analysis ◽  
Banking Sector ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Multilayer Perceptrons ◽  
Sigmoid Function ◽  
Churn Prediction ◽  
Heuristic Knowledge ◽  
Selection Of

Until recently, traditional machine learning techniques (TMLTs) such as multilayer perceptrons (MLPs) and support vector machines (SVMs) have been used successfully for churn prediction, but with significant efforts expended on the configuration of the training parameters. The selection of the right training parameters for supervised learning is almost always experimentally determined in an ad hoc manner. Deep neural networks (DNNs) have shown significant predictive strength over TMLTs when used for churn predictions. However, the more complex architecture of DNNs and their capacity to process huge amounts of non-linear input data demand more time and effort to configure the training hyperparameters for DNNs during churn modeling. This makes the process more challenging for inexperienced machine learning practitioners and researchers. So far, limited research has been done to establish the effects of different hyperparameters on the performance of DNNs during churn prediction. There is a lack of empirically derived heuristic knowledge to guide the selection of hyperparameters when DNNs are used for churn modeling. This paper presents an experimental analysis of the effects of different hyperparameters when DNNs are used for churn prediction in the banking sector. The results from three experiments revealed that the deep neural network (DNN) model performed better than the MLP when a rectifier function was used for activation in the hidden layers and a sigmoid function was used in the output layer. The performance of the DNN was better when the batch size was smaller than the size of the test set data, while the RemsProp training algorithm had better accuracy when compared with the stochastic gradient descent (SGD), Adam, AdaGrad, Adadelta, and AdaMax algorithms. The study provides heuristic knowledge that could guide researchers and practitioners in machine learning-based churn prediction from the tabular data for customer relationship management in the banking sector when DNNs are used.

scholarly journals Machine Learning for Sensorless Temperature Estimation of a BLDC Motor

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4655
Author(s):  
Dariusz Czerwinski ◽  
Jakub Gęca ◽  
Krzysztof Kolano
Keyword(s):  
Machine Learning ◽  
Temperature Measurement ◽  
Stochastic Gradient Descent ◽  
Estimation Accuracy ◽  
Coefficient Of Determination ◽  
Percentage Error ◽  
Support Vector ◽  
Bldc Motor ◽  
Temperature Estimation ◽  
Motor Operation

In this article, the authors propose two models for BLDC motor winding temperature estimation using machine learning methods. For the purposes of the research, measurements were made for over 160 h of motor operation, and then, they were preprocessed. The algorithms of linear regression, ElasticNet, stochastic gradient descent regressor, support vector machines, decision trees, and AdaBoost were used for predictive modeling. The ability of the models to generalize was achieved by hyperparameter tuning with the use of cross-validation. The conducted research led to promising results of the winding temperature estimation accuracy. In the case of sensorless temperature prediction (model 1), the mean absolute percentage error MAPE was below 4.5% and the coefficient of determination R2 was above 0.909. In addition, the extension of the model with the temperature measurement on the casing (model 2) allowed reducing the error value to about 1% and increasing R2 to 0.990. The results obtained for the first proposed model show that the overheating protection of the motor can be ensured without direct temperature measurement. In addition, the introduction of a simple casing temperature measurement system allows for an estimation with accuracy suitable for compensating the motor output torque changes related to temperature.

Utilizing Data-Driven Models to Predict Brittleness in Tuscaloosa Marine Shale: A Machine Learning Approach

2021 ◽  
Author(s):  
Jamal Ahmadov
Keyword(s):  
Machine Learning ◽  
Random Forest ◽  
Brittleness Index ◽  
Estimation Methods ◽  
Gradient Boosting ◽  
Average Error ◽  
Support Vector ◽  
Marine Shale ◽  
Effective Manner ◽  
Selection Of

Abstract The Tuscaloosa Marine Shale (TMS) formation is a clay- and liquid-rich emerging shale play across central Louisiana and southwest Mississippi with recoverable resources of 1.5 billion barrels of oil and 4.6 trillion cubic feet of gas. The formation poses numerous challenges due to its high average clay content (50 wt%) and rapidly changing mineralogy, making the selection of fracturing candidates a difficult task. While brittleness plays an important role in screening potential intervals for hydraulic fracturing, typical brittleness estimation methods require the use of geomechanical and mineralogical properties from costly laboratory tests. Machine Learning (ML) can be employed to generate synthetic brittleness logs and therefore, may serve as an inexpensive and fast alternative to the current techniques. In this paper, we propose the use of machine learning to predict the brittleness index of Tuscaloosa Marine Shale from conventional well logs. We trained ML models on a dataset containing conventional and brittleness index logs from 8 wells. The latter were estimated either from geomechanical logs or log-derived mineralogy. Moreover, to ensure mechanical data reliability, dynamic-to-static conversion ratios were applied to Young's modulus and Poisson's ratio. The predictor features included neutron porosity, density and compressional slowness logs to account for the petrophysical and mineralogical character of TMS. The brittleness index was predicted using algorithms such as Linear, Ridge and Lasso Regression, K-Nearest Neighbors, Support Vector Machine (SVM), Decision Tree, Random Forest, AdaBoost and Gradient Boosting. Models were shortlisted based on the Root Mean Square Error (RMSE) value and fine-tuned using the Grid Search method with a specific set of hyperparameters for each model. Overall, Gradient Boosting and Random Forest outperformed other algorithms and showed an average error reduction of 5 %, a normalized RMSE of 0.06 and a R-squared value of 0.89. The Gradient Boosting was chosen to evaluate the test set and successfully predicted the brittleness index with a normalized RMSE of 0.07 and R-squared value of 0.83. This paper presents the practical use of machine learning to evaluate brittleness in a cost and time effective manner and can further provide valuable insights into the optimization of completion in TMS. The proposed ML model can be used as a tool for initial screening of fracturing candidates and selection of fracturing intervals in other clay-rich and heterogeneous shale formations.

scholarly journals A Clinical-Radiomics Nomogram Based on Computed Tomography for Predicting Risk of Local Recurrence After Radiotherapy in Nasopharyngeal Carcinoma

2021 ◽  
Vol 11 ◽  
Author(s):  
Chaohua Zhu ◽  
Huixian Huang ◽  
Xu Liu ◽  
Hao Chen ◽  
Hailan Jiang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Computed Tomography ◽  
Nasopharyngeal Carcinoma ◽  
Local Recurrence ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Stable Model ◽  
Clinical Factors ◽  
Specificity And Sensitivity ◽  
Radiomics Signature

Purpose: We aimed to establish a nomogram model based on computed tomography (CT) imaging radiomic signature and clinical factors to predict the risk of local recurrence in nasopharyngeal carcinoma (NPC) after intensity-modulated radiotherapy (IMRT).Methods: This was a retrospective study consisting of 156 NPC patients treated with IMRT. Radiomics features were extracted from the gross tumor volume for nasopharynx (GTVnx) in pretreatment CT images for patients with or without local recurrence. Discriminative radiomics features were selected after t-test and the least absolute shrinkage and selection operator (LASSO) analysis. The most stable model was obtained to generate radiomics signature (Rad_Score) by using machine learning models including Logistic Regression, K-Nearest neighbor, Naive Bayes, Decision Tree, Stochastic Gradient Descent, Gradient Booting Tree and Linear Support Vector Classification. A nomogram for local recurrence was established based on Rad_Score and clinical factors. The predictive performance of nomogram was evaluated by discrimination ability and calibration ability. Decision Curve Analysis (DCA) was used to evaluate the clinical benefits of the multi-factor nomogram in predicting local recurrence after IMRT.Results: Local recurrence occurred in 42 patients. A total of 1,452 radiomics features were initially extracted and seven stable features finally selected after LASSO analysis were used for machine learning algorithm modeling to generate Rad_Score. The nomogram showed that the greater Rad_Score was associated with the higher risk of local recurrence. The concordance index, specificity and sensitivity in the training cohort were 0.931 (95%CI:0.8765–0.9856), 91.2 and 82.8%, respectively; whereas, in the validation cohort, they were 0.799 (95%CI: 0.6458–0.9515), 79.4, and 69.2%, respectively.Conclusion: The nomogram based on radiomics signature and clinical factors can predict the risk of local recurrence after IMRT in patients with NPC and provide evidence for early clinical intervention.

scholarly journals Modeling of Aboveground Biomass with Landsat 8 OLI and Machine Learning in Temperate Forests

Forests ◽  
2019 ◽  
Vol 11 (1) ◽  
pp. 11
Author(s):  
Pablito M. López-Serrano ◽  
José Luis Cárdenas Domínguez ◽  
José Javier Corral-Rivas ◽  
Enrique Jiménez ◽  
Carlos A. López-Sánchez ◽  
...  
Keyword(s):  
Machine Learning ◽  
Aboveground Biomass ◽  
Goodness Of Fit ◽  
Accurate Estimation ◽  
Support Vector ◽  
Landsat 8 ◽  
Sensing Applications ◽  
Learning Techniques ◽  
Physical Variables ◽  
Selection Of

An accurate estimation of forests’ aboveground biomass (AGB) is required because of its relevance to the carbon cycle, and because of its economic and ecological importance. The selection of appropriate variables from satellite information and physical variables is important for precise AGB prediction mapping. Because of the complex relationships for AGB prediction, non-parametric machine-learning techniques represent potentially useful techniques for AGB estimation, but their use and comparison in forest remote-sensing applications is still relatively limited. The objective of the present study was to evaluate the performance of automatic learning techniques, support vector regression (SVR) and random forest (RF), to predict the observed AGB (from 318 permanent sampling plots) from the Landsat 8 Landsat 8 Operational Land Imager (OLI) sensor, spectral indexes, texture indexes and physical variables the Sierra Madre Occidental in Mexico. The result showed that the best SVR model explained 80% of the total variance (root mean square error (RMSE) = 8.20 Mg ha−1). The variables that best predicted AGB, in order of importance, were the bands that belong to the region of red and near and middle infrared, and the average temperature. The results show that the SVR technique has a good potential for the estimation of the AGB and that the selection of the model hyperparameters has important implications for optimizing the goodness of fit.

scholarly journals CUSTOMER SEGMENTATION IN PRIVATE BANKING SECTOR USING MACHINE LEARNING TECHNIQUES

2013 ◽  
Vol 14 (5) ◽  
pp. 923-939 ◽  
Author(s):  
Ion Smeureanu ◽  
Gheorghe Ruxanda ◽  
Laura Maria Badea
Keyword(s):  
Machine Learning ◽  
Character Recognition ◽  
Foreign Exchange Market ◽  
Banking Sector ◽  
Customer Segmentation ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Learning Techniques ◽  
Fraud Prediction ◽  
Private Banking

Machine learning techniques have proven good performance in classification matters of all kinds: medical diagnosis, character recognition, credit default and fraud prediction, and also foreign exchange market prognosis. Customer segmentation in private banking sector is an important step for profitable business development, enabling financial institutions to address their products and services to homogeneous classes of customers. This paper approaches two of the most popular machine learning techniques, Neural Networks and Support Vector Machines, and describes how each of these perform in a segmentation process.

scholarly journals Credit Risk Assessment using Machine Learning Techniques

2019 ◽  
Vol 9 (1) ◽  
pp. 3482-3486
Keyword(s):  
Machine Learning ◽  
Risk Assessment ◽  
Random Forest ◽  
Credit Risk ◽  
Banking Sector ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Credit Risk Assessment ◽  
Learning Techniques ◽  
Cart Algorithm

Analysis of credit scoring is an effective credit risk assessment technique, which is one of the major research fields in the banking sector. Machine learning has a variety of applications in the banking sector and it has been widely used for data analysis. Modern techniques such as machine learning have provided a self-regulating process to analyze the data using classification techniques. The classification method is a supervised learning process in which the computer learns from the input data provided and makes use of this information to classify the new dataset. This research paper presents a comparison of various machine learning techniques used to evaluate the credit risk. A credit transaction that needs to be accepted or rejected is trained and implemented on the dataset using different machine learning algorithms. The techniques are implemented on the German credit dataset taken from UCI repository which has 1000 instances and 21 attributes, depending on which the transactions are either accepted or rejected. This paper compares algorithms such as Support Vector Network, Neural Network, Logistic Regression, Naive Bayes, Random Forest, and Classification and Regression Trees (CART) algorithm and the results obtained show that Random Forest algorithm was able to predict credit risk with higher accuracy

scholarly journals A Comparative Analysis of Enhanced Machine Learning Algorithms for Smart Grid Stability Prediction

2021 ◽  
Author(s):  
ANKIT GHOSH ◽  
ALOK KOLE
Keyword(s):  
Machine Learning ◽  
Comparative Analysis ◽  
Smart Grid ◽  
Smart Grids ◽  
Machine Learning Algorithms ◽  
Electricity Sector ◽  
Stochastic Gradient Descent ◽  
Energy Output ◽  
Gradient Boosting ◽  
Support Vector

<p>Smart grid is an essential concept in the transformation of the electricity sector into an intelligent digitalized energy network that can deliver optimal energy from the source to the consumers. Smart grids being self-sufficient systems are constructed through the integration of information, telecommunication, and advanced power technologies with the existing electricity systems. Artificial Intelligence (AI) is an important technology driver in smart grids. The application of AI techniques in smart grid is becoming more apparent because the traditional modelling optimization and control techniques have their own limitations. Machine Learning (ML) being a sub-set of AI enables intelligent decision-making and response to sudden changes in the customer energy demands, unexpected disruption of power supply, sudden variations in renewable energy output or any other catastrophic events in a smart grid. This paper presents the comparison among some of the state-of-the-art ML algorithms for predicting smart grid stability. The dataset that has been selected contains results from simulations of smart grid stability. Enhanced ML algorithms such as Support Vector Machine (SVM), Logistic Regression, K-Nearest Neighbour (KNN), Naïve Bayes (NB), Decision Tree (DT), Random Forest (RF), Stochastic Gradient Descent (SGD) classifier, XGBoost and Gradient Boosting classifiers have been implemented to forecast smart grid stability. A comparative analysis among the different ML models has been performed based on the following evaluation metrics such as accuracy, precision, recall, F1-score, AUC-ROC, and AUC-PR curves. The test results that have been obtained have been quite promising with the XGBoost classifier outperforming all the other models with an accuracy of 97.5%, recall of 98.4%, precision of 97.6%, F1-score of 97.9%, AUC-ROC of 99.8% and AUC-PR of 99.9%. </p>

scholarly journals Sentiment classification for employees reviews using regression vector- stochastic gradient descent classifier (RV-SGDC)

2021 ◽  
Vol 7 ◽  
pp. e712
Author(s):  
Babacar Gaye ◽  
Dezheng Zhang ◽  
Aziguli Wulamu
Keyword(s):  
Machine Learning ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Sentiment Classification ◽  
Majority Voting ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Hybrid Architecture ◽  
Accuracy Score ◽  
Learning Models

The satisfaction of employees is very important for any organization to make sufficient progress in production and to achieve its goals. Organizations try to keep their employees satisfied by making their policies according to employees’ demands which help to create a good environment for the collective. For this reason, it is beneficial for organizations to perform staff satisfaction surveys to be analyzed, allowing them to gauge the levels of satisfaction among employees. Sentiment analysis is an approach that can assist in this regard as it categorizes sentiments of reviews into positive and negative results. In this study, we perform experiments for the world’s big six companies and classify their employees’ reviews based on their sentiments. For this, we proposed an approach using lexicon-based and machine learning based techniques. Firstly, we extracted the sentiments of employees from text reviews and labeled the dataset as positive and negative using TextBlob. Then we proposed a hybrid/voting model named Regression Vector-Stochastic Gradient Descent Classifier (RV-SGDC) for sentiment classification. RV-SGDC is a combination of logistic regression, support vector machines, and stochastic gradient descent. We combined these models under a majority voting criteria. We also used other machine learning models in the performance comparison of RV-SGDC. Further, three feature extraction techniques: term frequency-inverse document frequency (TF-IDF), bag of words, and global vectors are used to train learning models. We evaluated the performance of all models in terms of accuracy, precision, recall, and F1 score. The results revealed that RV-SGDC outperforms with a 0.97 accuracy score using the TF-IDF feature due to its hybrid architecture.

scholarly journals APPLICATION OF MACHINE LEARNING ALGORITHMS FOR PROCESSING COMMENTS FROM THE YOUTUBE VIDEO HOSTING UNDER TRAINING VIDEOS

2021 ◽  
pp. 33-42
Author(s):  
L. S. Koriashkina ◽  
H. V. Symonets
Keyword(s):  
Machine Learning ◽  
Gradient Descent ◽  
Russian Language ◽  
Stochastic Gradient ◽  
Machine Learning Algorithms ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Gradient Enhancement

Purpose. Detecting toxic comments on YouTube video hosting under training videos by classifying unstructured text using a combination of machine learning methods. Methodology. To work with the specified type of data, machine learning methods were used for cleaning, normalizing, and presenting textual data in a form acceptable for processing on a computer. Directly to classify comments as “toxic”, we used a logistic regression classifier, a linear support vector classification method without and with a learning method – stochastic gradient descent, a random forest classifier and a gradient enhancement classifier. In order to assess the work of the classifiers, the methods of calculating the matrix of errors, accuracy, completeness and F-measure were used. For a more generalized assessment, a cross-validation method was used. Python programming language. Findings. Based on the assessment indicators, the most optimal methods were selected – support vector machine (Linear SVM), without and with the training method using stochastic gradient descent. The described technologies can be used to analyze the textual comments under any training videos to detect toxic reviews. Also, the approach can be useful for identifying unwanted or even aggressive information on social networks or services where reviews are provided. Originality. It consists in a combination of methods for preprocessing a specific type of text, taking into account such features as the possibility of having a timecode, emoji, links, and the like, as well as in the adaptation of classification methods of machine learning for the analysis of Russian-language comments. Practical value. It is about optimizing (simplification) the comment analysis process. The need for this processing is due to the growing volumes of text data, especially in the field of education through quarantine conditions and the transition to distance learning. The volume of educational Internet content already needs to automate the processing and analysis of feedback, over time this need will only grow.

scholarly journals A Hadoop Based Framework Integrating Machine Learning Classifiers for Anomaly Detection in the Internet of Things

Electronics ◽  
2021 ◽  
Vol 10 (16) ◽  
pp. 1955
Author(s):  
Ikram Sumaiya Thaseen ◽  
Vanitha Mohanraj ◽  
Sakthivel Ramachandran ◽  
Kishore Sanapala ◽  
Sang-Soo Yeo
Keyword(s):  
Machine Learning ◽  
Internet Of Things ◽  
Experimental Analysis ◽  
Parameter Tuning ◽  
Computational Time ◽  
Support Vector ◽  
Learning Approaches ◽  
K Nearest Neighbor ◽  
Machine Learning Classifiers ◽  
Learning Classifiers

In recent years, different variants of the botnet are targeting government, private organizations and there is a crucial need to develop a robust framework for securing the IoT (Internet of Things) network. In this paper, a Hadoop based framework is proposed to identify the malicious IoT traffic using a modified Tomek-link under-sampling integrated with automated Hyper-parameter tuning of machine learning classifiers. The novelty of this paper is to utilize a big data platform for benchmark IoT datasets to minimize computational time. The IoT benchmark datasets are loaded in the Hadoop Distributed File System (HDFS) environment. Three machine learning approaches namely naive Bayes (NB), K-nearest neighbor (KNN), and support vector machine (SVM) are used for categorizing IoT traffic. Artificial immune network optimization is deployed during cross-validation to obtain the best classifier parameters. Experimental analysis is performed on the Hadoop platform. The average accuracy of 99% and 90% is obtained for BoT_IoT and ToN_IoT datasets. The accuracy difference in ToN-IoT dataset is due to the huge number of data samples captured at the edge layer and fog layer. However, in BoT-IoT dataset only 5% of the training and test samples from the complete dataset are considered for experimental analysis as released by the dataset developers. The overall accuracy is improved by 19% in comparison with state-of-the-art techniques. The computational times for the huge datasets are reduced by 3–4 hours through Map Reduce in HDFS.

Sign in / Sign up

Export Citation Format

Share Document