scholarly journals Near-Optimal Sparse Neural Trees for Supervised Learning

2021 ◽  
Author(s):  
Tanujit Chakraborty
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Decision Tree ◽  
Classification Tree ◽  
Mathematical Formulation ◽  
Machine Learning Algorithms ◽  
Optimal Decision ◽  
Feed Forward ◽  
Feed Forward Network ◽  
Tree Algorithms

Decision tree algorithms have been among the most popular algorithms for interpretable (transparent) machine learning since the early 1980s. On the other hand, deep learning methods have boosted the capacity of machine learning algorithms and are now being used for non-trivial applications in various applied domains. But training a fully-connected deep feed-forward network by gradient-descent backpropagation is slow and requires arbitrary choices regarding the number of hidden units and layers. In this paper, we propose near-optimal neural regression trees, intending to make it much faster than deep feed-forward networks and for which it is not essential to specify the number of hidden units in the hidden layers of the neural network in advance. The key idea is to construct a decision tree and then simulate the decision tree with a neural network. This work aims to build a mathematical formulation of neural trees and gain the complementary benefits of both sparse optimal decision trees and neural trees. We propose near-optimal sparse neural trees (NSNT) that is shown to be asymptotically consistent and robust in nature. Additionally, the proposed NSNT model obtain a fast rate of convergence which is near-optimal up to some logarithmic factor. We comprehensively benchmark the proposed method on a sample of 80 datasets (40 classification datasets and 40 regression datasets) from the UCI machine learning repository. We establish that the proposed method is likely to outperform the current state-of-the-art methods (random forest, XGBoost, optimal classification tree, and near-optimal nonlinear trees) for the majority of the datasets.

scholarly journals Near-optimal Sparse Neural Trees

2021 ◽  
Author(s):  
Tanujit Chakraborty ◽  
Tanmoy Chakraborty
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Decision Tree ◽  
Classification Tree ◽  
Mathematical Formulation ◽  
Machine Learning Algorithms ◽  
Optimal Decision ◽  
Feed Forward ◽  
Feed Forward Network ◽  
Tree Algorithms

Decision tree algorithms have been among the most popular algorithms for interpretable (transparent) machine learning since the early 1980s. On the other hand, deep learning methods have boosted the capacity of machine learning algorithms and are now being used for non-trivial applications in various applied domains. But training a fully-connected deep feed-forward network by gradient-descent backpropagation is slow and requires arbitrary choices regarding the number of hidden units and layers. In this paper, we propose near-optimal neural regression trees, intending to make it much faster than deep feed-forward networks and for which it is not essential to specify the number of hidden units in the hidden layers of the neural network in advance. The key idea is to construct a decision tree and then simulate the decision tree with a neural network. This work aims to build a mathematical formulation of neural trees and gain the complementary benefits of both sparse optimal decision trees and neural trees. We propose near-optimal sparse neural trees (NSNT) that is shown to be asymptotically consistent and robust in nature. Additionally, the proposed NSNT model obtain a fast rate of convergence which is near-optimal upto some logarithmic factor. We comprehensively benchmark the proposed method on a sample of 80 datasets (40 classification datasets and 40 regression datasets) from the UCI machine learning repository. We establish that the proposed method is likely to outperform the current state-of-the-art methods (random forest, XGBoost, optimal classification tree, and near-optimal nonlinear trees) for the majority of the datasets.

scholarly journals COLOR RECOGNITION WEARABLE DEVICE USING MACHINE LEARNING FOR VISUALY IMPAIRED PERSON

2018 ◽  
Vol 19 (2) ◽  
pp. 213-220
Author(s):  
NIK NUR WAHIDAH NIK HASHIM ◽  
TAREK MOHAMED BOLAD ◽  
Noor Hazrin Hany Mohamad Hanif
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Computer Vision ◽  
Decision Tree ◽  
Open Source ◽  
Machine Learning Algorithms ◽  
Raspberry Pi ◽  
Tree Algorithms ◽  
Impaired Person ◽  
Color Conversion

ABSTRACT: Recognizing colors is a concerning problem for the visually impaired person. The aim of this paper is to convert colors to sound and vibration in order to allow fully/partially blind people to have a ‘feeling’ or better understanding of the different colors around them. The idea is to develop a device that can produce vibration for colors. The user can also hear the name of the color along with ‘feeling’ the vibration. Two algorithms were used to distinguish between colors;  RGB to HSV color conversion in comparison with neural network and decision tree based machine learning algorithms. Raspberry Pi 3 with Open Source Computer Vision (OpenCV) software handles the image processing. The results for RGB to HSV color conversion algorithm were performed with 3 different colors (red, blue, and green). In addition, neural network and decision tree algorithms were trained and tested with eight colors (red, green, blue, orange, yellow, purple, white, and black) for the conversion to sound and vibration. Neural network and decision tree algorithms achieved higher accuracy and efficiency for the majority of tested colors as compared to the RGB to HSV. ABSTRAK: Membezakan antara warna adalah masalah yang merunsingkan terutamanya kepada mereka yang buta, separa buta atau buta warna. Tujuan kertas penyelidikan ini adalah untuk membentangkan kaedah menukar warna kepada bunyi dan getaran bagi membolehkan individu yang buta, separa buta atau buta warna untuk mendapat ‘perasaan’ atau pemahaman yang lebih baik tentang warna-warna yang berbeza disekeliling mereka. Idea yang dicadangkan adalah dengan membuat sebuah alat yang dapat menghasilkan getaran bagi setiap warna yang berbeza. Disamping itu, pengguna juga dapat mendengar nama warna tersebut. Algoritma yang digunakan untuk membezakan antara warna adalah penukaran warna RGB kepada HSV yang dibandingkan dengan rangkaian neural dan algoritma pembelajaran mesin berasaskan pokok keputusan. Raspberry Pi 3 bersaiz kad kredit dengan perisian Open Source Computer Vision (OpenCV) mengendalikan pemprosesan imej. Hasil algoritma penukaran warna RGB kepada HSV telah dilakukan dengan tiga warna yang berbeza (merah, biru, dan hijau). Tambahan pula, hasil rangkaian neural dan algoritma berasaskan pokok keputusan telah dilakukan dengan lapan warna (merah, hijau, biru, oren, kuning, ungu, putih, dan hitam) dengan penukaran warna tersebut kepada bunyi dan getaran. Selain itu, hasil rangkaian neural dan algoritma berasaskan pokok keputusan mencapai hasil dapatan yang baik dengan ketepatan dan kecekapan yang tinggi bagi kebanyakan warna yang diuji berbanding RGB kepada HSV.

scholarly journals Deep Convolutional Neural Network Feed Forward and Back Prop a gation (DCNN F BP) Algorithm f or Predicting Heart Disease u sing Internet o f Things

2021 ◽  
Vol 11 (1) ◽  
pp. 283-287
Author(s):  
Saranya N ◽  
◽  
Kavi Priya S ◽  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Heart Disease ◽  
Convolutional Neural Network ◽  
Back Propagation ◽  
High Volume ◽  
Deep Convolutional Neural Network ◽  
Machine Learning Algorithms ◽  
Batch Size ◽  
Feed Forward

In recent years, due to the increasing amounts of data gathered from the medical area, the Internet of Things are majorly developed. But the data gathered are of high volume, velocity, and variety. In the proposed work the heart disease is predicted using wearable devices. To analyze the data efficiently and effectively, Deep Canonical Neural Network Feed-Forward and Back Propagation (DCNN-FBP) algorithm is used. The data are gathered from wearable gadgets and preprocessed by employing normalization. The processed features are analyzed using a deep convolutional neural network. The DCNN-FBP algorithm is exercised by applying forward and backward propagation algorithm. Batch size, epochs, learning rate, activation function, and optimizer are the parameters used in DCNN-FBP. The datasets are taken from the UCI machine learning repository. The performance measures such as accuracy, specificity, sensitivity, and precision are used to validate the performance. From the results, the model attains 89% accuracy. Finally, the outcomes are juxtaposed with the traditional machine learning algorithms to illustrate that the DCNN-FBP model attained higher accuracy.

scholarly journals An Attempt to Use Machine Learning Algorithms to Estimate the Rockburst Hazard in Underground Excavations of Hard Coal Mine

Energies ◽  
2021 ◽  
Vol 14 (21) ◽  
pp. 6928
Author(s):  
Łukasz Wojtecki ◽  
Sebastian Iwaszenko ◽  
Derek B. Apel ◽  
Tomasz Cichy
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Artificial Neural Network ◽  
Rock Mass ◽  
Decision Tree ◽  
Risk Prediction ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Hard Coal ◽  
Underground Excavations

Rockburst is a dynamic rock mass failure occurring during underground mining under unfavorable stress conditions. The rockburst phenomenon concerns openings in different rocks and is generally correlated with high stress in the rock mass. As a result of rockburst, underground excavations lose their functionality, the infrastructure is damaged, and the working conditions become unsafe. Assessing rockburst hazards in underground excavations becomes particularly important with the increasing mining depth and the mining-induced stresses. Nowadays, rockburst risk prediction is based mainly on various indicators. However, some attempts have been made to apply machine learning algorithms for this purpose. For this article, we employed an extensive range of machine learning algorithms, e.g., an artificial neural network, decision tree, random forest, and gradient boosting, to estimate the rockburst risk in galleries in one of the deep hard coal mines in the Upper Silesian Coal Basin, Poland. With the use of these algorithms, we proposed rockburst risk prediction models. Neural network and decision tree models were most effective in assessing whether a rockburst occurred in an analyzed case, taking into account the average value of the recall parameter. In three randomly selected datasets, the artificial neural network models were able to identify all of the rockbursts.

scholarly journals Static and Dynamic Activity Detection with Ambient Sensors in Smart Spaces

Sensors ◽  
2019 ◽  
Vol 19 (4) ◽  
pp. 804 ◽  
Author(s):  
Sagar Shelke ◽  
Baris Aksanli
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Real Time ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Connected Components ◽  
Activity Detection ◽  
Smart Space ◽  
Smart Spaces ◽  
Feed Forward

Convergence of Machine Learning, Internet of Things, and computationally powerful single-board computers has boosted research and implementation of smart spaces. Smart spaces make predictions based on historical data to enhance user experience. In this paper, we present a low-cost, low-energy smart space implementation to detect static and dynamic human activities that require simple motions. We use low-resolution (4 × 16) and non-intrusive thermal sensors to collect data. We train six machine learning algorithms, namely logistic regression, naive Bayes, support vector machine, decision tree, random forest and artificial neural network (vanilla feed-forward) on the dataset collected in our lab. Our experiments reveal a very high static activity detection rate with all algorithms, where the feed-forward neural network method gives the best accuracy of 99.96%. We also show how data collection methods and sensor placement plays an important role in the resulting accuracy of different machine learning algorithms. To detect dynamic activities in real time, we use cross-correlation and connected components of thermal images. Our smart space implementation, with its real-time properties, can be used in various domains and applications, such as conference room automation, elderly health-care, etc.

scholarly journals A new ML-based approach to enhance student engagement in online environment

PLoS ONE ◽  
2021 ◽  
Vol 16 (11) ◽  
pp. e0258788
Author(s):  
Sarra Ayouni ◽  
Fahima Hajjej ◽  
Mohamed Maddeh ◽  
Shaha Al-Otaibi
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Artificial Neural Network ◽  
Support Vector Machine ◽  
Student Engagement ◽  
Decision Tree ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Support Vector ◽  
Online Environment

The educational research is increasingly emphasizing the potential of student engagement and its impact on performance, retention and persistence. This construct has emerged as an important paradigm in the higher education field for many decades. However, evaluating and predicting the student’s engagement level in an online environment remains a challenge. The purpose of this study is to suggest an intelligent predictive system that predicts the student’s engagement level and then provides the students with feedback to enhance their motivation and dedication. Three categories of students are defined depending on their engagement level (Not Engaged, Passively Engaged, and Actively Engaged). We applied three different machine-learning algorithms, namely Decision Tree, Support Vector Machine and Artificial Neural Network, to students’ activities recorded in Learning Management System reports. The results demonstrate that machine learning algorithms could predict the student’s engagement level. In addition, according to the performance metrics of the different algorithms, the Artificial Neural Network has a greater accuracy rate (85%) compared to the Support Vector Machine (80%) and Decision Tree (75%) classification techniques. Based on these results, the intelligent predictive system sends feedback to the students and alerts the instructor once a student’s engagement level decreases. The instructor can identify the students’ difficulties during the course and motivate them through e-mail reminders, course messages, or scheduling an online meeting.

scholarly journals Examination of Assorted Social Engineering Attack by Different Types of Machine Learning Algorithms

2021 ◽  
Vol 10 (7) ◽  
pp. 56-60
Author(s):  
Amal Alhamad ◽  
Dalal Aldablan ◽  
Raghad Albahlal
Keyword(s):  
Machine Learning ◽  
Social Network ◽  
Decision Tree ◽  
Learning Algorithms ◽  
Social Engineering ◽  
Machine Learning Algorithms ◽  
Privacy And Security ◽  
Different Types ◽  
Tree Algorithms

The most powerful attack on the systems is Social Engineering Attack because of this attack deals with Psychology so that there is no hardware or software can prevent it or even can defend it and hence people need to be trained to defend against it.[1] Social engineering is mostly done by phone or email. In this research, which is based on previous research we have conducted, the aim of it was of it was to highlight the different social engineering attacks and how they can prevent in social network because social engineering is one of the biggest problems in social network, a concern the privacy and security. This project is using a set of data then analysis it uses the Weka tool, to defend against these attacks we have evaluated three decision tree algorithms, RandomForest, REPTree and RandomTree. It was also related to an J48 algorithm, On the contrary, here contains a complete overview of social engineering attacks, also more than one algorithm was searched.

scholarly journals Towards machine learning analytics for jet substructure

2020 ◽  
Vol 2020 (9) ◽  
Author(s):  
Gregor Kasieczka ◽  
Simone Marzani ◽  
Gregory Soyez ◽  
Giovanni Stagnitto
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Particle Physics ◽  
Expert Knowledge ◽  
Binary Classification ◽  
Rapid Development ◽  
Machine Learning Algorithms ◽  
Optimal Decision ◽  
Jet Substructure ◽  
Discrimination Power

Abstract The past few years have seen a rapid development of machine-learning algorithms. While surely augmenting performance, these complex tools are often treated as black-boxes and may impair our understanding of the physical processes under study. The aim of this paper is to move a first step into the direction of applying expert-knowledge in particle physics to calculate the optimal decision function and test whether it is achieved by standard training, thus making the aforementioned black-box more transparent. In particular, we consider the binary classification problem of discriminating quark-initiated jets from gluon-initiated ones. We construct a new version of the widely used N-subjettiness, which features a simpler theoretical behaviour than the original one, while maintaining, if not exceeding, the discrimination power. We input these new observables to the simplest possible neural network, i.e. the one made by a single neuron, or perceptron, and we analytically study the network behaviour at leading logarithmic accuracy. We are able to determine under which circumstances the perceptron achieves optimal performance. We also compare our analytic findings to an actual implementation of a perceptron and to a more realistic neural network and find very good agreement.

scholarly journals Application of Data Mining Algorithms for Dementia in People with HIV/AIDS

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Luana Ibiapina Cordeiro Calíope Pinheiro ◽  
Maria Lúcia Duarte Pereira ◽  
Marcial Porto Fernandez ◽  
Francisco Mardônio Vieira Filho ◽  
Wilson Jorge Correia Pinto de Abreu ◽  
...  
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Data Mining ◽  
Logistic Regression ◽  
Random Forest ◽  
Decision Tree ◽  
Learning Algorithms ◽  
Principal Component ◽  
Machine Learning Algorithms ◽  
Hiv Aids

Dementia interferes with the individual’s motor, behavioural, and intellectual functions, causing him to be unable to perform instrumental activities of daily living. This study is aimed at identifying the best performing algorithm and the most relevant characteristics to categorise individuals with HIV/AIDS at high risk of dementia from the application of data mining. Principal component analysis (PCA) algorithm was used and tested comparatively between the following machine learning algorithms: logistic regression, decision tree, neural network, KNN, and random forest. The database used for this study was built from the data collection of 270 individuals infected with HIV/AIDS and followed up at the outpatient clinic of a reference hospital for infectious and parasitic diseases in the State of Ceará, Brazil, from January to April 2019. Also, the performance of the algorithms was analysed for the 104 characteristics available in the database; then, with the reduction of dimensionality, there was an improvement in the quality of the machine learning algorithms and identified that during the tests, even losing about 30% of the variation. Besides, when considering only 23 characteristics, the precision of the algorithms was 86% in random forest, 56% logistic regression, 68% decision tree, 60% KNN, and 59% neural network. The random forest algorithm proved to be more effective than the others, obtaining 84% precision and 86% accuracy.

scholarly journals Index split decision tree and compositional deep neural network for text categorization

2017 ◽  
Vol 7 (1.1) ◽  
pp. 449
Author(s):  
N Ravikumar ◽  
Dr P. Tamil Selvan
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Decision Tree ◽  
Text Categorization ◽  
Deep Neural Network ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Decision Tree Model ◽  
Tree Model ◽  
The Impact

Text categorization with machine learning algorithms generally reckons to possess horizontal set of classes. Several advanced machine learning algorithms have been designed in the past few decades. With the growing research work for text categorization, it has become important to categorize the research outcome and provide the learners with an effective machine learning method, a framework called, Hierarchical Decision Tree and Deep Neural Network (HDT-DNN).It investigates machine learning algorithms to create horizontal set of classes and it is used for classification of text. With this objective, a novel and efficient text categorization framework based on decision tree model is used in order to categorize text according to superior and subordinate level. The text to be categorized is presented in the form of a tree with parent text category being superior to all. The intermediate level represents the text that is both superior and subordinate. Then Deep Neural Network model is presented initiating compositional model, where the text has to be categorized, as a layered integration of primitives from the constructed decision tree model. The extra layers enable composition of features from lower layers, potentially modeling complex text with fewer units than a similarly carried out shallow network producing hierarchical classification. The significance of the impact of HDT-DNN framework is evaluated through empirical study. Extensive experiments are carried out and the performance of HDT-DNN framework is evaluated and compared with existing state-of-art methods using parameters such as precision, classification accuracy, classification time, with respect to varied number of features and document size.

Sign in / Sign up

Export Citation Format

Share Document