scholarly journals One-Stage Tree: end-to-end tree builder and pruner

2021 ◽  
Author(s):  
Zhuoer Xu ◽  
Guanghui Zhu ◽  
Chunfeng Yuan ◽  
Yihua Huang
Keyword(s):  
Decision Tree ◽  
Greedy Algorithms ◽  
Back Propagation ◽  
Approximate Solutions ◽  
Bilevel Optimization ◽  
Training Data ◽  
One Stage ◽  
Continuous Relaxation ◽  
Np Complete ◽  
End To End

AbstractDecision trees have favorable properties, including interpretability, high computational efficiency, and the ability to learn from little training data. Learning a decision tree is known to be NP-complete. The researchers have proposed many greedy algorithms such as CART to learn approximate solutions. Inspired by the current popular neural networks, soft trees that support end-to-end training with back-propagation have attracted more and more attention. However, existing soft trees either lose the interpretability due to the continuous relaxation or employ the two-stage method of end-to-end building and then pruning. In this paper, we propose One-Stage Tree to build and prune the decision tree jointly through a bilevel optimization problem. Moreover, we leverage the reparameterization trick and proximal iterations to keep the tree discrete during end-to-end training. As a result, One-Stage Tree reduces the performance gap between training and testing and maintains the advantage of interpretability. Extensive experiments demonstrate that the proposed One-Stage Tree outperforms CART and the existing soft trees on classification and regression tasks.

Left ventricular hypertrophy detection system based on electrocardiogram signal

2020 ◽  
Vol 41 (Supplement_2) ◽  
Author(s):  
C.W Liu ◽  
R.H Pan ◽  
Y.L Hu
Keyword(s):  
Left Ventricular Hypertrophy ◽  
Decision Tree ◽  
Ventricular Hypertrophy ◽  
Detection System ◽  
Pearson Correlation ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Left Ventricular ◽  
Heart Beat ◽  
Knowledge Engineer

Abstract Background Left ventricular hypertrophy (LVH) is associated with increased risks of cardiovascular diseases. Electrocardiography (ECG) is generally used to screen LVH in general population and electrocardiographic LVH is further confirmed by transthoracic echocardiography (Echo). Purpose We aimed to establish an ECG LVH detection system that was validated by echo LVH. Methods We collected the data of ECGs and Echo from the previous database. The voltage of R- and S-amplitude in each ECG lead were measured twice by a study assistance blinded to the study design, (artificially measured). Another knowledge engineer analyzed row signals of ECG (the algorithm). We firstly check the correlation of R- and S-amplitude between the artificially measured and the algorythm. ECG LVH is defined by the voltage criteria and Echo LVH is defined by LV mass index >115 g/m2 in men and >95 g/m2 in women. Then we use decision tree, k-means, and back propagation neural network (BPNN) with or without heart beat segmentation to establish a rapid and accurate LVH detection system. The ratio of training set to test set was 7:3. Results The study consisted of a sample size of 953 individuals (90% male) with 173 Echo LVH. The R- and S-amplitude were highly correlated between artificially measured and the algorithm R- and S-amplitude regarding that the Pearson correlation coefficient were >0.9 in each lead (the highest r of 0.997 in RV5 and the lowest r of 0.904 in aVR). Without heart beat segmentation, the accuracy of decision tree, k-means, and BPNN to predict echo LVH were 0.74, 0.73 and 0.51, respectively. With heart beat segmentation, the signal of Echo LVH expanded to 1466, and the accuracy to predict ECG LVH were obviously improved (0.92 for decision tree, 0.96 for k-means, and 0.59 for BPNN). Conclusions Our study showed that machine-learning model by BPNN had the highest accuracy than decision trees and k-means based on ECG R- and S-amplitude signal analyses. Figure 1. Three layers of the decision tree Funding Acknowledgement Type of funding source: None

scholarly journals Rethinking the Random Cropping Data Augmentation Method Used in the Training of CNN-Based SAR Image Ship Detector

2020 ◽  
Vol 13 (1) ◽  
pp. 34
Author(s):  
Rong Yang ◽  
Robert Wang ◽  
Yunkai Deng ◽  
Xiaoxue Jia ◽  
Heng Zhang
Keyword(s):  
Neural Network ◽  
Data Augmentation ◽  
Back Propagation ◽  
Detection Performance ◽  
Training Data ◽  
Sar Image ◽  
Optical Images ◽  
The Neural Network ◽  
Effective Training ◽  
Standard Configuration

The random cropping data augmentation method is widely used to train convolutional neural network (CNN)-based target detectors to detect targets in optical images (e.g., COCO datasets). It can expand the scale of the dataset dozens of times while consuming only a small amount of calculations when training the neural network detector. In addition, random cropping can also greatly enhance the spatial robustness of the model, because it can make the same target appear in different positions of the sample image. Nowadays, random cropping and random flipping have become the standard configuration for those tasks with limited training data, which makes it natural to introduce them into the training of CNN-based synthetic aperture radar (SAR) image ship detectors. However, in this paper, we show that the introduction of traditional random cropping methods directly in the training of the CNN-based SAR image ship detector may generate a lot of noise in the gradient during back propagation, which hurts the detection performance. In order to eliminate the noise in the training gradient, a simple and effective training method based on feature map mask is proposed. Experiments prove that the proposed method can effectively eliminate the gradient noise introduced by random cropping and significantly improve the detection performance under a variety of evaluation indicators without increasing inference cost.

Application of ANN Back-Propagation for Fracture Design Parameters in Extra-Low Cycle Rotating Bending Fatigue

2009 ◽  
Vol 610-613 ◽  
pp. 450-453
Author(s):  
Hong Yan Duan ◽  
You Tang Li ◽  
Jin Zhang ◽  
Gui Ping He
Keyword(s):  
Neural Network ◽  
Neural Network Model ◽  
Back Propagation ◽  
Training Data ◽  
Design Parameters ◽  
Ann Model ◽  
Bending Fatigue ◽  
The Neural Network ◽  
Rotating Bending Fatigue ◽  
Rotating Bending

The fracture problems of ecomaterial (aluminum alloyed cast iron) under extra-low cycle rotating bending fatigue loading were studied using artificial neural networks (ANN) in this paper. The training data were used in the formation of training set of ANN. The ANN model exhibited excellent in results comparison with the experimental results. It was concluded that predicted fracture design parameters by the trained neural network model seem more reasonable compared to approximate methods. It is possible to claim that, ANN is fairly promising prediction technique if properly used. Training ANN model was introduced at first. And then the Training data for the development of the neural network model was obtained from the experiments. The input parameters, notch depth, the presetting deflection and tip radius of the notch, and the output parameters, the cycle times of fracture were used during the network training. The neural network architecture is designed. The ANN model was developed using back propagation architecture with three layers jump connections, where every layer was connected or linked to every previous layer. The number of hidden neurons was determined according to special formula. The performance of system is summarized at last. In order to facilitate the comparisons of predicted values, the error evaluation and mean relative error are obtained. The result show that the training model has good performance, and the experimental data and predicted data from ANN are in good coherence.

scholarly journals High-Performance Adaptive Neurofuzzy Classifier with a Parametric Tuning

2018 ◽  
Vol 155 ◽  
pp. 01037
Author(s):  
Sergey Gorbachev ◽  
Vladimir Syryamkin
Keyword(s):  
Decision Tree ◽  
High Performance ◽  
Fuzzy Inference ◽  
Back Propagation ◽  
Adaptive Algorithms ◽  
Fuzzy Classifier ◽  
Network Adaptation ◽  
Error Back Propagation ◽  
Multicriteria Problems ◽  
Neuro Fuzzy

The article is devoted to research and development of adaptive algorithms for neuro-fuzzy inference when solving multicriteria problems connected with analysis of expert (foresight) data to identify technological breakthroughs and strategic perspectives of scientific, technological and innovative development. The article describes the optimized structuralfunctional scheme of the high-performance adaptive neuro-fuzzy classifier with a logical output, which has such specific features as a block of decision tree-based fuzzy rules and a hybrid algorithm for neural network adaptation of parameters based on the error back-propagation to the root of the decision tree.

scholarly journals COMPARISON OF NAIVE BAYES ALGORITHM AND C.45 ALGORITHM IN CLASSIFICATION OF POOR COMMUNITIES RECEIVING NON CASH FOOD ASSISTANCE IN WANASARI VILLAGE KARAWANG REGENCY

2020 ◽  
Vol 17 (1) ◽  
pp. 37-42
Author(s):  
Yuris Alkhalifi ◽  
Ainun Zumarniansyah ◽  
Rian Ardianto ◽  
Nila Hardi ◽  
Annisa Elfina Augustia
Keyword(s):  
Decision Tree ◽  
Naive Bayes ◽  
Confusion Matrix ◽  
Total Sample ◽  
Naïve Bayes ◽  
Food Assistance ◽  
Training Data ◽  
Naive Bayes Classifier ◽  
Bayes Classifier ◽  
Naïve Bayes Classifier

Non-Cash Food Assistance or Bantuan Pangan Non-Tunai (BPNT) is food assistance from the government given to the Beneficiary Family (KPM) every month through an electronic account mechanism that is used only to buy food at the Electronic Shop Mutual Assistance Joint Business Group Hope Family Program (e-Warong KUBE PKH ) or food traders working with Bank Himbara. In its distribution, BPNT still has problems that occur that are experienced by the village apparatus especially the apparatus of Desa Wanasari on making decisions, which ones are worthy of receiving (poor) and not worthy of receiving (not poor). So one way that helps in making decisions can be done through the concept of data mining. In this study, a comparison of 2 algorithms will be carried out namely Naive Bayes Classifier and Decision Tree C.45. The total sample used is as much as 200 head of household data which will then be divided into 2 parts into validation techniques is 90% training data and 10% test data of the total sample used then the proposed model is made in the RapidMiner application and then evaluated using the Confusion Matrix table to find out the highest level of accuracy from 2 of these methods. The results in this classification indicate that the level of accuracy in the Naive Bayes Classifier method is 98.89% and the accuracy level in the Decision Tree C.45 method is 95.00%. Then the conclusion that in this study the algorithm with the highest level of accuracy is the Naive Bayes Classifier algorithm method with a difference in the accuracy rate of 3.89%.

Simple Convolutional-Based Models: Are They Learning the Task or the Data?

2021 ◽  
pp. 1-17
Author(s):  
Luis Sa-Couto ◽  
Andreas Wichert
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Training Data ◽  
Model Complexity ◽  
Data Sets ◽  
Simple Task ◽  
Data Set ◽  
Knowing That ◽  
Handwritten Digit ◽  
End To End

Abstract Convolutional neural networks (CNNs) evolved from Fukushima's neocognitron model, which is based on the ideas of Hubel and Wiesel about the early stages of the visual cortex. Unlike other branches of neocognitron-based models, the typical CNN is based on end-to-end supervised learning by backpropagation and removes the focus from built-in invariance mechanisms, using pooling not as a way to tolerate small shifts but as a regularization tool that decreases model complexity. These properties of end-to-end supervision and flexibility of structure allow the typical CNN to become highly tuned to the training data, leading to extremely high accuracies on typical visual pattern recognition data sets. However, in this work, we hypothesize that there is a flip side to this capability, a hidden overfitting. More concretely, a supervised, backpropagation based CNN will outperform a neocognitron/map transformation cascade (MTCCXC) when trained and tested inside the same data set. Yet if we take both models trained and test them on the same task but on another data set (without retraining), the overfitting appears. Other neocognitron descendants like the What-Where model go in a different direction. In these models, learning remains unsupervised, but more structure is added to capture invariance to typical changes. Knowing that, we further hypothesize that if we repeat the same experiments with this model, the lack of supervision may make it worse than the typical CNN inside the same data set, but the added structure will make it generalize even better to another one. To put our hypothesis to the test, we choose the simple task of handwritten digit classification and take two well-known data sets of it: MNIST and ETL-1. To try to make the two data sets as similar as possible, we experiment with several types of preprocessing. However, regardless of the type in question, the results align exactly with expectation.

THE STAG OIL FIELD FORMATION EVALUATION: A NEURAL NETWORK APPROACH

1999 ◽  
Vol 39 (1) ◽  
pp. 451 ◽  
Author(s):  
H. Crocker ◽  
C.C. Fung ◽  
K.W. Wong
Keyword(s):  
Neural Network ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Oil Field ◽  
Training Data ◽  
Ann Model ◽  
Neural Network Approach ◽  
Data Set ◽  
Formation Evaluation ◽  
Core Data

The producing M. australis Sandstone of the Stag Oil Field is a bioturbated glauconitic sandstone that is difficult to evaluate using conventional methods. Well log and core data are available for the Stag Field and for the nearby Centaur–1 well. Eight wells have log data; six also have core data.In the past few years artificial intelligence has been applied to formation evaluation. In particular, artificial neural networks (ANN) used to match log and core data have been studied. The ANN approach has been used to analyse the producing Stag Field sands. In this paper, new ways of applying the ANN are reported. Results from simple ANN approach are unsatisfactory. An integrated ANN approach comprising the unsupervised Self-Organising Map (SOM) and the Supervised Back Propagation Neural Network (BPNN) appears to give a more reasonable analysis.In this case study the mineralogical and petrophysical characteristics of a cored well are predicted from the 'training' data set of the other cored wells in the field. The prediction from the ANN model is then used for comparison with the known core data. In this manner, the accuracy of the prediction is determined and a prediction qualifier computed.This new approach to formation evaluation should provide a match between log and core data that may be used to predict the characteristics of a similar uncored interval. Although the results for the Stag Field are satisfactory, further study applying the method to other fields is required.

scholarly journals FD-GAN: Generative Adversarial Networks with Fusion-Discriminator for Single Image Dehazing

2020 ◽  
Vol 34 (07) ◽  
pp. 10729-10736 ◽  
Author(s):  
Yu Dong ◽  
Yihao Liu ◽  
He Zhang ◽  
Shifeng Chen ◽  
Yu Qiao
Keyword(s):  
Training Data ◽  
Training Dataset ◽  
Generative Adversarial Networks ◽  
Single Image ◽  
Image Dehazing ◽  
Atmospheric Scattering ◽  
Color Distortion ◽  
Adversarial Networks ◽  
End To End ◽  
Single Image Dehazing

Recently, convolutional neural networks (CNNs) have achieved great improvements in single image dehazing and attained much attention in research. Most existing learning-based dehazing methods are not fully end-to-end, which still follow the traditional dehazing procedure: first estimate the medium transmission and the atmospheric light, then recover the haze-free image based on the atmospheric scattering model. However, in practice, due to lack of priors and constraints, it is hard to precisely estimate these intermediate parameters. Inaccurate estimation further degrades the performance of dehazing, resulting in artifacts, color distortion and insufficient haze removal. To address this, we propose a fully end-to-end Generative Adversarial Networks with Fusion-discriminator (FD-GAN) for image dehazing. With the proposed Fusion-discriminator which takes frequency information as additional priors, our model can generator more natural and realistic dehazed images with less color distortion and fewer artifacts. Moreover, we synthesize a large-scale training dataset including various indoor and outdoor hazy images to boost the performance and we reveal that for learning-based dehazing methods, the performance is strictly influenced by the training data. Experiments have shown that our method reaches state-of-the-art performance on both public synthetic datasets and real-world images with more visually pleasing dehazed results.

IMPROVING STABILITY OF DECISION TREES

2002 ◽  
Vol 16 (02) ◽  
pp. 145-159 ◽  
Author(s):  
MARK LAST ◽  
ODED MAIMON ◽  
EINAT MINKOV
Keyword(s):  
Decision Tree ◽  
Predictive Accuracy ◽  
Statistical Significance ◽  
Training Data ◽  
Tree Level ◽  
Information Theoretic ◽  
Tree Algorithms ◽  
Tree Models ◽  
The Stability ◽  
Information Theoretic Method

Decision-tree algorithms are known to be unstable: small variations in the training set can result in different trees and different predictions for the same validation examples. Both accuracy and stability can be improved by learning multiple models from bootstrap samples of training data, but the "meta-learner" approach makes the extracted knowledge hardly interpretable. In the following paper, we present the Info-Fuzzy Network (IFN), a novel information-theoretic method for building stable and comprehensible decision-tree models. The stability of the IFN algorithm is ensured by restricting the tree structure to using the same feature for all nodes of the same tree level and by the built-in statistical significance tests. The IFN method is shown empirically to produce more compact and stable models than the "meta-learner" techniques, while preserving a reasonable level of predictive accuracy.

Sign in / Sign up

Export Citation Format

Share Document