scholarly journals Online Boosting Algorithm Based on Two-Phase SVM Training

2012 ◽  
Vol 2012 ◽  
pp. 1-8 ◽  
Author(s):  
Vsevolod Yugov ◽  
Itsuo Kumazawa
Keyword(s):  
Gradient Descent ◽  
Training Algorithms ◽  
Two Phase ◽  
Hinge Loss ◽  
Online Setting ◽  
Boosting Algorithm ◽  
The Times ◽  
Mercer Kernels ◽  
Fine Tune ◽  
Kernel Parameters

We describe and analyze a simple and effective two-step online boosting algorithm that allows us to utilize highly effective gradient descent-based methods developed for online SVM training without the need to fine-tune the kernel parameters, and we show its efficiency by several experiments. Our method is similar to AdaBoost in that it trains additional classifiers according to the weights provided by previously trained classifiers, but unlike AdaBoost, we utilize hinge-loss rather than exponential loss and modify algorithm for the online setting, allowing for varying number of classifiers. We show that our theoretical convergence bounds are similar to those of earlier algorithms, while allowing for greater flexibility. Our approach may also easily incorporate additional nonlinearity in form of Mercer kernels, although our experiments show that this is not necessary for most situations. The pre-training of the additional classifiers in our algorithms allows for greater accuracy while reducing the times associated with usual kernel-based approaches. We compare our algorithm to other online training algorithms, and we show, that for most cases with unknown kernel parameters, our algorithm outperforms other algorithms both in runtime and convergence speed.

scholarly journals Optical Recognition of Handwritten Logic Formulas Using Neural Networks

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2761
Author(s):  
Vaios Ampelakiotis ◽  
Isidoros Perikos ◽  
Ioannis Hatzilygeroudis ◽  
George Tsihrintzis
Keyword(s):  
Neural Networks ◽  
Character Recognition ◽  
Gradient Descent ◽  
Feedforward Neural Networks ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Training Algorithms ◽  
Gradient Descent Algorithm ◽  
Two Stages ◽  
And Training

In this paper, we present a handwritten character recognition (HCR) system that aims to recognize first-order logic handwritten formulas and create editable text files of the recognized formulas. Dense feedforward neural networks (NNs) are utilized, and their performance is examined under various training conditions and methods. More specifically, after three training algorithms (backpropagation, resilient propagation and stochastic gradient descent) had been tested, we created and trained an NN with the stochastic gradient descent algorithm, optimized by the Adam update rule, which was proved to be the best, using a trainset of 16,750 handwritten image samples of 28 × 28 each and a testset of 7947 samples. The final accuracy achieved is 90.13%. The general methodology followed consists of two stages: the image processing and the NN design and training. Finally, an application has been created that implements the methodology and automatically recognizes handwritten logic formulas. An interesting feature of the application is that it allows for creating new, user-oriented training sets and parameter settings, and thus new NN models.

scholarly journals Weight Optimization in Recurrent Neural Networks with Hybrid Metaheuristic Cuckoo Search Techniques for Data Classification

2015 ◽  
Vol 2015 ◽  
pp. 1-12 ◽  
Author(s):  
Nazri Mohd Nawi ◽  
Abdullah Khan ◽  
M. Z. Rehman ◽  
Haruna Chiroma ◽  
Tutut Herawan
Keyword(s):  
Gradient Descent ◽  
Search Algorithm ◽  
Error Function ◽  
Back Propagation ◽  
Data Classification ◽  
Cuckoo Search ◽  
Recurrent Network ◽  
Training Algorithms ◽  
Local Minima ◽  
Classification Problems

Recurrent neural network (RNN) has been widely used as a tool in the data classification. This network can be educated with gradient descent back propagation. However, traditional training algorithms have some drawbacks such as slow speed of convergence being not definite to find the global minimum of the error function since gradient descent may get stuck in local minima. As a solution, nature inspired metaheuristic algorithms provide derivative-free solution to optimize complex problems. This paper proposes a new metaheuristic search algorithm called Cuckoo Search (CS) based on Cuckoo bird’s behavior to train Elman recurrent network (ERN) and back propagation Elman recurrent network (BPERN) in achieving fast convergence rate and to avoid local minima problem. The proposed CSERN and CSBPERN algorithms are compared with artificial bee colony using BP algorithm and other hybrid variants algorithms. Specifically, some selected benchmark classification problems are used. The simulation results show that the computational efficiency of ERN and BPERN training process is highly enhanced when coupled with the proposed hybrid method.

scholarly journals Stochastic Subgradient for Large-Scale Support Vector Machine Using the Generalized Pinball Loss Function

Symmetry ◽  
2021 ◽  
Vol 13 (9) ◽  
pp. 1652
Author(s):  
Wanida Panup ◽  
Rabian Wangkeeree
Keyword(s):  
Support Vector Machine ◽  
Loss Function ◽  
Gradient Descent ◽  
Large Scale ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Support Vector ◽  
Hinge Loss ◽  
Gradient Descent Algorithm ◽  
Pinball Loss

In this paper, we propose a stochastic gradient descent algorithm, called stochastic gradient descent method-based generalized pinball support vector machine (SG-GPSVM), to solve data classification problems. This approach was developed by replacing the hinge loss function in the conventional support vector machine (SVM) with a generalized pinball loss function. We show that SG-GPSVM is convergent and that it approximates the conventional generalized pinball support vector machine (GPSVM). Further, the symmetric kernel method was adopted to evaluate the performance of SG-GPSVM as a nonlinear classifier. Our suggested algorithm surpasses existing methods in terms of noise insensitivity, resampling stability, and accuracy for large-scale data scenarios, according to the experimental results.

scholarly journals Local Regularizer Improves Generalization

2020 ◽  
Vol 34 (04) ◽  
pp. 6861-6868 ◽  
Author(s):  
Yikai Zhang ◽  
Hui Qu ◽  
Dimitris Metaxas ◽  
Chao Chen
Keyword(s):  
Deep Learning ◽  
Theoretical Analysis ◽  
Experimental Evidence ◽  
Gradient Descent ◽  
Stochastic Gradient ◽  
Stochastic Gradient Descent ◽  
Training Algorithms ◽  
Training Methods ◽  
Theoretical Understanding ◽  
Better Than

Regularization plays an important role in generalization of deep learning. In this paper, we study the generalization power of an unbiased regularizor for training algorithms in deep learning. We focus on training methods called Locally Regularized Stochastic Gradient Descent (LRSGD). An LRSGD leverages a proximal type penalty in gradient descent steps to regularize SGD in training. We show that by carefully choosing relevant parameters, LRSGD generalizes better than SGD. Our thorough theoretical analysis is supported by experimental evidence. It advances our theoretical understanding of deep learning and provides new perspectives on designing training algorithms. The code is available at https://github.com/huiqu18/LRSGD.

Determination of Engine Misfire Location using Artificial Neural Networks

2019 ◽  
Vol 11 (4) ◽  
Author(s):  
Nouby M. Ghazaly ◽  
Muhammad Abdel-Fattah ◽  
Mostafa M. Makrahy
Keyword(s):  
Gradient Descent ◽  
Recognition Rate ◽  
Network Size ◽  
Training Algorithms ◽  
Spark Ignition Engines ◽  
Average Recognition Rate ◽  
Marquardt Algorithm ◽  
Gradient Descent Algorithm ◽  
Artificial Neural ◽  
Quasi Newton

Misfire in spark-ignition engines is one of the major faults that affect the power produced by the engine and pollute the environment and may cause further engine damage. This paper presents an evaluation of an artificial neural network based performance system through three most popular training algorithms namely Gradient Descent, Lavenberg-Marquadt and Quasi-Newton to determine the misfire location. Misfire is simulated by removing ignition coil to that cylinder namely Cylinder 1,2,3,4 and Cylinders 1 and 2, 1 and 4 and 2 and 3 with three different conditions such as idle, 2000 rpm and 3000 rpm. The results showed that the Quasi-Newton is higher in recognition rate average of 98.19 % but it takes more time to train. The Lavenberg-Marquardt algorithm is also good with an average recognition rate of 96.09 % with the fastest performance than Quasi-Newton. The gradient descent algorithm requires the network size to be more complicated to perform well with least time and high recognition rate.

scholarly journals Predicting the Times of Retweeting in Microblogs

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Li Kuang ◽  
Xiang Tang ◽  
Kehua Guo
Keyword(s):  
Time Series ◽  
Time Distribution ◽  
Distribution Curve ◽  
Information Propagation ◽  
Web Portals ◽  
Traditional Media ◽  
Two Phase ◽  
Two Phases ◽  
The Times ◽  
The Given

Recently, microblog services accelerate the information propagation among peoples, leaving the traditional media like newspaper, TV, forum, blogs, and web portals far behind. Various messages are spread quickly and widely by retweeting in microblogs. In this paper, we take Sina microblog as an example, aiming to predict the possible number of retweets of an original tweet in one month according to the time series distribution of its topnretweets. In order to address the problem, we propose the concept of a tweet’s lifecycle, which is mainly decided by three factors, namely, the response time, the importance of content, and the interval time distribution, and then the given time series distribution curve of its topnretweets is fitted by a two-phase function, so as to predict the number of its retweets in one month. The phases in the function are divided by the lifecycle of the original tweet and different functions are used in the two phases. Experiment results show that our solution can address the problem of predicting the times of retweeting in microblogs with a satisfying precision.

Modeling of the Two-Phase Closed Thermosyphon

1987 ◽  
Vol 109 (3) ◽  
pp. 722-730 ◽  
Author(s):  
J. G. Reed ◽  
C. L. Tien
Keyword(s):  
Steady State ◽  
Analysis Tool ◽  
Governing Equations ◽  
Transfer Coefficients ◽  
Two Phase ◽  
Heat Transfer Coefficients ◽  
Proposed Model ◽  
The Times ◽  
Steady State Solutions ◽  
Liquid And Vapor Phases

A comprehensive model is developed to predict the steady-state and transient performance of the two-phase closed thermosyphon. One-dimensional governing equations for the liquid and vapor phases are developed using available correlations to specify the shear stress and heat transfer coefficients. Steady-state solutions agree well with thermosyphon flooding data from several sources and with film thickness data obtained in the present investigation. While no data are available with which to compare the transient analysis, the results indicate that, for most systems, the governing time scale for system transients is the film residence time, which is typically much longer than the times required for viscous and thermal diffusion through the film. The proposed model offers a versatile and comprehensive analysis tool which is relatively simple.

scholarly journals Experimental Study to Develop Writing skills through Blended Learning in the Times of Internet+

2018 ◽  
Vol 246 ◽  
pp. 03031
Author(s):  
Guangying Cui ◽  
Jiahao Hu
Keyword(s):  
Blended Learning ◽  
Teaching And Learning ◽  
Classroom Interaction ◽  
Writing Skills ◽  
Teacher Observations ◽  
Time Restrictions ◽  
Learning Speed ◽  
Online Setting ◽  
Computer Based ◽  
The Times

Blended learning emphasizes the role of computer‐based technologies in learning how to develop writing skills. Through BL, learners not only control the learning speed, but also do not suffer from the time restrictions of classroom interaction. Teaching and learning take place in both on-campus and online setting and various ways are offered to communicate with each other, either synchronously or asynchronously. Compared to paper teaching documents, the electronic resources are easier for the teachers to keep in order. On the other hand, blended learning leaves a significant role for students’ classroom learning. The effect of BL in developing writing skills has been done through a half-year empirical study. Over the period of half a year, the students’ writing skills have been tracked through interviews, learning contracts and teacher observations. The empirical application illustrates the features and advantages of BL approach.

Robust Loss Functions for Boosting

2007 ◽  
Vol 19 (8) ◽  
pp. 2183-2244 ◽  
Author(s):  
Takafumi Kanamori ◽  
Takashi Takenouchi ◽  
Shinto Eguchi ◽  
Noboru Murata
Keyword(s):  
Numerical Experiments ◽  
Gradient Descent ◽  
Robust Statistics ◽  
Noisy Data ◽  
Loss Functions ◽  
Descent Algorithm ◽  
Gradient Descent Algorithm ◽  
Boosting Algorithms ◽  
Boosting Algorithm ◽  
Decision Boundaries

Boosting is known as a gradient descent algorithm over loss functions. It is often pointed out that the typical boosting algorithm, Adaboost, is highly affected by outliers. In this letter, loss functions for robust boosting are studied. Based on the concept of robust statistics, we propose a transformation of loss functions that makes boosting algorithms robust against extreme outliers. Next, the truncation of loss functions is applied to contamination models that describe the occurrence of mislabels near decision boundaries. Numerical experiments illustrate that the proposed loss functions derived from the contamination models are useful for handling highly noisy data in comparison with other loss functions.

Sign in / Sign up

Export Citation Format

Share Document