Preventing Model Overfitting and Underfitting in Convolutional Neural Networks

2018 ◽  
Vol 10 (4) ◽  
pp. 19-28 ◽  
Author(s):  
Andrei Dmitri Gavrilov ◽  
Alex Jordache ◽  
Maya Vasdani ◽  
Jack Deng
Keyword(s):  
Machine Learning ◽  
Image Recognition ◽  
Recognition Performance ◽  
Square Lattice ◽  
Training Set ◽  
Learning Methods ◽  
Theoretical Understanding ◽  
The Past ◽  
Machine Learning Methods ◽  
The Impact

The current discourse in the machine learning domain converges to the agreement that machine learning methods emerged as some of the most prominent learning and classification approaches over the past decade. The CNN became one of most actively researched and broadly-applied deep machine learning methods. However, the training set has a large influence on the accuracy of a network and it is paramount to create an architecture that supports its maximum training and recognition performance. The problem considered in this article is how to prevent overfitting and underfitting. The deficiencies are addressed by comparing the statistics of CNN image recognition algorithms to the Ising model. Using a two-dimensional square-lattice array, the impact that the learning rate and regularization rate parameters have on the adaptability of CNNs for image classification are evaluated. The obtained results contribute to a better theoretical understanding of a CNN and provide concrete guidance on preventing model overfitting and underfitting when a CNN is applied for image recognition tasks.

scholarly journals The rise and fall of machine learning methods in biomedical research

F1000Research ◽  
2017 ◽  
Vol 6 ◽  
pp. 2012 ◽  
Author(s):  
Hashem Koohy
Keyword(s):  
Machine Learning ◽  
Biomedical Research ◽  
Life Sciences ◽  
Biological Data ◽  
Research Note ◽  
Machine Learning Techniques ◽  
Learning Methods ◽  
The Past ◽  
Machine Learning Methods ◽  
Learning Techniques

In the era of explosion in biological data, machine learning techniques are becoming more popular in life sciences, including biology and medicine. This research note examines the rise and fall of the most commonly used machine learning techniques in life sciences over the past three decades.

scholarly journals Machine Learning Methods Applied for Modeling the Process of Obtaining Bricks Using Silicon-Based Materials

Materials ◽  
2021 ◽  
Vol 14 (23) ◽  
pp. 7232
Author(s):  
Costel Anton ◽  
Silvia Curteanu ◽  
Cătălin Lisa ◽  
Florin Leon
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Raw Materials ◽  
Optimization Procedure ◽  
Exhaust Emission ◽  
Energy Potential ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Emission Changes ◽  
The Impact

Most of the time, industrial brick manufacture facilities are designed and commissioned for a particular type of manufacture mix and a particular type of burning process. Productivity and product quality maintenance and improvement is a challenge for process engineers. Our paper aims at using machine learning methods to evaluate the impact of adding new auxiliary materials on the amount of exhaust emissions. Experimental determinations made in similar conditions enabled us to build a database containing information about 121 brick batches. Various models (artificial neural networks and regression algorithms) were designed to make predictions about exhaust emission changes when auxiliary materials are introduced into the manufacture mix. The best models were feed-forward neural networks with two hidden layers, having MSE < 0.01 and r2 > 0.82 and, as regression model, kNN with error < 0.6. Also, an optimization procedure, including the best models, was developed in order to determine the optimal values for the parameters that assure the minimum quantities for the gas emission. The Pareto front obtained in the multi-objective optimization conducted with grid search method allows the user the chose the most convenient values for the dry product mass, clay, ash and organic raw materials which minimize gas emissions with energy potential.

Towards Behaviour Recognition with Unlabelled Sensor Data

2013 ◽  
pp. 86-110
Author(s):  
Sook-Ling Chua ◽  
Stephen Marsland ◽  
Hans W. Guesgen
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Inverse Problem ◽  
Sensor Data ◽  
Training Set ◽  
Learning Methods ◽  
Machine Learning Methods ◽  
Using Data ◽  
Symbolic Approach ◽  
Behaviour Recognition

The problem of behaviour recognition based on data from sensors is essentially an inverse problem: given a set of sensor observations, identify the sequence of behaviours that gave rise to them. In a smart home, the behaviours are likely to be the standard human behaviours of living, and the observations will depend upon the sensors that the house is equipped with. There are two main approaches to identifying behaviours from the sensor stream. One is to use a symbolic approach, which explicitly models the recognition process. Another is to use a sub-symbolic approach to behaviour recognition, which is the focus in this chapter, using data mining and machine learning methods. While there have been many machine learning methods of identifying behaviours from the sensor stream, they have generally relied upon a labelled dataset, where a person has manually identified their behaviour at each time. This is particularly tedious to do, resulting in relatively small datasets, and is also prone to significant errors as people do not pinpoint the end of one behaviour and commencement of the next correctly. In this chapter, the authors consider methods to deal with unlabelled sensor data for behaviour recognition, and investigate their use. They then consider whether they are best used in isolation, or should be used as preprocessing to provide a training set for a supervised method.

scholarly journals Quantified uncertainties in fission yields from machine learning

2020 ◽  
Vol 242 ◽  
pp. 05003
Author(s):  
A.E. Lovell ◽  
A.T. Mohan ◽  
P. Talou ◽  
M. Chertkov
Keyword(s):  
Machine Learning ◽  
Nuclear Physics ◽  
Experimental Error ◽  
Training Data ◽  
Learning Methods ◽  
Mixture Density ◽  
Machine Learning Methods ◽  
Predicted Values ◽  
The Impact ◽  
Fission Yields

As machine learning methods gain traction in the nuclear physics community, especially those methods that aim to propagate uncertainties to unmeasured quantities, it is important to understand how the uncertainty in the training data coming either from theory or experiment propagates to the uncertainty in the predicted values. Gaussian Processes and Bayesian Neural Networks are being more and more widely used, in particular to extrapolate beyond measured data. However, studies are typically not performed on the impact of the experimental errors on these extrapolated values. In this work, we focus on understanding how uncertainties propagate from input to prediction when using machine learning methods. We use a Mixture Density Network (MDN) to incorporate experimental error into the training of the network and construct uncertainties for the associated predicted quantities. Systematically, we study the effect of the size of the experimental error, both on the reproduced training data and extrapolated predictions for fission yields of actinides.

scholarly journals Smart literature review: a practical topic modelling approach to exploratory literature review

2019 ◽  
Vol 6 (1) ◽  
Author(s):  
Claus Boye Asmussen ◽  
Charles Møller
Keyword(s):  
Machine Learning ◽  
Literature Review ◽  
Latent Dirichlet Allocation ◽  
Topic Model ◽  
Topic Modelling ◽  
Learning Methods ◽  
The Past ◽  
Machine Learning Methods ◽  
Literature Reviews ◽  
Modelling Approach

Abstract Manual exploratory literature reviews should be a thing of the past, as technology and development of machine learning methods have matured. The learning curve for using machine learning methods is rapidly declining, enabling new possibilities for all researchers. A framework is presented on how to use topic modelling on a large collection of papers for an exploratory literature review and how that can be used for a full literature review. The aim of the paper is to enable the use of topic modelling for researchers by presenting a step-by-step framework on a case and sharing a code template. The framework consists of three steps; pre-processing, topic modelling, and post-processing, where the topic model Latent Dirichlet Allocation is used. The framework enables huge amounts of papers to be reviewed in a transparent, reliable, faster, and reproducible way.

scholarly journals Daily streamflow forecasting for Paraíba do Sul river using machine learning methods with hydrologic inputs

2018 ◽  
Author(s):  
Yulia Gorodetskaya ◽  
Leonardo Goliatt Da Fonseca ◽  
Gisele Goulart Tavares ◽  
Celso Bandeira de Melo Ribeiro
Keyword(s):  
Machine Learning ◽  
Lead Times ◽  
Streamflow Forecasting ◽  
Learning Methods ◽  
Daily Streamflow ◽  
Machine Learning Methods ◽  
Strategic Value ◽  
The Impact ◽  
Hydrologic Inputs ◽  
Paraíba Do Sul River

The Paraíba do Sul river flows through the most important industrial region of Brazil and its basin is characterized by conflicts of multiple uses of its water resources. The prediction of its natural flow has strategic value for water management in this basin. This research investigates the applicability of the two machine learning methods (Random Forest and Artificial Neural Networks) for daily streamflow forecasting of the Paraíba do Sul River at lead times of 1-7 days. The impact of fluviometric and pluviometric data from other basin sites on the quality of the forecast is also evaluated.

Sign in / Sign up

Export Citation Format

Share Document