scholarly journals Parametric Versus Semi and Nonparametric Regression Models

2021 ◽  
Vol 10 (2) ◽  
pp. 90
Author(s):  
Hamdy F. F. Mahmoud
Keyword(s):  
Nonparametric Regression ◽  
Robust Estimation ◽  
Regression Models ◽  
Linear Models ◽  
Kernel Smoothing ◽  
Estimation Method ◽  
Parametric Models ◽  
Estimation Methods ◽  
Explanatory Variables ◽  
Local Linear

There are three common types of regression models: parametric, semiparametric and nonparametric regression. The model should be used to fit the real data depends on how much information is available about the form of the relationship between the response variable and explanatory variables, and the random error distribution that is assumed. Researchers need to be familiar with each modeling approach requirements. In this paper, differences between these models, common estimation methods, robust estimation, and applications are introduced. For parametric models, there are many known methods of estimation, such as least squares and maximum likelihood methods which are extensively studied but they require strong assumptions. On the other hand, nonparametric regression models are free of assumptions regarding the form of the response-explanatory variables relationships but estimation methods, such as kernel and spline smoothing are computationally expensive and smoothing parameters need to be obtained. For kernel smoothing there two common estimators: local constant and local linear smoothing methods. In terms of bias, especially at the boundaries of the data range, local linear is better than local constant estimator.  Robust estimation methods for linear models are well studied, however the robust estimation methods in nonparametric regression methods are limited. A robust estimation method for the semiparametric and nonparametric regression models is introduced.

Several Aspects of Nonparametric Prediction of Nonlinear Time Series

2019 ◽  
Vol 65 (1) ◽  
pp. 7-24
Author(s):  
Witold Orzeszko
Keyword(s):  
Time Series ◽  
Nonparametric Regression ◽  
Kernel Smoothing ◽  
Kernel Estimator ◽  
Monte Carlo Study ◽  
Nonlinear Time Series ◽  
Fixed Number ◽  
Parametric Models ◽  
Parametric Approach ◽  
Local Linear

Nonparametric regression is an alternative to the parametric approach, which consists of applying parametric models, i.e. models of the certain functional form with a fixed number of parameters. As opposed to the parametric approach, nonparametric models have a general form, which can be approximated increasingly precisely when the sample size grows. Hereby they do not impose such restricted assumptions about the form of the modelling dependencies and in consequence, they are more flexible and let the data speak for themselves. That is why they are a promising tool for forecasting, especially in case of nonlinear time series. One of the most popular nonparametric regression method is the Nadaraya- Watson kernel smoothing. Nowadays, there are a number of variations of this method, like the local-linear kernel estimator, which combines the local linear approximation and the kernel estimator. In the paper a Monte Carlo study is conducted in order to assess the usefulness of the kernel smoothers to nonlinear time series forecasting and to compare them with the other techniques of forecasting.

Robust estimation of error scale in nonparametric regression models

2008 ◽  
Vol 138 (10) ◽  
pp. 3200-3216 ◽  
Author(s):  
Isabella Rodica Ghement ◽  
Marcelo Ruiz ◽  
Ruben Zamar
Keyword(s):  
Nonparametric Regression ◽  
Robust Estimation ◽  
Regression Models ◽  
Estimation Of Error

Lower Bound on the Accuracy of Parameter Estimation Methods for Linear Sensorimotor Synchronization Models

2015 ◽  
Vol 3 (1-2) ◽  
pp. 32-51 ◽  
Author(s):  
Nori Jacoby ◽  
Peter E. Keller ◽  
Bruno H. Repp ◽  
Merav Ahissar ◽  
Naftali Tishby
Keyword(s):  
Parameter Estimation ◽  
Lower Bound ◽  
Computational Models ◽  
Linear Models ◽  
Estimation Error ◽  
Explanatory Power ◽  
Estimation Method ◽  
Sensorimotor Synchronization ◽  
Estimation Methods ◽  
Unbiased Estimation

The mechanisms that support sensorimotor synchronization — that is, the temporal coordination of movement with an external rhythm — are often investigated using linear computational models. The main method used for estimating the parameters of this type of model was established in the seminal work of Vorberg and Schulze (2002), and is based on fitting the model to the observed auto-covariance function of asynchronies between movements and pacing events. Vorberg and Schulze also identified the problem of parameter interdependence, namely, that different sets of parameters might yield almost identical fits, and therefore the estimation method cannot determine the parameters uniquely. This problem results in a large estimation error and bias, thereby limiting the explanatory power of existing linear models of sensorimotor synchronization. We present a mathematical analysis of the parameter interdependence problem. By applying the Cramér–Rao lower bound, a general lower bound limiting the accuracy of any parameter estimation procedure, we prove that the mathematical structure of the linear models used in the literature determines that this problem cannot be resolved by any unbiased estimation method without adopting further assumptions. We then show that adding a simple and empirically justified constraint on the parameter space — assuming a relationship between the variances of the noise terms in the model — resolves the problem. In a follow-up paper in this volume, we present a novel estimation technique that uses this constraint in conjunction with matrix algebra to reliably estimate the parameters of almost all linear models used in the literature.

scholarly journals Moments estimators and omnibus chi-square tests for some usual probability laws

2021 ◽  
Vol 16 (4) ◽  
pp. 3061-3094
Author(s):  
Gorgui Gning ◽  
Aladji Babacar Niang ◽  
Modou Ngom ◽  
Gane Lo
Keyword(s):  
Statistical Tests ◽  
Estimation Method ◽  
Parametric Models ◽  
Likelihood Method ◽  
Estimation Methods ◽  
Chi Square ◽  
Moment Estimation ◽  
Chi Square Test ◽  
Gamma Law ◽  
The Moment

For many probability laws, in parametric models, the estimation of the parameters can be done in the frame of the maximum likelihood method, or in the frame of moment estimation methods, or by using the plug-in method, etc. Usually, for estimating more than one parameter, the same frame is used. We focus on the moment estimation method in this paper. We use the instrumental tool of the functional empirical process (fep) in Lo (2016) to show how it is practical to derive, almost algebraically, the joint distribution Gaussian law and to derive omnibus chi-square asymptotic laws from it. We choose four distributions to illustrate the method (Gamma law, beta law, Uniform law and Fisher law) and completely describe the asymptotic laws of the moment estimators whenever possible. Simulations studies are performed to investigate for each case the smallest sizes for which the obtained statistical tests are recommendable. Generally, the omnibus chi-square test proposed here work fine with sample sizes around fifty.

scholarly journals Robust Estimation methods of Generalized Exponential Distribution with Outliers

2020 ◽  
pp. 545-559
Author(s):  
Hisham Mohamed Almongy ◽  
Ehab M. Almetwally
Keyword(s):  
Robust Estimation ◽  
Estimation Method ◽  
Likelihood Estimation ◽  
Point Estimation ◽  
Least Square ◽  
Estimation Methods ◽  
Monte Carlo Simulation Study ◽  
Least Absolute Deviations ◽  
Complete Dataset ◽  
M Estimation

This paper discussed robust estimation for point estimation of the shape and scale parameters for generalized exponential (GE) distribution using a complete dataset in the presence of various percentages of outliers. In the case of outliers, it is known that classical methods such as maximum likelihood estimation (MLE), least square (LS) and maximum product spacing (MPS) in case of outliers cannot reach the best estimator. To confirm this fact, these classical methods were applied to the data of this study and compared with non-classical estimation methods. The non-classical (Robust) methods such as least absolute deviations (LAD), and M-estimation (using M. Huber (MH) weight and M. Bisquare (MB) weight) had been introduced to obtain the best estimation method for the parameters of the GE distribution. The comparison was done numerically by using the Monte Carlo simulation study. The two real datasets application confirmed that the M-estimation method is very much suitable for estimating the GE parameters. We concluded that the M-estimation method using Huber object function is a suitable estimation method in estimating the parameters of the GE distribution for a complete dataset in the presence of various percentages of outliers.

CHOOSING A METHOD FOR ESTIMATING THE PARAMETERS OF LINEAR REGRESSION BASED ON IDENTIFICATION OF ANOMALOUS OBSERVATIONS

2021 ◽  
pp. 24-29
Author(s):  
С.И. Носков
Keyword(s):  
Parameter Estimation ◽  
Least Squares ◽  
Robust Estimation ◽  
Regression Models ◽  
Regression Line ◽  
Estimation Method ◽  
Model Parameters ◽  
Method Of Least Squares ◽  
Sharp Drop ◽  
Modulus Method

Описываются свойства методов оценивания параметров регрессионных моделей - наименьших квадратов, модулей, антиробастного, а также их применения для решения конкретных практических проблем. При этом метод наименьших модулей не реагирует на аномальные наблюдения выборки, метод антиробастного оценивания сильно отклоняет линию регрессии в их направлении, метод наименьших квадратов занимает промежуточное положение. Показано, что если целью построения модели является проведение на ее основе многовариантных прогнозных расчетов значений зависимой переменной, то выбор метода численной идентификации параметров модели следует производить на основе анализа характера выбросов. Если есть основания полагать, что подобные им ситуации могут иметь место в будущем, следует выбрать метод антиробастного оценивания, в противном же случае - метод наименьших модулей. Построена регрессионная модель грузооборота Красноярской железной дороги на основе применения всех трех методов оценивания параметров. Проведен анализ причин, имеющих место в 2010 году в ситуации резкого падения величины грузооборота, которая вполне может характеризоваться как аномальное наблюдение в данных. Сделаны рекомендации по выбору метода оценивания параметров в этом случае The article describes the properties of methods for estimating the parameters of regression models - least squares, moduli, anti-robust - as well as their application for solving specific practical problems. At the same time, the method of least modules does not respond to anomalous observations of the sample, the method of anti-robust estimation strongly deviates the regression line in their direction, the method of least squares occupies an intermediate position. I show that if the purpose of constructing a model is to carry out multivariate predictive calculations of the values of the dependent variable on its basis, then the choice of a method for the numerical identification of model parameters should be based on an analysis of the nature of emissions. If there is a reason to believe that similar situations may occur in the future, the anti-robust estimation method should be chosen, otherwise - the least modulus method. I built a regression model of the freight turnover of the Krasnoyarsk railway on the basis of the application of all three methods of parameter estimation. I carried out the analysis of the reasons for the situation of a sharp drop in the value of cargo turnover in 2010, which may well be characterized as anomalous observation in the data. I give recommendations on the choice of the parameter estimation method in this case

Robust Estimation Peculiarity of Robust Estimation Methods in Observations to Obey Generalized Gaussian Distribution

2014 ◽  
Vol 556-562 ◽  
pp. 4380-4385
Author(s):  
Peng Fei Xing ◽  
Yong Hui Ge ◽  
Yan Li
Keyword(s):  
Robust Estimation ◽  
Gaussian Distribution ◽  
Estimation Method ◽  
Simulation Method ◽  
Estimation Methods ◽  
Generalized Gaussian Distribution ◽  
The Impact

Robust estimation method in generalized Gaussian distribution of observations under obedience can effectively eliminate or reduce the influence of gross errors, however, peculiarity of different estimation methods are not the same. In this paper, it’s used simulation method, the commonly used 13 kinds of robust features robust estimation methods were compared. The results showed that: L1 method, Danish method, German-McClure method and IGGIII program is more efficient robust estimation methods in Observations to obey generalized gaussian distribution, which method is more effective than other commonly used to eliminate the impact of robust estimation of gross errors or weaken .

scholarly journals A Comparison of Different Estimation Methods to Handle Missing Data in Explanatory Variables

2020 ◽  
pp. 3327-3336
Author(s):  
Manal Jabbar Salman
Keyword(s):  
Missing Data ◽  
Regression Models ◽  
Missing Values ◽  
Mean Squared Error ◽  
General Pattern ◽  
Error Variance ◽  
Small Sample ◽  
Estimation Methods ◽  
Sample Sizes ◽  
Explanatory Variables

Missing data is one of the problems that may occur in regression models. This problem is usually handled by deletion mechanism available in statistical software. This method reduces statistical inference values because deletion affects sample size. In this paper, Expectation Maximization algorithm (EM), Multicycle-Expectation-Conditional Maximization algorithm (MC-ECM), Expectation-Conditional Maximization Either (ECME), and Recurrent Neural Networks (RNN) are used to estimate multiple regression models when explanatory variables have some missing values. Experimental dataset were generated using Visual Basic programming language with missing values of explanatory variables according to a missing mechanism at random general pattern and some ratios of missing values (10%, 20%, and 30%) with error variance values of 0.5, 1. 5, and 2, which were included in sample sizes of 25, 50, 100, and 500 and evaluated using Mean Squared Error (MSE). Simulation results show that RNN outperforms the other methods, followed by EM at small sample sizes.

Sign in / Sign up

Export Citation Format

Share Document