scholarly journals Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature

2014 ◽  
Vol 7 (3) ◽  
pp. 1247-1250 ◽  
Author(s):  
T. Chai ◽  
R. R. Draxler
Keyword(s):  
Root Mean Square Error ◽  
Mean Square Error ◽  
Root Mean Square ◽  
Model Evaluation ◽  
Mean Absolute Error ◽  
Evaluation Studies ◽  
Model Performance ◽  
Absolute Error ◽  
Average Error ◽  
Mean Square

Abstract. Both the root mean square error (RMSE) and the mean absolute error (MAE) are regularly employed in model evaluation studies. Willmott and Matsuura (2005) have suggested that the RMSE is not a good indicator of average model performance and might be a misleading indicator of average error, and thus the MAE would be a better metric for that purpose. While some concerns over using RMSE raised by Willmott and Matsuura (2005) and Willmott et al. (2009) are valid, the proposed avoidance of RMSE in favor of MAE is not the solution. Citing the aforementioned papers, many researchers chose MAE over RMSE to present their model evaluation statistics when presenting or adding the RMSE measures could be more beneficial. In this technical note, we demonstrate that the RMSE is not ambiguous in its meaning, contrary to what was claimed by Willmott et al. (2009). The RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian. In addition, we show that the RMSE satisfies the triangle inequality requirement for a distance metric, whereas Willmott et al. (2009) indicated that the sums-of-squares-based statistics do not satisfy this rule. In the end, we discussed some circumstances where using the RMSE will be more beneficial. However, we do not contend that the RMSE is superior over the MAE. Instead, a combination of metrics, including but certainly not limited to RMSEs and MAEs, are often required to assess model performance.

scholarly journals Root mean square error (RMSE) or mean absolute error (MAE)?

2014 ◽  
Vol 7 (1) ◽  
pp. 1525-1534 ◽  
Author(s):  
T. Chai ◽  
R. R. Draxler
Keyword(s):  
Root Mean Square Error ◽  
Mean Square Error ◽  
Root Mean Square ◽  
Model Evaluation ◽  
Mean Absolute Error ◽  
Evaluation Studies ◽  
Model Performance ◽  
Absolute Error ◽  
Average Error ◽  
Mean Square

Abstract. Both the root mean square error (RMSE) and the mean absolute error (MAE) are regularly employed in model evaluation studies. Willmott and Matsuura (2005) have suggested that the RMSE is not a good indicator of average model performance and might be a misleading indicator of average error and thus the MAE would be a better metric for that purpose. Their paper has been widely cited and may have influenced many researchers in choosing MAE when presenting their model evaluation statistics. However, we contend that the proposed avoidance of RMSE and the use of MAE is not the solution to the problem. In this technical note, we demonstrate that the RMSE is not ambiguous in its meaning, contrary to what was claimed by Willmott et al. (2009). The RMSE is more appropriate to represent model performance than the MAE when the error distribution is expected to be Gaussian. In addition, we show that the RMSE satisfies the triangle inequality requirement for a distance metric.

Metabolic Rate in Adolescent Athletes: The Development and Validation of New Equations, and Comparison to Previous Models

2020 ◽  
Vol 30 (4) ◽  
pp. 249-257
Author(s):  
Reid J. Reale ◽  
Timothy J. Roberts ◽  
Khalil A. Lee ◽  
Justina L. Bonsignore ◽  
Melissa L. Anderson
Keyword(s):  
Root Mean Square Error ◽  
Metabolic Rate ◽  
Mean Square Error ◽  
Root Mean Square ◽  
Mean Absolute Error ◽  
Compartment Model ◽  
Absolute Error ◽  
Mean Square ◽  
Adolescent Athletes ◽  
Two Compartment Model

We sought to assess the accuracy of current or developing new prediction equations for resting metabolic rate (RMR) in adolescent athletes. RMR was assessed via indirect calorimetry, alongside known predictors (body composition via dual-energy X-ray absorptiometry, height, age, and sex) and hypothesized predictors (race and maturation status assessed via years to peak height velocity), in a diverse cohort of adolescent athletes (n = 126, 77% male, body mass = 72.8 ± 16.6 kg, height = 176.2 ± 10.5 cm, age = 16.5 ± 1.4 years). Predictive equations were produced and cross-validated using repeated k-fold cross-validation by stepwise multiple linear regression (10 folds, 100 repeats). Performance of the developed equations was compared with several published equations. Seven of the eight published equations examined performed poorly, underestimating RMR in >75% to >90% of cases. Root mean square error of the six equations ranged from 176 to 373, mean absolute error ranged from 115 to 373 kcal, and mean absolute error SD ranged from 103 to 185 kcal. Only the Schofield equation performed reasonably well, underestimating RMR in 51% of cases. A one- and two-compartment model were developed, both r2 of .83, root mean square error of 147, and mean absolute error of 114 ± 26 and 117 ± 25 kcal for the one- and two-compartment model, respectively. Based on the models’ performance, as well as visual inspection of residual plots, the following model predicts RMR in adolescent athletes with better precision than previous models; RMR = 11.1 × body mass (kg) + 8.4 × height (cm) − (340 male or 537 female).

scholarly journals AZƏRBAYCANDA COVID-19 KORONAVİRUSUNA YOLUXMANIN PROQNOZLAŞDIRILMASI ÜÇÜN ARIMA MODELLƏRİNİN TƏTBİQİ

2021 ◽  
Vol 12 (1) ◽  
pp. 95-104
Author(s):  
Firəngiz Sadıyeva ◽  
Keyword(s):  
Root Mean Square Error ◽  
Mean Square Error ◽  
Root Mean Square ◽  
Mean Absolute Error ◽  
Moving Average ◽  
Absolute Error ◽  
Mean Square ◽  
Autoregressive Integrated Moving Average

Məqalədə COVID-19 pandemiyasını proqnozlaşdırmaq üçün avtoreqressiv inteqrasiya edilmiş hərəkətli ortalama (ing. ARIMA. Autoregressive İntegrated Moving Average) modeli təklif edilmişdir. COVID-19 dünyada sürətlə yayılan və hazırda davam edən yeni növ pandemiyadır. Son dövrlərdə pandemiyaya yoluxanların sayı Azərbaycanda rekord həddə çatmışdır. Məhz bu səbəbdən COVID-19 pandemiyasının proqnozu məsələsinə baxılmışdır və bir neçə ayı əhatə edən real verilənlərlə eksperimentlərdə təklif edilmiş ARIMA modelinin COVID-19 zaman sıralarının proqnozlaşdırılması üçün müxtəlif parametrlərlə tətbiq edilmişdir. Verilənlər dedikdə, 22.01.2020 – 22.10.2020 tarixləri arasında Azərbaycan Respublikasının Səhiyyə Nazirliyi (www.sehiyye.gov.az) tərəfindən rəsmi olaraq qeydiyyata alınan gündəlik yoluxma hallarının sayı nəzərdə tutulur. Bu verilənlərdən istifadə etməklə, növbəti zaman aralığında ölkəmizdə baş verəcək yoluxma halları proqnoz edilmişdir. Bunun üçün ARIMA modelinə müxtəlif parametrlər verilmiş və uyğun olaraq hər bir modelin səhv dərəcəsi qiymətləndirilmişdir. Səhvin qiymətləndirilməsi üçün MAPE (Mean Absolute Persentace Error), MAE (Mean Absolute Error) və RMSE (Root Mean Square Error) funksiyaları istifadə edilib. Müqayisələr nəticəsində ən uyğun model seçilmişdir. Alınmış nəticələr ölkəmizdə pandemiya dövründə həm səhiyyə sistemi, həm də adi vətəndaşlar üçün vacib amildir. Əldə edilmiş nəticələr statistik metodların koronavirusa aid qeyri-stasionar zaman sıralarının proqnozlaşdırılmasının digər məsələlərə tətbiqində də məhsuldar ola biləcəyini təsdiqləyir.

PEMODELAN PREDIKTIF KONSUMSI ENERGI BANGUNAN GEDUNG KOMERSIAL DENGAN ALGORITMA SUPPORT VECTOR MACHINE

2018 ◽  
Vol 14 (2) ◽  
pp. 225
Author(s):  
Indriyanti Indriyanti ◽  
Agus Subekti
Keyword(s):  
Machine Learning ◽  
Support Vector Machine ◽  
Root Mean Square Error ◽  
Mean Square Error ◽  
Root Mean Square ◽  
Mean Absolute Error ◽  
Absolute Error ◽  
Support Vector ◽  
Mean Square

Konsumsi energi bangunan yang semakin meningkat mendorong para peneliti untuk membangun sebuah model prediksi dengan menerapkan metode machine learning, namun masih belum diketahui model yang paling akurat. Model prediktif untuk konsumsi energi bangunan komersial penting untuk konservasi energi. Dengan menggunakan model yang tepat, kita dapat membuat desain bangunan yang lebih efisien dalam penggunaan energi. Dalam tulisan ini, kami mengusulkan model prediktif berdasarkan metode pembelajaran mesin untuk mendapatkan model terbaik dalam memprediksi total konsumsi energi. Algoritma yang digunakan yaitu SMOreg dan LibSVM dari kelas Support Vector Machine, kemudian untuk evaluasi model berdasarkan nilai Mean Absolute Error dan Root Mean Square Error. Dengan menggunakan dataset publik yang tersedia, kami mengembangkan model berdasarkan pada mesin vektor pendukung untuk regresi. Hasil pengujian kedua algoritma tersebut diketahui bahwa algoritma SMOreg memiliki akurasi lebih baik karena memiliki nilai MAE dan RMSE sebesar 4,70 dan 10,15, sedangkan untuk model LibSVM memiliki nilai MAE dan RMSE sebesar 9,37 dan 14,45. Kami mengusulkan metode berdasarkan algoritma SMOreg karena kinerjanya lebih baik.

scholarly journals UJI AKURASI PRODUK ESTIMASI CURAH HUJAN SATELIT GPM IMERG DI SURABAYA, INDONESIA

2018 ◽  
Vol 19 (2) ◽  
pp. 83
Author(s):  
Mukhamad Adib Azka ◽  
Prabu Aditya Sugianto ◽  
Andreas Kurniawan Silitonga ◽  
Imma Redha Nugraheni
Keyword(s):  
Root Mean Square Error ◽  
Mean Square Error ◽  
Root Mean Square ◽  
Mean Absolute Error ◽  
Absolute Error ◽  
Mean Square

Curah hujan merupakan parameter meteorologi yang sangat berpengaruh dalam kehidupan. Saat ini, pengamatan secara in situ sangat kurang representatif untuk digunakan sebagai analisis karena jangkauannya yang sangat sempit sehingga memerlukan instrumen pendukung seperti satelit agar dapat memberikan gambaran yang lebih baik terkait distribusi hujan. Namun, data satelit juga belum tentu sepenuhnya benar karena resolusi dan kondisi dari setiap wilayah berbeda. Penelitian ini bertujuan untuk mendapatkan nilai akurasi, bias, korelasi, root mean square error (RMSE), dan mean absolute error (MAE) data estimasi curah hujan GPM IMERG dengan data curah hujan pengamatan langsung. Penelitian ini dilakukkan di Surabaya dengan menggunakan data estimasi curah hujan GPM IMERG dan data curah hujan pengamatan langsung dari Stasiun Meteorologi Kelas I Juanda Surabaya selama tahun 2017 mewakili musim hujan, musim kemarau, dan periode transisi. Hasil penelitian menunjukkan bahwa data curah hujan produk GPM IMERG memiliki korelasi yang sangat baik untuk memperkirakan akumulasi curah hujan bulanan. Sedangkan, untuk akumulasi harian, memiliki korelasi yang sangat rendah. Sementara itu untuk akumulasi sepuluh harian, data curah hujan produk satelit GPM IMERG memiliki korelasi yang baik terutama di periode musim hujan dan musim kemarau, akan tetapi memiliki korelasi yang rendah selama periode transisi dari musim hujan ke musim kemarau atau sebaliknya. Pada umumnya, produk ini sangat bagus dalam menentukan ada atau tidaknya hujan, tetapi performanya sangat rendah dalam menentukan besarnya intensitas curah hujan.

scholarly journals An Appraisal of Dynamic Bayesian Model Averaging-based Merged Multi-Satellite Precipitation Datasets Over Complex Topography and the Diverse Climate of Pakistan

2019 ◽  
Vol 12 (1) ◽  
pp. 10 ◽  
Author(s):  
Khalil Ur Rahman ◽  
Songhao Shang ◽  
Muhammad Shahid ◽  
Yeqiang Wen
Keyword(s):  
Root Mean Square Error ◽  
Mean Square Error ◽  
Correlation Coefficient ◽  
Root Mean Square ◽  
Mean Absolute Error ◽  
Model Averaging ◽  
Absolute Error ◽  
Efficiency Score ◽  
Mean Square ◽  
Satellite Precipitation

Merging satellite precipitation products tends to reduce the errors associated with individual satellite precipitation products and has higher potential for hydrological applications. The current study evaluates the performance of merged multi-satellite precipitation dataset (daily temporal and 0.25° spatial resolution) developed using the Dynamic Bayesian Model Averaging algorithm across four different climate regions, i.e., glacial, humid, arid and hyper-arid regions, of Pakistan during 2000–2015. Four extensively evaluated SPPs over Pakistan, i.e., Tropical Rainfall Measurement Mission (TRMM) Multi-satellite Precipitation Analysis (TMPA) 3B42V7, Precipitation Estimation from Remotely Sensed Information using Artificial Neural Networks-Climate Data Record (PERSIANN-CDR), Climate Prediction Center MORPHing technique (CMORPH), and Era-Interim, are used to develop the merged multi-satellite precipitation dataset. Six statistical indices, including Mean Bias Error, Mean Absolute Error, Root Mean Square Error, Correlation Coefficient, Kling-Gupta efficiency, and Theil’s U coefficient, are used to evaluate the performance of merged multi-satellite precipitation dataset over 102 ground precipitation gauges both spatially and temporally. Moreover, the ensemble spread score and standard deviation are also used to depict the spread and variation of precipitation of merged multi-satellite precipitation dataset. Skill scores for all statistical indices are also included in the analyses, which shows improvement of merged multi-satellite precipitation dataset against Simple Model Averaging. The results revealed that DBMA-MSPD assigned higher weights to TMPA (0.32) and PERSIANN-CDR (0.27). TMPA presented higher skills in glacial and humid regions with average weights of 0.32 and 0.37 as compared to PERSIANN-CDR of 0.27 and 0.25, respectively. TMPA and Era-Interim depicted higher skills during pre-monsoon and monsoon seasons, with average weights of 0.31 and 0.52 (TMPA) and 0.25 and 0.21 (Era-Interim), respectively. Merged multi-satellite precipitation dataset overestimated precipitation in glacial/humid regions and showed poor performance, with the poorest values of mean absolute error (2.69 mm/day), root mean square error (11.96 mm/day), correlation coefficient (0.41), Kling-Gupta efficiency score (0.33) and Theil’s U (0.70) at some stations in glacial/humid regions. Higher performance is observed in hyper-arid region, with the best values of 0.71 mm/day, 1.72 mm/day, 0.84, 0.93, and 0.37 for mean absolute error, root mean square error, correlation coefficient, Kling-Gupta Efficiency score, and Theil’s U, respectively. Merged multi-Satellite Precipitation Dataset demonstrated significant improvements as compared to TMPA across all climate regions with average improvements of 45.26% (mean bias error), 30.99% (mean absolute error), 30.1% (root mean square error), 11.34% (correlation coefficient), 9.53% (Kling-Gupta efficiency score) and 8.86% (Theil’s U). The ensemble spread and variation of DBMA-MSPD calculated using ensemble spread score and standard deviation demonstrates high spread (11.38 mm/day) and variation (12.58 mm/day) during monsoon season in the humid and glacial regions, respectively. Moreover, the improvements of DBMA-MSPD quantified against fixed weight SMA-MSPD reveals supremacy of DBMA-MSPD, higher improvements (40–50%) in glacial and humid regions.

scholarly journals Forecasting Exports and Imports by using Autoregressive (AR) with Seasonal Dummies and Box-Jenkins Approaches: A Case of Pakistan

2020 ◽  
Vol 31 (3) ◽  
pp. 291-301
Author(s):  
Sahir Pervaiz Ghauri ◽  
Rizwan Raheem Ahmed ◽  
Dalia Streimikiene ◽  
Justas Streimikis
Keyword(s):  
Root Mean Square Error ◽  
Mean Square Error ◽  
Root Mean Square ◽  
Mean Absolute Error ◽  
Moving Average ◽  
Trade Deficit ◽  
Absolute Error ◽  
Ar Model ◽  
Mean Square ◽  
Auto Regressive

This research aims to evaluate two econometric models to forecast imports and exports for the financial year (FY) 2020. For this purpose, we used the annual exports and imports data of Pakistan from FY2002 to FY2019. Thus, in this regard, we employed, and compared the results of two econometrics models such as Box Jenkins or Autoregressive Integrated Moving Average (ARIMA), and Auto-Regressive (AR) with seasonal dummies. For examining the precision of forecasting, we employed mean absolute error and root mean square error approaches. The findings of Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) reveal that the ARIMA or Box Jenkins approach provides better accuracy of the forecast for the exports as compared to the AR model with dummies. However, Auto-Regressive (AR) model has demonstrated more precision for the imports as compared to the Box Jenkins model. Hence, the projected forecasting for the growth of export is 1.87% for the FY2020 and projected forecasting for the import demonstrates a negative variation of -1.61% for the FY2020. The findings of the undertaken study recommend the policymakers of Pakistan to take corrective measures to increase exports and to prevent the country from the trade deficit. The policymakers of Pakistan should give more incentives to the exporters and decrease the cost of doing business to be more competitive than the regional economies such as India, Bangladesh, and China.

Sign in / Sign up

Export Citation Format

Share Document