scholarly journals Assessing Point Forecast Bias Across Multiple Time Series: Measures and Visual Tools

2021 ◽  
Vol 10 (5) ◽  
pp. 46
Author(s):  
Andrey Davydenko ◽  
Paul Goodwin

Measuring bias is important as it helps identify flaws in quantitative forecasting methods or judgmental forecasts. It can, therefore, potentially help improve forecasts. Despite this, bias tends to be under-represented in the literature: many studies focus solely on measuring accuracy. Methods for assessing bias in single series are relatively well-known and well-researched, but for datasets containing thousands of observations for multiple series, the methodology for measuring and reporting bias is less obvious. We compare alternative approaches against a number of criteria when rolling-origin point forecasts are available for different forecasting methods and for multiple horizons over multiple series. We focus on relatively simple, yet interpretable and easy-to-implement metrics and visualization tools that are likely to be applicable in practice. To study the statistical properties of alternative measures we use theoretical concepts and simulation experiments based on artificial data with predetermined features. We describe the difference between mean and median bias, describe the connection between metrics for accuracy and bias, provide suitable bias measures depending on the loss function used to optimise forecasts, and suggest which measures for accuracy should be used to accompany bias indicators. We propose several new measures and provide our recommendations on how to evaluate forecast bias across multiple series.

Author(s):  
Andrey Davydenko ◽  
Paul Goodwin

Measuring bias is important as it helps identify flaws in quantitative forecasting methods or judgmental forecasts. It can, therefore, potentially help improve forecasts. Despite this, bias tends to be under-represented in the literature: many studies focus solely on measuring accuracy. Methods for assessing bias in single series are relatively well-known and well-researched, but for datasets containing thousands of observations for multiple series, the methodology for measuring and reporting bias is less obvious. We compare alternative approaches against a number of criteria when rolling-origin point forecasts are available for different forecasting methods and for multiple horizons over multiple series. We focus on relatively simple, yet interpretable and easy-to-implement metrics and visualization tools that are likely to be applicable in practice. To study the statistical properties of alternative measures we use theoretical concepts and simulation experiments based on artificial data with predetermined features. We describe the difference between mean and median bias, describe the connection between metrics for accuracy and bias, provide suitable bias measures depending on the loss function used to optimise forecasts, and suggest which measures for accuracy should be used to accompany bias indicators. We propose several new measures and provide our recommendations on how to evaluate forecast bias across multiple series.


2021 ◽  
Author(s):  
Andrey Davydenko ◽  
Paul Goodwin

Measuring bias is important as it helps identify flaws in quantitative forecasting methods or judgmental forecasts. It can, therefore, potentially help improve forecasts. Despite this, bias tends to be under represented in the literature: many studies focus solely on measuring accuracy. Methods for assessing bias in single series are relatively well known and well researched, but for datasets containing thousands of observations for multiple series, the methodology for measuring and reporting bias is less obvious. We compare alternative approaches against a number of criteria when rolling origin point forecasts are available for different forecasting methods and for multiple horizons over multiple series. We focus on relatively simple, yet interpretable and easy to implement metrics and visualization tools that are likely to be applicable in practice. To study the statistical properties of alternative measures we use theoretical concepts and simulation experiments based on artificial data with predetermined features. We describe the difference between mean and median bias, describe the connection between metrics for accuracy and bias, provide suitable bias measures depending on the loss function used to optimise forecasts, and suggest which measures for accuracy should be used to accompany bias indicators. We propose several new measures and provide our recommendations on how to evaluate forecast bias across multiple series. /// Note: This is the final version of the paper, which appeared in the International Journal of Statistics and Probability. The first draft of this paper was uploaded to Preprints.org on 11 May, 2021: https://doi.org/10.20944/preprints202105.0261.v1 /// Copyrights: This is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).


1991 ◽  
Vol 261 (3) ◽  
pp. E312-E318 ◽  
Author(s):  
C. M. Barrera ◽  
A. J. Kastin ◽  
M. B. Fasold ◽  
W. A. Banks

Systemic administration of luteinizing hormone-releasing hormone (LHRH) in rats has been found to influence behavior independently of pituitary or ovarian function. A previous study has shown that LHRH can cross the blood-brain barrier in one direction, but it was not known whether this was due to a saturable transport system. The rate of entry of 125I-labeled LHRH from blood to brain was determined by two different single-pass methods of carotid perfusion. The first, a multiple time point method, measures Ki from the slope of the linear regression when brain-to-blood ratios of radioiodinated LHRH are plotted against time. Saturable transport was determined by the difference between the Ki of rats perfused with 125I-LHRH (12.51 X 10(-3) mg.g-1.min-1) vs. rats perfused with 125I-LHRH and unlabeled LHRH (10 nmol/ml; 2.20 X 10(-3) ml.g-1.min-1). The inhibition by the unlabeled peptide was statistically significant (P less than 0.001). The second method, a single time point technique, measures the cerebrovascular permeability-surface area coefficient (PA). Saturable transport was determined in rats by the competition of unlabeled LHRH with 125I-LHRH. The PA value for 125I-LHRH (20.00 X 10(-3) ml.g-1.min-1) was significantly greater (P less than 0.05) than for 125I-LHRH with the addition of 10 nmol/ml unlabeled LHRH (4.14 X 10(-3) ml.g-1.min-1). Saturable transport of LHRH from brain to blood in mice was also determined.(ABSTRACT TRUNCATED AT 250 WORDS)


2015 ◽  
Vol 20 (1) ◽  
pp. 11-14 ◽  
Author(s):  
Jill Bradshaw ◽  
Peter McGill

Purpose – The purpose of this paper is to provide a commentary on “Why study the history of learning disability?” by Goodey. Design/methodology/approach – In his paper, Goodey makes the point that the term “learning disability” is not a fixed, stable concept, but one which has changed and evolved over time. This commentary explores the difference between care staff and professional understandings of people with learning disabilities including how these have changed in recent times. Findings – Care staff knowledge is likely to be based on direct experiences of particular people with learning disabilities whilst professional knowledge is likely to be based on theoretical concepts of general application. These differences in source of knowledge may interfere with desired collaboration between staff and professionals. Originality/value – This commentary suggests that differences in understandings should be recognised and the application of theory to practice is far from simple.


2019 ◽  
Vol 3 (3) ◽  
pp. 199-210
Author(s):  
Annur Aliyyu

DISRUPTIVE BEHAVIOR AND TEACHER TREATMENTS CONFIGURATION. This study aims to determine the theoretical concepts, acceptability, practicability, and effectiveness of guidelines for rational emotive counseling, homework assignment techniques, and confrontation techniques. This research is development research with steps: design preparation, design validation, design testing, and design revision. Questionnaire and interview methods collected data, and the data collected were analyzed using descriptive analysis techniques. The results showed: (1) Theoretically rational emotive counseling procedures homework assignment techniques and confrontation techniques include steps: showing the counselee that the problem faced is related to irrational beliefs, bringing the counselee to the opposite stage of consciousness by showing that the counselee is maintained emotional disturbances to stay active by always thinking illogically, correcting the thoughts of the counselee and leaving irrational ideas, challenging the counselee to develop rational philosophies. The difference between the two lies in the things done at each step;  (2) Acceptability of rational emotive counseling guidelines for homework assignment techniques and confrontation techniques; (3) The practicality of rational emotive counseling guidelines for homework assignment techniques and confrontation techniques is very high; (4) The effectiveness of rational emotive counseling guidelines for homework assignment techniques and confrontation techniques is very high.


Author(s):  
Zuwati Hasim ◽  
Shi Di ◽  
Roger Barnard

Ministries of Education in many countries have adopted various forms of school-based assessment (SBA) to replace (for example, New Zealand) or complement (for example, England, Australia and Malaysia) more conventional forms of assessment such as tests and examinations. Central to these alternative approaches to SBA is formative assessment. In recent years, a body of research has been built investigating various aspects of SBA in Malaysia, but there has been a dearth of studies exploring what practising teachers believe and do regarding implementing formative assessment in their own classrooms. The present article reports some of the findings of a case study in which ten Malaysian primary school teachers of English were interviewed to identify the extent of their understanding of formative assessment and their reported practices of providing feedback in an SBA environment. Initially, the teachers revealed a general lack of understanding of the difference between formative and summative assessment. In such a situation, it would seem that the teachers are unready to implement SBA at the present stage, despite it having been mandated in Malaysian schools since 2011-12. However, later in the interview, they reported implementing various forms of feedback, some of which might be regarded as formative. There is a need, therefore, to differentiate between teachers’ explicit knowledge and their implicit understanding of matters such as formative feedback. The inherent limitations of self-report data emerging from interviews will be discussed and how these might be overcome.


Geografie ◽  
1995 ◽  
Vol 100 (2) ◽  
pp. 115-121
Author(s):  
David Uhlíř

This article deals with two theoretical concepts: flexible specialization and flexible accumulation. It starts with a very brief description of the changes in organization of production that occurred in the recent decades as a consequence of the 1970s and early 1980s crises. Their single most important characteristic is a great flexibility. Subsequently, the ways in which the two theories explain the changes themselves and their consequences are described in a more detailed manner. This includes reactions of the economic subjects involved, i.e. the state and the individual enterprises. Further on, the author aims to clarify the difference between two "flexible concepts" that are often misinterpreted in geographical literature. The distinction leads to an evaluation of the flexible specialization theory as an important contribution to the theories of regional development: on the other hand the flexible accumulation theorizes more generally the social and economic change. Several critical remarks concerning both theories are quoted in the last part of this article.


Author(s):  
Keisuke Kamiya ◽  
Kimihiko Yasuda

This paper studies spatial motion of a stretched string subjected to two harmonic excitations with different frequencies. Two cases are considered, one in which both excitations are in a plane, and one in which they are perpendicular to each other. First, theoretical analysis by the method of multiple time scales is conducted. It is found that in the former case sub-combination resonance and summed-and-differential combination resonance of whirling type can occur, and in the latter case sub-combination resonances parallel and perpendicular to the excitations and of whirling type can occur. Then, experiment is conducted. In the experiment, it was confirmed that the same types of motions as those predicted in the theoretical analysis occured. In addition, it was found that for the case in which the excitations are in a plane the response is sensitive to the difference between the natural frequencies in the two perpendicular directions. Finally, effect of the difference between the natural frequencies is studied.


2006 ◽  
Vol 05 (04) ◽  
pp. 329-335 ◽  
Author(s):  
José Guajardo ◽  
Richard Weber ◽  
Jaime Miranda

Various techniques have been proposed to forecast a given time series. Models from the ARIMA family have been successfully used, as well as regression approaches based on e.g. linear, non-linear regression, neural networks, and Support Vector Regression. What makes the difference in many real-world applications, however, is not the technique but an appropriate forecasting methodology. Here, we propose such a methodology for the regression-based forecasting approach. A hybrid system is presented that iteratively selects the most relevant features and constructs the regression model optimizing its parameters dynamically. We develop a particular technique for feature selection as well as for model construction. The methodology, however, is a generic one providing the opportunity to employ alternative approaches within our framework. The application to several time series underlines its usefulness.


Motor Control ◽  
2015 ◽  
Vol 19 (4) ◽  
pp. 355-369 ◽  
Author(s):  
Ruud J. R. Den Hartigh ◽  
Ralf F. A. Cox ◽  
Christophe Gernigon ◽  
Nico W. Van Yperen ◽  
Paul L. C. Van Geert

The aim of this study was to examine (1) the temporal structures of variation in rowers’ (natural) ergometer strokes to make inferences about the underlying motor organization, and (2) the relation between these temporal structures and skill level. Four high-skilled and five lower-skilled rowers completed 550 strokes on a rowing ergometer. Detrended Fluctuation Analysis was used to quantify the temporal structure of the intervals between force peaks. Results showed that the temporal structure differed from random, and revealed prominent patterns of pink noise for each rower. Furthermore, the high-skilled rowers demonstrated more pink noise than the lower-skilled rowers. The presence of pink noise suggests that rowing performance emerges from the coordination among interacting component processes across multiple time scales. The difference in noise pattern between high-skilled and lower-skilled athletes indicates that the complexity of athletes’ motor organization is a potential key characteristic of elite performance.


Sign in / Sign up

Export Citation Format

Share Document