The Effectiveness of Expectation Models in Recognizing Error Patterns and Generating and Eliminating Hypotheses While Conducting Analytical Procedures

2003 ◽  
Vol 22 (2) ◽  
pp. 147-170 ◽  
Author(s):  
Robert A Leitch ◽  
Yining Chen
Keyword(s):  
Business Processes ◽  
Structural Model ◽  
Type I Error ◽  
Moving Average ◽  
Hypothesis Generation ◽  
Type I ◽  
Accounting System ◽  
Lower Efficiency ◽  
Analytical Procedures ◽  
Joint Consideration

This study analyzes the hypothesis generation and elimination capability of analytical procedures using Structural, Stepwise, Martingale, and ARIMA (Auto Regressive Integrated Moving Average) expectation models. We seed 14 errors into 36 complete sets of actual monthly financial statements from three companies. The bivariate pattern of differences, along with the structure of the accounting, business process, and economic system, are used to analytically determine (hypothesize) the most likely cause of the error. We then test the capability of these expectation models to generate correct hypotheses or to eliminate incorrect hypotheses. Positive and negative testing approaches, founded on multivariate normal theory, are examined. From a hypothesis generation perspective using the positive testing approach, the results indicate that the Structural and Stepwise models, yield lower effectiveness risks (type II error) than the ARIMA and Martingale models, with the edge going to the Structural model. From a hypothesis elimination perspective using the negative approach, the results indicate that the Structural and Stepwise models yield lower efficiency risks (type I error) than the Martingale and ARIMA models, with the edge going to the Stepwise model. This study provides strong evidence to support the use of the structure of an organization's business processes (McCarthy 1982; Bell et al. 1997), its associated accounting system, and economic structure to build an expectation model. Moreover, the joint consideration of errors is found to be superior to the marginal approach advocated by Kinney (1987). The results presented here have the potential to assist auditors in directing audit efforts.

Smoothing Facilitates the Detection of Coupled Responses in Psychophysiological Time Series

2000 ◽  
Vol 14 (1) ◽  
pp. 1-10 ◽  
Author(s):  
Joni Kettunen ◽  
Niklas Ravaja ◽  
Liisa Keltikangas-Järvinen
Keyword(s):  
Time Series ◽  
Type I Error ◽  
Electrodermal Activity ◽  
Moving Average ◽  
Type I ◽  
Facial Electromyography ◽  
Response Characteristics ◽  
Smoothing Methods ◽  
Response Systems ◽  
Different Response

Abstract We examined the use of smoothing to enhance the detection of response coupling from the activity of different response systems. Three different types of moving average smoothers were applied to both simulated interbeat interval (IBI) and electrodermal activity (EDA) time series and to empirical IBI, EDA, and facial electromyography time series. The results indicated that progressive smoothing increased the efficiency of the detection of response coupling but did not increase the probability of Type I error. The power of the smoothing methods depended on the response characteristics. The benefits and use of the smoothing methods to extract information from psychophysiological time series are discussed.

scholarly journals Operating Characteristics of Statistical Methods for Detecting Gene-by-Measured Environment Interaction in the Presence of Gene-Environment Correlation under Violations of Distributional Assumptions

2015 ◽  
Vol 18 (1) ◽  
pp. 19-27 ◽  
Author(s):  
Carol A. Van Hulle ◽  
Paul J. Rathouz
Keyword(s):  
Structural Model ◽  
Behavior Genetics ◽  
Type I Error ◽  
Genetic Research ◽  
Error Rates ◽  
Type I ◽  
Environment Interaction ◽  
Operating Characteristics ◽  
Type I Error Rates ◽  
Distributional Assumptions

Accurately identifying interactions between genetic vulnerabilities and environmental factors is of critical importance for genetic research on health and behavior. In the previous work of Van Hulle et al. (Behavior Genetics, Vol. 43, 2013, pp. 71–84), we explored the operating characteristics for a set of biometric (e.g., twin) models of Rathouz et al. (Behavior Genetics, Vol. 38, 2008, pp. 301–315), for testing gene-by-measured environment interaction (GxM) in the presence of gene-by-measured environment correlation (rGM) where data followed the assumed distributional structure. Here we explore the effects that violating distributional assumptions have on the operating characteristics of these same models even when structural model assumptions are correct. We simulated N = 2,000 replicates of n = 1,000 twin pairs under a number of conditions. Non-normality was imposed on either the putative moderator or on the ultimate outcome by ordinalizing or censoring the data. We examined the empirical Type I error rates and compared Bayesian information criterion (BIC) values. In general, non-normality in the putative moderator had little impact on the Type I error rates or BIC comparisons. In contrast, non-normality in the outcome was often mistaken for or masked GxM, especially when the outcome data were censored.

One Size Fits All?

Methodology ◽  
2012 ◽  
Vol 8 (1) ◽  
pp. 23-38 ◽  
Author(s):  
Manuel C. Voelkle ◽  
Patrick E. McKnight
Keyword(s):  
Repeated Measures ◽  
Structural Equation ◽  
Type I Error ◽  
Equation Modeling ◽  
Type I ◽  
Finite Sample ◽  
Traditional Methods ◽  
Repeated Measures Anova ◽  
Special Cases ◽  
Latent Curve

The use of latent curve models (LCMs) has increased almost exponentially during the last decade. Oftentimes, researchers regard LCM as a “new” method to analyze change with little attention paid to the fact that the technique was originally introduced as an “alternative to standard repeated measures ANOVA and first-order auto-regressive methods” (Meredith & Tisak, 1990, p. 107). In the first part of the paper, this close relationship is reviewed, and it is demonstrated how “traditional” methods, such as the repeated measures ANOVA, and MANOVA, can be formulated as LCMs. Given that latent curve modeling is essentially a large-sample technique, compared to “traditional” finite-sample approaches, the second part of the paper addresses the question to what degree the more flexible LCMs can actually replace some of the older tests by means of a Monte-Carlo simulation. In addition, a structural equation modeling alternative to Mauchly’s (1940) test of sphericity is explored. Although “traditional” methods may be expressed as special cases of more general LCMs, we found the equivalence holds only asymptotically. For practical purposes, however, no approach always outperformed the other alternatives in terms of power and type I error, so the best method to be used depends on the situation. We provide detailed recommendations of when to use which method.

Assessing Person Fit With the Information Matrix Test

Methodology ◽  
2015 ◽  
Vol 11 (1) ◽  
pp. 3-12 ◽  
Author(s):  
Jochen Ranger ◽  
Jörg-Tobias Kuhn
Keyword(s):  
Simulation Study ◽  
Type I Error ◽  
Information Matrix ◽  
Small Samples ◽  
Type I ◽  
Person Fit ◽  
Power Of The Test ◽  
Order Expansion ◽  
Trait Stability ◽  
Information Matrix Test

In this manuscript, a new approach to the analysis of person fit is presented that is based on the information matrix test of White (1982) . This test can be interpreted as a test of trait stability during the measurement situation. The test follows approximately a χ2-distribution. In small samples, the approximation can be improved by a higher-order expansion. The performance of the test is explored in a simulation study. This simulation study suggests that the test adheres to the nominal Type-I error rate well, although it tends to be conservative in very short scales. The power of the test is compared to the power of four alternative tests of person fit. This comparison corroborates that the power of the information matrix test is similar to the power of the alternative tests. Advantages and areas of application of the information matrix test are discussed.

How to Detect Publication Bias in Psychological Research

2019 ◽  
Vol 227 (4) ◽  
pp. 261-279 ◽  
Author(s):  
Frank Renkewitz ◽  
Melanie Keiner
Keyword(s):  
Publication Bias ◽  
Effect Size ◽  
Statistical Power ◽  
Type I Error ◽  
Psychological Research ◽  
Type I ◽  
True Effect Size ◽  
Questionable Research Practices ◽  
True Effect ◽  
Meta Analyses

Abstract. Publication biases and questionable research practices are assumed to be two of the main causes of low replication rates. Both of these problems lead to severely inflated effect size estimates in meta-analyses. Methodologists have proposed a number of statistical tools to detect such bias in meta-analytic results. We present an evaluation of the performance of six of these tools. To assess the Type I error rate and the statistical power of these methods, we simulated a large variety of literatures that differed with regard to true effect size, heterogeneity, number of available primary studies, and sample sizes of these primary studies; furthermore, simulated studies were subjected to different degrees of publication bias. Our results show that across all simulated conditions, no method consistently outperformed the others. Additionally, all methods performed poorly when true effect sizes were heterogeneous or primary studies had a small chance of being published, irrespective of their results. This suggests that in many actual meta-analyses in psychology, bias will remain undiscovered no matter which detection method is used.

Correction: “Influence of Selection Bias on the Test Decision – A Simulation Study”

2014 ◽  
Vol 53 (05) ◽  
pp. 343-343
Keyword(s):  
Selection Bias ◽  
Simulation Study ◽  
Error Rate ◽  
Type I Error ◽  
Block Size ◽  
Error Rates ◽  
Type I ◽  
Type I Error Rate ◽  
Representation Error ◽  
Numeric Representation

We have to report marginal changes in the empirical type I error rates for the cut-offs 2/3 and 4/7 of Table 4, Table 5 and Table 6 of the paper “Influence of Selection Bias on the Test Decision – A Simulation Study” by M. Tamm, E. Cramer, L. N. Kennes, N. Heussen (Methods Inf Med 2012; 51: 138 –143). In a small number of cases the kind of representation of numeric values in SAS has resulted in wrong categorization due to a numeric representation error of differences. We corrected the simulation by using the round function of SAS in the calculation process with the same seeds as before. For Table 4 the value for the cut-off 2/3 changes from 0.180323 to 0.153494. For Table 5 the value for the cut-off 4/7 changes from 0.144729 to 0.139626 and the value for the cut-off 2/3 changes from 0.114885 to 0.101773. For Table 6 the value for the cut-off 4/7 changes from 0.125528 to 0.122144 and the value for the cut-off 2/3 changes from 0.099488 to 0.090828. The sentence on p. 141 “E.g. for block size 4 and q = 2/3 the type I error rate is 18% (Table 4).” has to be replaced by “E.g. for block size 4 and q = 2/3 the type I error rate is 15.3% (Table 4).”. There were only minor changes smaller than 0.03. These changes do not affect the interpretation of the results or our recommendations.

Sign in / Sign up

Export Citation Format

Share Document