Confirming Testlet Effects

2012 ◽  
Vol 36 (2) ◽  
pp. 104-121 ◽  
Author(s):  
Christine E. DeMars
Keyword(s):  
False Alarm ◽  
Sample Size ◽  
Statistical Significance ◽  
Information Criteria ◽  
False Alarms ◽  
Parameter Estimates ◽  
Significance Tests ◽  
Item Responses ◽  
False Alarm Rates ◽  
Essential Unidimensionality

A testlet is a cluster of items that share a common passage, scenario, or other context. These items might measure something in common beyond the trait measured by the test as a whole; if so, the model for the item responses should allow for this testlet trait. But modeling testlet effects that are negligible makes the model unnecessarily complicated and risks capitalization on chance, increasing the error in parameter estimates. Checking each testlet to see if the items within the testlet share something beyond the primary trait could therefore be useful. This study included (a) comparison between a model with no testlets and a model with testlet g, (b) comparison between a model with all suspected testlets and a model with all suspected testlets except testlet g, and (c) a test of essential unidimensionality. Overall, Comparison b was most useful for detecting testlet effects. Model comparisons based on information criteria, specifically the sample-size adjusted Bayesian Information Criteria (SSA-BIC) and BIC, resulted in fewer false alarms than statistical significance tests. The test of essential unidimensionality had true hit rates and false alarm rates similar to the SSA-BIC when the testlet effect was zero for all testlets except the studied testlet. But the presence of additional testlet effects in the partitioning test led to higher false alarm rates for the test of essential unidimensionality.

Interpreting Statistical Significance and Meaningfulness in Adapted Physical Activity Research

1998 ◽  
Vol 15 (2) ◽  
pp. 103-118 ◽  
Author(s):  
Vinson H. Sutlive ◽  
Dale A. Ulrich
Keyword(s):  
Physical Activity ◽  
Sample Size ◽  
Recent Literature ◽  
Statistical Significance ◽  
Effect Sizes ◽  
Significance Tests ◽  
Adapted Physical Activity ◽  
Research Designs ◽  
Alpha Level ◽  
Research Findings

The unqualified use of statistical significance tests for interpreting the results of empirical research has been called into question by researchers in a number of behavioral disciplines. This paper reviews what statistical significance tells us and what it does not, with particular attention paid to criticisms of using the results of these tests as the sole basis for evaluating the overall significance of research findings. In addition, implications for adapted physical activity research are discussed. Based on the recent literature of other disciplines, several recommendations for evaluating and reporting research findings are made. They include calculating and reporting effect sizes, selecting an alpha level larger than the conventional .05 level, placing greater emphasis on replication of results, evaluating results in a sample size context, and employing simple research designs. Adapted physical activity researchers are encouraged to use specific modifiers when describing findings as significant.

scholarly journals Effects of fixed- and varied-context repetition on associative recognition in amnesia

2010 ◽  
Vol 16 (4) ◽  
pp. 596-602 ◽  
Author(s):  
DANIEL L. GREENBERG ◽  
MIEKE VERFAELLIE
Keyword(s):  
False Alarm ◽  
Recognition Task ◽  
Associative Recognition ◽  
False Alarms ◽  
Study Phase ◽  
Repetition Condition ◽  
Context Condition ◽  
False Alarm Rates

AbstractThis study compared the effects of fixed- and varied-context repetition on associative recognition in amnesia. Controls and amnesic participants were presented with a set of three-word phrases. Each was presented three times. In the varied-context condition, the verb changed with each presentation; in the fixed-context condition, it remained constant. At test, participants performed an associative-recognition task in which they were shown pairs of words from the study phase and asked to distinguish between intact and recombined pairs. For corrected recognition (hits – false alarms), controls performed better in the varied-context than in the fixed-context repetition condition, whereas amnesic participants’ performance did not differ between conditions. Similarly, controls had lower false-alarm rates in the varied-context condition, but there was no significant effect of condition for the amnesic participants. Thus, varied-context repetition does not improve amnesic participants’ performance on a recollection-dependent associative-recognition task, possibly because the amnesic participants were unable to take advantage of the additional cues that the varied-context encoding condition provided. (JINS, 2010, 16, 596–602.)

scholarly journals Reducing false alarm rates during change detection by modeling relief, shade and shadow of multi-temporal imagery.

2014 ◽  
Vol XL-4 ◽  
pp. 65-70 ◽  
Author(s):  
B. Gorte ◽  
C. van der Sande
Keyword(s):  
Change Detection ◽  
False Alarm ◽  
3D Model ◽  
Integrated Approach ◽  
False Alarms ◽  
The Sun ◽  
Geometric Image ◽  
Multi Temporal ◽  
False Alarm Rates

Change detection on the basis of multi-temporal imagery may lead to false alarms when the image has changed, whereas the scene has not. Geometric image differerences in an unchanged scene may be due to relief displacement, caused by diferent camera positions. Radiometric differences may be caused by changes in illumimation and shadow between the images, caused by a different position of the sun. The effects may be predicted, and after that compensated, if a 3d model of the scene is available. The paper presents an integrated approach to prediction of and compensation for relief displacement, shading and shadow.

scholarly journals Insights into Criteria for Statistical Significance from Signal Detection Analysis

2019 ◽  
Vol 3 ◽  
Author(s):  
Jessica K. Witt
Keyword(s):  
Signal Detection ◽  
Sample Size ◽  
Effect Size ◽  
Statistical Significance ◽  
Bayes Factors ◽  
False Alarms ◽  
P Values ◽  
Questionable Research Practices ◽  
Detection Analysis ◽  
Signal Detection Analysis

What is best criterion for determining statistical significance? In psychology, the criterion has been p < .05. This criterion has been criticized since its inception, and the criticisms have been rejuvenated with recent failures to replicate studies published in top psychology journals. Several replacement criteria have been suggested including reducing the alpha level to .005 or switching to other types of criteria such as Bayes factors or effect sizes. Here, various decision criteria for statistical significance were evaluated using signal detection analysis on the outcomes of simulated data. The signal detection measure of area under the curve (AUC) is a measure of discriminability with a value of 1 indicating perfect discriminability and 0.5 indicating chance performance. Applied to criteria for statistical significance, it provides an estimate of the decision criterion’s performance in discriminating real effects from null effects. AUCs were high (M = .96, median = .97) for p values, suggesting merit in using p values to discriminate significant effects. AUCs can be used to assess methodological questions such as how much improvement will be gained with increased sample size, how much discriminability will be lost with questionable research practices, and whether it is better to run a single high-powered study or a study plus a replication at lower powers. AUCs were also used to compare performance across p values, Bayes factors, and effect size (Cohen’s d). AUCs were equivalent for p values and Bayes factors and were slightly higher for effect size. Signal detection analysis provides separate measures of discriminability and bias. With respect to bias, the specific thresholds that produced maximally-optimal utility depended on sample size, although this dependency was particularly notable for p values and less so for Bayes factors. The application of signal detection theory to the issue of statistical significance highlights the need to focus on both false alarms and misses, rather than false alarms alone.

Lag Sequential Analysis Using Pronet: Effects of Series Length and Noise

2005 ◽  
Vol 49 (24) ◽  
pp. 2115-2119
Author(s):  
Preston A. Kiekel ◽  
Jennifer Winner
Keyword(s):  
Time Series ◽  
False Alarm ◽  
Sample Size ◽  
False Alarm Rate ◽  
Sequential Analysis ◽  
Future Research ◽  
False Alarms ◽  
Event Data ◽  
Series Length ◽  
Lag Sequential Analysis

Procedural Networks (ProNet) is presented as a form of lag sequential analysis, but using the Pathfinder algorithm as its kernel. In general, Lag Sequential Analysis is a tool used to identify representative chains of events in a time series of event data. A simulation is used to compare the performance of ProNet, as sample size and noise are varied in the time series. Noise in the data leads to fewer hits, but does not affect false alarm rate. Increased sample size leads to fewer false alarms and more hits. The method is defined and developed, and future research is discussed.

scholarly journals Machine Learning Approach to Combat False Alarms in Wireless Intrusion Detection System

2018 ◽  
Vol 11 (3) ◽  
pp. 67 ◽  
Author(s):  
D. Sudaroli Vijayakumar ◽  
S. Ganapathy
Keyword(s):  
Machine Learning ◽  
Wireless Networks ◽  
Intrusion Detection ◽  
False Alarm ◽  
False Alarm Rate ◽  
Intrusion Detection Systems ◽  
False Alarms ◽  
Detection Systems ◽  
High Detection ◽  
False Alarm Rates

Wireless Networks facilitate the ease of communication for sharing the crucial information. Recently, most of the small and large-scale companies, educational institutions, government organizations, medical sectors, military and banking sectors are using the wireless networks. Security threats, a common term found both in wired as well as in wireless networks. However, it holds lot of importance in wireless networks because of its susceptible nature to threats. Security concerns in WLAN are studied and many organizations concluded that Wireless Intrusion Detection Systems (WIDS) is an essential element in network security infrastructure to monitor wireless activity for signs of attacks. However, it is an indisputable fact that the art of detecting attacks remains in its infancy. WIDS generally collect the activities within the protected network and analyze them to detect intrusions and generates an intrusion alarm. Irrespective of the different types of Intrusion Detection Systems, the major problems arising with WIDS is its inability to handle large volumes of alarms and more prone to false alarm attacks. Reducing the false alarms can improve the overall efficiency of the WIDS. Many techniques have been proposed in the literature to reduce the false alarm rates. However, most of the existing techniques are failed to provide desirable result and the high complexity to achieve high detection rate with less false alarm rates. This is the right time to propose a new technique for providing high detection accuracy with less false alarm rate. This paper made an extensive survey about the role of machine learning techniques to reduce the false alarm rate in WLAN IEEE 802.11. This survey proved that the substantial improvement has been achieved by reducing false alarm rate through machine learning algorithms. In addition to that, advancements specific to machine learning approaches is studied meticulously and a filtration technique is proposed.

Memory for Radio Advertisements: the Effect of Program and Typicality

2013 ◽  
Vol 16 ◽  
Author(s):  
Beatriz Martín-Luengo ◽  
Karlos Luna ◽  
Malen Migueles
Keyword(s):  
False Alarm ◽  
Recognition Test ◽  
False Recognition ◽  
Response Criterion ◽  
False Alarms ◽  
Filler Task ◽  
Radio Program ◽  
False Alarm Rates ◽  
True False

AbstractWe examined the influence of the type of radio program on the memory for radio advertisements. We also investigated the role in memory of the typicality (high or low) of the elements of the products advertised. Participants listened to three types of programs (interesting, boring, enjoyable) with two advertisements embedded in each. After completing a filler task, the participants performed a true/false recognition test. Hits and false alarm rates were higher for the interesting and enjoyable programs than for the boring one. There were also more hits and false alarms for the high-typicality elements. The response criterion for the advertisements embedded in the boring program was stricter than for the advertisements in other types of programs. We conclude that the type of program in which an advertisement is inserted and the nature of the elements of the advertisement affect both the number of hits and false alarms and the response criterion, but not the accuracy of the memory.

Standard Asymptotic Theory

2017 ◽  
Author(s):  
Russell Cheng
Keyword(s):  
Asymptotic Theory ◽  
Asymptotic Variance ◽  
Statistical Significance ◽  
Regularity Conditions ◽  
Parameter Estimates ◽  
Asymptotic Results ◽  
Ml Estimation ◽  
True Parameter ◽  
Log Likelihood ◽  
Ml Estimator

This book relies on maximum likelihood (ML) estimation of parameters. Asymptotic theory assumes regularity conditions hold when the ML estimator is consistent. Typically an additional third derivative condition is assumed to ensure that the ML estimator is also asymptotically normally distributed. Standard asymptotic results that then hold are summarized in this chapter; for example, the asymptotic variance of the ML estimator is then given by the Fisher information formula, and the log-likelihood ratio, the Wald and the score statistics for testing the statistical significance of parameter estimates are all asymptotically equivalent. Also, the useful profile log-likelihood then behaves exactly as a standard log-likelihood only in a parameter space of just one dimension. Further, the model can be reparametrized to make it locally orthogonal in the neighbourhood of the true parameter value. The large exponential family of models is briefly reviewed where a unified set of regular conditions can be obtained.

scholarly journals Realizing Target Detection in SAR Images Based on Multiscale Superpixel Fusion

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1643
Author(s):  
Ming Liu ◽  
Shichao Chen ◽  
Fugang Lu ◽  
Mengdao Xing ◽  
Jingbiao Wei
Keyword(s):  
Synthetic Aperture Radar ◽  
False Alarm ◽  
False Alarm Rate ◽  
Target Detection ◽  
Synthetic Aperture ◽  
False Alarms ◽  
Sar Image ◽  
Constant False Alarm Rate ◽  
Sar Images ◽  
Complex Scenes

For target detection in complex scenes of synthetic aperture radar (SAR) images, the false alarms in the land areas are hard to eliminate, especially for the ones near the coastline. Focusing on the problem, an algorithm based on the fusion of multiscale superpixel segmentations is proposed in this paper. Firstly, the SAR images are partitioned by using different scales of superpixel segmentation. For the superpixels in each scale, the land-sea segmentation is achieved by judging their statistical properties. Then, the land-sea segmentation results obtained in each scale are combined with the result of the constant false alarm rate (CFAR) detector to eliminate the false alarms located on the land areas of the SAR image. In the end, to enhance the robustness of the proposed algorithm, the detection results obtained in different scales are fused together to realize the final target detection. Experimental results on real SAR images have verified the effectiveness of the proposed algorithm.

Sign in / Sign up

Export Citation Format

Share Document