scholarly journals Classification Accuracy of Multidimensional Tests: Quantifying the Impact of Noninvariance

2022 ◽  
pp. 1-10
Author(s):  
Mark H. C. Lai ◽  
Yichi Zhang
Keyword(s):  
Classification Accuracy ◽  
Multidimensional Tests ◽  
The Impact

scholarly journals Investigating the Impact of Noneffortful Responses on Individual-Level Scores: Can the Effort-Moderated IRT Model Serve as a Solution?

2021 ◽  
pp. 014662162110138
Author(s):  
Joseph A. Rios ◽  
James Soland
Keyword(s):  
Classification Accuracy ◽  
Negative Impact ◽  
Parameter Recovery ◽  
Individual Level ◽  
Irt Model ◽  
Ability Estimates ◽  
Misclassification Rates ◽  
Individual Scores ◽  
3Pl Model ◽  
The Impact

Suboptimal effort is a major threat to valid score-based inferences. While the effects of such behavior have been frequently examined in the context of mean group comparisons, minimal research has considered its effects on individual score use (e.g., identifying students for remediation). Focusing on the latter context, this study addressed two related questions via simulation and applied analyses. First, we investigated how much including noneffortful responses in scoring using a three-parameter logistic (3PL) model affects person parameter recovery and classification accuracy for noneffortful responders. Second, we explored whether improvements in these individual-level inferences were observed when employing the Effort Moderated IRT (EM-IRT) model under conditions in which its assumptions were met and violated. Results demonstrated that including 10% noneffortful responses in scoring led to average bias in ability estimates and misclassification rates by as much as 0.15 SDs and 7%, respectively. These results were mitigated when employing the EM-IRT model, particularly when model assumptions were met. However, once model assumptions were violated, the EM-IRT model’s performance deteriorated, though still outperforming the 3PL model. Thus, findings from this study show that (a) including noneffortful responses when using individual scores can lead to potential unfounded inferences and potential score misuse, and (b) the negative impact that noneffortful responding has on person ability estimates and classification accuracy can be mitigated by employing the EM-IRT model, particularly when its assumptions are met.

scholarly journals Classification of standing and sitting phases based on in-socket piezoelectric sensors in a transfemoral amputee

2020 ◽  
Vol 0 (0) ◽  
Author(s):  
Tawfik Yahya ◽  
Nur Azah Hamzaid ◽  
Sadeeq Ali ◽  
Farahiyah Jasni ◽  
Hanie Nadia Shasmin
Keyword(s):  
Time Domain ◽  
Classification Accuracy ◽  
Sensory System ◽  
Metabolic Energy ◽  
Support Vector ◽  
Data Set ◽  
Transfemoral Amputee ◽  
Sit To Stand ◽  
Optimal Classifier ◽  
The Impact

AbstractA transfemoral prosthesis is required to assist amputees to perform the activity of daily living (ADL). The passive prosthesis has some drawbacks such as utilization of high metabolic energy. In contrast, the active prosthesis consumes less metabolic energy and offers better performance. However, the recent active prosthesis uses surface electromyography as its sensory system which has weak signals with microvolt-level intensity and requires a lot of computation to extract features. This paper focuses on recognizing different phases of sitting and standing of a transfemoral amputee using in-socket piezoelectric-based sensors. 15 piezoelectric film sensors were embedded in the inner socket wall adjacent to the most active regions of the agonist and antagonist knee extensor and flexor muscles, i. e. region with the highest level of muscle contractions of the quadriceps and hamstring. A male transfemoral amputee wore the instrumented socket and was instructed to perform several sitting and standing phases using an armless chair. Data was collected from the 15 embedded sensors and went through signal conditioning circuits. The overlapping analysis window technique was used to segment the data using different window lengths. Fifteen time-domain and frequency-domain features were extracted and new feature sets were obtained based on the feature performance. Eight of the common pattern recognition multiclass classifiers were evaluated and compared. Regression analysis was used to investigate the impact of the number of features and the window lengths on the classifiers’ accuracies, and Analysis of Variance (ANOVA) was used to test significant differences in the classifiers’ performances. The classification accuracy was calculated using k-fold cross-validation method, and 20% of the data set was held out for testing the optimal classifier. The results showed that the feature set (FS-5) consisting of the root mean square (RMS) and the number of peaks (NP) achieved the highest classification accuracy in five classifiers. Support vector machine (SVM) with cubic kernel proved to be the optimal classifier, and it achieved a classification accuracy of 98.33 % using the test data set. Obtaining high classification accuracy using only two time-domain features would significantly reduce the processing time of controlling a prosthesis and eliminate substantial delay. The proposed in-socket sensors used to detect sit-to-stand and stand-to-sit movements could be further integrated with an active knee joint actuation system to produce powered assistance during energy-demanding activities such as sit-to-stand and stair climbing. In future, the system could also be used to accurately predict the intended movement based on their residual limb’s muscle and mechanical behaviour as detected by the in-socket sensory system.

scholarly journals Leveraging Road Characteristics and Contributor Behaviour for Assessing Road Type Quality in OSM

2021 ◽  
Vol 10 (7) ◽  
pp. 436
Author(s):  
Amerah Alghanim ◽  
Musfira Jilani ◽  
Michela Bertolotto ◽  
Gavin McArdle
Keyword(s):  
Machine Learning ◽  
Spatial Data ◽  
Classification Accuracy ◽  
Supervised Machine Learning ◽  
Machine Learning Techniques ◽  
Data Set ◽  
Semantic Inference ◽  
Road Type ◽  
The Impact

Volunteered Geographic Information (VGI) is often collected by non-expert users. This raises concerns about the quality and veracity of such data. There has been much effort to understand and quantify the quality of VGI. Extrinsic measures which compare VGI to authoritative data sources such as National Mapping Agencies are common but the cost and slow update frequency of such data hinder the task. On the other hand, intrinsic measures which compare the data to heuristics or models built from the VGI data are becoming increasingly popular. Supervised machine learning techniques are particularly suitable for intrinsic measures of quality where they can infer and predict the properties of spatial data. In this article we are interested in assessing the quality of semantic information, such as the road type, associated with data in OpenStreetMap (OSM). We have developed a machine learning approach which utilises new intrinsic input features collected from the VGI dataset. Specifically, using our proposed novel approach we obtained an average classification accuracy of 84.12%. This result outperforms existing techniques on the same semantic inference task. The trustworthiness of the data used for developing and training machine learning models is important. To address this issue we have also developed a new measure for this using direct and indirect characteristics of OSM data such as its edit history along with an assessment of the users who contributed the data. An evaluation of the impact of data determined to be trustworthy within the machine learning model shows that the trusted data collected with the new approach improves the prediction accuracy of our machine learning technique. Specifically, our results demonstrate that the classification accuracy of our developed model is 87.75% when applied to a trusted dataset and 57.98% when applied to an untrusted dataset. Consequently, such results can be used to assess the quality of OSM and suggest improvements to the data set.

scholarly journals Patient-specific optimization of automated detection improves seizure onset zone localization based on high frequency oscillations

2020 ◽  
Author(s):  
Casey L. Trevino ◽  
Jack J. Lin ◽  
Indranil Sen-Gupta ◽  
Beth A. Lopour
Keyword(s):  
High Frequency ◽  
Classification Accuracy ◽  
Patient Specific ◽  
Seizure Onset ◽  
Data Set ◽  
Localization Accuracy ◽  
High Frequency Oscillations ◽  
Seizure Onset Zone ◽  
Wide Range ◽  
The Impact

AbstractHigh frequency oscillations (HFOs) are a promising biomarker of epileptogenicity, and automated algorithms are critical tools for their detection. However, previously validated algorithms often exhibit decreased HFO detection accuracy when applied to a new data set, if the parameters are not optimized. This likely contributes to decreased seizure localization accuracy, but this has never been tested. Therefore, we evaluated the impact of parameter selection on seizure onset zone (SOZ) localization using automatically detected HFOs. We detected HFOs in intracranial EEG from twenty medically refractory epilepsy patients with seizure free surgical outcomes using an automated algorithm. For each patient, we assessed classification accuracy of channels inside/outside the SOZ using a wide range of detection parameters and identified the parameters associated with maximum classification accuracy. We found that only three out of twenty patients achieved maximal localization accuracy using conventional HFO detection parameters, and optimal parameter ranges varied significantly across patients. The parameters for amplitude threshold and root-mean-square window had the greatest impact on SOZ localization accuracy; minimum event duration and rejection of false positive events did not significantly affect the results. Using individualized optimal parameters led to substantial improvements in localization accuracy, particularly in reducing false positives from non-SOZ channels. We conclude that optimal HFO detection parameters are patient-specific, often differ from conventional parameters, and have a significant impact on SOZ localization. This suggests that individual variability should be considered when implementing automatic HFO detection as a tool for surgical planning.

scholarly journals Impact of Low Resolution on Image Recognition with Deep Neural Networks: An Experimental Study

2018 ◽  
Vol 28 (4) ◽  
pp. 735-744 ◽  
Author(s):  
Michał Koziarski ◽  
Bogusław Cyganek
Keyword(s):  
Neural Networks ◽  
Image Recognition ◽  
Classification Accuracy ◽  
Deep Neural Networks ◽  
Dynamic Range ◽  
Super Resolution ◽  
Image Resolution ◽  
Quality Data ◽  
Low Resolution ◽  
The Impact

Abstract Due to the advances made in recent years, methods based on deep neural networks have been able to achieve a state-of-the-art performance in various computer vision problems. In some tasks, such as image recognition, neural-based approaches have even been able to surpass human performance. However, the benchmarks on which neural networks achieve these impressive results usually consist of fairly high quality data. On the other hand, in practical applications we are often faced with images of low quality, affected by factors such as low resolution, presence of noise or a small dynamic range. It is unclear how resilient deep neural networks are to the presence of such factors. In this paper we experimentally evaluate the impact of low resolution on the classification accuracy of several notable neural architectures of recent years. Furthermore, we examine the possibility of improving neural networks’ performance in the task of low resolution image recognition by applying super-resolution prior to classification. The results of our experiments indicate that contemporary neural architectures remain significantly affected by low image resolution. By applying super-resolution prior to classification we were able to alleviate this issue to a large extent as long as the resolution of the images did not decrease too severely. However, in the case of very low resolution images the classification accuracy remained considerably affected.

scholarly journals Coupling Uncertainties with Accuracy Assessment in Object-based Slum Detections, Case Study: Jakarta, Indonesia

2017 ◽  
Author(s):  
Jati Pratomo ◽  
Monika Kuffer ◽  
Javier Martinez ◽  
Divyani Kohli
Keyword(s):  
Land Tenure ◽  
Classification Accuracy ◽  
Accuracy Assessment ◽  
False Negative ◽  
National Policy ◽  
True Positive ◽  
True Negative ◽  
Object Based ◽  
The Impact

Object-Based Image Analysis (OBIA) has been successfully used to map slums. In general, the occurrence of uncertainties in producing geographic data is inevitable. However, most studies concentrated solely on assessing the classification accuracy and neglecting the inherent uncertainties. Our research analyses the impact of uncertainties in measuring the accuracy of OBIA-based slum detection. We selected Jakarta as our case study area, because of a national policy of slum eradication, which is causing rapid changes in slum areas. Our research comprises of four parts: slum conceptualization, ruleset development, implementation, and accuracy and uncertainty measurements. Existential and extensional uncertainty arise when producing reference data. The comparison of a manual expert delineations of slums with OBIA slum classification results into four combinations: True Positive, False Positive, True Negative and False Negative. However, the higher the True Positive (which lead to a better accuracy), the lower the certainty of the results. This demonstrates the impact of extensional uncertainties. Our study also demonstrates the role of non-observable indicators (i.e., land tenure), to assist slum detection, particularly in areas where uncertainties exist. In conclusion, uncertainties are increasing when aiming to achieve a higher classification accuracy by matching manual delineation and OBIA classification.

ADAPTIVE FUZZY LOGIC-BASED FRAMEWORK FOR HANDLING IMPRECISION AND UNCERTAINTY IN CLASSIFICATION OF BIOINFORMATICS DATASETS

2011 ◽  
Vol 08 (03) ◽  
pp. 513-534
Author(s):  
TAREK HELMY ◽  
ZEHASHEEM RASHEED ◽  
MOHAMED AL-MULHEM
Keyword(s):  
Fuzzy Logic ◽  
Classification Accuracy ◽  
Empirical Studies ◽  
Data Sets ◽  
Training Algorithms ◽  
Classification Framework ◽  
Different Types ◽  
Adaptive Fuzzy Logic ◽  
And Performance ◽  
The Impact

Classification in the emerging field of bioinformatics is a challenging task, because the information about different diseases is either insufficient or lacking in authenticity as data is collected from different types of medical equipments. In addition, the limitation of human expertise in manual diagnoses leads to incorrect diagnoses. Moreover, the information gathered from various sources is subject to imprecision and uncertainty. Imprecision arises when the data is not validated by experts. This paper presents an adaptive Type-2 Fuzzy Logic System-based (FLS) classification framework for multivariate data to diagnose different types of diseases. This framework is capable of handling imprecision and uncertainty, and its classification accuracy and performance are measured by using University of California Irvine (UCI), well-known medical data sets. The results are compared with the most common existing classifiers in both computer science and statistics literatures. This classification is performed based on the nature of inputs (e.g., singleton or nonsingleton) and on whether uncertainty is present in the system or absent. Empirical results have shown that our proposed FLS classification framework outperforms earlier implemented models with better classification accuracy. In addition, we conducted empirical studies on this classifier regarding the impact of various parameters of FLS such as training algorithms and defuzzification methods.

scholarly journals DAMPAK REDENOMINASI TERHADAP INFLASI INDONESIA: PENANGANAN MISSING MENGGUNAKAN METODE CASE DELETION, PMM, RF DAN BAYESIAN

2019 ◽  
Vol 3 (3) ◽  
pp. 272-286
Author(s):  
Windri Wucika Bemi ◽  
Rani Nooraeni
Keyword(s):  
Economic Growth ◽  
Classification Accuracy ◽  
Historical Data ◽  
Negative Impact ◽  
Economic Factors ◽  
Macroeconomic Variables ◽  
Case Deletion ◽  
The Third ◽  
Bayesian Linear Regression ◽  
The Impact

Indonesia is the country with the third largest currency digit after Vietnam and Zimbabwe. In 2010, Indonesia conveyed a discourse on the application of rupiah redenomination, but in its implementation it was necessary to estimate the economic factors that would be affected, especially inflation, where inflation was one of the decisive indicators of the success of the redenomination policy of the currency. To estimate the impact of redenomination on inflation, Indonesia can reflect on the historical data of countries that have implemented the policy. Based on historical data, a model can be applied to Indonesia. Historical data includes macroeconomic variables and forms of government. To get a model with better precision, complete data needs to be considered. The historical missing will make the inferencing obtained invalid and important information that can be used for analysis also diminishes. The case deletion method, mean matching predictive, random forest, and bayesian linear regression can be used to handle it. The results showed that there were 38.18% missing data from total observations and the case deletion method as the best method. Then the condition of hyperinflation, economic growth, and the index of government forms significantly impacted inflation after the implementation of redenomination. So, if Indonesia applies redenomination between the period 2010-2017, with the classification accuracy of 64.71%, it is estimated that it will have a negative impact because the inflation will increase after redenomination is implemented.

Sign in / Sign up

Export Citation Format

Share Document