An Automatic Evaluation Method for Parkinson's Dyskinesia Using Finger Tapping Video for Small Samples

PurposeThe assessment of dyskinesia in Parkinson's disease (PD) based on Artificial Intelligence technology is a significant and challenging task. At present, doctors usually use MDS-UPDRS scale to assess the severity of patients. This method is time-consuming and laborious, and there are subjective differences. The evaluation method based on sensor equipment is also widely used, but this method is expensive and needs professional guidance, which is not suitable for remote evaluation and patient self-examination. In addition, it is difficult to collect patient data in medical research, so it is of great significance to find an objective and automatic assessment method for Parkinson's dyskinesia based on small samples.MethodsIn this study, we design an automatic evaluation method combining manual features and convolutional neural network (CNN), which is suitable for small sample classification. Based on the finger tapping video of Parkinson's patients, we use the pose estimation model to obtain the action skeleton information and calculate the feature data. We then use the 5-folds cross validation training model to achieve optimum trade-of between bias and variance, and finally make multi-class prediction through fully connected network (FCN). ResultsOur proposed method achieves the current optimal accuracy of 79.7% in this research. We have compared with the latest methods of related research, and our method is superior to them in terms of accuracy, number of parameters and FLOPs. ConclusionThe method in this paper does not require patients to wear sensor devices, and has obvious advantages in remote clinical evaluation. At the same time, the method of using motion feature data to train CNN model obtains the optimal accuracy, effectively solves the problem of difficult data acquisition in medicine, and provides a new idea for small sample classification.

Guidance to Pre-tokeniztion for SacreBLEU: Meta-Evaluation in Korean

SacreBLEU, by incorporating a text normalizing step in the pipeline, has been well-received as an automatic evaluation metric in recent years. With agglutinative languages such as Korean, however, the metric cannot provide a conceivable result without the help of customized pre-tokenization. In this regard, this paper endeavors to examine the influence of diversified pre-tokenization schemes –word, morpheme, character, and subword– on the aforementioned metric by performing a meta-evaluation with manually-constructed into-Korean human evaluation data. Our empirical study demonstrates that the correlation of SacreBLEU (to human judgment) fluctuates consistently by the token type. The reliability of the metric even deteriorates due to some tokenization, and MeCab is not an exception. Guiding through the proper usage of tokenizer for each metric, we stress the significance of a character level and the insignificance of a Jamo level in MT evaluation.

Study on Automatic Evaluation Method of Spoken English Based on Multimodal Discourse Analysis Theory

Aiming at the low confidence of traditional spoken English automatic evaluation methods, this study designs an automatic evaluation method of spoken English based on multimodal discourse analysis theory. This evaluation method uses sound sensors to collect spoken English pronunciation signals, decomposes the spoken English speech signals by multilayer wavelet feature scale transform, and carries out adaptive filter detection and spectrum analysis on spoken English speech signals according to the results of feature decomposition. Based on multimodal discourse analysis theory, this evaluation method can extract the automatic evaluation features of spoken English and automatically recognize the speech quality according to the results. The experimental results show that, compared with the control group, the designed evaluation method has obvious advantages in confidence evaluation and can solve the problem of low confidence of traditional oral automatic evaluation methods.

Developing Methods to Evaluate Content Quality of Dementia Websites

With the popularity of the Internet, consumers are likely to resort to websites for dementia information. However, they may not have the knowledge or experience in distinguishing quality information from opinion pieces. This study investigated the developing methods, instruments and parameters for evaluating the content quality of dementia websites. By reviewing 18 existing instruments from the relevant literature, we identified four developing methods – questionnaire survey, automatic evaluation, Delphi method and focus group discussion. These instruments include six parameters – reliability, currency, readability, disclosure, objectivity and relevance – to evaluate the content quality. With the significant social and economic impact of dementia, developing specific instruments to measure the content quality of dementia websites is necessary.

Subjective Answer Evaluator

The automatic assessment of subjective replies necessitates the use of Natural Language Processing and automated assessment. Ontology, semantic similarity matching, and statistical approaches are among the strategies employed. But most of the methods are based on an unsupervised approach. The proposed system uses an unsupervised method and is divided into two modules. The first one is extracting the essential data through text summarization and the second is applying various Natural Language models to the text retrieved from the above step and giving marks to them. Keywords: Automatic Evaluation, NLP, Text Summarization, Similarity Measure, Marks Scoring

I know what you coded last summer

The outbreak of the COVID-19 pandemic caused a surge in enrollments in online courses. Consequently, this boost in numbers of students affected teachers ability to evaluate exercises and resolve doubts. In this context, tools designed to evaluate and provide feedback on code solutions can be used in programming courses to reduce teachers workload. Nonetheless, even with using such tools, the literature shows that learning how to program is a challenging task. Programming is complex and the programming language employed can also affect students outcomes. Thus, designing good exercises can reduce students difficulties in identifying the problem and help reduce syntax challenges. This research employs learning analytics processes on automatic evaluation tools interaction logs and code solutions to find metrics capable of identifying problematic exercises and their difficulty. In this context, an exercise is considered problematic if students have problems interpreting its description or its solution requires complex programming structures like loops, conditionals and recursion. The data comes from online introductory programming courses. Results show that the computed metrics can identify problematic exercises, as well as those that are being challenging.