scholarly journals Detecting Symptom Errors in Neural Machine Translation of Patient Health Information on Depressive Disorders: Developing Interpretable Bayesian Machine Learning Classifiers

2021 ◽  
Vol 12 ◽  
Author(s):  
Wenxiu Xie ◽  
Meng Ji ◽  
Mengdan Zhao ◽  
Tianqi Zhou ◽  
Fan Yang ◽  
...  
Keyword(s):  
Machine Learning ◽  
Health Information ◽  
Machine Translation ◽  
Posterior Probability ◽  
Depressive Disorders ◽  
Diagnostic Utility ◽  
Neural Machine Translation ◽  
Machine Learning Classifiers ◽  
Patient Health ◽  
Bayesian Machine Learning

Background: Due to its convenience, wide availability, low usage cost, neural machine translation (NMT) has increasing applications in diverse clinical settings and web-based self-diagnosis of diseases. Given the developing nature of NMT tools, this can pose safety risks to multicultural communities with limited bilingual skills, low education, and low health literacy. Research is needed to scrutinise the reliability, credibility, usability of automatically translated patient health information.Objective: We aimed to develop high-performing Bayesian machine learning classifiers to assist clinical professionals and healthcare workers in assessing the quality and usability of NMT on depressive disorders. The tool did not require any prior knowledge from frontline health and medical professionals of the target language used by patients.Methods: We used Relevance Vector Machine (RVM) to increase generalisability and clinical interpretability of classifiers. It is a typical sparse Bayesian classifier less prone to overfitting with small training datasets. We optimised RVM by leveraging automatic recursive feature elimination and expert feature refinement from the perspective of health linguistics. We evaluated the diagnostic utility of the Bayesian classifier under different probability cut-offs in terms of sensitivity, specificity, positive and negative likelihood ratios against clinical thresholds for diagnostic tests. Finally, we illustrated interpretation of RVM tool in clinic using Bayes' nomogram.Results: After automatic and expert-based feature optimisation, the best-performing RVM classifier (RVM_DUFS12) gained the highest AUC (0.8872) among 52 competing models with distinct optimised, normalised features sets. It also had statistically higher sensitivity and specificity compared to other models. We evaluated the diagnostic utility of the best-performing model using Bayes' nomogram: it had a positive likelihood ratio (LR+) of 4.62 (95% C.I.: 2.53, 8.43), and the associated posterior probability (odds) was 83% (5.0) (95% C.I.: 73%, 90%), meaning that approximately 10 in 12 English texts with positive test are likely to contain information that would cause clinically significant conceptual errors if translated by Google; it had a negative likelihood ratio (LR-) of 0.18 (95% C.I.: 0.10,0.35) and associated posterior probability (odds) was 16% (0.2) (95% C.I: 10%, 27%), meaning that about 10 in 12 English texts with negative test can be safely translated using Google.

Detecting Critical Conceptual Mistakes in Google Translated Medical Information on Infectious Diseases: using Bayesian Machine Learning Classifiers (Preprint)

2021 ◽  
Author(s):  
Wenxiu Xie ◽  
Meng Ji ◽  
Tianyong Hao ◽  
Chi-Yin Chow
Keyword(s):  
Machine Learning ◽  
Health Education ◽  
Infectious Diseases ◽  
Machine Translation ◽  
Semantic Features ◽  
Machine Learning Classifiers ◽  
Combined Features ◽  
Sensitivity Specificity ◽  
Translation Errors ◽  
Bayesian Machine Learning

UNSTRUCTURED Objective: To determine the linguistic/textual features of English health educational materials for predicting the probabilistic distribution of critical conceptual mistakes in neural machine translations (Google Translate: English to Chinese) of public-oriented online health resources on infectious diseases and viruses. Methods: We collected 200 English source texts on infectious diseases and their human translations to Chinese from HON. Net certified health education websites. Human translations were compared with machine translations (Google Translate) by native Chinese speakers to identify critical conceptual mistakes. To overcome overfitting issues of machine learning with small, high-dimensional datasets, Bayesian machine learning classifiers (relevance vector machine, RVM) was trained (70% and 30% train/test data split; 5-fold cross-validation) on English source texts classified as linked or not with machine translation outputs containing critical conceptual mistakes, to identify possible source text features causing clinically significant machine translation errors. We compared the performance of RVM with the combined features through separate optimization (CFSO: 21), to RVM trained on the original combined features (OCF: 135) (20 structural; 115 semantic features), combined features through joint optimization (CFJO: 48); optimized structural features (OTF: 5), and optimized semantic features (OSF: 16). In addition, RVM (CFSO) was compared to classifiers using individual standard (currently available) parameters to measure English complexity (Flesch Reading Ease FRE; Gunning Fog Index - GFI; SMOG Readability Index-SMOG). Results: The AUC, sensitivity, specificity and accuracy of RVM MLCs trained on different features sets were: CFSO (AUC: 0.685; sensitivity: 0.73, specificity: 0.63; accuracy: 0.68); OCF (AUC: 0.7; sensitivity: 0.42, specificity: 0.8; accuracy: 0.625); CFJO (AUC: 0.690; sensitivity: 0.54, specificity: 0.73; accuracy: 0.64); OTF (AUC: 0.587; sensitivity: 0.58, specificity: 0.53; accuracy: 0.55); OSF (AUC: 0.679; sensitivity: 0.58, specificity: 0.67; accuracy: 0.625). The best-performing model was RVM trained on the combined features through separate optimisation (CFSO) (16% of the original combined features). RVM (CFSO) outperformed binary classifiers (BCs) using standard English readability tests. The accuracy, sensitivity, specificity of the three BCs were FRE (accuracy 0.457; sensitivity 0.903, specificity 0.011); GFI (accuracy 0.5735; sensitivity 0.685, specificity 0.462); SMOG (accuracy 0.568; sensitivity 0.674, specificity 0.462). Conclusion: Our study found that machine-generated Chinese medical translation errors were not caused by difficult medical jargon or a lack of readability of source language information. It was certain English structures (passive voices; sentences starting with conjunctions), semantic polysemy (different meanings of a word when used in common versus specialized domains) which tend to cause critical conceptual mistakes in neural machine translation systems (English to Chinese) of health education information on infectious diseases.

scholarly journals Forecasting the Suitability of Online Mental Health Information for Effective Self-Care Developing Machine Learning Classifiers Using Natural Language Features

2021 ◽  
Vol 18 (19) ◽  
pp. 10048
Author(s):  
Meng Ji ◽  
Wenxiu Xie ◽  
Riliu Huang ◽  
Xiaobo Qian
Keyword(s):  
Mental Health ◽  
Machine Learning ◽  
Health Information ◽  
Self Care ◽  
Health Resources ◽  
Health Issues ◽  
Mental Health Information ◽  
Learning Classifier ◽  
Mental Health Issues ◽  
Bayesian Machine Learning

Background: Online mental health information represents important resources for people living with mental health issues. Suitability of mental health information for effective self-care remains understudied, despite the increasing needs for more actionable mental health resources, especially among young people. Objective: We aimed to develop Bayesian machine learning classifiers as data-based decision aids for the assessment of the actionability of credible mental health information for people with mental health issues and diseases. Methods: We collected and classified creditable online health information on mental health issues into generic mental health (GEN) information and patient-specific (PAS) mental health information. GEN and PAS were both patient-oriented health resources developed by health authorities of mental health and public health promotion. GENs were non-classified online health information without indication of targeted readerships; PASs were developed purposefully for specific populations (young, elderly people, pregnant women, and men) as indicated by their website labels. To ensure the generalisability of our model, we chose to develop a sparse Bayesian machine learning classifier using Relevance Vector Machine (RVM). Results: Using optimisation and normalisation techniques, we developed a best-performing classifier through joint optimisation of natural language features and min-max normalisation of feature frequencies. The AUC (0.957), sensitivity (0.900), and specificity (0.953) of the best model were statistically higher (p < 0.05) than other models using parallel optimisation of structural and semantic features with or without feature normalisation. We subsequently evaluated the diagnostic utility of our model in the clinic by comparing its positive (LR+) and negative likelihood ratios (LR−) and 95% confidence intervals (95% C.I.) as we adjusted the probability thresholds with the range of 0.1 and 0.9. We found that the best pair of LR+ (18.031, 95% C.I.: 10.992, 29.577) and LR− (0.100, 95% C.I.: 0.068, 0.148) was found when the probability threshold was set to 0.45 associated with a sensitivity of 0.905 (95%: 0.867, 0.942) and specificity of 0.950 (95% C.I.: 0.925, 0.975). These statistical properties of our model suggested its applicability in the clinic. Conclusion: Our study found that PAS had significant advantage over GEN mental health information regarding information actionability, engagement, and suitability for specific populations with distinct mental health issues. GEN is more suitable for general mental health information acquisition, whereas PAS can effectively engage patients and provide more effective and needed self-care support. The Bayesian machine learning classifier developed provided automatic tools to support decision making in the clinic to identify more actionable resources, effective to support self-care among different populations.

scholarly journals Supporting Risk-Aware Use of Online Translation Tools in Delivering Mental Healthcare Services among Spanish-Speaking Populations

2021 ◽  
Vol 2021 ◽  
pp. 1-13
Author(s):  
Wenxiu Xie ◽  
Meng Ji ◽  
Mengdan Zhao ◽  
Xiaobo Qian ◽  
Chi-Yin Chow ◽  
...  
Keyword(s):  
Machine Learning ◽  
Machine Translation ◽  
Mental Healthcare ◽  
Healthcare Services ◽  
Likelihood Ratios ◽  
Spanish Speaking ◽  
Healthcare Settings ◽  
Learning Classifier ◽  
Translation Tools ◽  
Bayesian Machine Learning

Neural machine translation technologies are having increasing applications in clinical and healthcare settings. In multicultural countries, automatic translation tools provide critical support to medical and health professionals in their interaction and exchange of health messages with migrant patients with limited or non-English proficiency. While research has mainly explored the usability and limitations of state-of-the-art machine translation tools in the detection and diagnosis of physical diseases and conditions, there is a persistent lack of evidence-based studies on the applicability of machine translation tools in the delivery of mental healthcare services for vulnerable populations. Our study developed Bayesian machine learning algorithms using relevance vector machine to support frontline health workers and medical professionals to make better informed decisions between risks and convenience of using online translation tools when delivering mental healthcare services to Spanish-speaking minority populations living in English-speaking countries. Major strengths of the machine learning classifier that we developed include scalability, interpretability, and adaptability of the classifier for diverse mental healthcare settings. In this paper, we report on the process of the Bayesian machine learning classifier development through automatic feature optimisation and the interpretation of the classifier-enabled assessment of the suitability of original English mental health information for automatic online translation. We elaborate on the interpretation of the assessment results in clinical settings using statistical tools such as positive likelihood ratios and negative likelihood ratios.

scholarly journals A Machine Learning Technique to Analyze Depressive Disorders

2021 ◽  
Author(s):  
Dixita Mali ◽  
Kritika Kumawat ◽  
Gaurav Kumawat ◽  
Prasun Chakrabarti ◽  
Sandeep Poddar ◽  
...  
Keyword(s):  
Neural Network ◽  
Mental Health ◽  
Machine Learning ◽  
Depressive Disorders ◽  
Random Trees ◽  
Machine Learning Technique ◽  
Machine Learning Classifiers ◽  
Large Numbers ◽  
Learning Technique ◽  
The Impact

Abstract Depression is an ordinary mental health care problem and the usual cause of disability worldwide. The main purpose of this research was to determine that how depression affects the life of an individual. It is a leading cause of morbidity and death. Over the last 50–60 years, large numbers of studies published various aspects including the impact of depression. The main purpose of this research is to determine whether the person is suffering from depression or not. The dataset of Depression has been taken from the Kaggle website. Guided Machine Learning classifiers have helped in the highest accuracy of a dataset. Classifiers like XGBoost Tree, Random Trees, Neural Network, SVM, Random Forest, C5.0, and Bay Net. From the result, it is evident that the C5.0 classifier is giving the highest accuracy with 83.94 % and for each classifier, the result is derived based without pre-processing.

Bayesian Machine Learning

2005 ◽  
pp. 229-235
Author(s):  
Eitel J.M. Lauria
Keyword(s):  
Machine Learning ◽  
Observational Data ◽  
Bayesian Methods ◽  
Posterior Probability ◽  
Prior Probability ◽  
Probabilistic Approach ◽  
Bayes Theorem ◽  
Bayesian Framework ◽  
Probability Models ◽  
Bayesian Machine Learning

Bayesian methods provide a probabilistic approach to machine learning. The Bayesian framework allows us to make inferences from data using probability models for values we observe and about which we want to draw some hypotheses. Bayes theorem provides the means of calculating the probability of a hypothesis (posterior probability) based on its prior probability, the probability of the observations and the likelihood that the observational data fit the hypothesis.

scholarly journals Machine learning: Implications for translator education

2017 ◽  
Vol 62 (2) ◽  
Author(s):  
Gary Massey ◽  
Maureen Ehrensberger-Dow
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Machine Translation ◽  
New Technologies ◽  
Added Value ◽  
Neural Machine Translation ◽  
Computer Aided ◽  
Translator Education

AbstractMachines are learning fast, and human translators must keep pace by learning with, from and about them. Deep learning (DL) and neural machine translation (NMT) are set to change the reality of translation and the distributions of tasks. Although theoretical and practical courses on computer-aided and/or machine translation abound, less attention has been paid to DL and NMT in most translation programmes. The challenge for translation education is to give students the knowledge and toolkits to learn when and how to embrace the new technologies, and to exploit how and when the added value of human intuition, creativity and ethics can and should be deployed.

scholarly journals Neural Machine Translation using Recurrent Neural Network

2020 ◽  
Vol 9 (4) ◽  
pp. 1395-1400
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Machine Translation ◽  
Recurrent Neural Network ◽  
Machine Learning Algorithms ◽  
The Other ◽  
Neural Machine Translation ◽  
To Come ◽  
Artificial Neural Network Ann ◽  
Translation Systems

In this era of globalization, it is quite likely to come across people or community who do not share the same language for communication as us. To acknowledge the problems caused by this, we have machine translation systems being developed. Developers of several reputed organizations like Google LLC, have been working to bring algorithms to support machine translations using machine learning algorithms like Artificial Neural Network (ANN) in order to facilitate machine translation. Several Neural Machine Translations have been developed in this regard, but Recurrent Neural Network (RNN), on the other hand, has not grown much in this field. In our work, we have tried to bring RNN in the field of machine translations, in order to acknowledge the benefits of RNN over ANN. The results show how RNN is able to perform machine translations with proper accuracy.

Bayesian Machine Learning Classifiers for Combining Structural and Functional Measurements to Classify Healthy and Glaucomatous Eyes

2008 ◽  
Vol 49 (3) ◽  
pp. 945 ◽  
Author(s):  
Christopher Bowd ◽  
Jiucang Hao ◽  
Ivan M. Tavares ◽  
Felipe A. Medeiros ◽  
Linda M. Zangwill ◽  
...  
Keyword(s):  
Machine Learning ◽  
Machine Learning Classifiers ◽  
Learning Classifiers ◽  
Bayesian Machine Learning
Sign in / Sign up

Export Citation Format

Share Document