Assessment of Agreement Between Human Ratings and Lexicon-Based Sentiment Ratings of Open-Ended Responses on a Behavioral Rating Scale

Assessment ◽  
2021 ◽  
pp. 107319112199646
Author(s):  
Olivia Gratz ◽  
Duncan Vos ◽  
Megan Burke ◽  
Neelkamal Soares
Keyword(s):  
Sentiment Analysis ◽  
Language Processing ◽  
Rating Scales ◽  
Clinical Decision Making ◽  
Rating Scale ◽  
Clinical Decision ◽  
Assessment System ◽  
Word Count ◽  
Positive Correlation ◽  
Sentiment Score

To date, there is a paucity of research conducting natural language processing (NLP) on the open-ended responses of behavior rating scales. Using three NLP lexicons for sentiment analysis of the open-ended responses of the Behavior Assessment System for Children-Third Edition, the researchers discovered a moderately positive correlation between the human composite rating and the sentiment score using each of the lexicons for strengths comments and a slightly positive correlation for the concerns comments made by guardians and teachers. In addition, the researchers found that as the word count increased for open-ended responses regarding the child’s strengths, there was a greater positive sentiment rating. Conversely, as word count increased for open-ended responses regarding child concerns, the human raters scored comments more negatively. The authors offer a proof-of-concept to use NLP-based sentiment analysis of open-ended comments to complement other data for clinical decision making.

Patterns of admission to acute psychiatric in-patient facilities: a national survey in Italy

2008 ◽  
Vol 39 (3) ◽  
pp. 485-496 ◽  
Author(s):  
A. Preti ◽  
P. Rucci ◽  
G. Santone ◽  
A. Picardi ◽  
R. Miglio ◽  
...  
Keyword(s):  
Rating Scales ◽  
Clinical Decision Making ◽  
Rating Scale ◽  
Social Withdrawal ◽  
Social Performance ◽  
Family Members ◽  
Clinical Decision ◽  
Public And Private ◽  
Affective Psychoses ◽  
Psychiatric Rating Scale

BackgroundA proper understanding of patterns of care represents a crucial step in improving clinical decision making and enhancing service provision. Only a few studies, however, have explored global patterns of psychiatric admissions nationwide, and none have been undertaken in Italy.MethodSociodemographic, clinical and treatment-related information was collected for 1577 patients admitted to 130 public and 36 private in-patient facilities in Italy during an index period in the year 2004. All patients were also rated using the 24-item Brief Psychiatric Rating Scale (BPRS) and the Personal and Social Performance (PSP) rating scales.ResultsNon-affective psychoses (36%) were the most common diagnoses and accounted to a large extent for compulsory admissions. Private facilities were more likely to admit patients with organic mental disorders and substance abuse/dependence and less likely to admit patients with non-affective psychoses. Overall, 77.8% of patients had been receiving treatment by a mental health professional in the month prior to admission. In 54% of cases, the admission was solicited by patients' family members. The main factors preceding admission were impairment in work or social functioning, social withdrawal, and conflict with family members. Agitation, delusions and/or hallucinations, and the presence of multiple problems were associated with compulsory admissions, whereas depressive and anxiety symptoms were associated with voluntary admissions.ConclusionsIn a mixed, public–private psychiatric care system, like the Italian one, public and private facilities admit patients with widely different clinical characteristics and needs. Family support represents an important resource for most patients, and interventions specifically addressed to relieving family burden are warranted.

Preliminary Findings on the Use of Session Questionnaires to Improve Clinician-Client Alliance With Special Consideration to Clinical Education

2015 ◽  
Vol 25 (1) ◽  
pp. 50-60
Author(s):  
Anu Subramanian
Keyword(s):  
Graduate Students ◽  
Clinical Education ◽  
Clinical Decision Making ◽  
Rating Scale ◽  
Treatment Effectiveness ◽  
Clinical Decision ◽  
Added Value ◽  
Therapeutic Outcomes ◽  
Specific Feedback ◽  
Stakeholder Perspective

ASHA's focus on evidence-based practice (EBP) includes the family/stakeholder perspective as an important tenet in clinical decision making. The common factors model for treatment effectiveness postulates that clinician-client alliance positively impacts therapeutic outcomes and may be the most important factor for success. One strategy to improve alliance between a client and clinician is the use of outcome questionnaires. In the current study, eight parents of toddlers who attended therapy sessions at a university clinic responded to a session outcome questionnaire that included both rating scale and descriptive questions. Six graduate students completed a survey that included a question about the utility of the questionnaire. Results indicated that the descriptive questions added value and information compared to using only the rating scale. The students were varied in their responses regarding the effectiveness of the questionnaire to increase their comfort with parents. Information gathered from the questionnaire allowed for specific feedback to graduate students to change behaviors and created opportunities for general discussions regarding effective therapy techniques. In addition, the responses generated conversations between the client and clinician focused on clients' concerns. Involving the stakeholder in identifying both effective and ineffective aspects of therapy has advantages for clinical practice and education.

scholarly journals Desiderata for computable representations of electronic health records-driven phenotype algorithms

2015 ◽  
Vol 22 (6) ◽  
pp. 1220-1230 ◽  
Author(s):  
Huan Mo ◽  
William K Thompson ◽  
Luke V Rasmussen ◽  
Jennifer A Pacheco ◽  
Guoqian Jiang ◽  
...  
Keyword(s):  
Electronic Health Records ◽  
Language Processing ◽  
Clinical Decision Making ◽  
Clinical Decision ◽  
Relational Algebra ◽  
Common Data Model ◽  
Health Records ◽  
Electronic Health ◽  
Value Sets ◽  
Text Searching

Abstract Background Electronic health records (EHRs) are increasingly used for clinical and translational research through the creation of phenotype algorithms. Currently, phenotype algorithms are most commonly represented as noncomputable descriptive documents and knowledge artifacts that detail the protocols for querying diagnoses, symptoms, procedures, medications, and/or text-driven medical concepts, and are primarily meant for human comprehension. We present desiderata for developing a computable phenotype representation model (PheRM). Methods A team of clinicians and informaticians reviewed common features for multisite phenotype algorithms published in PheKB.org and existing phenotype representation platforms. We also evaluated well-known diagnostic criteria and clinical decision-making guidelines to encompass a broader category of algorithms. Results We propose 10 desired characteristics for a flexible, computable PheRM: (1) structure clinical data into queryable forms; (2) recommend use of a common data model, but also support customization for the variability and availability of EHR data among sites; (3) support both human-readable and computable representations of phenotype algorithms; (4) implement set operations and relational algebra for modeling phenotype algorithms; (5) represent phenotype criteria with structured rules; (6) support defining temporal relations between events; (7) use standardized terminologies and ontologies, and facilitate reuse of value sets; (8) define representations for text searching and natural language processing; (9) provide interfaces for external software algorithms; and (10) maintain backward compatibility. Conclusion A computable PheRM is needed for true phenotype portability and reliability across different EHR products and healthcare systems. These desiderata are a guide to inform the establishment and evolution of EHR phenotype algorithm authoring platforms and languages.

scholarly journals Early Identification of Patients with Acute Gastrointestinal Bleeding using Electronic Health Record Phenotyping

2020 ◽  
Author(s):  
Dennis Shung ◽  
Cynthia Tsay ◽  
Loren Laine ◽  
Prem Thomas ◽  
Caitlin Partridge ◽  
...  
Keyword(s):  
Risk Stratification ◽  
Gastrointestinal Bleeding ◽  
Real Time ◽  
Decision Rule ◽  
Language Processing ◽  
Clinical Decision Making ◽  
External Validation ◽  
Clinical Decision ◽  
Internal Validation ◽  
Electronic Health

Background and AimGuidelines recommend risk stratification scores in patients presenting with gastrointestinal bleeding (GIB), but such scores are uncommonly employed in practice. Automation and deployment of risk stratification scores in real time within electronic health records (EHRs) would overcome a major impediment. This requires an automated mechanism to accurately identify (“phenotype”) patients with GIB at the time of presentation. The goal is to identify patients with acute GIB by developing and evaluating EHR-based phenotyping algorithms for emergency department (ED) patients.MethodsWe specified criteria using structured data elements to create rules for identifying patients, and also developed a natural-language-processing (NLP)-based algorithm for automated phenotyping of patients, tested them with tenfold cross-validation (n=7144) and external validation (n=2988), and compared them with the standard method for encoding patient conditions in the EHR, Systematized Nomenclature of Medicine (SNOMED). The gold standard for GIB diagnosis was independent dual manual review of medical records. The primary outcome was positive predictive value (PPV).ResultsA decision rule using GIB-specific terms from ED triage and from ED review-of-systems assessment performed better than SNOMED on internal validation (PPV=91% [90%-93%] vs. 74% [71%-76%], P<0.001) and external validation (PPV=85% [84%-87%] vs. 69% [67%-71%], P<0.001). The NLP algorithm (external validation PPV=80% [79-82%]) was not superior to the structured-datafields decision rule.ConclusionsAn automated decision rule employing GIB-specific triage and review-of-systems terms can be used to trigger EHR-based deployment of risk stratification models to guide clinical decision-making in real time for patients with acute GIB presenting to the ED.

Aspect-Based Sentiment Analysis of Online Product Reviews

2017 ◽  
pp. 175-191 ◽  
Author(s):  
Vinod Kumar Mishra ◽  
Himanshu Tiruwa
Keyword(s):  
Sentiment Analysis ◽  
Computational Linguistics ◽  
Language Processing ◽  
Future Trend ◽  
Product Reviews ◽  
Customer Reviews ◽  
Part Of Speech ◽  
Lexical Approach ◽  
Sentiment Score ◽  
Pos Tagger

Sentiment analysis is a part of computational linguistics concerned with extracting sentiment and emotion from text. It is also considered as a task of natural language processing and data mining. Sentiment analysis mainly concentrate on identifying whether a given text is subjective or objective and if it is subjective, then whether it is negative, positive or neutral. This chapter provide an overview of aspect based sentiment analysis with current and future trend of research on aspect based sentiment analysis. This chapter also provide a aspect based sentiment analysis of online customer reviews of Nokia 6600. To perform aspect based classification we are using lexical approach on eclipse platform which classify the review as a positive, negative or neutral on the basis of features of product. The Sentiwordnet is used as a lexical resource to calculate the overall sentiment score of each sentence, pos tagger is used for part of speech tagging, frequency based method is used for extraction of the aspects/features and used negation handling for improving the accuracy of the system.

scholarly journals P071: Artificial intelligence in emergency medicine: A scoping review

CJEM ◽  
2020 ◽  
Vol 22 (S1) ◽  
pp. S90-S90
Author(s):  
A. Kirubarajan ◽  
A. Taher ◽  
S. Khan ◽  
S. Masood
Keyword(s):  
Artificial Intelligence ◽  
Emergency Medicine ◽  
Language Processing ◽  
Scoping Review ◽  
Clinical Decision Making ◽  
Data Extraction ◽  
A Priori ◽  
Clinical Decision ◽  
Hospital Environment ◽  
Clinical Implementation

Introduction: The study of artificial intelligence (AI) in medicine has become increasingly popular over the last decade. The emergency department (ED) is uniquely situated to benefit from AI due to its power of diagnostic prediction, and its ability to continuously improve with time. However, there is a lack of understanding of the breadth and scope of AI applications in emergency medicine, and evidence supporting its use. Methods: Our scoping review was completed according to PRISMA-ScR guidelines and was published a priori on Open Science Forum. We systematically searched databases (Medline-OVID, EMBASE, CINAHL, and IEEE) for AI interventions relevant to the ED. Study selection and data extraction was performed independently by two investigators. We categorized studies based on type of AI model used, location of intervention, clinical focus, intervention sub-type, and type of comparator. Results: Of the 1483 original database citations, a total of 181 studies were included in the scoping review. Inter-rater reliability for study screening for titles and abstracts was 89.1%, and for full-text review was 77.8%. Overall, we found that 44 (24.3%) studies utilized supervised learning, 63 (34.8%) studies evaluated unsupervised learning, and 13 (7.2%) studies utilized natural language processing. 17 (9.4%) studies were conducted in the pre-hospital environment, with the remainder occurring either in the ED or the trauma bay. The majority of interventions centered around prediction (n = 73, 40.3%). 48 studies (25.5%) analyzed AI interventions for diagnosis. 23 (12.7%) interventions focused on diagnostic imaging. 89 (49.2%) studies did not have a comparator to their AI intervention. 63 (34.8%) studies used statistical models as a comparator, 19 (10.5%) of which were clinical decision making tools. 15 (8.3%) studies used humans as comparators, with 12 of the 15 (80%) studies showing superiority in favour of the AI intervention when compared to a human. Conclusion: AI-related research is rapidly increasing in emergency medicine. AI interventions are heterogeneous in both purpose and design, but primarily focus on predictive modeling. Most studies do not involve a human comparator and lack information on patient-oriented outcomes. While some studies show promising results for AI-based interventions, there remains uncertainty regarding their superiority over standard practice, and further research is needed prior to clinical implementation.

Mutation landscape of Chinese melanoma patients.

2019 ◽  
Vol 37 (15_suppl) ◽  
pp. 9563-9563
Author(s):  
Fuxue Huang ◽  
Dandan Li ◽  
Xizhi Wen ◽  
Fang Wang ◽  
Xiaoshi Zhang ◽  
...  
Keyword(s):  
Cutaneous Melanoma ◽  
Clinical Decision Making ◽  
Treatment Options ◽  
Clinical Decision ◽  
Genomic Profiling ◽  
Primary Tumors ◽  
Positive Correlation ◽  
Mutation Burden ◽  
Frequent Mutations ◽  
Melanoma Patients

9563 Background: Treatment options for melanoma, which has the highest mutation burden among common cancers, has proliferated in the past decade. Genomic profiling has becoming essential to clinical decision making. However, limited studies have interrogated the genomic landscape of Chinese melanoma patients. We also investigated the correlation between tumor mutation burden (TMB) and clinical outcomes of immunotherapy (IO). Methods: In this study, we retrospectively surveyed the genomic profiling of primary tumors of 81 (40 males, 41 females) metastatic Chinese melanoma patients with a median age of 52, using a panel consisting of 295 cancer-related genes, spanning 2.02MB of human genome. Patients used IO as first line treatment (n = 25) were enrolled for survival analyses. Results: In this cohort, 15, 24 and 42 were acral, mucosal and cutaneous melanoma, respectively. Collectively, we identified 1,114 mutations, spanning 248 genes, with BRAF, MYC and NBN being the most frequently mutated genes, occurring in 40%, 27% and 21% of patients, respectively. Mutation spectrum varied significantly by subtypes. BRAF (57%) and LRP1B (26%) were the most frequently mutated genes in cutaneous melanoma (CM). KIT and NRAS, reported to be frequently mutated in CM, each occurred in only 12% patients in this cohort. MYC amplification was the most commonly seen mutation in acral and mucosal melanoma (MM). Other frequent mutations in MM included: NBN (38%) RUNX1T1(29%) and TP53 (29%). In acral melanoma (AM), CCND1, FGF3/19, NF1and NBN were frequently mutated. It is interesting to note that no TP53 mutation was observed in AM. AM and MM had significantly more CNVs than CM. Of the 25 patients underwent IO, our data revealed a positive correlation between TMB and PFS (p = 0.007). Such correlation also exited in each subtype. Furthermore, we derived a cutoff of 15, which can effectively distinguish clinical response. Patients with TMB > 15 mut/Mb had a significantly longer PFS than patients harboring TMB < 15 mut/Mb (P = 0.02). Patients with CM had a longer PFS than patients with AM or MM (p = 0.018). No correlation between PDL1 expression and PFS was observed. Conclusions: Our study revealed a distinctive mutation landscape for each subtype. Furthermore, we also revealed a positive correlation between TMB and PFS as well as a lack of correlation between PDL1 expression and PFS.

scholarly journals DICE: A Drug Indication Classification and Encyclopedia for AI-Based Indication Extraction

2021 ◽  
Vol 4 ◽  
Author(s):  
Arjun Bhatt ◽  
Ruth Roberts ◽  
Xi Chen ◽  
Ting Li ◽  
Skylar Connor ◽  
...  
Keyword(s):  
Language Processing ◽  
Clinical Decision Making ◽  
Short Term Memory ◽  
Drug Repositioning ◽  
External Validation ◽  
Clinical Decision ◽  
Language Models ◽  
Free Text ◽  
Drug Labeling ◽  
Real World Evidence

Drug labeling contains an ‘INDICATIONS AND USAGE’ that provides vital information to support clinical decision making and regulatory management. Effective extraction of drug indication information from free-text based resources could facilitate drug repositioning projects and help collect real-world evidence in support of secondary use of approved medicines. To enable AI-powered language models for the extraction of drug indication information, we used manual reading and curation to develop a Drug Indication Classification and Encyclopedia (DICE) based on FDA approved human prescription drug labeling. A DICE scheme with 7,231 sentences categorized into five classes (indications, contradictions, side effects, usage instructions, and clinical observations) was developed. To further elucidate the utility of the DICE, we developed nine different AI-based classifiers for the prediction of indications based on the developed DICE to comprehensively assess their performance. We found that the transformer-based language models yielded an average MCC of 0.887, outperforming the word embedding-based Bidirectional long short-term memory (BiLSTM) models (0.862) with a 2.82% improvement on the test set. The best classifiers were also used to extract drug indication information in DrugBank and achieved a high enrichment rate (&gt;0.930) for this task. We found that domain-specific training could provide more explainable models without performance sacrifices and better generalization for external validation datasets. Altogether, the proposed DICE could be a standard resource for the development and evaluation of task-specific AI-powered, natural language processing (NLP) models.

Sign in / Sign up

Export Citation Format

Share Document