What makes the difference? An empirical comparison of fusion strategies for multimodal language analysis

2021 ◽  
Vol 66 ◽  
pp. 184-197
Author(s):  
Dimitris Gkoumas ◽  
Qiuchi Li ◽  
Christina Lioma ◽  
Yijun Yu ◽  
Dawei Song
Keyword(s):  
Empirical Comparison ◽  
Language Analysis ◽  
Multimodal Language ◽  
The Difference

scholarly journals Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis

2020 ◽  
Author(s):  
Yao-Hung Hubert Tsai ◽  
Martin Ma ◽  
Muqiao Yang ◽  
Ruslan Salakhutdinov ◽  
Louis-Philippe Morency
Keyword(s):  
Language Analysis ◽  
Multimodal Language

scholarly journals Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis

2020 ◽  
Vol 34 (05) ◽  
pp. 8992-8999
Author(s):  
Zhongkai Sun ◽  
Prathusha Sarma ◽  
William Sethares ◽  
Yingyu Liang
Keyword(s):  
Emotion Recognition ◽  
Sentiment Analysis ◽  
Canonical Correlation ◽  
Language Models ◽  
Outer Product ◽  
Language Analysis ◽  
Benchmark Datasets ◽  
Text Features ◽  
Multimodal Language ◽  
Multimodal Sentiment Analysis

Multimodal language analysis often considers relationships between features based on text and those based on acoustical and visual properties. Text features typically outperform non-text features in sentiment analysis or emotion recognition tasks in part because the text features are derived from advanced language models or word embeddings trained on massive data sources while audio and video features are human-engineered and comparatively underdeveloped. Given that the text, audio, and video are describing the same utterance in different ways, we hypothesize that the multimodal sentiment analysis and emotion recognition can be improved by learning (hidden) correlations between features extracted from the outer product of text and audio (we call this text-based audio) and analogous text-based video. This paper proposes a novel model, the Interaction Canonical Correlation Network (ICCN), to learn such multimodal embeddings. ICCN learns correlations between all three modes via deep canonical correlation analysis (DCCA) and the proposed embeddings are then tested on several benchmark datasets and against other state-of-the-art multimodal embedding algorithms. Empirical results and ablation studies confirm the effectiveness of ICCN in capturing useful information from all three views.

scholarly journals Multimodal Language Analysis in the Wild: CMU-MOSEI Dataset and Interpretable Dynamic Fusion Graph

2018 ◽  
Author(s):  
AmirAli Bagher Zadeh ◽  
Paul Pu Liang ◽  
Soujanya Poria ◽  
Erik Cambria ◽  
Louis-Philippe Morency
Keyword(s):  
Language Analysis ◽  
In The Wild ◽  
Multimodal Language

Suicide Risk Factors: A Language Analysis Approach in Social Media

2021 ◽  
pp. 0261927X2110361
Author(s):  
Gerardo Sierra ◽  
Patricia Andrade-Palos ◽  
Gemma Bel-Enguix ◽  
Alejandro Osornio-Arteaga ◽  
Adriana Cabrera-Mora ◽  
...  
Keyword(s):  
Risk Factors ◽  
Social Media ◽  
Suicide Risk ◽  
Language Use ◽  
Public Health Issue ◽  
Mexican Culture ◽  
Language Analysis ◽  
Social Media Platforms ◽  
Suicide Risk Factors ◽  
The Difference

Suicide represents a public health issue that requires new preventive strategies. Therefore, this study analyzes differences in language use between a themed posts group (suicide and depression) and a random posts group (non-specific topics) from different social media platforms. In addition, the similarity of the texts of themed posts group with the set of phrases linked to suicide risk factors is analyzed. Texts were processed using the Linguistic Inquiry and Word Count (LIWC) software. A 95% bootstrap confidence interval (CI) was built for the difference in means for the resulting values per word category across groups. Significant differences in the use of language were observed between the themed post group and the random post group, whereas no differences were found between the themed post group and the set of phrases linked to suicide risk factors. These results can contribute to the development of risk-detection tools, which identify users at risk based on the analysis of language use embedded in social media. Such opportune detection would significantly increase the suicide prevention work. However, there is still a latent need for doing more research in this field, especially for the Mexican culture.

scholarly journals Analisis Butir Soal Ujian Semester Ganjil Kimia Kelas X SMA Negeri 1 Perbaungan

2019 ◽  
Vol 1 (1) ◽  
pp. 38
Author(s):  
Ria Atika Purba ◽  
Nora Susanti ◽  
Rosna Rosna
Keyword(s):  
Quantitative Analysis ◽  
Quantitative Research ◽  
Cognitive Domain ◽  
School Year ◽  
Language Analysis ◽  
Power Difference ◽  
The Difference ◽  
Very High ◽  
Domain Level

The purpose of this study is to find out the quality of the odd chemistry semester exam questions of class X IPA 1 Perbaungan 2017/2018 school year based on qualitative analysis (material, construction, and language), analysis of cognitive level distribution, and quantitative analysis (validity, reliability, power difference, level of difficulty, and effectiveness of deception) using the ANATES V4 program. This study uses a type of quantitative research with descriptive methods. The results of this study are that the content validity in the material is in a very high category, in the construction aspect in the low category and the language aspect in the category is very high. Results based on cognitive domain level distribution are not in accordance with domain level proportions. The results of the study are based on quantitative analysis which is based on validity is 55% valid, and 45% is invalid. Based on reliability is 0.76. Based on the difference in power is good and based on effectiveness the deception is effective.

scholarly journals Multimodal Language Analysis with Recurrent Multistage Fusion

2018 ◽  
Author(s):  
Paul Pu Liang ◽  
Ziyin Liu ◽  
AmirAli Bagher Zadeh ◽  
Louis-Philippe Morency
Keyword(s):  
Language Analysis ◽  
Multimodal Language

scholarly journals Clinician vs. Machine: Estimating Vocalizations Rates in Young Children With Developmental Disorders

2018 ◽  
Vol 27 (3) ◽  
pp. 1066-1072
Author(s):  
Shelley L. Bredin-Oja ◽  
Heather Fielding ◽  
Kandace K. Fleming ◽  
Steven F. Warren
Keyword(s):  
Developmental Disorders ◽  
Environment Analysis ◽  
Language Analysis ◽  
Significant Difference ◽  
The Difference ◽  
Highly Correlated ◽  
Nonverbal Children ◽  
Analysis System ◽  
Language Environment ◽  
Digital Language

Purpose The purpose of this study was to investigate the reliability of an automated language analysis system, the Language Environment Analysis (LENA), compared with a human transcriber to determine the rate of child vocalizations during recording sessions that were significantly shorter than recommended for the automated device. Method Participants were 6 nonverbal male children between the ages of 28 and 46 months. Two children had autism diagnoses, 2 had Down syndrome, 1 had a chromosomal deletion, and 1 had developmental delay. Participants were recorded by the LENA digital language processor during 14 play-based interactions with a responsive adult. Rate of child vocalizations during each of the 84 recordings was determined by both a human transcriber and the LENA software. Results A statistically significant difference between the 2 methods was observed for 4 of the 6 participants. Effect sizes were moderate to large. Variation in syllable structure did not explain the difference between the 2 methods. Vocalization rates from the 2 methods were highly correlated for 5 of the 6 participants. Conclusions Estimates of vocalization rates from nonverbal children produced by the LENA system differed from human transcription during sessions that were substantially shorter than the recommended recording length. These results confirm the recommendation of the LENA Foundation to record sessions of at least 1 hr.

The Origin of the Moon

1962 ◽  
Vol 14 ◽  
pp. 149-155 ◽  
Author(s):  
E. L. Ruskol
Keyword(s):  
Chemical Composition ◽  
Solar System ◽  
The Moon ◽  
The Earth ◽  
The Difference ◽  
Origin Of The Moon

The difference between average densities of the Moon and Earth was interpreted in the preceding report by Professor H. Urey as indicating a difference in their chemical composition. Therefore, Urey assumes the Moon's formation to have taken place far away from the Earth, under conditions differing substantially from the conditions of Earth's formation. In such a case, the Earth should have captured the Moon. As is admitted by Professor Urey himself, such a capture is a very improbable event. In addition, an assumption that the “lunar” dimensions were representative of protoplanetary bodies in the entire solar system encounters great difficulties.

Sign in / Sign up

Export Citation Format

Share Document