scholarly journals Buzz Tweet Classification Based on Text and Image Features of Tweets Using Multi-Task Learning

2021 ◽  
Vol 11 (22) ◽  
pp. 10567
Author(s):  
Reishi Amitani ◽  
Kazuyuki Matsumoto ◽  
Minoru Yoshida ◽  
Kenji Kita
Keyword(s):  
Neural Network ◽  
Social Media ◽  
Detailed Analysis ◽  
Image Features ◽  
Analysis Method ◽  
Text And Image ◽  
Multiple Features ◽  
Single Feature ◽  
Text Features ◽  
Tweet Classification

This study investigates social media trends and proposes a buzz tweet classification method to explore the factors causing the buzz phenomenon on Twitter. It is difficult to identify the causes of the buzz phenomenon based solely on texts posted on Twitter. It is expected that by limiting the tweets to those with attached images and using the characteristics of the images and the relationships between the text and images, a more detailed analysis than that of with text-only tweets can be conducted. Therefore, an analysis method was devised based on a multi-task neural network that uses both the features extracted from the image and text as input and the buzz class (buzz/non-buzz) and the number of “likes (favorites)” and “retweets (RTs)” as output. The predictions made using a single feature of the text and image were compared with the predictions using a combination of multiple features. The differences between buzz and non-buzz features were analyzed based on the cosine similarity between the text and the image. The buzz class was correctly identified with a correctness rate of approximately 80% for all combinations of image and text features, with the combination of BERT and VGG16 providing the highest correctness rate.

scholarly journals Predicting Implicit User Preferences with Multimodal Feature Fusion for Similar User Recommendation in Social Media

2021 ◽  
Vol 11 (3) ◽  
pp. 1064
Author(s):  
Jenq-Haur Wang ◽  
Yen-Tsang Wu ◽  
Long Wang
Keyword(s):  
Social Media ◽  
Feature Fusion ◽  
Relevant Information ◽  
Image Features ◽  
User Preferences ◽  
User Preference ◽  
Late Fusion ◽  
Multimodal Features ◽  
Fusion Methods ◽  
Text Features

In social networks, users can easily share information and express their opinions. Given the huge amount of data posted by many users, it is difficult to search for relevant information. In addition to individual posts, it would be useful if we can recommend groups of people with similar interests. Past studies on user preference learning focused on single-modal features such as review contents or demographic information of users. However, such information is usually not easy to obtain in most social media without explicit user feedback. In this paper, we propose a multimodal feature fusion approach to implicit user preference prediction which combines text and image features from user posts for recommending similar users in social media. First, we use the convolutional neural network (CNN) and TextCNN models to extract image and text features, respectively. Then, these features are combined using early and late fusion methods as a representation of user preferences. Lastly, a list of users with the most similar preferences are recommended. The experimental results on real-world Instagram data show that the best performance can be achieved when we apply late fusion of individual classification results for images and texts, with the best average top-k accuracy of 0.491. This validates the effectiveness of utilizing deep learning methods for fusing multimodal features to represent social user preferences. Further investigation is needed to verify the performance in different types of social media.

Truncated attention mechanism and cascade loss for cross-modal person re-identification

2021 ◽  
pp. 1-13
Author(s):  
Shuo Shi ◽  
Changwei Huo ◽  
Yingchun Guo ◽  
Stephen Lean ◽  
Gang Yan ◽  
...  
Keyword(s):  
Natural Language ◽  
Short Term Memory ◽  
Principal Component ◽  
Image Features ◽  
Attention Mechanism ◽  
Text And Image ◽  
Level Information ◽  
Text Features ◽  
Lstm Network ◽  
Language Description

Person re-identification with natural language description is a process of retrieving the corresponding person’s image from an image dataset according to a text description of the person. The key challenge in this cross-modal task is to extract visual and text features and construct loss functions to achieve cross-modal matching between text and image. Firstly, we designed a two-branch network framework for person re-identification with natural language description. In this framework we include the following: a Bi-directional Long Short-Term Memory (Bi-LSTM) network is used to extract text features and a truncated attention mechanism is proposed to select the principal component of the text features; a MobileNet is used to extract image features. Secondly, we proposed a Cascade Loss Function (CLF), which includes cross-modal matching loss and single modal classification loss, both with relative entropy function, to fully exploit the identity-level information. The experimental results on the CUHK-PEDES dataset demonstrate that our method achieves better results in Top-5 and Top-10 than other current 10 state-of-the-art algorithms.

scholarly journals Bahasa Agama di Media Sosial: Analisis Framing pada Media Sosial “Islam Populer”

2020 ◽  
Vol 13 (2) ◽  
pp. 203-234
Author(s):  
Moch. Fakhruroji ◽  
Ridwan Rustandi ◽  
Busro Busro
Keyword(s):  
Social Media ◽  
Religious Practice ◽  
Religious Language ◽  
Analysis Method ◽  
Framing Analysis ◽  
Text And Image ◽  
Image Construction ◽  
Subjective Reality ◽  
The Core ◽  
The Media

Abstrak Penelitian ini bertujuan untuk menganalisis bahasa agama di media sosial yang dikontstruksi oleh akun Islam Populer. Penelitian menggunakan pendekatan kualitatif dengan analisis teks media. Analisis media package dilakukan dengan metode analisis framing model William A. Gamson dan Modigliani. Unit analisis dilakukan terhadap core frame dan condensing symbol pada akun “Islam Populer” baik di kanal Youtube, Fanpage Facebook maupun Instagram. Hasil penelitian menyimpulkan bahwa pengemasan bahasa agama pada akun “Islam Populer” dilakukan dengan merujuk pada sumber normativitas Islam, yakni alqurân dan alhadîts. Konstruksi bahasa agama di media sosial di bangun sebagai sebuah praktik keagamaan yang dikemas melalui serangkaian simbol baik bersifat verbal maupun non verbal. Dalam konteks budaya siber, konstruksi gagasan ini dilakukan dengan membentuk ulang realitas secara subjektif. Realitas subjektif terbangun melalui rekayasa teks dan image. Pada titik inilah, bahasa agama di media sosial menjadi sebuah imagologi keagamaan yang berada pada wilayah ambiguitas, yakni antara wilayah sakral dan profan.   Kata Kunci: Bahasa agama; Media Sosial; Islam Populer; Imagologi agama.   Abstract This study aims to analyze the religious language on social media of "Islam Populer." This study uses a qualitative approach by analysing media texts. The media package analysis is performed using the William A. Gamson and Modigliani framing analysis method. The unit of analysis is carried out on the core frame and condensing symbols on "Islam Populer" social media accounts; Youtube channel, Facebook Fanpage and Instagram as well. The results of this study conclude that the packaging of religious language in "Islam Populer" accounts was carried out by referring to the sources of Islamic normativity, namely alqurân and alhadîts. The construction of religious language on social media was built as a religious practice that is packaged through a series of symbols both verbal and non-verbal. In the context of cyberculture, the construction of this idea is conducted by subjectively reshaping reality. Subjective reality is built through text and image construction. At this point, the language of religion in social media becomes a kind of religious imagology that is in the area of ambiguity, namely between the sacred and the profane.. Keywords: Religious languages, social media, Islam Populer, imagology of religion

scholarly journals BlueMemo: Depression Analysis through Twitter Posts

2020 ◽  
Author(s):  
Pengwei Hu ◽  
Chenhao Lin ◽  
Hui Su ◽  
Shaochun Li ◽  
Xue Han ◽  
...  
Keyword(s):  
Social Media ◽  
Potential Risk ◽  
Image Features ◽  
Classification Model ◽  
Hidden Information ◽  
Visual Attributes ◽  
Depressed Patients ◽  
Text Features ◽  
Use Of Social Media ◽  
New System

The use of social media runs through our lives, and users' emotions are also affected by it. Previous studies have reported social organizations and psychologists using social media to find depressed patients. However, due to the variety of content published by users, it isn't effortless for the system to consider the text, image, and even the hidden information behind the image. To address this problem, we proposed a new system for social media screening of depressed patients named BlueMemo. We collected real-time posts from Twitter. Based on the posts, learned text features, image features, and visual attributes were extracted as three modalities and were fed into a multi-modal fusion and classification model to implement our system. The proposed BlueMemo has the power to help physicians and clinicians quickly and accurately identify users at potential risk for depression.

Digitalization system of ancient architecture decoration art based on neural network and image features

2020 ◽  
pp. 1-12
Author(s):  
Wu Xin ◽  
Qiu Daping
Keyword(s):  
Neural Network ◽  
Construction Industry ◽  
Three Dimensional ◽  
Performance Testing ◽  
Image Features ◽  
Three Dimensional Model ◽  
Performance Effect ◽  
Data Process ◽  
And Performance ◽  
Construction Mode

The inheritance and innovation of ancient architecture decoration art is an important way for the development of the construction industry. The data process of traditional ancient architecture decoration art is relatively backward, which leads to the obvious distortion of the digitalization of ancient architecture decoration art. In order to improve the digital effect of ancient architecture decoration art, based on neural network, this paper combines the image features to construct a neural network-based ancient architecture decoration art data system model, and graphically expresses the static construction mode and dynamic construction process of the architecture group. Based on this, three-dimensional model reconstruction and scene simulation experiments of architecture groups are realized. In order to verify the performance effect of the system proposed in this paper, it is verified through simulation and performance testing, and data visualization is performed through statistical methods. The result of the study shows that the digitalization effect of the ancient architecture decoration art proposed in this paper is good.

Sosyal medyada Suriye’li sığınmacılara yönelik algı: bir sözlük değerlendirmesi

2017 ◽  
Vol 4 (2) ◽  
pp. 185-200 ◽  
Author(s):  
Servet Kardeş ◽  
Çağla Banko ◽  
Berrin Akman
Keyword(s):  
Qualitative Research ◽  
Social Media ◽  
Content Analysis ◽  
Syrian Refugees ◽  
Analysis Method ◽  
Social Media Platform ◽  
Content Analysis Method ◽  
Negative News ◽  
Media Platform ◽  
Made In

Bu araştırmada sığınmacılara yönelik paylaşımların yapıldığı sosyal medyada yer alan sözlüklerden birinde sığınmacılara yönelik algıya bakılmıştır. Yöntem olarak nitel desende olan bu çalışmada, bir sosyal medya sitesinde yer alan paylaşımlar içerik analizi yoluyla derinlemesine incelenip yorumlanmıştır. Araştırmanın sonucunda sosyal medya kullanıcılarının sığınmacıları büyük bir güvensizlik ortamı ve huzursuzluk yaratan bireyler olarak gördükleri saptanmış, sığınmacılarla yaşanan deneyimlerin ve medyadaki haberlerin bu düşüncelerin oluşmasında etkisinin olduğu belirlenmiştir. Bunun yanında sosyal medya kullanıcılarının devletin sığınmacılar konusunda yanlış politika izlediğini düşündükleri ve sığınmacılar için etkili bir planlama yapılmadığını ifade ettikleri görülmüştür. Çalışmanın sonuçları doğrultusunda medyada sığınmacılar hakkında çıkan haberlerde olumsuz ve şiddet temalı haberlerin azaltılması, Suriyeli sığınmacıların durumu, sahip oldukları haklar ve topluma yansımaları hakkında doğru ve bilgilendirici kamu spotları hazırlanması ayrıca sığınmacıların topluma entegre olma sürecinin her basamağında daha planlı ve etkili bir yol izlenmesi önerilebilir.ABSTRACT IN ENGLISHPerceptions about Syrian refugees on social media: an evaluation of a social media platformIn this research, posts which are about Syrian refugees were published in a social media platform, called as “sözlük” were investigated. The research is a qualitative research. The posts in this platform are analyzed with content analysis method. According to results of analyses, social media users see Syrian refugees as people who create an insecure and a restless environment. The experiences people had with them and news have an effect on this view. In addition, social media users think that government made inappropriate policies and ineffective plans about Syrian refugees. It is suggested negative news about Syrian refugees should be decreased and government should make safer policies. In addition, adaptation of refugees to society should be made in more planned and effective way.

Classification of papillary thyroid carcinoma histological images based on deep learning

2021 ◽  
pp. 1-11
Author(s):  
Yaning Liu ◽  
Lin Han ◽  
Hexiang Wang ◽  
Bo Yin
Keyword(s):  
Neural Network ◽  
Differential Diagnosis ◽  
Deep Learning ◽  
Papillary Thyroid Carcinoma ◽  
Thyroid Carcinoma ◽  
Image Features ◽  
Papillary Thyroid ◽  
Histological Image ◽  
Histological Images

Papillary thyroid carcinoma (PTC) is a common carcinoma in thyroid. As many benign thyroid nodules have the papillary structure which could easily be confused with PTC in morphology. Thus, pathologists have to take a lot of time on differential diagnosis of PTC besides personal diagnostic experience and there is no doubt that it is subjective and difficult to obtain consistency among observers. To address this issue, we applied deep learning to the differential diagnosis of PTC and proposed a histological image classification method for PTC based on the Inception Residual convolutional neural network (IRCNN) and support vector machine (SVM). First, in order to expand the dataset and solve the problem of histological image color inconsistency, a pre-processing module was constructed that included color transfer and mirror transform. Then, to alleviate overfitting of the deep learning model, we optimized the convolution neural network by combining Inception Network and Residual Network to extract image features. Finally, the SVM was trained via image features extracted by IRCNN to perform the classification task. Experimental results show effectiveness of the proposed method in the classification of PTC histological images.

scholarly journals Image Restoration by Learning Morphological Opening-Closing Network

2020 ◽  
Vol 4 (1) ◽  
pp. 87-107
Author(s):  
Ranjan Mondal ◽  
Moni Shankar Dey ◽  
Bhabatosh Chanda
Keyword(s):  
Neural Network ◽  
Image Restoration ◽  
State Of The Art ◽  
Source Code ◽  
Back Propagation ◽  
Image Features ◽  
Main Difficulty ◽  
The Right ◽  
Right Order ◽  
Morphological Opening

AbstractMathematical morphology is a powerful tool for image processing tasks. The main difficulty in designing mathematical morphological algorithm is deciding the order of operators/filters and the corresponding structuring elements (SEs). In this work, we develop morphological network composed of alternate sequences of dilation and erosion layers, which depending on learned SEs, may form opening or closing layers. These layers in the right order along with linear combination (of their outputs) are useful in extracting image features and processing them. Structuring elements in the network are learned by back-propagation method guided by minimization of the loss function. Efficacy of the proposed network is established by applying it to two interesting image restoration problems, namely de-raining and de-hazing. Results are comparable to that of many state-of-the-art algorithms for most of the images. It is also worth mentioning that the number of network parameters to handle is much less than that of popular convolutional neural network for similar tasks. The source code can be found here https://github.com/ranjanZ/Mophological-Opening-Closing-Net

Sign in / Sign up

Export Citation Format

Share Document