Explicit Document Modeling through Weighted Multiple-Instance Learning

Representing documents is a crucial component in many NLP tasks, for instance predicting aspect ratings in reviews. Previous methods for this task treat documents globally, and do not acknowledge that target categories are often assigned by their authors with generally no indication of the specific sentences that motivate them. To address this issue, we adopt a weakly supervised learning model, which jointly learns to focus on relevant parts of a document according to the context along with a classifier for the target categories. Derived from the weighted multiple-instance regression (MIR) framework, the model learns decomposable document vectors for each individual category and thus overcomes the representational bottleneck in previous methods due to a fixed-length document vector. During prediction, the estimated relevance or saliency weights explicitly capture the contribution of each sentence to the predicted rating, thus offering an explanation of the rating. Our model achieves state-of-the-art performance on multi-aspect sentiment analysis, improving over several baselines. Moreover, the predicted saliency weights are close to human estimates obtained by crowdsourcing, and increase the performance of lexical and topical features for review segmentation and summarization.

Download Full-text

Non-I.I.D. Multi-Instance Learning for Predicting Instance and Bag Labels with Variational Auto-Encoder

Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ◽

10.24963/ijcai.2021/465 ◽

2021 ◽

Author(s):

Weijia Zhang

Keyword(s):

Medical Imaging ◽

Supervised Learning ◽

Real World ◽

State Of The Art ◽

Experimental Results ◽

Weakly Supervised Learning ◽

Variational Autoencoder ◽

Label Prediction ◽

Weakly Supervised ◽

Better Than

Multi-instance learning is a type of weakly supervised learning. It deals with tasks where the data is a set of bags and each bag is a set of instances. Only the bag labels are observed whereas the labels for the instances are unknown. An important advantage of multi-instance learning is that by representing objects as a bag of instances, it is able to preserve the inherent dependencies among parts of the objects. Unfortunately, most existing algorithms assume all instances to be identically and independently distributed, which violates real-world scenarios since the instances within a bag are rarely independent. In this work, we propose the Multi-Instance Variational Autoencoder (MIVAE) algorithm which explicitly models the dependencies among the instances for predicting both bag labels and instance labels. Experimental results on several multi-instance benchmarks and end-to-end medical imaging datasets demonstrate that MIVAE performs better than state-of-the-art algorithms for both instance label and bag label prediction tasks.

Download Full-text

Weakly Supervised Sentiment-Specific Region Discovery for VSA

The Computer Journal ◽

10.1093/comjnl/bxaa112 ◽

2020 ◽

Author(s):

Luoyang Xue ◽

Ang Xu ◽

Qirong Mao ◽

Lijian Gao ◽

Jie Chen

Keyword(s):

Sentiment Analysis ◽

State Of The Art ◽

Multiple Instance Learning ◽

Local Information ◽

Specific Region ◽

Convolution Kernels ◽

Weakly Supervised ◽

Candidate Regions ◽

Region Discovery ◽

Object Level

Abstract Local information has significant contributions to visual sentiment analysis (VSA). Recent studies about local region discovery need manually annotate region location. Affective local information learning and automatic discovery of sentiment-specific region are still the challenges in VSA. In this paper, we propose an end-to-end VSA method for weakly supervised sentiment-specific region discovery. Our method contains two branches: an automatic sentiment-specific region discovery branch and a sentiment analysis branch. In the sentiment-specific region discovery branch, a region proposal network with multiple convolution kernels is proposed to generate candidate affective regions. Then, we design the multiple instance learning (MIL) loss to remove redundant and noisy candidate regions. Finally, the sentiment analysis branch integrates both holistic and localized information obtained in the first branch by feature map coupling for final sentiment classification. Our method automatically discovers sentiment-specific regions by the constraint of MIL loss function without object-level labels. Quantitative and qualitative evaluations on four benchmark affective datasets demonstrate that our proposed method outperforms the state-of-the-art methods.

Download Full-text

Cross-lingual Projected Expectation Regularization for Weakly Supervised Learning

Transactions of the Association for Computational Linguistics ◽

10.1162/tacl_a_00165 ◽

2014 ◽

Vol 2 ◽

pp. 55-66 ◽

Cited By ~ 10

Author(s):

Mengqiu Wang ◽

Christopher D. Manning

Keyword(s):

Supervised Learning ◽

Model Uncertainty ◽

State Of The Art ◽

New Method ◽

Weakly Supervised Learning ◽

Supervised Methods ◽

Weakly Supervised ◽

Language Boundaries ◽

Standard Chinese ◽

Cross Lingual

We consider a multilingual weakly supervised learning scenario where knowledge from annotated corpora in a resource-rich language is transferred via bitext to guide the learning in other languages. Past approaches project labels across bitext and use them as features or gold labels for training. We propose a new method that projects model expectations rather than labels, which facilities transfer of model uncertainty across language boundaries. We encode expectations as constraints and train a discriminative CRF model using Generalized Expectation Criteria (Mann and McCallum, 2010). Evaluated on standard Chinese-English and German-English NER datasets, our method demonstrates F1 scores of 64% and 60% when no labeled data is used. Attaining the same accuracy with supervised CRFs requires 12k and 1.5k labeled sentences. Furthermore, when combined with labeled examples, our method yields significant improvements over state-of-the-art supervised methods, achieving best reported numbers to date on Chinese OntoNotes and German CoNLL-03 datasets.

Download Full-text