scholarly journals Supervised and Unsupervised Neural Approaches to Text Readability

2021 ◽  
Vol 47 (1) ◽  
pp. 141-179
Author(s):  
Matej Martinc ◽  
Senja Pollak ◽  
Marko Robnik-Šikonja
Keyword(s):  
State Of The Art ◽  
Comprehensive Analysis ◽  
Language Models ◽  
Feature Engineering ◽  
Data Sets ◽  
Data Set ◽  
Systematic Comparison ◽  
Current State ◽  
Text Readability ◽  
Unsupervised Approach

Abstract We present a set of novel neural supervised and unsupervised approaches for determining the readability of documents. In the unsupervised setting, we leverage neural language models, whereas in the supervised setting, three different neural classification architectures are tested. We show that the proposed neural unsupervised approach is robust, transferable across languages, and allows adaptation to a specific readability task and data set. By systematic comparison of several neural architectures on a number of benchmark and new labeled readability data sets in two languages, this study also offers a comprehensive analysis of different neural approaches to readability classification. We expose their strengths and weaknesses, compare their performance to current state-of-the-art classification approaches to readability, which in most cases still rely on extensive feature engineering, and propose possibilities for improvements.

scholarly journals Modeling clinical assessor intervariability using deep hypersphere encoder–decoder networks

2019 ◽  
Vol 32 (14) ◽  
pp. 10705-10717 ◽  
Author(s):  
Joost van der Putten ◽  
Fons van der Sommen ◽  
Jeroen de Groof ◽  
Maarten Struyvenberg ◽  
Svitlana Zinger ◽  
...  
Keyword(s):  
State Of The Art ◽  
Ground Truth ◽  
Observer Agreement ◽  
Data Sets ◽  
Observer Variability ◽  
Data Set ◽  
Current State ◽  
Inter Observer Variability ◽  
Proposed Model ◽  
Segmentation Models

AbstractIn medical imaging, a proper gold-standard ground truth as, e.g., annotated segmentations by assessors or experts is lacking or only scarcely available and suffers from large intervariability in those segmentations. Most state-of-the-art segmentation models do not take inter-observer variability into account and are fully deterministic in nature. In this work, we propose hypersphere encoder–decoder networks in combination with dynamic leaky ReLUs, as a new method to explicitly incorporate inter-observer variability into a segmentation model. With this model, we can then generate multiple proposals based on the inter-observer agreement. As a result, the output segmentations of the proposed model can be tuned to typical margins inherent to the ambiguity in the data. For experimental validation, we provide a proof of concept on a toy data set as well as show improved segmentation results on two medical data sets. The proposed method has several advantages over current state-of-the-art segmentation models such as interpretability in the uncertainty of segmentation borders. Experiments with a medical localization problem show that it offers improved biopsy localizations, which are on average 12% closer to the optimal biopsy location.

scholarly journals MSA Transformer

2021 ◽  
Author(s):  
Roshan Rao ◽  
Jason Liu ◽  
Robert Verkuil ◽  
Joshua Meier ◽  
John F. Canny ◽  
...  
Keyword(s):  
Structure Learning ◽  
State Of The Art ◽  
Language Model ◽  
Language Modeling ◽  
Language Models ◽  
Multiple Sequence ◽  
Wide Margin ◽  
Current State ◽  
Individual Sequences ◽  
And Function

AbstractUnsupervised protein language models trained across millions of diverse sequences learn structure and function of proteins. Protein language models studied to date have been trained to perform inference from individual sequences. The longstanding approach in computational biology has been to make inferences from a family of evolutionarily related sequences by fitting a model to each family independently. In this work we combine the two paradigms. We introduce a protein language model which takes as input a set of sequences in the form of a multiple sequence alignment. The model interleaves row and column attention across the input sequences and is trained with a variant of the masked language modeling objective across many protein families. The performance of the model surpasses current state-of-the-art unsupervised structure learning methods by a wide margin, with far greater parameter efficiency than prior state-of-the-art protein language models.

scholarly journals Unsupervised Action Proposals Using Support Vector Classifiers for Online Video Processing

Sensors ◽  
2020 ◽  
Vol 20 (10) ◽  
pp. 2953
Author(s):  
Marcos Baptista Ríos ◽  
Roberto Javier López-Sastre ◽  
Francisco Javier Acevedo-Rodríguez ◽  
Pilar Martín-Martín ◽  
Saturnino Maldonado-Bascón
Keyword(s):  
Video Processing ◽  
State Of The Art ◽  
Learning To Rank ◽  
Support Vector ◽  
New Approach ◽  
Support Vector Classifier ◽  
Current State ◽  
Video Frames ◽  
Unsupervised Approach ◽  
Video Sensor

In this work, we introduce an intelligent video sensor for the problem of Action Proposals (AP). AP consists of localizing temporal segments in untrimmed videos that are likely to contain actions. Solving this problem can accelerate several video action understanding tasks, such as detection, retrieval, or indexing. All previous AP approaches are supervised and offline, i.e., they need both the temporal annotations of the datasets during training and access to the whole video to effectively cast the proposals. We propose here a new approach which, unlike the rest of the state-of-the-art models, is unsupervised. This implies that we do not allow it to see any labeled data during learning nor to work with any pre-trained feature on the used dataset. Moreover, our approach also operates in an online manner, which can be beneficial for many real-world applications where the video has to be processed as soon as it arrives at the sensor, e.g., robotics or video monitoring. The core of our method is based on a Support Vector Classifier (SVC) module which produces candidate segments for AP by distinguishing between sets of contiguous video frames. We further propose a mechanism to refine and filter those candidate segments. This filter optimizes a learning-to-rank formulation over the dynamics of the segments. An extensive experimental evaluation is conducted on Thumos’14 and ActivityNet datasets, and, to the best of our knowledge, this work supposes the first unsupervised approach on these main AP benchmarks. Finally, we also provide a thorough comparison to the current state-of-the-art supervised AP approaches. We achieve 41% and 59% of the performance of the best-supervised model on ActivityNet and Thumos’14, respectively, confirming our unsupervised solution as a correct option to tackle the AP problem. The code to reproduce all our results will be publicly released upon acceptance of the paper.

scholarly journals QASC: A Dataset for Question Answering via Sentence Composition

2020 ◽  
Vol 34 (05) ◽  
pp. 8082-8090
Author(s):  
Tushar Khot ◽  
Peter Clark ◽  
Michal Guerquin ◽  
Peter Jansen ◽  
Ashish Sabharwal
Keyword(s):  
Common Sense ◽  
Human Performance ◽  
Question Answering ◽  
State Of The Art ◽  
Multiple Choice ◽  
Training Data ◽  
Language Models ◽  
Current State ◽  
New Concepts ◽  
Large Corpus

Composing knowledge from multiple pieces of texts is a key challenge in multi-hop question answering. We present a multi-hop reasoning dataset, Question Answering via Sentence Composition (QASC), that requires retrieving facts from a large corpus and composing them to answer a multiple-choice question. QASC is the first dataset to offer two desirable properties: (a) the facts to be composed are annotated in a large corpus, and (b) the decomposition into these facts is not evident from the question itself. The latter makes retrieval challenging as the system must introduce new concepts or relations in order to discover potential decompositions. Further, the reasoning model must then learn to identify valid compositions of these retrieved facts using common-sense reasoning. To help address these challenges, we provide annotation for supporting facts as well as their composition. Guided by these annotations, we present a two-step approach to mitigate the retrieval challenges. We use other multiple-choice datasets as additional training data to strengthen the reasoning model. Our proposed approach improves over current state-of-the-art language models by 11% (absolute). The reasoning and retrieval problems, however, remain unsolved as this model still lags by 20% behind human performance.

scholarly journals A review of forecasting techniques for large datasets

2008 ◽  
Vol 203 ◽  
pp. 109-115 ◽  
Author(s):  
Jana Eklund ◽  
George Kapetanios
Keyword(s):  
State Of The Art ◽  
Large Data ◽  
Large Datasets ◽  
Large Data Sets ◽  
Multiple Models ◽  
Small Subset ◽  
Data Sets ◽  
Single Model ◽  
Data Set ◽  
Forecasting Techniques

This paper aims to provide a brief and relatively non-technical overview of state-of-the-art forecasting with large data sets. We classify existing methods into four groups depending on whether data sets are used wholly or partly, whether a single model or multiple models are used and whether a small subset or the whole data set is being forecast. In particular, we provide brief descriptions of the methods and short recommendations where appropriate, without going into detailed discussions of their merits or demerits.

scholarly journals Modified Deep Neural Networks for Dog Breeds Identification

2018 ◽  
Author(s):  
Aydin Ayanzadeh ◽  
Sahand Vahidnia
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
The State ◽  
Fine Tuning ◽  
Test Accuracy ◽  
Data Sets ◽  
Data Set

In this paper, we leverage state of the art models on Imagenet data-sets. We use the pre-trained model and learned weighs to extract the feature from the Dog breeds identification data-set. Afterwards, we applied fine-tuning and dataaugmentation to increase the performance of our test accuracy in classification of dog breeds datasets. The performance of the proposed approaches are compared with the state of the art models of Image-Net datasets such as ResNet-50, DenseNet-121, DenseNet-169 and GoogleNet. we achieved 89.66% , 85.37% 84.01% and 82.08% test accuracy respectively which shows thesuperior performance of proposed method to the previous works on Stanford dog breeds datasets.

scholarly journals Bokeh Effect Rendering with Vision Transformers

2022 ◽  
Author(s):  
Hariharan Nagasubramaniam ◽  
Rabih Younes
Keyword(s):  
Neural Networks ◽  
State Of The Art ◽  
Large Diameter ◽  
Depth Map ◽  
Global Information ◽  
Initial Training ◽  
Data Set ◽  
Current State ◽  
Mobile Cameras ◽  
Transformer Model

Bokeh effect is growing to be an important feature in photography, essentially to choose an object of interest to be in focus with the rest of the background being blurred. While naturally rendering this effect requires a DSLR with large diameter of aperture, with the current advancements in Deep Learning, this effect can also be produced in mobile cameras. Most of the existing methods use Convolutional Neural Networks while some relying on the depth map to render this effect. In this paper, we propose an end-to-end Vision Transformer model for Bokeh rendering of images from monocular camera. This architecture uses vision transformers as backbone, thus learning from the entire image rather than just the parts from the filters in a CNN. This property of retaining global information coupled with initial training of the model for image restoration before training to render the blur effect for the background, allows our method to produce clearer images and outperform the current state-of-the-art models on the EBB! Data set. The code to our proposed method can be found at: https://github.com/Soester10/ Bokeh-Rendering-with-Vision-Transformers.

scholarly journals Constructing a Lightweight Key-Value Store Based on the Windows Native Features

2019 ◽  
Vol 9 (18) ◽  
pp. 3801 ◽  
Author(s):  
Hyuk-Yoon Kwon
Keyword(s):  
State Of The Art ◽  
Main Idea ◽  
Real Data ◽  
Data Sets ◽  
Parameter Setting ◽  
Data Set ◽  
Multi Level ◽  
Windows Registry ◽  
Best Parameter ◽  
Better Than

In this paper, we propose a method to construct a lightweight key-value store based on the Windows native features. The main idea is providing a thin wrapper for the key-value store on top of a built-in storage in Windows, called Windows registry. First, we define a mapping of the components in the key-value store onto the components in the Windows registry. Then, we present a hash-based multi-level registry index so as to distribute the key-value data balanced and to efficiently access them. Third, we implement basic operations of the key-value store (i.e., Get, Put, and Delete) by manipulating the Windows registry using the Windows native APIs. We call the proposed key-value store WR-Store. Finally, we propose an efficient ETL (Extract-Transform-Load) method to migrate data stored in WR-Store into any other environments that support existing key-value stores. Because the performance of the Windows registry has not been studied much, we perform the empirical study to understand the characteristics of WR-Store, and then, tune the performance of WR-Store to find the best parameter setting. Through extensive experiments using synthetic and real data sets, we show that the performance of WR-Store is comparable to or even better than the state-of-the-art systems (i.e., RocksDB, BerkeleyDB, and LevelDB). Especially, we show the scalability of WR-Store. That is, WR-Store becomes much more efficient than the other key-value stores as the size of data set increases. In addition, we show that the performance of WR-Store is maintained even in the case of intensive registry workloads where 1000 processes accessing to the registry actively are concurrently running.

scholarly journals A logic programming approach to predict effective compiler settings for embedded software

2015 ◽  
Vol 15 (4-5) ◽  
pp. 481-494 ◽  
Author(s):  
CRAIG BLACKMORE ◽  
OLIVER RAY ◽  
KERSTIN EDER
Keyword(s):  
Embedded System ◽  
Logic Programming ◽  
State Of The Art ◽  
System Development ◽  
Hybrid Approach ◽  
Inductive Logic ◽  
Programming Approach ◽  
Data Set ◽  
Current State ◽  
Execution Times

AbstractThis paper introduces a new logic-based method for optimising the selection of compiler flags on embedded architectures. In particular, we use Inductive Logic Programming (ILP) to learn logical rules that relate effective compiler flags to specific program features. Unlike earlier work, we aim to infer human-readable rules and we seek to develop a relational first-order approach which automatically discovers relevant features rather than relying on a vector of predetermined attributes. To this end we generated a data set by measuring execution times of 60 benchmarks on an embedded system development board and we developed an ILP prototype which outperforms the current state-of-the-art learning approach in 34 of the 60 benchmarks. Finally, we combined the strengths of the current state of the art and our ILP method in a hybrid approach which reduced execution times by an average of 8% and up to 50% in some cases.

scholarly journals Interpretable Rumor Detection in Microblogs by Attending to User Interactions

2020 ◽  
Vol 34 (05) ◽  
pp. 8783-8790 ◽  
Author(s):  
Ling Min Serena Khoo ◽  
Hai Leong Chieu ◽  
Zhong Qian ◽  
Jing Jiang
Keyword(s):  
State Of The Art ◽  
Attention Mechanism ◽  
Data Sets ◽  
Long Distance ◽  
User Interactions ◽  
Data Set ◽  
Structure Information ◽  
Attention Model ◽  
Tree Models ◽  
Rumor Detection

We address rumor detection by learning to differentiate between the community's response to real and fake claims in microblogs. Existing state-of-the-art models are based on tree models that model conversational trees. However, in social media, a user posting a reply might be replying to the entire thread rather than to a specific user. We propose a post-level attention model (PLAN) to model long distance interactions between tweets with the multi-head attention mechanism in a transformer network. We investigated variants of this model: (1) a structure aware self-attention model (StA-PLAN) that incorporates tree structure information in the transformer network, and (2) a hierarchical token and post-level attention model (StA-HiTPLAN) that learns a sentence representation with token-level self-attention. To the best of our knowledge, we are the first to evaluate our models on two rumor detection data sets: the PHEME data set as well as the Twitter15 and Twitter16 data sets. We show that our best models outperform current state-of-the-art models for both data sets. Moreover, the attention mechanism allows us to explain rumor detection predictions at both token-level and post-level.

Sign in / Sign up

Export Citation Format

Share Document