scholarly journals Improving Multilingual Sentence Embedding using Bi-directional Dual Encoder with Additive Margin Softmax

2019 ◽  
Author(s):  
Yinfei Yang ◽  
Gustavo Hernandez Abrego ◽  
Steve Yuan ◽  
Mandy Guo ◽  
Qinlan Shen ◽  
...  
Keyword(s):  
United Nations ◽  
State Of The Art ◽  
Cosine Similarity ◽  
Retrieval Task ◽  
Parallel Corpus ◽  
Similar Performance ◽  
Second Stage ◽  
Current State ◽  
Proposed Model ◽  
Document Level

In this paper, we present an approach to learn multilingual sentence embeddings using a bi-directional dual-encoder with additive margin softmax. The embeddings are able to achieve state-of-the-art results on the United Nations (UN) parallel corpus retrieval task. In all the languages tested, the system achieves P@1 of 86% or higher. We use pairs retrieved by our approach to train NMT models that achieve similar performance to models trained on gold pairs. We explore simple document-level embeddings constructed by averaging our sentence embeddings. On the UN document-level retrieval task, document embeddings achieve around 97% on P@1 for all experimented language pairs. Lastly, we evaluate the proposed model on the BUCC mining task. The learned embeddings with raw cosine similarity scores achieve competitive results compared to current state-of-the-art models, and with a second-stage scorer we achieve a new state-of-the-art level on this task.

scholarly journals Exploring the Role of Stress in Bayesian Word Segmentation using Adaptor Grammars

2014 ◽  
Vol 2 ◽  
pp. 93-104 ◽  
Author(s):  
Benjamin Börschinger ◽  
Mark Johnson
Keyword(s):  
State Of The Art ◽  
Word Segmentation ◽  
Stress Constraint ◽  
Frequent Patterns ◽  
Current State ◽  
Proposed Model ◽  
Stress Patterns ◽  
A Current ◽  
Over Time

Stress has long been established as a major cue in word segmentation for English infants. We show that enabling a current state-of-the-art Bayesian word segmentation model to take advantage of stress cues noticeably improves its performance. We find that the improvements range from 10 to 4%, depending on both the use of phonotactic cues and, to a lesser extent, the amount of evidence available to the learner. We also find that in particular early on, stress cues are much more useful for our model than phonotactic cues by themselves, consistent with the finding that children do seem to use stress cues before they use phonotactic cues. Finally, we study how the model’s knowledge about stress patterns evolves over time. We not only find that our model correctly acquires the most frequent patterns relatively quickly but also that the Unique Stress Constraint that is at the heart of a previously proposed model does not need to be built in but can be acquired jointly with word segmentation.

scholarly journals SWIM: A Simple Word Interaction Model for Implicit Discourse Relation Recognition

2017 ◽  
Author(s):  
Wenqiang Lei ◽  
Xuancong Wang ◽  
Meichun Liu ◽  
Ilija Ilievski ◽  
Xiangnan He ◽  
...  
Keyword(s):  
Short Term Memory ◽  
State Of The Art ◽  
Interaction Model ◽  
Weighted Averaging ◽  
Short Term ◽  
Current State ◽  
Proposed Model ◽  
Order Of Magnitude ◽  
Long Short Term Memory ◽  
Semantic Interaction

Capturing the semantic interaction of pairs of words across arguments and proper argument representation are both crucial issues in implicit discourse relation recognition. The current state-of-the-art represents arguments as distributional vectors that are computed via bi-directional Long Short-Term Memory networks (BiLSTMs), known to have significant model complexity.In contrast, we demonstrate that word-weighted averaging can encode argument representation which can incorporate word pair information efficiently. By saving an order of magnitude in parameters, our proposed model achieves equivalent performance, but trains seven times faster.

scholarly journals A Deep Learning Based Approach for Localization and Recognition of Pakistani Vehicle License Plates

Sensors ◽  
2021 ◽  
Vol 21 (22) ◽  
pp. 7696
Author(s):  
Umair Yousaf ◽  
Ahmad Khan ◽  
Hazrat Ali ◽  
Fiaz Gul Khan ◽  
Zia ur Rehman ◽  
...  
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
The Other ◽  
License Plate ◽  
Standard Size ◽  
Current State ◽  
Bounding Box ◽  
Proposed Model ◽  
Plate Area

License plate localization is the process of finding the license plate area and drawing a bounding box around it, while recognition is the process of identifying the text within the bounding box. The current state-of-the-art license plate localization and recognition approaches require license plates of standard size, style, fonts, and colors. Unfortunately, in Pakistan, license plates are non-standard and vary in terms of the characteristics mentioned above. This paper presents a deep-learning-based approach to localize and recognize Pakistani license plates with non-uniform and non-standardized sizes, fonts, and styles. We developed a new Pakistani license plate dataset (PLPD) to train and evaluate the proposed model. We conducted extensive experiments to compare the accuracy of the proposed approach with existing techniques. The results show that the proposed method outperformed the other methods to localize and recognize non-standard license plates.

Neural Steganalysis with Spatial Rich Models for Image Steganography Detection

2020 ◽  
Author(s):  
Jonathan Lwowski ◽  
Isaac Corley ◽  
Justin Hoffman
Keyword(s):  
Deep Neural Networks ◽  
State Of The Art ◽  
Principal Component ◽  
Image Steganalysis ◽  
Moving Image ◽  
Advanced Persistent Threat ◽  
Current State ◽  
File Formats ◽  
Proposed Model ◽  
Industry Applications

Digital image steganalysis is the process of detecting if an image contains concealed data embedded within its pixel space inserted via a steganography algorithm. The detection of these images is highly motivated by Advanced Persistent Threat (APT) groups, such as APT37 Reaper, commonly utilizing these techniques to transmit malicious shellcode to perform further post-exploitation activity on a compromised host. Performing detection has become increasingly difficult due to modern steganography algorithms advancing at a greater rate than the steganalysis techniques designed to combat them. The task of detection is challenging due to modern steganography techniques that embed messages into images with only minor modifications to the original content which varies from image to image. In this paper, we pipeline Spatial Rich Models (SRM) feature extraction, Principal Component Analysis (PCA), and Deep Neural Networks (DNNs) to perform image steganalysis. Our proposed model, Neural Spatial Rich Models (NSRM) is an ensemble of DNN classifiers trained to detect 4 different state-of-the-art steganography algorithms at 5 different embedding rates, allowing for an end-to-end model which can be more easily deployed at scale. Additionally our results show our proposed model outperforms other current state-of-the-art neural network based image steganalysis techniques. Lastly, we provide an analysis of the current academic steganalysis benchmark dataset, BOSSBase, as well as performance of detection of steganography in various file formats with the hope of moving image steganalysis algorithms towards the point they can be utilized in actual industry applications.

Reduced Base Fuzzy Logic Controller for Robot Actuators

2014 ◽  
Vol 555 ◽  
pp. 249-258 ◽  
Author(s):  
Victor Vladareanu ◽  
Paul Schiopu ◽  
Shuang Cang ◽  
Hong Nian Yu
Keyword(s):  
Fuzzy Logic ◽  
Fuzzy Logic Controller ◽  
Fuzzy Controller ◽  
State Of The Art ◽  
Slight Modification ◽  
Rule Base ◽  
Matlab Simulink ◽  
Current State ◽  
Proposed Model

The paper proposes an innovative type of fuzzy logic controller for robot actuators, building upon the current state of the art fuzzy architectures and various observations from work with Fuzzy Logic and Extenics Theory. This leads to a modified fuzzy controller, with a significantly simpler rule base, which shows comparable results. The effect is achieved by taking advantage of the rule base makeup of a regular linear fuzzy controller. Some slight modification is needed to the controller architecture, which is explained in detail. The rationality and validity of the proposed model are demonstrated through simulation in the Matlab/Simulink environment. The results show that the proposed new controller architecture obtains remarkable results, while having the advantage of increased simplicity in design and setting of parameters. Throughout the paper, opportunities for further improvement and research are highlighted and discussed.

scholarly journals Neural Steganalysis with Spatial Rich Models for Image Steganography Detection

2020 ◽  
Author(s):  
Jonathan Lwowski ◽  
Isaac Corley ◽  
Justin Hoffman
Keyword(s):  
Deep Neural Networks ◽  
State Of The Art ◽  
Principal Component ◽  
Image Steganalysis ◽  
Moving Image ◽  
Advanced Persistent Threat ◽  
Current State ◽  
File Formats ◽  
Proposed Model ◽  
Industry Applications

Digital image steganalysis is the process of detecting if an image contains concealed data embedded within its pixel space inserted via a steganography algorithm. The detection of these images is highly motivated by Advanced Persistent Threat (APT) groups, such as APT37 Reaper, commonly utilizing these techniques to transmit malicious shellcode to perform further post-exploitation activity on a compromised host. Performing detection has become increasingly difficult due to modern steganography algorithms advancing at a greater rate than the steganalysis techniques designed to combat them. The task of detection is challenging due to modern steganography techniques that embed messages into images with only minor modifications to the original content which varies from image to image. In this paper, we pipeline Spatial Rich Models (SRM) feature extraction, Principal Component Analysis (PCA), and Deep Neural Networks (DNNs) to perform image steganalysis. Our proposed model, Neural Spatial Rich Models (NSRM) is an ensemble of DNN classifiers trained to detect 4 different state-of-the-art steganography algorithms at 5 different embedding rates, allowing for an end-to-end model which can be more easily deployed at scale. Additionally our results show our proposed model outperforms other current state-of-the-art neural network based image steganalysis techniques. Lastly, we provide an analysis of the current academic steganalysis benchmark dataset, BOSSBase, as well as performance of detection of steganography in various file formats with the hope of moving image steganalysis algorithms towards the point they can be utilized in actual industry applications.

Expansion dataset COVID-19 chest X-ray using data augmentation and histogram equalization

2022 ◽  
Vol 12 (2) ◽  
pp. 1904
Author(s):  
Farah Flayeh Alkhalid ◽  
Abdulhakeem Qusay Albayati ◽  
Ahmed Ali Alhammad
Keyword(s):  
Data Augmentation ◽  
State Of The Art ◽  
Virus Disease ◽  
Histogram Equalization ◽  
Vital Role ◽  
X Ray ◽  
Current State ◽  
Proposed Model ◽  
Chest X Ray ◽  
Using Data

The main important factor that plays vital role in success the deep learning is the deep training by many and many images, if neural networks are getting bigger and bigger but the training datasets are not, then it sounds like going to hit an accuracy wall. Briefly, this paper investigates the current state of the art of approaches used for a data augmentation for expansion the corona virus disease 2019 (COVID-19) chest X-ray images using different data augmentation methods (transformation and enhancement) the dataset expansion helps to rise numbers of images from 138 to 5520, the increasing rate is 3,900%, this proposed model can be used to expand any type of image dataset, in addition, the dataset have used with convolutional neural network (CNN) model to make classification if detected infection with COVID-19 in X-ray, the results have gotten high training accuracy=99%

DSC_NEXT: Document-Level Sentiment ClassificatioN with EXTensional-Information

2021 ◽  
pp. 2150001
Author(s):  
Pengyuan Liu ◽  
Chenghao Zhu ◽  
Yi Wu
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Network Models ◽  
User Preferences ◽  
Sentiment Classification ◽  
Learning Models ◽  
Neural Network Models ◽  
Product Features ◽  
Current State ◽  
Document Level

Document-level sentiment classification is to assign an overall sentiment polarity to an opinion document. Some researchers have already realized that, in addition to document texts, extensional-information such as product features and user preferences can be quite useful. Many previous studies represent them as ID-type extensional-information and incorporate them into deep learning models. However, they ignore the descriptive extensional information that is also useful for document representations. This paper covers the following aspects: (1) introduces the Description of Opinion Target (DOT), a new extensional-information for document-level sentiment classification, (2) builds the Document-level Sentiment ClassificatioN with EXTensional-information (DSC_NEXT) dataset which consists of three datasets: IMDB_NEXT, Yelp_NEXT and CMRDB_NEXT and (3) validates the effectiveness of DOT by performing experiments based on current state-of-the-art (SOTA) document-level sentiment analysis methods. Implications for using extensional-information in neural network models are also considered.

scholarly journals Image Restoration by Second-Order Total Generalized Variation and Wavelet Frame Regularization

Complexity ◽  
2019 ◽  
Vol 2019 ◽  
pp. 1-16 ◽  
Author(s):  
Jianguang Zhu ◽  
Kai Li ◽  
Binbin Hao
Keyword(s):  
State Of The Art ◽  
Second Order ◽  
Wavelet Frame ◽  
Smooth Functions ◽  
Total Generalized Variation ◽  
Current State ◽  
Proposed Model ◽  
Staircase Effect ◽  
Alternative Direction Method ◽  
Generalized Variation

It has been proved that total generalized variation (TGV) can better preserve edges while suppressing staircase effect. In this paper, we propose an effective hybrid regularization model based on second-order TGV and wavelet frame. The proposed model inherits the advantages of TGV regularization and wavelet frame regularization, can eliminate staircase effect while protecting the sharp edge, and simultaneously has good capability of sparsely estimating the piecewise smooth functions. The alternative direction method of multiplier (ADMM) is employed to solve the new model. Numerical results show that our proposed model can preserve more details and get higher image visual quality than some current state-of-the-art methods.

scholarly journals Modeling clinical assessor intervariability using deep hypersphere encoder–decoder networks

2019 ◽  
Vol 32 (14) ◽  
pp. 10705-10717 ◽  
Author(s):  
Joost van der Putten ◽  
Fons van der Sommen ◽  
Jeroen de Groof ◽  
Maarten Struyvenberg ◽  
Svitlana Zinger ◽  
...  
Keyword(s):  
State Of The Art ◽  
Ground Truth ◽  
Observer Agreement ◽  
Data Sets ◽  
Observer Variability ◽  
Data Set ◽  
Current State ◽  
Inter Observer Variability ◽  
Proposed Model ◽  
Segmentation Models

AbstractIn medical imaging, a proper gold-standard ground truth as, e.g., annotated segmentations by assessors or experts is lacking or only scarcely available and suffers from large intervariability in those segmentations. Most state-of-the-art segmentation models do not take inter-observer variability into account and are fully deterministic in nature. In this work, we propose hypersphere encoder–decoder networks in combination with dynamic leaky ReLUs, as a new method to explicitly incorporate inter-observer variability into a segmentation model. With this model, we can then generate multiple proposals based on the inter-observer agreement. As a result, the output segmentations of the proposed model can be tuned to typical margins inherent to the ambiguity in the data. For experimental validation, we provide a proof of concept on a toy data set as well as show improved segmentation results on two medical data sets. The proposed method has several advantages over current state-of-the-art segmentation models such as interpretability in the uncertainty of segmentation borders. Experiments with a medical localization problem show that it offers improved biopsy localizations, which are on average 12% closer to the optimal biopsy location.

Sign in / Sign up

Export Citation Format

Share Document