scholarly journals Improving Neural Question Generation Using Answer Separation

2019 ◽  
Vol 33 ◽  
pp. 6602-6609 ◽  
Author(s):  
Yanghoon Kim ◽  
Hwanhee Lee ◽  
Joongbo Shin ◽  
Kyomin Jung
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Significant Proportion ◽  
Separation Method ◽  
Experimental Results ◽  
Question Generation ◽  
Previous State

Neural question generation (NQG) is the task of generating a question from a given passage with deep neural networks. Previous NQG models suffer from a problem that a significant proportion of the generated questions include words in the question target, resulting in the generation of unintended questions. In this paper, we propose answer-separated seq2seq, which better utilizes the information from both the passage and the target answer. By replacing the target answer in the original passage with a special token, our model learns to identify which interrogative word should be used. We also propose a new module termed keyword-net, which helps the model better capture the key information in the target answer and generate an appropriate question. Experimental results demonstrate that our answer separation method significantly reduces the number of improper questions which include answers. Consequently, our model significantly outperforms previous state-of-the-art NQG models.

Learning robust features by extended generative stochastic networks

2018 ◽  
Vol 09 (01) ◽  
pp. 1850004
Author(s):  
Da Teng ◽  
Xiao Song ◽  
Guanghong Gong ◽  
Junhua Zhou
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Random Noise ◽  
Stochastic Networks ◽  
Experimental Results ◽  
Feedforward Networks ◽  
Adversarial Examples ◽  
Art Performance

Deep neural networks have achieved state-of-the-art performance on many object recognition tasks, but they are vulnerable to small adversarial perturbations. In this paper, several extensions of generative stochastic networks (GSNs) are proposed to improve the robustness of neural networks to random noise and adversarial perturbations. Experimental results show that compared to normal GSN method, the extensions using adversarial examples, lateral connections and feedforward networks can improve the performance of GSNs by making the models more resistant to overfitting and noise.

scholarly journals Convolutional Neural Networks for Classifying Melanoma Images

2020 ◽  
Author(s):  
Abhinav Sagar ◽  
J Dheeba
Keyword(s):  
Neural Networks ◽  
Skin Cancer ◽  
Transfer Learning ◽  
Convolutional Neural Networks ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Cancer Classification ◽  
Previous State ◽  
Better Than

AbstractIn this work, we address the problem of skin cancer classification using convolutional neural networks. A lot of cancer cases early on are misdiagnosed as something else leading to severe consequences including the death of a patient. Also there are cases in which patients have some other problems and doctors think they might have skin cancer. This leads to unnecessary time and money spent for further diagnosis. In this work, we address both of the above problems using deep neural networks and transfer learning architecture. We have used publicly available ISIC databases for both training and testing our model. Our work achieves an accuracy of 0.935, precision of 0.94, recall of 0.77, F1 score of 0.85 and ROC-AUC of 0.861 which is better than the previous state of the art approaches.

scholarly journals Neural Question Generation with Answer Pivot

2020 ◽  
Vol 34 (05) ◽  
pp. 9138-9145
Author(s):  
Bingning Wang ◽  
Xiaochuan Wang ◽  
Ting Tao ◽  
Qi Zhang ◽  
Jingfang Xu
Keyword(s):  
Neural Networks ◽  
Degrees Of Freedom ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Selection Process ◽  
Question Generation ◽  
Named Entities ◽  
Human Evaluation ◽  
The Given ◽  
Machine Reading

Neural question generation (NQG) is the task of generating questions from the given context with deep neural networks. Previous answer-aware NQG methods suffer from the problem that the generated answers are focusing on entity and most of the questions are trivial to be answered. The answer-agnostic NQG methods reduce the bias towards named entities and increasing the model's degrees of freedom, but sometimes result in generating unanswerable questions which are not valuable for the subsequent machine reading comprehension system. In this paper, we treat the answers as the hidden pivot for question generation and combine the question generation and answer selection process in a joint model. We achieve the state-of-the-art result on the SQuAD dataset according to automatic metric and human evaluation.

scholarly journals Visual Dialogue State Tracking for Question Generation

2020 ◽  
Vol 34 (07) ◽  
pp. 11831-11838 ◽  
Author(s):  
Wei Pang ◽  
Xiaojie Wang
Keyword(s):  
State Of The Art ◽  
Experimental Results ◽  
Question Generation ◽  
Art Methods ◽  
Previous State ◽  
State Tracking ◽  
Art Performance

GuessWhat?! is a visual dialogue task between a guesser and an oracle. The guesser aims to locate an object supposed by the oracle oneself in an image by asking a sequence of Yes/No questions. Asking proper questions with the progress of dialogue is vital for achieving successful final guess. As a result, the progress of dialogue should be properly represented and tracked. Previous models for question generation pay less attention on the representation and tracking of dialogue states, and therefore are prone to asking low quality questions such as repeated questions. This paper proposes visual dialogue state tracking (VDST) based method for question generation. A visual dialogue state is defined as the distribution on objects in the image as well as representations of objects. Representations of objects are updated with the change of the distribution on objects. An object-difference based attention is used to decode new question. The distribution on objects is updated by comparing the question-answer pair and objects. Experimental results on GuessWhat?! dataset show that our model significantly outperforms existing methods and achieves new state-of-the-art performance. It is also noticeable that our model reduces the rate of repeated questions from more than 50% to 21.9% compared with previous state-of-the-art methods.

Filter pruning - deeper layers need fewer filters

2021 ◽  
pp. 1-14
Author(s):  
Heng Wang ◽  
Xiang Ye ◽  
Yong Li
Keyword(s):  
Neural Networks ◽  
Redundancy Analysis ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
The State ◽  
Experimental Results ◽  
Existing Structures ◽  
Pruning Strategy ◽  
Treat All ◽  
Three Phases

Model pruning aims to reduce the parameter amount of deep neural networks while retaining the performance. Existing strategies often treat all layers equally and all layers simply share the same pruning rate. However, it is observed from our experiments that the redundancy degree differs from layer to layer. Based on this observation, this work proposes a pruning strategy depending on the layer-wise redundancy degree. Firstly, we define the redundancy degree for each layer by the norm and similarity redundancy of filters. Then a novel layer-wise strategy, Redundancy-dependent Filter Pruning (RedFiP), is proposed which prunes different proportion of filters at different layers according to the defined redundancy degree. Since the redundancy analysis and experimental results of RedFiP show that deeper layers need fewer filters, a phase-wise strategy, Phased Filter Pruning (PFP), is proposed that divides the layers into three phases and layers in each phase share the same pruning rate. The phase-wise PFP allows the layer-wise RedFiP to be easily implemented in existing structures of deep neural networks. Experimental results show that when total parameters are pruned by 40%, RedFiP outperforms the state-of-the-art strategy FPGM-Mixed by 1.83% on CIFAR-100, and even slightly outperforms the non-pruned model by 0.11% on CIFAR-10. On ImageNet-1k, RedFiP (30%) and PFP (30%) outperform FPGM-Mixed (30%) by 1.3% and 0.8% with ResNet-18.

scholarly journals Using spatial-temporal ensembles of convolutional neural networks for lumen segmentation in ureteroscopy

2021 ◽  
Author(s):  
Jorge F. Lazo ◽  
Aldo Marzullo ◽  
Sara Moccia ◽  
Michele Catellani ◽  
Benoit Rosa ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Automatic Segmentation ◽  
Temporal Information ◽  
Invasive Technique ◽  
Dice Similarity Coefficient ◽  
Specular Reflections ◽  
Lumen Segmentation ◽  
Previous State

Abstract Purpose Ureteroscopy is an efficient endoscopic minimally invasive technique for the diagnosis and treatment of upper tract urothelial carcinoma. During ureteroscopy, the automatic segmentation of the hollow lumen is of primary importance, since it indicates the path that the endoscope should follow. In order to obtain an accurate segmentation of the hollow lumen, this paper presents an automatic method based on convolutional neural networks (CNNs). Methods The proposed method is based on an ensemble of 4 parallel CNNs to simultaneously process single and multi-frame information. Of these, two architectures are taken as core-models, namely U-Net based in residual blocks ($$m_1$$ m 1 ) and Mask-RCNN ($$m_2$$ m 2 ), which are fed with single still-frames I(t). The other two models ($$M_1$$ M 1 , $$M_2$$ M 2 ) are modifications of the former ones consisting on the addition of a stage which makes use of 3D convolutions to process temporal information. $$M_1$$ M 1 , $$M_2$$ M 2 are fed with triplets of frames ($$I(t-1)$$ I ( t - 1 ) , I(t), $$I(t+1)$$ I ( t + 1 ) ) to produce the segmentation for I(t). Results The proposed method was evaluated using a custom dataset of 11 videos (2673 frames) which were collected and manually annotated from 6 patients. We obtain a Dice similarity coefficient of 0.80, outperforming previous state-of-the-art methods. Conclusion The obtained results show that spatial-temporal information can be effectively exploited by the ensemble model to improve hollow lumen segmentation in ureteroscopic images. The method is effective also in the presence of poor visibility, occasional bleeding, or specular reflections.

scholarly journals Representing Deep Neural Networks Latent Space Geometries with Graphs

Algorithms ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 39
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Objective Function ◽  
Learning Process ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
The Core ◽  
Learning Tasks ◽  
Latent Space

Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists of training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more detail, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: (i) reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, (ii) designing efficient embeddings for classification is achieved by targeting specific geometries, and (iii) robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.

scholarly journals Reconfigurable Binary Neural Network Accelerator with Adaptive Parallelism Scheme

Electronics ◽  
2021 ◽  
Vol 10 (3) ◽  
pp. 230
Author(s):  
Jaechan Cho ◽  
Yongchul Jung ◽  
Seongjoo Lee ◽  
Yunho Jung
Keyword(s):  
Neural Networks ◽  
High Throughput ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Throughput Performance ◽  
Adaptive Parallelism ◽  
Sensor Applications ◽  
Binary Neural Network ◽  
Target Layer ◽  
Network Topologies

Binary neural networks (BNNs) have attracted significant interest for the implementation of deep neural networks (DNNs) on resource-constrained edge devices, and various BNN accelerator architectures have been proposed to achieve higher efficiency. BNN accelerators can be divided into two categories: streaming and layer accelerators. Although streaming accelerators designed for a specific BNN network topology provide high throughput, they are infeasible for various sensor applications in edge AI because of their complexity and inflexibility. In contrast, layer accelerators with reasonable resources can support various network topologies, but they operate with the same parallelism for all the layers of the BNN, which degrades throughput performance at certain layers. To overcome this problem, we propose a BNN accelerator with adaptive parallelism that offers high throughput performance in all layers. The proposed accelerator analyzes target layer parameters and operates with optimal parallelism using reasonable resources. In addition, this architecture is able to fully compute all types of BNN layers thanks to its reconfigurability, and it can achieve a higher area–speed efficiency than existing accelerators. In performance evaluation using state-of-the-art BNN topologies, the designed BNN accelerator achieved an area–speed efficiency 9.69 times higher than previous FPGA implementations and 24% higher than existing VLSI implementations for BNNs.

scholarly journals Label Distribution for Learning with Noisy Labels

2020 ◽  
Author(s):  
Yun-Peng Liu ◽  
Ning Xu ◽  
Yu Zhang ◽  
Xin Geng
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Learning Algorithm ◽  
State Of The Art ◽  
Confidence Estimation ◽  
Novel Method ◽  
Real World Datasets ◽  
Label Distribution ◽  
Noisy Labels

The performances of deep neural networks (DNNs) crucially rely on the quality of labeling. In some situations, labels are easily corrupted, and therefore some labels become noisy labels. Thus, designing algorithms that deal with noisy labels is of great importance for learning robust DNNs. However, it is difficult to distinguish between clean labels and noisy labels, which becomes the bottleneck of many methods. To address the problem, this paper proposes a novel method named Label Distribution based Confidence Estimation (LDCE). LDCE estimates the confidence of the observed labels based on label distribution. Then, the boundary between clean labels and noisy labels becomes clear according to confidence scores. To verify the effectiveness of the method, LDCE is combined with the existing learning algorithm to train robust DNNs. Experiments on both synthetic and real-world datasets substantiate the superiority of the proposed algorithm against state-of-the-art methods.

Framework for TCAD augmented machine learning on multi- I–V characteristics using convolutional neural network and multiprocessing

2021 ◽  
Vol 42 (12) ◽  
pp. 124101
Author(s):  
Thomas Hirtz ◽  
Steyn Huurman ◽  
He Tian ◽  
Yi Yang ◽  
Tian-Ling Ren
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Information Technologies ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Data Driven ◽  
Sufficient Data ◽  
Learning Models ◽  
Simulation Tools ◽  
New Information

Abstract In a world where data is increasingly important for making breakthroughs, microelectronics is a field where data is sparse and hard to acquire. Only a few entities have the infrastructure that is required to automate the fabrication and testing of semiconductor devices. This infrastructure is crucial for generating sufficient data for the use of new information technologies. This situation generates a cleavage between most of the researchers and the industry. To address this issue, this paper will introduce a widely applicable approach for creating custom datasets using simulation tools and parallel computing. The multi-I–V curves that we obtained were processed simultaneously using convolutional neural networks, which gave us the ability to predict a full set of device characteristics with a single inference. We prove the potential of this approach through two concrete examples of useful deep learning models that were trained using the generated data. We believe that this work can act as a bridge between the state-of-the-art of data-driven methods and more classical semiconductor research, such as device engineering, yield engineering or process monitoring. Moreover, this research gives the opportunity to anybody to start experimenting with deep neural networks and machine learning in the field of microelectronics, without the need for expensive experimentation infrastructure.

Sign in / Sign up

Export Citation Format

Share Document