scholarly journals Catastrophic Interference in Predictive Neural Network Models of Distributional Semantics

2020 ◽  
Author(s):  
Willa Mannering ◽  
Michael N. Jones
Keyword(s):  
Neural Network ◽  
Semantic Representation ◽  
Word Meaning ◽  
Network Models ◽  
Random Order ◽  
Artificial Language ◽  
Neural Network Models ◽  
Catastrophic Interference ◽  
The Impact ◽  
Language Corpus

The semantic memory literature has recently seen the emergence of predictive neural network models that use principles of reinforcement learning to create a “neural embedding” of word meaning when trained on a language corpus. These models have taken the field by storm, partially due to the resurgence of connectionist architectures, but also due to their remarkable success at fitting human data. However, predictive embedding models also inherit the weaknesses of their ancestors. In this paper, we explore the effect of catastrophic interference (CI), long known to be a flaw with neural network models, on a modern neural embedding model of semantic representation (word2vec). We use homonyms as an index of bias as a function of the order in which a corpus is learned. If the corpus is learned in random order, the final representation will tend towards the dominant sense of the word (bank > money) as opposed to the subordinate sense (bank > river). However, if the subordinate sense is presented to the network after learning the dominant sense, CI almost completely erases the dominant sense and the final representation strongly tends towards the more recent subordinate sense. We demonstrate the impact of CI and sequence of learning on the final neural embeddings learned by word2vec in both an artificial language and in an English corpus and evaluate the effectiveness of a recently proposed solution to CI from neuroscience, elastic weight consolidation, on mitigating the effects of CI.

Mathematics anxiety and cognition: an integrated neural network model

2020 ◽  
Vol 31 (3) ◽  
pp. 287-296
Author(s):  
Ahmed A. Moustafa ◽  
Angela Porter ◽  
Ahmed M. Megreya
Keyword(s):  
Neural Network ◽  
Educational Outcomes ◽  
Mathematics Anxiety ◽  
Network Models ◽  
Neural Mechanism ◽  
Neural Network Models ◽  
Cognitive Studies ◽  
Behavioral Studies ◽  
Negative Effect ◽  
The Impact

AbstractMany students suffer from anxiety when performing numerical calculations. Mathematics anxiety is a condition that has a negative effect on educational outcomes and future employment prospects. While there are a multitude of behavioral studies on mathematics anxiety, its underlying cognitive and neural mechanism remain unclear. This article provides a systematic review of cognitive studies that investigated mathematics anxiety. As there are no prior neural network models of mathematics anxiety, this article discusses how previous neural network models of mathematical cognition could be adapted to simulate the neural and behavioral studies of mathematics anxiety. In other words, here we provide a novel integrative network theory on the links between mathematics anxiety, cognition, and brain substrates. This theoretical framework may explain the impact of mathematics anxiety on a range of cognitive and neuropsychological tests. Therefore, it could improve our understanding of the cognitive and neurological mechanisms underlying mathematics anxiety and also has important applications. Indeed, a better understanding of mathematics anxiety could inform more effective therapeutic techniques that in turn could lead to significant improvements in educational outcomes.

scholarly journals The Impact of an Urban Growth Boundary on Land Development in Knox County, Tennessee: A Comparison of Two-Stage Probit Least Squares and Multilayer Neural Network Models

2007 ◽  
Vol 39 (3) ◽  
pp. 701-717 ◽  
Author(s):  
Seong-Hoon Cho ◽  
Olufemi A. Omitaomu ◽  
Neelam C. Poudyal ◽  
David B. Eastwood
Keyword(s):  
Neural Network ◽  
Urban Growth ◽  
Land Development ◽  
Network Models ◽  
Probit Model ◽  
Neural Network Models ◽  
Two Stage ◽  
Urban Growth Boundary ◽  
Knox County ◽  
The Impact

The impact of an urban growth boundary (UGB) on land development in Knox County, TN is estimated via two-stage probit and neural-network models. The insignificance of UGB variable in the two-stage probit model and more visible development patterns in the western part of Knoxville and the neighboring town of Farragut during the post-UGB period in both models suggest that the UGB has not curtailed urban sprawl. Although the network model is found to be a viable alternative to more conventional discrete choice approach for improving the predictability of land development, it is at the cost of evaluating marginal effects.

scholarly journals Artificial Neural Network Models for Material Classification by Photon Scattering Analysis

2020 ◽  
Vol 9 (03) ◽  
pp. 01-04
Author(s):  
Dr. Naveen Jain
Keyword(s):  
Neural Network ◽  
Risk Factors ◽  
Artificial Neural Network ◽  
Network Models ◽  
Photon Scattering ◽  
Neural Network Models ◽  
Material Classification ◽  
Artificial Neural ◽  
Artificial Neural Network Models ◽  
The Impact

This article explains the risk factors involved in a business. In each type of business, there are certain risk factors for the implementation of anything in the business. The type of risks involved can depend upon many factors. It also depends on the type of business an organisation is doing. But it is very important that the risk analyst does all the analysis of the risks that might arise in future and must take necessary actions in order to avoid those risks. The risk analyst can also try to reduce the impact of the risks on the business. Therefore, it is very important that the risk analyst should have the knowledge of how to analyse risk and then can act upon them.

scholarly journals The impact of extraneous features on the performance of recurrent neural network models in clinical tasks

2020 ◽  
Vol 102 ◽  
pp. 103351 ◽  
Author(s):  
Eugene Laksana ◽  
Melissa Aczon ◽  
Long Ho ◽  
Cameron Carlin ◽  
David Ledbetter ◽  
...  
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Network Models ◽  
Neural Network Models ◽  
The Impact

A Large-Scale Analysis of the Semantic Password Model and Linguistic Patterns in Passwords

2021 ◽  
Vol 24 (3) ◽  
pp. 1-21
Author(s):  
Rafael Veras ◽  
Christopher Collins ◽  
Julie Thorpe
Keyword(s):  
Neural Network ◽  
Graphical Models ◽  
Large Scale ◽  
Network Models ◽  
Neural Network Models ◽  
Part Of Speech ◽  
Large Scale Analysis ◽  
Linguistic Patterns ◽  
High Level ◽  
The Impact

In this article, we present a thorough evaluation of semantic password grammars. We report multifactorial experiments that test the impact of sample size, probability smoothing, and linguistic information on password cracking. The semantic grammars are compared with state-of-the-art probabilistic context-free grammar ( PCFG ) and neural network models, and tested in cross-validation and A vs. B scenarios. We present results that reveal the contributions of part-of-speech (syntactic) and semantic patterns, and suggest that the former are more consequential to the security of passwords. Our results show that in many cases PCFGs are still competitive models compared to their latest neural network counterparts. In addition, we show that there is little performance gain in training PCFGs with more than 1 million passwords. We present qualitative analyses of four password leaks (Mate1, 000webhost, Comcast, and RockYou) based on trained semantic grammars, and derive graphical models that capture high-level dependencies between token classes. Finally, we confirm the similarity inferences from our qualitative analysis by examining the effectiveness of grammars trained and tested on all pairs of leaks.

scholarly journals Towards Physics-Based Generalizable Convolutional Neural Network Models for Indoor Propagation

2021 ◽  
Author(s):  
Aristeidis Seretis
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Convolutional Neural Network ◽  
Large Scale ◽  
Network Models ◽  
Radiowave Propagation ◽  
Indoor Propagation ◽  
Neural Network Models ◽  
Small Set ◽  
The Impact

A fundamental challenge for machine learning models for electromagnetics is their ability to predict output quantities of interest (such as fields and scattering parameters) in geometries that the model has not been trained for. Addressing this challenge is a key to fulfilling one of the most appealing promises of machine learning for computational electromagnetics: the rapid solution of problems of interest just by processing the geometry and the sources involved. The impact of such models that can "generalize" to new geometries is more profound for large-scale computations, such as those encountered in wireless propagation scenarios. We present generalizable models for indoor propagation that can predict received signal strengths within new geometries, beyond those of the training set of the model, for transmitters and receivers of multiple positions, and for new frequencies. We show that a convolutional neural network can "learn" the physics of indoor radiowave propagation from ray-tracing solutions of a small set of training geometries, so that it can eventually deal with substantially different geometries. We emphasize the role of exploiting physical insights in the training of the network, by defining input parameters and cost functions that assist the network to efficiently learn basic and complex propagation mechanisms.

scholarly journals On the Impact of Interpretability Methods in Active Image Augmentation Method

2021 ◽  
Author(s):  
Flávio Arthur Oliveira Santos ◽  
Cleber Zanchettin ◽  
Leonardo Nogueira Matos ◽  
Paulo Novais
Keyword(s):  
Neural Network ◽  
Deep Neural Network ◽  
Network Models ◽  
Training Data ◽  
Learning Models ◽  
Neural Network Models ◽  
Wide Range ◽  
The Impact ◽  
Activation Mapping ◽  
Machine Learning Models

Abstract Robustness is a significant constraint in machine learning models. The performance of the algorithms must not deteriorate when training and testing with slightly different data. Deep neural network models achieve awe-inspiring results in a wide range of applications of computer vision. Still, in the presence of noise or region occlusion, some models exhibit inaccurate performance even with data handled in training. Besides, some experiments suggest deep learning models sometimes use incorrect parts of the input information to perform inference. Active image augmentation (ADA) is an augmentation method that uses interpretability methods to augment the training data and improve its robustness to face the described problems. Although ADA presented interesting results, its original version only used the vanilla backpropagation interpretability to train the U-Net model. In this work, we propose an extensive experimental analysis of the interpretability method’s impact on ADA. We use five interpretability methods: vanilla backpropagation, guided backpropagation, gradient-weighted class activation mapping (GradCam), guided GradCam and InputXGradient. The results show that all methods achieve similar performance at the ending of training, but when combining ADA with GradCam, the U-Net model presented an impressive fast convergence.

Sign in / Sign up

Export Citation Format

Share Document