An Application of a Recurrent Neural Model for Parsing Natural Language

1997 ◽  
Vol 06 (03) ◽  
pp. 397-419
Author(s):  
John J. Salerno
Keyword(s):  
Learning Algorithm ◽  
Neural Model ◽  
Error Rates ◽  
Training Phase ◽  
Initial Training ◽  
Neural Models ◽  
Fixed Size ◽  
Small Set ◽  
Hidden Layer ◽  
Basic Learning

Neural models for dealing with symbolic processing are in their infancy. Success thus far can be defined by the parsing of very simple phrases and a small set of words into small, fixed size frames. Many of these systems do not scale well as one increases the number of words or the phrase length. These models are limited with respect to the large number of epochs required to train and the error rates. In the discussion that follows we will address the issue of training. We will present an analysis which will provide a lower bound on the error rate. The approach presents simple extensions to the basic learning algorithm and make use of a closest neighbor algorithm for correctness. Other issues such as generalization versus memorization, optimum hidden layer size and teaching the network after the initial training phase are also discussed.

scholarly journals An Integrated Approach of Mechanistic-Modeling and Machine-Learning for Thickness Optimization of Frozen Microwaveable Foods

Foods ◽  
2021 ◽  
Vol 10 (4) ◽  
pp. 763
Author(s):  
Ran Yang ◽  
Zhenbo Wang ◽  
Jiajia Chen
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Food Product ◽  
Integrated Approach ◽  
Mechanistic Modeling ◽  
Bayesian Optimization ◽  
Initial Training ◽  
Thickness Optimization ◽  
Heating Uniformity ◽  
Food Design

Mechanistic-modeling has been a useful tool to help food scientists in understanding complicated microwave-food interactions, but it cannot be directly used by the food developers for food design due to its resource-intensive characteristic. This study developed and validated an integrated approach that coupled mechanistic-modeling and machine-learning to achieve efficient food product design (thickness optimization) with better heating uniformity. The mechanistic-modeling that incorporated electromagnetics and heat transfer was previously developed and validated extensively and was used directly in this study. A Bayesian optimization machine-learning algorithm was developed and integrated with the mechanistic-modeling. The integrated approach was validated by comparing the optimization performance with a parametric sweep approach, which is solely based on mechanistic-modeling. The results showed that the integrated approach had the capability and robustness to optimize the thickness of different-shape products using different initial training datasets with higher efficiency (45.9% to 62.1% improvement) than the parametric sweep approach. Three rectangular-shape trays with one optimized thickness (1.56 cm) and two non-optimized thicknesses (1.20 and 2.00 cm) were 3-D printed and used in microwave heating experiments, which confirmed the feasibility of the integrated approach in thickness optimization. The integrated approach can be further developed and extended as a platform to efficiently design complicated microwavable foods with multiple-parameter optimization.

scholarly journals Assessment of the Content of Dry Matter and Dry Organic Matter in Compost with Neural Modelling Methods

Agriculture ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. 307
Author(s):  
Dawid Wojcieszak ◽  
Maciej Zaborowicz ◽  
Jacek Przybył ◽  
Piotr Boniecki ◽  
Aleksander Jędruś
Keyword(s):  
Image Analysis ◽  
Organic Matter ◽  
Dry Matter ◽  
Neural Model ◽  
Original Method ◽  
Neural Models ◽  
Test Error ◽  
Effective Assessment ◽  
Modelling Methods

Neural image analysis is commonly used to solve scientific problems of biosystems and mechanical engineering. The method has been applied, for example, to assess the quality of foodstuffs such as fruit and vegetables, cereal grains, and meat. The method can also be used to analyse composting processes. The scientific problem lets us formulate the research hypothesis: it is possible to identify representative traits of the image of composted material that are necessary to create a neural model supporting the process of assessment of the content of dry matter and dry organic matter in composted material. The effect of the research is the identification of selected features of the composted material and the methods of neural image analysis resulted in a new original method enabling effective assessment of the content of dry matter and dry organic matter. The content of dry matter and dry organic matter can be analysed by means of parameters specifying the colour of compost. The best developed neural models for the assessment of the content of dry matter and dry organic matter in compost are: in visible light RBF 19:19-2-1:1 (test error 0.0922) and MLP 14:14-14-11-1:1 (test error 0.1722), in mixed light RBF 30:30-8-1:1 (test error 0.0764) and MLP 7:7-9-7-1:1 (test error 0.1795). The neural models generated for the compost images taken in mixed light had better qualitative characteristics.

scholarly journals The Rare Word Issue in Natural Language Generation: A Character-Based Solution

Informatics ◽  
2021 ◽  
Vol 8 (1) ◽  
pp. 20
Author(s):  
Giovanni Bonetta ◽  
Marco Roberti ◽  
Rossella Cancelliere ◽  
Patrick Gallinari
Keyword(s):  
Natural Language ◽  
Error Probability ◽  
Essential Feature ◽  
Proper Names ◽  
Neural Model ◽  
Natural Language Generation ◽  
Training Phase ◽  
Tabular Data ◽  
Rare Word ◽  
Language Generation

In this paper, we analyze the problem of generating fluent English utterances from tabular data, focusing on the development of a sequence-to-sequence neural model which shows two major features: the ability to read and generate character-wise, and the ability to switch between generating and copying characters from the input: an essential feature when inputs contain rare words like proper names, telephone numbers, or foreign words. Working with characters instead of words is a challenge that can bring problems such as increasing the difficulty of the training phase and a bigger error probability during inference. Nevertheless, our work shows that these issues can be solved and efforts are repaid by the creation of a fully end-to-end system, whose inputs and outputs are not constrained to be part of a predefined vocabulary, like in word-based models. Furthermore, our copying technique is integrated with an innovative shift mechanism, which enhances the ability to produce outputs directly from inputs. We assess performance on the E2E dataset, the benchmark used for the E2E NLG challenge, and on a modified version of it, created to highlight the rare word copying capabilities of our model. The results demonstrate clear improvements over the baseline and promising performance compared to recent techniques in the literature.

scholarly journals Unsupervised learning by competing hidden units

2019 ◽  
Vol 116 (16) ◽  
pp. 7723-7731 ◽  
Author(s):  
Dmitry Krotov ◽  
John J. Hopfield
Keyword(s):  
Learning Algorithm ◽  
Lower Layer ◽  
Learning Rule ◽  
Backpropagation Algorithm ◽  
Feedforward Networks ◽  
Feature Detectors ◽  
End To End ◽  
Hidden Layer ◽  
Full Network ◽  
Global Inhibition

It is widely believed that end-to-end training with the backpropagation algorithm is essential for learning good feature detectors in early layers of artificial neural networks, so that these detectors are useful for the task performed by the higher layers of that neural network. At the same time, the traditional form of backpropagation is biologically implausible. In the present paper we propose an unusual learning rule, which has a degree of biological plausibility and which is motivated by Hebb’s idea that change of the synapse strength should be local—i.e., should depend only on the activities of the pre- and postsynaptic neurons. We design a learning algorithm that utilizes global inhibition in the hidden layer and is capable of learning early feature detectors in a completely unsupervised way. These learned lower-layer feature detectors can be used to train higher-layer weights in a usual supervised way so that the performance of the full network is comparable to the performance of standard feedforward networks trained end-to-end with a backpropagation algorithm on simple tasks.

Study on Deep Structure of Extreme Learning Machine (DS-ELM) for Datasets with Noise

2014 ◽  
Vol 989-994 ◽  
pp. 3679-3682 ◽  
Author(s):  
Meng Meng Ma ◽  
Bo He
Keyword(s):  
Extreme Learning Machine ◽  
Simple Structure ◽  
Learning Algorithm ◽  
Deep Structure ◽  
Noisy Data ◽  
Machine Learning Algorithm ◽  
Feed Forward Neural Networks ◽  
Learning Machine ◽  
Hidden Layer ◽  
Auto Associative Neural Network

Extreme learning machine (ELM), a relatively novel machine learning algorithm for single hidden layer feed-forward neural networks (SLFNs), has been shown competitive performance in simple structure and superior training speed. To improve the effectiveness of ELM for dealing with noisy datasets, a deep structure of ELM, short for DS-ELM, is proposed in this paper. DS-ELM contains three level networks (actually contains three nets ): the first level network is trained by auto-associative neural network (AANN) aim to filter out noise as well as reduce dimension when necessary; the second level network is another AANN net aim to fix the input weights and bias of ELM; and the last level network is ELM. Experiments on four noisy datasets are carried out to examine the new proposed DS-ELM algorithm. And the results show that DS-ELM has higher performance than ELM when dealing with noisy data.

The TP Chromatic Aberration Estimation by Using Neural Networks

2011 ◽  
Vol 121-126 ◽  
pp. 4239-4243 ◽  
Author(s):  
Du Jou Huang ◽  
Yu Ju Chen ◽  
Huang Chu Huang ◽  
Yu An Lin ◽  
Rey Chue Hwang
Keyword(s):  
Neural Networks ◽  
Learning Algorithm ◽  
Back Propagation ◽  
Physical Property ◽  
Chromatic Aberration ◽  
Neural Model ◽  
Artificial Intelligent ◽  
Error Back Propagation ◽  
The Neural Networks ◽  
Simulation Results

The chromatic aberration estimations of touch panel (TP) film by using neural networks are presented in this paper. The neural networks with error back-propagation (BP) learning algorithm were used to catch the complex relationship between the chromatic aberration, i.e., L.A.B. values, and the relative parameters of TP decoration film. An artificial intelligent (AI) estimator based on neural model for the estimation of physical property of TP film is expected to be developed. From the simulation results shown, the estimations of chromatic aberration of TP film are very accurate. In other words, such an AI estimator is quite promising and potential in commercial using.

A FILTER PROPOSAL FOR INCLUDING FEATURE CONSTRUCTION IN A GENETIC LEARNING ALGORITHM

2012 ◽  
Vol 20 (supp02) ◽  
pp. 31-49 ◽  
Author(s):  
DAVID GARCIA ◽  
ANTONIO GONZALEZ ◽  
RAUL PEREZ
Keyword(s):  
Learning Algorithm ◽  
A Priori ◽  
Fuzzy Rule ◽  
Feature Construction ◽  
Rule Based ◽  
Filter Model ◽  
Information Measures ◽  
Construction Techniques ◽  
Basic Learning ◽  
Current Proposal

In system identification process often a predetermined set of features is used. However, in many cases it is difficult to know a priori whether the selected features were really the more appropriate ones. This is the reason why the feature construction techniques have been very interesting in many applications. Thus, the current proposal introduces the use of these techniques in order to improve the description of fuzzy rule-based systems. In particular, the idea is to include feature construction in a genetic learning algorithm. The construction of attributes in this study will be restricted to the inclusion of functions defined on the initial attributes of the system. Since the number of functions and the number of attributes can be very large, a filter model, based on the use of information measures, is introduced. In this way, the genetic algorithm only needs to explore the particular new features that may be of greater interest to the final identification of the system. In order to manage the knowledge provided by the new attributes based on the use of functions we propose a new model of rule by extending a basic learning fuzzy rule-based model. Finally, we show the experimental study associated with this work.

scholarly journals A Comparative Assessment of Listening Span and Self-Efficacy in Simultaneous Interpreting Training

Sendebar ◽  
2021 ◽  
Vol 32 ◽  
pp. 167-176
Author(s):  
Saida Anssari Naim
Keyword(s):  
Self Efficacy ◽  
Comparative Assessment ◽  
Cognitive Factors ◽  
Personality Factors ◽  
Training Phase ◽  
Initial Training ◽  
Predictive Capacity ◽  
Personality Dimension ◽  
Cognitive Dimension ◽  
Simultaneous Interpreting

This study compares the results obtained from a group of interpreting trainees in an adapted listening span test (cognitive dimension), with those obtained from this same group in two self-efficacy tests (personality dimension). The objective is to assess the predictive value of these tests with respect to the scores obtained by the subjects in a simultaneous interpreting test. The analysis of data leads us to the conclusion that, at least in an initial training phase in simultaneous interpreting, only the cognitive factors (measured here in terms of listening span) show a significant and positive predictive capacity of the quality in simultaneous interpreting. In this regard, personality factors (measured here in terms of self-efficacy) offer results contradictory to the expectations or no significant correlation values.

scholarly journals Verification of filter efficiency of horizontal roughing filter by Weglin's design criteria and Artificial Neural Network

2009 ◽  
Vol 2 (1) ◽  
pp. 21-27 ◽  
Author(s):  
◽  
◽  
◽  
◽  
Keyword(s):  
Genetic Model ◽  
Neural Model ◽  
Model Verification ◽  
Design Criteria ◽  
Experimental Setup ◽  
Mean Square ◽  
Raw Water ◽  
Neural Models ◽  
Sand Filter ◽  
Slow Sand Filter

Abstract. The general objective of this study is to estimate the performance of the Horizontal Roughing Filter (HRF) by using Weglin's design criteria based on 1/3–2/3 filter theory. The main objective of the present study is to validate HRF developed in the laboratory with Slow Sand Filter (SSF) as a pretreatment unit with the help of Weglin's design criteria for HRF with respect to raw water condition and neuro-genetic model developed based on the filter dataset. The results achieved from the three different models were compared to find whether the performance of the experimental HRF with SSF output conforms to the other two models which will verify the validity of the former. According to the results, the experimental setup was coherent with the neural model but incoherent with the results from Weglin's formula as lowest mean square error was observed in case of the neuro-genetic model while comparing with the values found from the experimental SSF-HRF unit. As neural models are known to learn a problem with utmost efficiency, the model verification result was taken as positive.

scholarly journals A Goal-Driven Tree-Structured Neural Model for Math Word Problems

2019 ◽  
Author(s):  
Zhipeng Xie ◽  
Shichao Sun
Keyword(s):  
Word Problems ◽  
State Of The Art ◽  
Neural Model ◽  
Neural Models ◽  
Top Down ◽  
Feedforward Networks ◽  
Whole Process ◽  
Expression Tree ◽  
Generate Solution ◽  
Human Problem

Most existing neural models for math word problems exploit Seq2Seq model to generate solution expressions sequentially from left to right, whose results are far from satisfactory due to the lack of goal-driven mechanism commonly seen in human problem solving. This paper proposes a tree-structured neural model to generate expression tree in a goal-driven manner. Given a math word problem, the model first identifies and encodes its goal to achieve, and then the goal gets decomposed into sub-goals combined by an operator in a top-down recursive way. The whole process is repeated until the goal is simple enough to be realized by a known quantity as leaf node. During the process, two-layer gated-feedforward networks are designed to implement each step of goal decomposition, and a recursive neural network is used to encode fulfilled subtrees into subtree embeddings, which provides a better representation of subtrees than the simple goals of subtrees. Experimental results on the dataset Math23K have shown that our tree-structured model outperforms significantly several state-of-the-art models.

Sign in / Sign up

Export Citation Format

Share Document