Hybrid Neural-global Minimization Method of Logical Rule Extraction

Methodology of extraction of optimal sets of logical rules using neural networks and global minimization procedures has been developed. Initial rules are extracted using density estimation neural networks with rectangular functions or multilayered perceptron (MLP) networks trained with constrained backpropagation algorithm, transforming MLPs into simpler networks performing logical functions. A constructive algorithm called CMLP2LN is proposed, in which rules of increasing specificity are generated consecutively by adding more nodes to the network. Neural rule extraction is followed by optimization of rules using global minimization techniques. Estimation of confidence of various sets of rules is discussed. The hybrid approach to rule extraction has been applied to a number of benchmark and real life problems with very good results.

Download Full-text

NEURAL NETWORK RULE EXTRACTION BY A NEW ENSEMBLE CONCEPT AND ITS THEORETICAL AND HISTORICAL BACKGROUND: A REVIEW

International Journal of Computational Intelligence and Applications ◽

10.1142/s1469026813400063 ◽

2013 ◽

Vol 12 (04) ◽

pp. 1340006 ◽

Cited By ~ 13

Author(s):

YOICHI HAYASHI

Keyword(s):

Neural Network ◽

Neural Networks ◽

High Performance ◽

Real Life ◽

Rule Extraction ◽

Historical Background ◽

Neural Network Ensemble ◽

Extraction Algorithm ◽

Neural Network Ensembles ◽

Network Ensembles

This paper presents theoretical and historical backgrounds related to neural network rule extraction. It also investigates approaches for neural network rule extraction by ensemble concepts. Bologna pointed out that although many authors had generated comprehensive models from individual networks, much less work had been done to explain ensembles of neural networks. This paper carefully surveyed the previous work on rule extraction from neural network ensembles since 1988. We are aware of three major research groups i.e., Bologna' group, Zhou' group and Hayashi' group. The reason of these situations is obvious. Since the structures of previous neural network ensembles were quite complicated, the research on the efficient rule extraction algorithm from neural network ensembles was few although their learning capability was extremely high. Thus, these issues make rule extraction algorithm for neural network ensemble difficult task. However, there is a practical need for new ideas for neural network ensembles in order to realize the extremely high-performance needs of various rule extraction problems in real life. This paper successively explain nature of artificial neural networks, origin of neural network rule extraction, incorporating fuzziness in neural network rule extraction, theoretical foundation of neural network rule extraction, computational complexity of neural network rule extraction, neuro-fuzzy hybridization, previous rule extraction from neural network ensembles and difficulties of previous neural network ensembles. Next, this paper address three principles of proposed neural network rule extraction: to increase recognition rates, to extract rules from neural network ensembles, and to minimize the use of computing resources. We also propose an ensemble-recursive-rule extraction (E-Re-RX) by two or three standard backpropagation to train multi-layer perceptrons (MLPs), which enabled extremely high recognition accuracy and the extraction of comprehensible rules. Furthermore, this enabled rule extraction that resulted in fewer rules than those in previously proposed methods. This paper summarizes experimental results of rule extraction using E-Re-RX by multiple standard backpropagation MLPs and provides deep discussions. The results make it possible for the output from a neural network ensemble to be in the form of rules, thus open the "black box" of trained neural networks ensembles. Finally, we provide valuable conclusions and as future work, three open questions on the E-Re-RX algorithm.

Download Full-text

Model of Human Actions Recognition Based on 2D Kernel

ASME 2021 30th Conference on Information Storage and Processing Systems ◽

10.1115/isps2021-65031 ◽

2021 ◽

Author(s):

Bogdan Alexandru Radulescu ◽

Victorita Radulescu

Keyword(s):

Neural Networks ◽

Action Recognition ◽

Real Life ◽

Sensor Data ◽

Human Actions ◽

Advantages And Disadvantages ◽

Life Problems ◽

Starting Point ◽

Rgb Images ◽

Human Actions Recognition

Abstract Action Recognition is a domain that gains interest along with the development of specific motion capture equipment, hardware and power of processing. Its many applications in domains such as national security and behavior analysis make it even more popular among the scientific community, especially considering the ascending trend of machine learning methods. Nowadays approaches necessary to solve real life problems through human actions recognition became more interesting. To solve this problem are mainly two approaches when attempting to build a classifier, either using RGB images or sensor data, or where possible a combination of these two. Both methods have advantages and disadvantages and domains of utilization in real life problems, solvable through actions recognition. Using RGB input makes it possible to adopt a classifier on almost any infrastructure without specialized equipment, whereas combining video with sensor data provides a higher accuracy, albeit at a higher cost. Neural networks and especially convolutional neural networks are the starting point for human action recognition. By their nature, they can recognize very well spatial and temporal features, making them ideal for RGB images or sequences of RGB images. In the present paper is proposed the convolutional neural network architecture based on 2D kernels. Its structure, along with metrics measuring the performance, advantages and disadvantages are here illustrated. This solution based on 2D convolutions is fast, but has lower performance compared to other known solutions. The main problem when dealing with videos is the context extraction from a sequence of frames. Video classification using 2D Convolutional Layers is realized either by the most significant frame or by frame to frame, applying a probability distribution over the partial classes to obtain the final prediction. To classify actions, especially when differences between them are subtle, and consists of only a small part of the overall image is difficult. When classifying via the key frames, the total accuracy obtained is around 10%. The other approach, classifying each frame individually, proved to be too computationally expensive with negligible gains.

Download Full-text

A FRAMEWORK FOR INTEGRATING ARTIFICIAL NEURAL NETWORKS AND LOGIC PROGRAMMING

International Journal of Artificial Intelligence Tools ◽

10.1142/s0218213095000024 ◽

1995 ◽

Vol 04 (01n02) ◽

pp. 3-32 ◽

Cited By ~ 9

Author(s):

J.H.M. LEE ◽

V.W.L. TAM

Keyword(s):

Neural Networks ◽

Artificial Neural Networks ◽

Logic Programming ◽

Operational Semantics ◽

Real Life ◽

Constraint Logic Programming ◽

Constraint Satisfaction Problems ◽

Problem Size ◽

Life Problems ◽

Artificial Neural

Many real-life problems belong to the class of constraint satisfaction problems (CSP’s), which are NP-complete, and some NP-hard, in general. When the problem size grows, it becomes difficult to program solutions and to execute the solution in a timely manner. In this paper, we present a general framework for integrating artificial neural networks and logic programming so as to provide an efficient and yet expressive programming environment for solving CSP’s. To realize this framework, we propose PROCLANN, a novel constraint logic programming language. The PROCLANN language retains the simple and elegant declarative semantics of constraint logic programming. Operationally, PROCLANN uses the standard goal reduction strategy in the frontend to generate constraints, and an efficient backend constraint-solver based on artificial neural networks. Its operational semantics is probabilistic in nature. We show that PROCLANN is sound and weakly complete. A novelty of PROCLANN is that while it is a committed-choice language, PROCLANN supports non-determinism, allowing the generation of multiple answers to a query. An initial prototype implementation of PROCLANN is constructed and demonstrates empirically that PROCLANN out-performs the state of art in constraint logic programming implementation on certain hard instances of CSP’s.

Download Full-text

Direct-fit to nature: an evolutionary perspective on biological (and artificial) neural networks

10.1101/764258 ◽

2019 ◽

Author(s):

Uri Hasson ◽

Samuel A. Nastase ◽

Ariel Goldstein

Keyword(s):

Neural Networks ◽

Real Life ◽

Scientific Models ◽

Mammalian Brain ◽

Underlying Structure ◽

Brute Force ◽

Dimensional Parameter ◽

Robust Solution ◽

Life Problems ◽

Action Coordination

AbstractEvolution is a blind fitting process by which organisms, over generations, adapt to the niches of an ever-changing environment. Does the mammalian brain use similar brute-force fitting processes to learn how to perceive and act upon the world? Recent advances in training deep neural networks has exposed the power of optimizing millions of synaptic weights to map millions of observations along ecologically relevant objective functions. This class of models has dramatically outstripped simpler, more intuitive models, operating robustly in real-life contexts spanning perception, language, and action coordination. These models do not learn an explicit, human-interpretable representation of the underlying structure of the data; rather, they use local computations to interpolate over task-relevant manifolds in a high-dimensional parameter space. Furthermore, counterintuitively, over-parameterized models, similarly to evolutionary processes, can be simple and parsimonious as they provide a versatile, robust solution for learning a diverse set of functions. In contrast to traditional scientific models, where the ultimate goal is interpretability, over-parameterized models eschew interpretability in favor of solving real-life problems or tasks. We contend that over-parameterized blind fitting presents a radical challenge to many of the underlying assumptions and practices in computational neuroscience and cognitive psychology. At the same time, this shift in perspective informs longstanding debates and establishes unexpected links with evolution, ecological psychology, and artificial life.

Download Full-text

An Fusion of Whale and Sine Cosine Algorithms for solving optimization Functions

10.20944/preprints202010.0319.v1 ◽

2020 ◽

Author(s):

Gurjeet Singh Walia ◽

Narinder Singh ◽

Sharandeep Singh

Keyword(s):

Computer Science ◽

Hybrid Approach ◽

Real Life ◽

Optimal Solutions ◽

Science And Engineering ◽

Exploration Process ◽

Life Problems ◽

Medical Computer ◽

Statistical Solutions

We developed a novel hybrid approach for solving global optimization, computer science, bio-medical and engineering real life applications that is based on the coupling of the Whale Optimizer and Sine Cosine Algorithms via a surrogate model. We relate the whale optimizer algorithm to balance between the exploitation and the exploration process in the proposed method. There exist confirmed techniques for searching approximate best optimal solutions, but our algorithm will further guarantee that such numerical and statistical solutions satisfy physical bounds of the standard and real life functions. Our experiments with the benchmark, bio-medical, computer science and engineering real life problems have illustrated the advantages of using a newly hybrid approach based on mixing Whale Optimizer and Sine Cosine algorithms. It holds considerable potential for reducing execution time for solving standard and real life problems and at the same time improving the quality of the solution.

Download Full-text

Information Feedback Appraoch for the Simulation of Service Quality in the Inter-Object Communications

Simulation and Modeling ◽

10.4018/978-1-59904-198-8.ch006 ◽

2008 ◽

pp. 175-198

Author(s):

R. Manjunath

Keyword(s):

Neural Networks ◽

Real Life ◽

System Level ◽

New Class ◽

Model Based ◽

Multiple Flows ◽

Life Problems ◽

Reference Design ◽

The Impact ◽

Level Performance

Simulation of a system with limited data is challenging. It calls for a certain degree of intelligence built into the system. This chapter provides a new model-based simulation methodology that may be customized and used in the simulation of a wide variety of problems involving multiple source-destination flows with intermediate agents. It explains the model based on a new class of neural networks called differentially fed artificial neural networks and the system level performance of the same. Next, as an example, the impact of system level differential feedback on multiple flows and the application of the concept are presented, followed by the simulation results. The author hopes that a variety of real life problems that involve multiple flows may be mapped onto this simulation model and optimal performance may be obtained. The model serves as a reference design that may be fine-tuned based on the application.

Download Full-text

Rule Extraction From Binary Neural Networks With Convolutional Rules for Model Validation

Frontiers in Artificial Intelligence ◽

10.3389/frai.2021.642263 ◽

2021 ◽

Vol 4 ◽

Author(s):

Sophie Burkhardt ◽

Jannis Brugger ◽

Nicolas Wagner ◽

Zahra Ahmadi ◽

Kristian Kersting ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Rule Learning ◽

Rule Extraction ◽

High Dimensional ◽

Stochastic Local Search ◽

Correct Operation ◽

Rule Based ◽

The Individual ◽

Logical Rules

Classification approaches that allow to extract logical rules such as decision trees are often considered to be more interpretable than neural networks. Also, logical rules are comparatively easy to verify with any possible input. This is an important part in systems that aim to ensure correct operation of a given model. However, for high-dimensional input data such as images, the individual symbols, i.e. pixels, are not easily interpretable. Therefore, rule-based approaches are not typically used for this kind of high-dimensional data. We introduce the concept of first-order convolutional rules, which are logical rules that can be extracted using a convolutional neural network (CNN), and whose complexity depends on the size of the convolutional filter and not on the dimensionality of the input. Our approach is based on rule extraction from binary neural networks with stochastic local search. We show how to extract rules that are not necessarily short, but characteristic of the input, and easy to visualize. Our experiments show that the proposed approach is able to model the functionality of the neural network while at the same time producing interpretable logical rules. Thus, we demonstrate the potential of rule-based approaches for images which allows to combine advantages of neural networks and rule learning.

Download Full-text