Minimum description length pruning and maximum mutual information training of adaptive probabilistic neural networks

2002 ◽  
Author(s):  
W. Fakhr ◽  
M.I. Elmasry
Keyword(s):  
Neural Networks ◽  
Mutual Information ◽  
Minimum Description Length ◽  
Probabilistic Neural Networks ◽  
Maximum Mutual Information

scholarly journals Supposed Maximum Mutual Information for Improving Generalization and Interpretation of Multi-Layered Neural Networks

2019 ◽  
Vol 9 (2) ◽  
pp. 123-147 ◽  
Author(s):  
Ryotaro Kamimura
Keyword(s):  
Neural Networks ◽  
Mutual Information ◽  
Data Sets ◽  
Data Set ◽  
Information Theoretic ◽  
Information Maximization ◽  
Maximum Mutual Information ◽  
Information Theoretic Method ◽  
Mutual Information Maximization ◽  
Inputs And Outputs

Abstract The present paper1 aims to propose a new type of information-theoretic method to maximize mutual information between inputs and outputs. The importance of mutual information in neural networks is well known, but the actual implementation of mutual information maximization has been quite difficult to undertake. In addition, mutual information has not extensively been used in neural networks, meaning that its applicability is very limited. To overcome the shortcoming of mutual information maximization, we present it here in a very simplified manner by supposing that mutual information is already maximized before learning, or at least at the beginning of learning. The method was applied to three data sets (crab data set, wholesale data set, and human resources data set) and examined in terms of generalization performance and connection weights. The results showed that by disentangling connection weights, maximizing mutual information made it possible to explicitly interpret the relations between inputs and outputs.

The maximum mutual information without coding for binary quantum-state signals

1998 ◽  
Vol 45 (2) ◽  
pp. 269-282 ◽  
Author(s):  
MASAO OSAKI, OSAMU HIROTA MASASHI BAN
Keyword(s):  
Mutual Information ◽  
Quantum State ◽  
Maximum Mutual Information

Human Action Detection Based on Tracking Region of Maximum Mutual Information

2012 ◽  
Vol 38 (12) ◽  
pp. 2023 ◽  
Author(s):  
Tai-Qing WANG ◽  
Sheng-Jin WANG ◽  
Xiao-Qing DING
Keyword(s):  
Mutual Information ◽  
Human Action ◽  
Action Detection ◽  
Human Action Detection ◽  
Maximum Mutual Information
Sign in / Sign up

Export Citation Format

Share Document