Achieving upper mutual information bound precoding for V-OFDM with MMSE receiver

Task-nuisance decomposition describes why the information bottleneck loss I(z;x)−βI(z;y) is a suitable objective for supervised learning. The true category y is predicted for input x using latent variables z. When n is a nuisance independent from y, I(z;n) can be decreased by reducing I(z;x) since the latter upper bounds the former. We extend this framework by demonstrating that conditional mutual information I(z;x|y) provides an alternative upper bound for I(z;n). This bound is applicable even if z is not a sufficient representation of x, that is, I(z;y)≠I(x;y). We used mutual information neural estimation (MINE) to estimate I(z;x|y). Experiments demonstrated that I(z;x|y) is smaller than I(z;x) for layers closer to the input, matching the claim that the former is a tighter bound than the latter. Because of this difference, the information plane differs when I(z;x|y) is used instead of I(z;x).

Download Full-text

Individually Conditional Individual Mutual Information Bound on Generalization Error

10.1109/isit45174.2021.9518016 ◽

2021 ◽

Author(s):

Ruida Zhou ◽

Chao Tian ◽

Tie Liu

Keyword(s):

Mutual Information ◽

Generalization Error ◽

Information Bound

Download Full-text

The maximum mutual information without coding for binary quantum-state signals

Journal of Modern Optics ◽

10.1080/095003498151960 ◽

1998 ◽

Vol 45 (2) ◽

pp. 269-282 ◽

Cited By ~ 2

Author(s):

MASAO OSAKI, OSAMU HIROTA MASASHI BAN

Keyword(s):

Mutual Information ◽

Quantum State ◽

Maximum Mutual Information

Download Full-text

A mutual information‐based likelihood function for particle filter flood extent assimilation

Water Resources Research ◽

10.1029/2020wr027859 ◽

2021 ◽

Author(s):

Antara Dasgupta ◽

Renaud Hostache ◽

RAAJ Ramasankaran ◽

Guy J.‐P Schumann ◽

Stefania Grimaldi ◽

...

Keyword(s):

Mutual Information ◽

Particle Filter ◽

Likelihood Function

Download Full-text

Nonlinear Pattern Analysis of Ventricular Premature Beats by Mutual Information

Methods of Information in Medicine ◽

10.1055/s-0038-1636872 ◽

1997 ◽

Vol 36 (04/05) ◽

pp. 257-260 ◽

Cited By ~ 1

Author(s):

H. Saitoh ◽

T. Yokoshima ◽

H. Kishida ◽

H. Hayakawa ◽

R. J. Cohen ◽

...

Keyword(s):

Mutual Information ◽

Temporal Pattern ◽

Pattern Analysis ◽

Autonomic Activity ◽

Correlation Measure ◽

Ventricular Premature Beats ◽

Risk Of Mortality ◽

Premature Beats ◽

Information Values ◽

Nonlinear Pattern

Abstract:The frequency of ventricular premature beats (VPBs) has been related to the risk of mortality. However, little is known about the temporal pattern of occurrence of VPBs and its relationship to autonomic activity. Hence, we applied a general correlation measure, mutual information, to quantify how VPBs are generated over time. We also used mutual information to determine the correlation between VPB production and heart rate in order to evaluate effects of autonomic activity on VPB production. We examined twenty subjects with more than 3000 VPBs/day and simulated ran-( dom time series of VPB occurrence. We found that mutual information values could be used to characterize quantitatively the temporal patterns of VPB generation. Our data suggest that VPB production is not random and VPBs generated with a higher value of mutual information may be more greatly affected by autonomic activity.

Download Full-text

Data Screening Methods – Application to Differential Diagnosis in Pancreatic Pathology from Radiological Signs

Methods of Information in Medicine ◽

10.1055/s-0038-1636606 ◽

1978 ◽

Vol 17 (01) ◽

pp. 36-40 ◽

Cited By ~ 4

Author(s):

J.-P. Durbec ◽

Jaqueline Cornée ◽

P. Berthezene

Keyword(s):

Differential Diagnosis ◽

Mutual Information ◽

Data Processing ◽

Screening Methods ◽

Automatic Data ◽

Chronic Calcifying Pancreatitis ◽

Number Of Patients ◽

Calcifying Pancreatitis ◽

Pancreatic Pathology ◽

Data Screening

The practice of systematic examinations in hospitals and the increasing development of automatic data processing permits the storing of a great deal of information about a large number of patients belonging to different diagnosis groups.To predict or to characterize these diagnosis groups some descriptors are particularly useful, others carry no information. Data screening based on the properties of mutual information and on the log cross products ratios in contingency tables is developed. The most useful descriptors are selected. For each one the characterized groups are specified.This approach has been performed on a set of binary (presence—absence) radiological variables. Four diagnoses groups are concerned: cancer of pancreas, chronic calcifying pancreatitis, non-calcifying pancreatitis and probable pancreatitis. Only twenty of the three hundred and forty initial radiological variables are selected. The presence of each corresponding sign is associated with one or more diagnosis groups.

Download Full-text