Feature space maximum a posteriori linear regression for adaptation of deep neural networks

Convolutional Neural Networks (CNNs) have shown great power in various classification tasks and have achieved remarkable results in practical applications. However, the distinct learning difficulties in discriminating different pairs of classes are largely ignored by the existing networks. For instance, in CIFAR-10 dataset, distinguishing cats from dogs is usually harder than distinguishing horses from ships. By carefully studying the behavior of CNN models in the training process, we observe that the confusion level of two classes is strongly correlated with their angular separability in the feature space. That is, the larger the inter-class angle is, the lower the confusion will be. Based on this observation, we propose a novel loss function dubbed “Inter-Class Angular Loss” (ICAL), which explicitly models the class correlation and can be directly applied to many existing deep networks. By minimizing the proposed ICAL, the networks can effectively discriminate the examples in similar classes by enlarging the angle between their corresponding class vectors. Thorough experimental results on a series of vision and nonvision datasets confirm that ICAL critically improves the discriminative ability of various representative deep neural networks and generates superior performance to the original networks with conventional softmax loss.

Download Full-text

Effects of Food Contamination on Gastrointestinal Morbidity: Comparison of Different Machine-Learning Methods

International Journal of Environmental Research and Public Health ◽

10.3390/ijerph16050838 ◽

2019 ◽

Vol 16 (5) ◽

pp. 838 ◽

Cited By ~ 4

Author(s):

Qin Song ◽

Yu-Jun Zheng ◽

Jun Yang

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Linear Regression ◽

Deep Neural Networks ◽

Gastrointestinal Disease ◽

Food Contamination ◽

Gastrointestinal Diseases ◽

Central China ◽

Learning Models

Morbidity prediction can be useful in improving the effectiveness and efficiency of medical services, but accurate morbidity prediction is often difficult because of the complex relationships between diseases and their influencing factors. This study investigates the effects of food contamination on gastrointestinal-disease morbidities using eight different machine-learning models, including multiple linear regression, a shallow neural network, and three deep neural networks and their improved versions trained by an evolutionary algorithm. Experiments on the datasets from ten cities/counties in central China demonstrate that deep neural networks achieve significantly higher accuracy than classical linear-regression and shallow neural-network models, and the deep denoising autoencoder model with evolutionary learning exhibits the best prediction performance. The results also indicate that the prediction accuracies on acute gastrointestinal diseases are generally higher than those on other diseases, but the models are difficult to predict the morbidities of gastrointestinal tumors. This study demonstrates that evolutionary deep-learning models can be utilized to accurately predict the morbidities of most gastrointestinal diseases from food contamination, and this approach can be extended for the morbidity prediction of many other diseases.

Download Full-text

SVM Based Speaker Recognition Using Maximum a posteriori Linear Regression

2009 International Conference on Electronic Computer Technology ◽

10.1109/icect.2009.82 ◽

2009 ◽

Author(s):

Xiang Zhang ◽

Qingwei Zhao ◽

Yonghong Yan

Keyword(s):

Linear Regression ◽

Speaker Recognition ◽

Maximum A Posteriori ◽

A Posteriori

Download Full-text

New approach to Bayesian high-dimensional linear regression

Information and Inference A Journal of the IMA ◽

10.1093/imaiai/iax016 ◽

2018 ◽

Vol 7 (4) ◽

pp. 605-655 ◽

Cited By ~ 2

Author(s):

Shirin Jalali ◽

Arian Maleki

Keyword(s):

Linear Regression ◽

Sampling Rate ◽

High Dimensional ◽

Maximum A Posteriori ◽

Map Estimation ◽

A Posteriori ◽

Fundamental Limits ◽

New Approach ◽

Estimation Scheme ◽

Response Variables

Abstract Consider the problem of estimating parameters $X^n \in \mathbb{R}^n $, from $m$ response variables $Y^m = AX^n+Z^m$, under the assumption that the distribution of $X^n$ is known. Lack of computationally feasible algorithms that employ generic prior distributions and provide a good estimate of $X^n$ has limited the set of distributions researchers use to model the data. To address this challenge, in this article, a new estimation scheme named quantized maximum a posteriori (Q-MAP) is proposed. The new method has the following properties: (i) In the noiseless setting, it has similarities to maximum a posteriori (MAP) estimation. (ii) In the noiseless setting, when $X_1,\ldots,X_n$ are independent and identically distributed, asymptotically, as $n$ grows to infinity, its required sampling rate ($m/n$) for an almost zero-distortion recovery approaches the fundamental limits. (iii) It scales favorably with the dimensions of the problem and therefore is applicable to high-dimensional setups. (iv) The solution of the Q-MAP optimization can be found via a proposed iterative algorithm that is provably robust to error (noise) in response variables.

Download Full-text