Structure injected weight normalization for training deep networks

2021 ◽  
Author(s):  
Xu Yuan ◽  
Xiangjun Shen ◽  
Sumet Mehta ◽  
Teng Li ◽  
Shiming Ge ◽  
...  
Keyword(s):  
Deep Networks

Joint Deep Networks based Multi-source Feature Learning for QoS Prediction

2021 ◽  
pp. 1-1
Author(s):  
Youhao Xia ◽  
Ding Ding ◽  
Zhenhua Chang ◽  
Fan Li
Keyword(s):  
Feature Learning ◽  
Qos Prediction ◽  
Deep Networks

Variational Methods for Machine Learning with Applications to Deep Networks

2021 ◽  
Author(s):  
Lucas Pinheiro Cinelli ◽  
Matheus Araújo Marins ◽  
Eduardo Antônio Barros da Silva ◽  
Sérgio Lima Netto
Keyword(s):  
Machine Learning ◽  
Variational Methods ◽  
Deep Networks

Bayesian deep networks for absolute permeability and porosity uncertainty prediction from image borehole logs from brazilian carbonate reservoirs

2021 ◽  
pp. 108361
Author(s):  
Clécio R. Bom ◽  
Manuel Blanco Valentín ◽  
Bernardo M.O. Fraga ◽  
Jorge Campos ◽  
Bernardo Coutinho ◽  
...  
Keyword(s):  
Carbonate Reservoirs ◽  
Absolute Permeability ◽  
Uncertainty Prediction ◽  
Deep Networks ◽  
Borehole Logs

scholarly journals Learning deep networks with crowdsourcing for relevance evaluation

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Ming Wu ◽  
Xiaochun Yin ◽  
Qianmu Li ◽  
Jing Zhang ◽  
Xinqi Feng ◽  
...  
Keyword(s):  
Deep Networks ◽  
Relevance Evaluation

scholarly journals Heuristic rank selection with progressively searching tensor ring network

2021 ◽  
Author(s):  
Nannan Li ◽  
Yu Pan ◽  
Yaran Chen ◽  
Zixiang Ding ◽  
Dongbin Zhao ◽  
...  
Keyword(s):  
Genetic Algorithm ◽  
Compression Ratio ◽  
State Of The Art ◽  
Heuristic Method ◽  
Ring Networks ◽  
Ring Network ◽  
Narrow Region ◽  
Network Search ◽  
Deep Networks ◽  
Evolutionary Phase

AbstractRecently, tensor ring networks (TRNs) have been applied in deep networks, achieving remarkable successes in compression ratio and accuracy. Although highly related to the performance of TRNs, rank selection is seldom studied in previous works and usually set to equal in experiments. Meanwhile, there is not any heuristic method to choose the rank, and an enumerating way to find appropriate rank is extremely time-consuming. Interestingly, we discover that part of the rank elements is sensitive and usually aggregate in a narrow region, namely an interest region. Therefore, based on the above phenomenon, we propose a novel progressive genetic algorithm named progressively searching tensor ring network search (PSTRN), which has the ability to find optimal rank precisely and efficiently. Through the evolutionary phase and progressive phase, PSTRN can converge to the interest region quickly and harvest good performance. Experimental results show that PSTRN can significantly reduce the complexity of seeking rank, compared with the enumerating method. Furthermore, our method is validated on public benchmarks like MNIST, CIFAR10/100, UCF11 and HMDB51, achieving the state-of-the-art performance.

scholarly journals Data Independent Acquisition Based Bi-Directional Deep Networks for Biometric ECG Authentication

2021 ◽  
Vol 11 (3) ◽  
pp. 1125
Author(s):  
Htet Myet Lynn ◽  
Pankoo Kim ◽  
Sung Bum Pan
Keyword(s):  
Short Term Memory ◽  
Feature Learning ◽  
Biometric Authentication ◽  
Qrs Complex ◽  
Data Types ◽  
Ecg Signals ◽  
Data Independent Acquisition ◽  
Deep Networks ◽  
Classification Tasks ◽  
Electrocardiogram Ecg

In this report, the study of non-fiducial based approaches for Electrocardiogram(ECG) biometric authentication is examined, and several excessive techniques are proposed to perform comparative experiments for evaluating the best possible approach for all the classification tasks. Non-fiducial methods are designed to extract the discriminative information of a signal without annotating fiducial points. However, this process requires peak detection to identify a heartbeat signal. Based on recent studies that usually rely on heartbeat segmentation, QRS detection is required, and the process can be complicated for ECG signals for which the QRS complex is absent. Thus, many studies only conduct biometric authentication tasks on ECG signals with QRS complexes, and are hindered by similar limitations. To overcome this issue, we proposed a data-independent acquisition method to facilitate highly generalizable signal processing and feature learning processes. This is achieved by enhancing random segmentation to avoid complicated fiducial feature extraction, along with auto-correlation to eliminate the phase difference due to random segmentation. Subsequently, a bidirectional recurrent neural network (RNN) with long short-term memory (LSTM) deep networks is utilized to automatically learn the features associated with the signal and to perform an authentication task. The experimental results suggest that the proposed data-independent approach using a BLSTM network achieves a relatively high classification accuracy for every dataset relative to the compared techniques. Moreover, it exhibited a significantly higher accuracy rate in experiments using ECG signals without the QRS complex. The results also revealed that data-dependent methods can only perform well for specified data types and amendments of data variations, whereas the presented approach can also be considered for generalization to other quasi-periodical biometric signal-based classification tasks in future studies.

scholarly journals Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Abdulkadir Canatar ◽  
Blake Bordelon ◽  
Cengiz Pehlevan
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Kernel Regression ◽  
Learning Task ◽  
Learning Curves ◽  
Generalization Error ◽  
Theoretical Understanding ◽  
Classical Statistics ◽  
Deep Networks ◽  
Model Alignment

AbstractA theoretical understanding of generalization remains an open problem for many machine learning models, including deep networks where overparameterization leads to better performance, contradicting the conventional wisdom from classical statistics. Here, we investigate generalization error for kernel regression, which, besides being a popular machine learning method, also describes certain infinitely overparameterized neural networks. We use techniques from statistical mechanics to derive an analytical expression for generalization error applicable to any kernel and data distribution. We present applications of our theory to real and synthetic datasets, and for many kernels including those that arise from training deep networks in the infinite-width limit. We elucidate an inductive bias of kernel regression to explain data with simple functions, characterize whether a kernel is compatible with a learning task, and show that more data may impair generalization when noisy or not expressible by the kernel, leading to non-monotonic learning curves with possibly many peaks.

Sign in / Sign up

Export Citation Format

Share Document