scholarly journals VPNET: Variable Projection Networks

2021 ◽  
Author(s):  
Péter Kovács ◽  
Gergő Bognár ◽  
Christian Huber ◽  
Mario Huemer
Keyword(s):  
Signal Processing ◽  
Network Architecture ◽  
Computational Cost ◽  
Learning Ability ◽  
Ecg Signals ◽  
Variable Projection ◽  
Model Driven ◽  
Convolutional Networks ◽  
Fast Learning ◽  
Fully Connected

In this paper, we introduce VPNet, a novel model-driven neural network architecture based on variable projection (VP). Applying VP operators to neural networks results in learnable features, interpretable parameters, and compact network structures. This paper discusses the motivation and mathematical background of VPNet and presents experiments. The VPNet approach was evaluated in the context of signal processing, where we classified a synthetic dataset and real electrocardiogram (ECG) signals. Compared to fully connected and one-dimensional convolutional networks, VPNet offers fast learning ability and good accuracy at a low computational cost of both training and inference. Based on these advantages and the promising results obtained, we anticipate a profound impact on the broader field of signal processing, in particular on classification, regression and clustering problems.

scholarly journals Applying Fully Convolutional Architectures for Semantic Segmentation of a Single Tree Species in Urban Environment on High Resolution UAV Optical Imagery

Sensors ◽  
2020 ◽  
Vol 20 (2) ◽  
pp. 563 ◽  
Author(s):  
Daliana Lobo Torres ◽  
Raul Queiroz Feitosa ◽  
Patrick Nigri Happ ◽  
Laura Elena Cué La Rosa ◽  
José Marcato Junior ◽  
...  
Keyword(s):  
Tree Species ◽  
Conditional Random Fields ◽  
Computational Cost ◽  
Semantic Segmentation ◽  
Convolutional Networks ◽  
Single Tree ◽  
Fully Convolutional Networks ◽  
Processing Step ◽  
Fully Connected ◽  
High Computational Cost

This study proposes and evaluates five deep fully convolutional networks (FCNs) for the semantic segmentation of a single tree species: SegNet, U-Net, FC-DenseNet, and two DeepLabv3+ variants. The performance of the FCN designs is evaluated experimentally in terms of classification accuracy and computational load. We also verify the benefits of fully connected conditional random fields (CRFs) as a post-processing step to improve the segmentation maps. The analysis is conducted on a set of images captured by an RGB camera aboard a UAV flying over an urban area. The dataset also contains a mask that indicates the occurrence of an endangered species called Dipteryx alata Vogel, also known as cumbaru, taken as the species to be identified. The experimental analysis shows the effectiveness of each design and reports average overall accuracy ranging from 88.9% to 96.7%, an F1-score between 87.0% and 96.1%, and IoU from 77.1% to 92.5%. We also realize that CRF consistently improves the performance, but at a high computational cost.

scholarly journals Dynamically Optimizing Network Structure Based on Synaptic Pruning in the Brain

2021 ◽  
Vol 15 ◽  
Author(s):  
Feifei Zhao ◽  
Yi Zeng
Keyword(s):  
Neural Networks ◽  
Network Architecture ◽  
Network Performance ◽  
Computational Cost ◽  
Deep Convolutional Neural Networks ◽  
Learning Speed ◽  
Proposed Model ◽  
The Cost ◽  
Fully Connected ◽  
Synaptic Pruning

Most neural networks need to predefine the network architecture empirically, which may cause over-fitting or under-fitting. Besides, a large number of parameters in a fully connected network leads to the prohibitively expensive computational cost and storage overhead, which makes the model hard to be deployed on mobile devices. Dynamically optimizing the network architecture by pruning unused synapses is a promising technique for solving this problem. Most existing pruning methods focus on reducing the redundancy of deep convolutional neural networks by pruning unimportant filters or weights, at the cost of accuracy drop. In this paper, we propose an effective brain-inspired synaptic pruning method to dynamically modulate the network architecture and simultaneously improve network performance. The proposed model is biologically inspired as it dynamically eliminates redundant connections based on the synaptic pruning rules used during the brain's development. Connections are pruned if they are not activated or less activated multiple times consecutively. Extensive experiments demonstrate the effectiveness of our method on classification tasks of different complexity with the MNIST, Fashion MNIST, and CIFAR-10 datasets. Experimental results reveal that even for a compact network, the proposed method can also remove up to 59–90% of the connections, with relative improvement in learning speed and accuracy.

scholarly journals Multirate Processing with Selective Subbands and Machine Learning for Efficient Arrhythmia Classification

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1511
Author(s):  
Saeed Mian Qaisar ◽  
Alaeddine Mihoub ◽  
Moez Krichen ◽  
Humaira Nisar
Keyword(s):  
Machine Learning ◽  
Signal Reconstruction ◽  
Computational Cost ◽  
Machine Learning Techniques ◽  
Features Selection ◽  
Frequency Content ◽  
Fixed Rate ◽  
Ecg Signals ◽  
Learning Techniques ◽  
Multirate Processing

The usage of wearable gadgets is growing in the cloud-based health monitoring systems. The signal compression, computational and power efficiencies play an imperative part in this scenario. In this context, we propose an efficient method for the diagnosis of cardiovascular diseases based on electrocardiogram (ECG) signals. The method combines multirate processing, wavelet decomposition and frequency content-based subband coefficient selection and machine learning techniques. Multirate processing and features selection is used to reduce the amount of information processed thus reducing the computational complexity of the proposed system relative to the equivalent fixed-rate solutions. Frequency content-dependent subband coefficient selection enhances the compression gain and reduces the transmission activity and computational cost of the post cloud-based classification. We have used MIT-BIH dataset for our experiments. To avoid overfitting and biasness, the performance of considered classifiers is studied by using five-fold cross validation (5CV) and a novel proposed partial blind protocol. The designed method achieves more than 12-fold computational gain while assuring an appropriate signal reconstruction. The compression gain is 13 times compared to fixed-rate counterparts and the highest classification accuracies are 97.06% and 92.08% for the 5CV and partial blind cases, respectively. Results suggest the feasibility of detecting cardiac arrhythmias using the proposed approach.

Assessment of Machine Learning-Based Audiovisual Quality Predictors

2021 ◽  
Vol 17 (2) ◽  
pp. 1-22
Author(s):  
Mythili K. ◽  
Manish Narwaria
Keyword(s):  
Machine Learning ◽  
Signal Processing ◽  
Quality Assessment ◽  
Computational Models ◽  
Model Development ◽  
Multimedia Communication ◽  
Current Approach ◽  
Learning Ability ◽  
Data Uncertainty ◽  
Attractive Alternative

Quality assessment of audiovisual (AV) signals is important from the perspective of system design, optimization, and management of a modern multimedia communication system. However, automatic prediction of AV quality via the use of computational models remains challenging. In this context, machine learning (ML) appears to be an attractive alternative to the traditional approaches. This is especially when such assessment needs to be made in no-reference (i.e., the original signal is unavailable) fashion. While development of ML-based quality predictors is desirable, we argue that proper assessment and validation of such predictors is also crucial before they can be deployed in practice. To this end, we raise some fundamental questions about the current approach of ML-based model development for AV quality assessment and signal processing for multimedia communication in general. We also identify specific limitations associated with the current validation strategy which have implications on analysis and comparison of ML-based quality predictors. These include a lack of consideration of: (a) data uncertainty, (b) domain knowledge, (c) explicit learning ability of the trained model, and (d) interpretability of the resultant model. Therefore, the primary goal of this article is to shed some light into mentioned factors. Our analysis and proposed recommendations are of particular importance in the light of significant interests in ML methods for multimedia signal processing (specifically in cases where human-labeled data is used), and a lack of discussion of mentioned issues in existing literature.

scholarly journals Automatic ECG Classification Using Continuous Wavelet Transform and Convolutional Neural Network

Entropy ◽  
2021 ◽  
Vol 23 (1) ◽  
pp. 119
Author(s):  
Tao Wang ◽  
Changhua Lu ◽  
Yining Sun ◽  
Mei Yang ◽  
Chun Liu ◽  
...  
Keyword(s):  
Neural Network ◽  
Wavelet Transform ◽  
Convolutional Neural Network ◽  
Continuous Wavelet Transform ◽  
Continuous Wavelet ◽  
Time Frequency ◽  
Ecg Signals ◽  
Rr Interval ◽  
Frequency Components ◽  
Fully Connected

Early detection of arrhythmia and effective treatment can prevent deaths caused by cardiovascular disease (CVD). In clinical practice, the diagnosis is made by checking the electrocardiogram (ECG) beat-by-beat, but this is usually time-consuming and laborious. In the paper, we propose an automatic ECG classification method based on Continuous Wavelet Transform (CWT) and Convolutional Neural Network (CNN). CWT is used to decompose ECG signals to obtain different time-frequency components, and CNN is used to extract features from the 2D-scalogram composed of the above time-frequency components. Considering the surrounding R peak interval (also called RR interval) is also useful for the diagnosis of arrhythmia, four RR interval features are extracted and combined with the CNN features to input into a fully connected layer for ECG classification. By testing in the MIT-BIH arrhythmia database, our method achieves an overall performance of 70.75%, 67.47%, 68.76%, and 98.74% for positive predictive value, sensitivity, F1-score, and accuracy, respectively. Compared with existing methods, the overall F1-score of our method is increased by 4.75~16.85%. Because our method is simple and highly accurate, it can potentially be used as a clinical auxiliary diagnostic tool.

scholarly journals Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2852
Author(s):  
Parvathaneni Naga Srinivasu ◽  
Jalluri Gnana SivaSai ◽  
Muhammad Fazal Ijaz ◽  
Akash Kumar Bhoi ◽  
Wonjoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Skin Disease ◽  
Network Architecture ◽  
Large Scale ◽  
Short Term Memory ◽  
Convolutional Networks ◽  
Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

scholarly journals Recurrent Multi-Fiber Network for 3D MRI Brain Tumor Segmentation

Symmetry ◽  
2021 ◽  
Vol 13 (2) ◽  
pp. 320
Author(s):  
Yue Zhao ◽  
Xiaoqiang Ren ◽  
Kun Hou ◽  
Wentao Li
Keyword(s):  
Brain Tumor ◽  
Network Architecture ◽  
Contextual Information ◽  
Computational Cost ◽  
Disease Diagnosis ◽  
Tumor Segmentation ◽  
Brain Tumor Segmentation ◽  
Fiber Network ◽  
3D Network ◽  
The Brain

Automated brain tumor segmentation based on 3D magnetic resonance imaging (MRI) is critical to disease diagnosis. Moreover, robust and accurate achieving automatic extraction of brain tumor is a big challenge because of the inherent heterogeneity of the tumor structure. In this paper, we present an efficient semantic segmentation 3D recurrent multi-fiber network (RMFNet), which is based on encoder–decoder architecture to segment the brain tumor accurately. 3D RMFNet is applied in our paper to solve the problem of brain tumor segmentation, including a 3D recurrent unit and 3D multi-fiber unit. First of all, we propose that recurrent units segment brain tumors by connecting recurrent units and convolutional layers. This quality enhances the model’s ability to integrate contextual information and is of great significance to enhance the contextual information. Then, a 3D multi-fiber unit is added to the overall network to solve the high computational cost caused by the use of a 3D network architecture to capture local features. 3D RMFNet combines both advantages from a 3D recurrent unit and 3D multi-fiber unit. Extensive experiments on the Brain Tumor Segmentation (BraTS) 2018 challenge dataset show that our RMFNet remarkably outperforms state-of-the-art methods, and achieves average Dice scores of 89.62%, 83.65% and 78.72% for the whole tumor, tumor core and enhancing tumor, respectively. The experimental results prove our architecture to be an efficient tool for brain tumor segmentation accurately.

PYRAMIDAL HYBRID APPROACH: WAVELET NETWORK WITH OLS ALGORITHM-BASED IMAGE CLASSIFICATION

2011 ◽  
Vol 09 (01) ◽  
pp. 111-130 ◽  
Author(s):  
OLFA JEMAI ◽  
MOURAD ZAIED ◽  
CHOKRI BEN AMAR ◽  
MOHAMED ADEL ALIMI
Keyword(s):  
Least Squares ◽  
Image Classification ◽  
Network Architecture ◽  
Hybrid Approach ◽  
Learning Ability ◽  
Wavelet Network ◽  
Wavelet Networks ◽  
Orthogonal Least Squares ◽  
Colour Image ◽  
Beta Wavelet

Taking advantage of both the scaling property of wavelets and the high learning ability of neural networks, wavelet networks have recently emerged as a powerful tool in many applications in the field of signal processing such as data compression, function approximation as well as image recognition and classification. A novel wavelet network-based method for image classification is presented in this paper. The method combines the Orthogonal Least Squares algorithm (OLS) with the Pyramidal Beta Wavelet Network architecture (PBWN). First, the structure of the Pyramidal Beta Wavelet Network is proposed and the OLS method is used to design it by presetting the widths of the hidden units in PBWN. Then, to enhance the performance of the obtained PBWN, a novel learning algorithm based on orthogonal least squares and frames theory is proposed, in which we use OLS to select the hidden nodes. In the simulation part, the proposed method is employed to classify colour images. Comparisons with some typical wavelet networks are presented and discussed. Simulations also show that the PBWN-orthogonal least squares (PBWN-OLS) algorithm, which combines PBWN with the OLS algorithm, results in better performance for colour image classification.

Pruning In Time (PIT): A Lightweight Network Architecture Optimizer for Temporal Convolutional Networks

2021 ◽  
Author(s):  
Matteo Risso ◽  
Alessio Burrello ◽  
Daniele Jahier Pagliari ◽  
Francesco Conti ◽  
Lorenzo Lamberti ◽  
...  
Keyword(s):  
Network Architecture ◽  
Convolutional Networks
Sign in / Sign up

Export Citation Format

Share Document