scholarly journals Phase harmonic correlations and convolutional neural networks

2019 ◽  
Vol 9 (3) ◽  
pp. 721-747 ◽  
Author(s):  
Stéphane Mallat ◽  
Sixin Zhang ◽  
Gaspar Rochette
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Coherent Structures ◽  
Phase Dependence ◽  
Wavelet Filters ◽  
Convolutional Networks ◽  
Nonlinear Phase ◽  
Phase Filter ◽  
Multiplicative Operator ◽  
Space Frequency

Abstract A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters that are well localized in the frequency domain, with different phases. We show that a rectifier then acts as a filter on the phase of the resulting coefficients. It computes signal descriptors that are local in space, frequency and phase. The nonlinear phase filter becomes a multiplicative operator over phase harmonics computed with a Fourier transform along the phase. We prove that it defines a bi-Lipschitz and invertible representation. The correlations of phase harmonics coefficients characterize coherent structures from their phase dependence across frequencies. For wavelet filters, we show numerically that signals having sparse wavelet coefficients can be recovered from few phase harmonic correlations, which provide a compressive representation.

scholarly journals Single shot detector application for image disease localization

2021 ◽  
Author(s):  
shrikant pawar ◽  
Aditya Stanam ◽  
Rushikesh Chopade
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Feature Detection ◽  
Single Shot ◽  
X Ray ◽  
Convolutional Networks ◽  
Fully Convolutional Networks ◽  
Bounding Box ◽  
Unique Approach ◽  
Detection And Localization

Bounding box algorithms are useful in localization of image patterns. Recently, utilization of convolutional neural networks on X-ray images has proven a promising disease prediction technique. However, pattern localization over prediction has always been a challenging task with inconsistent coordinates, sizes, resolution and capture positions of an image. Several model architectures like Fast R-CNN, Faster R-CNN, Histogram of Oriented Gradients (HOG), You only look once (YOLO), Region-based Convolutional Neural Networks (R-CNN), Region-based Fully Convolutional Networks (R-FCN), Single Shot Detector (SSD), etc. are used for object detection and localization in modern-day computer vision applications. SSD and region-based detectors like Fast R-CNN or Faster R-CNN are very similar in design and implementation, but SSD have shown to work efficiently with larger frames per second (FPS) and lower resolution images. In this article, we present a unique approach of SSD with a VGG-16 network as a backbone for feature detection of bounding box algorithm to predict the location of an anomaly within chest X-ray image.

scholarly journals Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

2019 ◽  
Author(s):  
Chunlei Liu ◽  
Wenrui Ding ◽  
Xin Xia ◽  
Yuan Hu ◽  
Baochang Zhang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Superior Performance ◽  
Feature Maps ◽  
Unified Framework ◽  
Deep Convolutional Neural Networks ◽  
Convolutional Networks ◽  
Computation Efficiency ◽  
Significant Performance ◽  
Binary Network

Binarized  convolutional neural networks (BCNNs) are widely used to improve memory and computation efficiency of deep convolutional neural networks (DCNNs) for mobile and AI chips based applications. However, current BCNNs are not able to fully explore their corresponding full-precision models, causing a significant performance gap between them. In this paper, we propose rectified binary convolutional networks (RBCNs), towards optimized BCNNs, by combining full-precision kernels and feature maps to rectify the binarization process in a unified framework. In particular, we use a GAN to train the 1-bit binary network with the guidance of its corresponding full-precision model, which significantly improves the performance of BCNNs. The rectified convolutional layers are generic and flexible, and can be easily incorporated into existing DCNNs such as WideResNets and ResNets. Extensive experiments demonstrate the superior performance of the proposed RBCNs over state-of-the-art BCNNs. In particular, our method shows strong generalization on the object tracking task.

scholarly journals Modelling cross-sectional tabular data using convolutional neural networks: Prediction of corporate bankruptcy in Poland

2021 ◽  
Vol 8 (55) ◽  
pp. 352-377
Author(s):  
Aneta Dzik-Walczak ◽  
Maciej Odziemczyk
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Random Search ◽  
Learning Curves ◽  
Corporate Bankruptcy ◽  
Tabular Data ◽  
Cross Sectional ◽  
Data Transformations ◽  
Convolutional Networks ◽  
Observation Vector

Abstract The paper deals with the topic of modelling the probability of bankruptcy of Polish enterprises using convolutional neural networks. Convolutional networks take images as input, so it was thus necessary to apply the method of converting the observation vector to a matrix. Benchmarks for convolutional networks were logit models, random forests, XGBoost, and dense neural networks. Hyperparameters and model architecture were selected based on a random search and analysis of learning curves and experiments in folded, stratified cross-validation. In addition, the sensitivity of the results to data preprocessing was investigated. It was found that convolutional neural networks can be used to analyze cross-sectional tabular data, especially for the problem of modelling the probability of corporate bankruptcy. In order to achieve good results with models based on parameters updated by a gradient (neural networks and logit), it is necessary to use appropriate preprocessing techniques. Models based on decision trees have been shown to be insensitive to the data transformations used.

scholarly journals Alzheimer’s Detection through 3D Convolutional Neural Networks

2021 ◽  
Vol 34 (1) ◽  
Author(s):  
Ryan Hogan ◽  
Christoforos Christoforou
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Three Dimensional ◽  
Multilayer Perceptrons ◽  
Brain Scans ◽  
Convolutional Networks ◽  
3D Network ◽  
Efficient Resource ◽  
Mri Scans

To inform a proper diagnosis and understanding of Alzheimer’s Disease (AD), deep learning has emerged as an alternate approach for detecting physical brain changes within magnetic resonance imaging (MRI). The advancement of deep learning within biomedical imaging, particularly in MRI scans, has proven to be an efficient resource for abnormality detection while utilizing convolutional neural networks (CNN) to perform feature mapping within multilayer perceptrons. In this study, we aim to test the feasibility of using three-dimensional convolutional neural networks to identify neurophysiological degeneration in the entire-brain scans that differentiate between AD patients and controls. In particular, we propose and train a 3D-CNN model to classify between MRI scans of cognitively-healthy individuals and AD patients. We validate our proposed model on a large dataset composed of more than seven hundred MRI scans (half AD). Our results show a validation accuracy of 79% which is at par with the current state-of-the-art. The benefits of our proposed 3D network are that it can assist in the exploration and detection of AD by mapping the complex heterogeneity of the brain, particularly in the limbic system and temporal lobe. The goal of this research is to measure the efficacy and predictability of 3D convolutional networks in detecting the progression of neurodegeneration within MRI brain scans of HC and AD patients.

scholarly journals Graph Convolutional Networks for Text Classification

2019 ◽  
Vol 33 ◽  
pp. 7370-7377 ◽  
Author(s):  
Liang Yao ◽  
Chengsheng Mao ◽  
Yuan Luo
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Language Processing ◽  
Text Classification ◽  
State Of The Art ◽  
Classical Problem ◽  
Experimental Results ◽  
Training Data ◽  
Convolutional Networks ◽  
Single Text

Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with one-hot representation for word and document, it then jointly learns the embeddings for both words and documents, as supervised by the known class labels for documents. Our experimental results on multiple benchmark datasets demonstrate that a vanilla Text GCN without any external word embeddings or knowledge outperforms state-of-the-art methods for text classification. On the other hand, Text GCN also learns predictive word and document embeddings. In addition, experimental results show that the improvement of Text GCN over state-of-the-art comparison methods become more prominent as we lower the percentage of training data, suggesting the robustness of Text GCN to less training data in text classification.

scholarly journals A Biometric AsymmetricCryptosystem Software Module Based on Convolutional Neural Networks

2021 ◽  
Vol 13 (6) ◽  
pp. 1-12
Author(s):  
Ilyenko Anna ◽  
◽  
Ilyenko Sergii ◽  
Herasymenko Marharyta
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Associative Memory ◽  
Bidirectional Associative Memory ◽  
Software Module ◽  
Convolutional Networks ◽  
Cryptographic Key ◽  
Authentication System ◽  
And Training

During the research, the analysis of the existing biometric cryptographic systems was carried out. Some methods that help to generate biometric features were considered and compared with a cryptographic key. For comparing compact vectors of biometric images and cryptographic keys, the following methods are analyzed: designing and training of bidirectional associative memory; designing and training of single-layer and multilayer neural networks. As a result of comparative analysis of algorithms for extracting primary biometric features and comparing the generated image to a private key within the proposed authentication system, it was found that deep convolutional networks and neural network bidirectional associative memory are the most effective approach to process the data. In the research, an approach based on the integration of a biometric system and a cryptographic module was proposed, which allows using of a generated secret cryptographic key based on a biometric sample as the output of a neural network. The RSA algorithm is chosen to generate a private cryptographic key by use of convolutional neural networks and Python libraries. The software authentication module is implemented based on the client-server architecture using various internal Python libraries. Such authentication system should be used in systems where the user data and his valuable information resources are stored or where the user can perform certain valuable operations for which a cryptographic key is required. Proposed software module based on convolutional neural networks will be a perfect tool for ensuring the confidentiality of information and for all information-communication systems, because protecting information system from unauthorized access is one of the most pressing problems. This approach as software module solves the problem of secure generating and storing the secret key and author propose combination of the convolutional neural network with bidirectional associative memory, which is used to recognize the biometric sample, generate the image, and match it with a cryptographic key. The use of this software approach allows today to reduce the probability of errors of the first and second kind in authentication system and absolute number of errors was minimized by an average of 1,5 times. The proportion of correctly recognized images by the comparating together convolutional networks and neural network bidirectional associative memory in the authentication software module increased to 96,97%, which is on average from 1,08 times up to 1,01 times The authors further plan a number of scientific and technical solutions to develop and implement effective methods, tools to meet the requirements, principles and approaches to cybersecurity and cryptosystems for provide integrity and onfidentiality of information in experimental computer systems and networks.

scholarly journals Comparative Analysis of the Application of Multilayer and Convolutional Neural Networks for Recognition of Handwritten Letters of the Azerbaijani Alphabet

2021 ◽  
pp. 65-73
Author(s):  
Elshan Mustafayev ◽  
Rustam Azimov
Keyword(s):  
Neural Networks ◽  
Feature Extraction ◽  
Comparative Analysis ◽  
Convolutional Neural Networks ◽  
Information Technologies ◽  
Test Sample ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Feature Extraction Algorithm ◽  
Extraction Algorithm

Introduction. The implementation of information technologies in various spheres of public life dictates the creation of efficient and productive systems for entering information into computer systems. In such systems it is important to build an effective recognition module. At the moment, the most effective method for solving this problem is the use of artificial multilayer neural and convolutional networks. The purpose of the paper. This paper is devoted to a comparative analysis of the recognition results of handwritten characters of the Azerbaijani alphabet using neural and convolutional neural networks. Results. The analysis of the dependence of the recognition results on the following parameters is carried out: the architecture of neural networks, the size of the training base, the choice of the subsampling algorithm, the use of the feature extraction algorithm. To increase the training sample, the image augmentation technique was used. Based on the real base of 14000 characters, the bases of 28000, 42000 and 72000 characters were formed. The description of the feature extraction algorithm is given. Conclusions. Analysis of recognition results on the test sample showed: as expected, convolutional neural networks showed higher results than multilayer neural networks; the classical convolutional network LeNet-5 showed the highest results among all types of neural networks. However, the multi-layer 3-layer network, which was input by the feature extraction results; showed rather high results comparable with convolutional networks; there is no definite advantage in the choice of the method in the subsampling layer. The choice of the subsampling method (max-pooling or average-pooling) for a particular model can be selected experimentally; increasing the training database for this task did not give a tangible improvement in recognition results for convolutional networks and networks with preliminary feature extraction. However, for networks learning without feature extraction, an increase in the size of the database led to a noticeable improvement in performance. Keywords: neural networks, feature extraction, OCR.

Sign in / Sign up

Export Citation Format

Share Document