Conquering the CNN Over-Parameterization Dilemma: A Volterra Filtering Approach for Action Recognition

The importance of inference in Machine Learning (ML) has led to an explosive number of different proposals in ML, and particularly in Deep Learning. In an attempt to reduce the complexity of Convolutional Neural Networks, we propose a Volterra filter-inspired Network architecture. This architecture introduces controlled non-linearities in the form of interactions between the delayed input samples of data. We propose a cascaded implementation of Volterra Filtering so as to significantly reduce the number of parameters required to carry out the same classification task as that of a conventional Neural Network. We demonstrate an efficient parallel implementation of this Volterra Neural Network (VNN), along with its remarkable performance while retaining a relatively simpler and potentially more tractable structure. Furthermore, we show a rather sophisticated adaptation of this network to nonlinearly fuse the RGB (spatial) information and the Optical Flow (temporal) information of a video sequence for action recognition. The proposed approach is evaluated on UCF-101 and HMDB-51 datasets for action recognition, and is shown to outperform state of the art CNN approaches.

Download Full-text

ThriftyNets: Convolutional Neural Networks with Tiny Parameter Budget

IoT ◽

10.3390/iot2020012 ◽

2021 ◽

Vol 2 (2) ◽

pp. 222-235

Author(s):

Guillaume Coiffier ◽

Ghouthi Boukli Hacene ◽

Vincent Gripon

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Convolutional Neural Network ◽

Spatial Resolution ◽

Network Architecture ◽

Deep Neural Networks ◽

State Of The Art ◽

Feature Maps ◽

Neural Network Architecture

Deep Neural Networks are state-of-the-art in a large number of challenges in machine learning. However, to reach the best performance they require a huge pool of parameters. Indeed, typical deep convolutional architectures present an increasing number of feature maps as we go deeper in the network, whereas spatial resolution of inputs is decreased through downsampling operations. This means that most of the parameters lay in the final layers, while a large portion of the computations are performed by a small fraction of the total parameters in the first layers. In an effort to use every parameter of a network at its maximum, we propose a new convolutional neural network architecture, called ThriftyNet. In ThriftyNet, only one convolutional layer is defined and used recursively, leading to a maximal parameter factorization. In complement, normalization, non-linearities, downsamplings and shortcut ensure sufficient expressivity of the model. ThriftyNet achieves competitive performance on a tiny parameters budget, exceeding 91% accuracy on CIFAR-10 with less than 40 k parameters in total, 74.3% on CIFAR-100 with less than 600 k parameters, and 67.1% On ImageNet ILSVRC 2012 with no more than 4.15 M parameters. However, the proposed method typically requires more computations than existing counterparts.

Download Full-text

SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks

ACM Transactions on Graphics ◽

10.1145/3450284 ◽

2021 ◽

Vol 40 (3) ◽

pp. 1-13

Author(s):

Lumin Yang ◽

Jiajie Zhuang ◽

Hongbo Fu ◽

Xiangzhi Wei ◽

Kun Zhou ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Network Architecture ◽

Large Scale ◽

State Of The Art ◽

Semantic Segmentation ◽

Structure Information ◽

Graph Neural Networks ◽

Node Labels ◽

Point Level

We introduce SketchGNN , a convolutional graph neural network for semantic segmentation and labeling of freehand vector sketches. We treat an input stroke-based sketch as a graph with nodes representing the sampled points along input strokes and edges encoding the stroke structure information. To predict the per-node labels, our SketchGNN uses graph convolution and a static-dynamic branching network architecture to extract the features at three levels, i.e., point-level, stroke-level, and sketch-level. SketchGNN significantly improves the accuracy of the state-of-the-art methods for semantic sketch segmentation (by 11.2% in the pixel-based metric and 18.2% in the component-based metric over a large-scale challenging SPG dataset) and has magnitudes fewer parameters than both image-based and sequence-based methods.

Download Full-text

Concatenated Convolutional Neural Network Model for Cuffless Blood Pressure Estimation Using Fuzzy Recurrence Properties of PPG Signals

10.21203/rs.3.rs-1231567/v1 ◽

2022 ◽

Author(s):

Ali Bahari Malayeri ◽

Mohammad Bagher Khodabakhshi

Keyword(s):

Neural Network ◽

Blood Pressure ◽

Neural Networks ◽

Network Architecture ◽

Spatial Information ◽

Recurrence Plot ◽

Estimation Model ◽

British Hypertension Society ◽

Blood Pressure Estimation ◽

Cuffless Blood Pressure

Abstract Due to the importance of continuous monitoring of blood pressure (BP) in controlling hypertension, the topic of cuffless blood pressure (BP) estimation has been widely studied in recent years. A most important approach is to explore the nonlinear mapping between the recorded peripheral signals and the BP values which is usually conducted by deep neural networks. Because of the sequence-based pseudo periodic nature of peripheral signals such as photoplethysmogram (PPG), a proper estimation model needed to be equipped with the 1-dimensional (1-D) and recurrent layers. This, in turn, limits the usage of 2-dimensional (2-D) layers adopted in convolutional neural networks (CNN) for embedding spatial information in the model. In this study, considering the advantage of chaotic approaches, the recurrence characterization of peripheral signals was taken into account by a visual 2-D representation of PPG in phase space through fuzzy recurrence plot (FRP). FRP not only provides a beneficial framework for capturing the spatial properties of input signals but also creates a reliable approach for embedding the pseudo periodic properties to the neural models without using recurrent layers. Moreover, this study proposes a novel deep neural network architecture that combines the morphological features extracted simultaneously from two upgraded 1-D and 2-D CNNs capturing the temporal and spatial dependencies of PPGs in systolic and diastolic BP estimation. The model has been fed with the 1-D PPG sequences and the corresponding 2-D FRPs from two separate routes. The performance of the proposed framework was examined on the well-known public dataset, namely, Multi-Parameter Intelligent in Intensive Care II. Our scheme is analyzed and compared with the literature in terms of the requirements of the standards set by the British Hypertension Society (BHS) and the Association for the Advancement of Medical Instrumentation (AAMI). The proposed model met the AAMI requirements, and it achieved a grade of A as stated by the BHS standard. In addition, its mean absolute errors (MAE) and standard deviation for both systolic and diastolic blood pressure estimations were considerably low, 3.05±5.26 mmHg and 1.58±2.6 mmHg, in turn.

Download Full-text

Quantification of the suitable rooftop area for solar panel installation from overhead imagery using Convolutional Neural Networks

Journal of Physics Conference Series ◽

10.1088/1742-6596/2042/1/012002 ◽

2021 ◽

Vol 2042 (1) ◽

pp. 012002

Author(s):

Roberto Castello ◽

Alina Walch ◽

Raphaël Attias ◽

Riccardo Cadei ◽

Shasha Jiang ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Computer Vision ◽

State Of The Art ◽

Solar Panel ◽

Post Processing ◽

Processing Step ◽

Recent Method

Abstract The integration of solar technology in the built environment is realized mainly through rooftop-installed panels. In this paper, we leverage state-of-the-art Machine Learning and computer vision techniques applied on overhead images to provide a geo-localization of the available rooftop surfaces for solar panel installation. We further exploit a 3D building database to associate them to the corresponding roof geometries by means of a geospatial post-processing approach. The stand-alone Convolutional Neural Network used to segment suitable rooftop areas reaches an intersection over union of 64% and an accuracy of 93%, while a post-processing step using building database improves the rejection of false positives. The model is applied to a case study area in the canton of Geneva and the results are compared with another recent method used in the literature to derive the realistic available area.

Download Full-text

CapsTM: capsule network for Chinese medical text matching

BMC Medical Informatics and Decision Making ◽

10.1186/s12911-021-01442-9 ◽

2021 ◽

Vol 21 (S2) ◽

Author(s):

Xiaoming Yu ◽

Yedan Shen ◽

Yuan Ni ◽

Xiaowei Huang ◽

Xiaolong Wang ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Network Architecture ◽

State Of The Art ◽

Interaction Matrix ◽

Application Systems ◽

Input Layer ◽

Art Methods ◽

Text Matching

Abstract Background Text Matching (TM) is a fundamental task of natural language processing widely used in many application systems such as information retrieval, automatic question answering, machine translation, dialogue system, reading comprehension, etc. In recent years, a large number of deep learning neural networks have been applied to TM, and have refreshed benchmarks of TM repeatedly. Among the deep learning neural networks, convolutional neural network (CNN) is one of the most popular networks, which suffers from difficulties in dealing with small samples and keeping relative structures of features. In this paper, we propose a novel deep learning architecture based on capsule network for TM, called CapsTM, where capsule network is a new type of neural network architecture proposed to address some of the short comings of CNN and shows great potential in many tasks. Methods CapsTM is a five-layer neural network, including an input layer, a representation layer, an aggregation layer, a capsule layer and a prediction layer. In CapsTM, two pieces of text are first individually converted into sequences of embeddings and are further transformed by a highway network in the input layer. Then, Bidirectional Long Short-Term Memory (BiLSTM) is used to represent each piece of text and attention-based interaction matrix is used to represent interactive information of the two pieces of text in the representation layer. Subsequently, the two kinds of representations are fused together by BiLSTM in the aggregation layer, and are further represented with capsules (vectors) in the capsule layer. Finally, the prediction layer is a connected network used for classification. CapsTM is an extension of ESIM by adding a capsule layer before the prediction layer. Results We construct a corpus of Chinese medical question matching, which contains 36,360 question pairs. This corpus is randomly split into three parts: a training set of 32,360 question pairs, a development set of 2000 question pairs and a test set of 2000 question pairs. On this corpus, we conduct a series of experiments to evaluate the proposed CapsTM and compare it with other state-of-the-art methods. CapsTM achieves the highest F-score of 0.8666. Conclusion The experimental results demonstrate that CapsTM is effective for Chinese medical question matching and outperforms other state-of-the-art methods for comparison.

Download Full-text

ColNet: Embedding the Semantics of Web Tables for Column Type Prediction

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v33i01.330129 ◽

2019 ◽

Vol 33 ◽

pp. 29-36 ◽

Cited By ~ 9

Author(s):

Jiaoyan Chen ◽

Ernesto Jiménez-Ruiz ◽

Ian Horrocks ◽

Charles Sutton

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

State Of The Art ◽

Column Type ◽

Basic Understanding ◽

Word Representation ◽

Meta Information ◽

A Cell ◽

Multiple Cells

Automatically annotating column types with knowledge base (KB) concepts is a critical task to gain a basic understanding of web tables. Current methods rely on either table metadata like column name or entity correspondences of cells in the KB, and may fail to deal with growing web tables with incomplete meta information. In this paper we propose a neural network based column type annotation framework named ColNet which is able to integrate KB reasoning and lookup with machine learning and can automatically train Convolutional Neural Networks for prediction. The prediction model not only considers the contextual semantics within a cell using word representation, but also embeds the semantics of a column by learning locality features from multiple cells. The method is evaluated with DBPedia and two different web table datasets, T2Dv2 from the general Web and Limaye from Wikipedia pages, and achieves higher performance than the state-of-the-art approaches.

Download Full-text

Cross-situational word learning with multimodal neural networks

10.31234/osf.io/udbh2 ◽

2021 ◽

Author(s):

Wai Keen Vong ◽

Brenden M. Lake

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Word Learning ◽

Network Architecture ◽

Question Answering ◽

Cognitive Models ◽

Learning Approaches ◽

Neural Network Learning ◽

Single Epoch

In order to learn the mappings from words to referents, children must integrate co-occurrence information across individually ambiguous pairs of scenes and utterances, a challenge known as cross-situational word learning. In machine learning, recent multimodal neural networks have been shown to learn meaningful visual-linguistic mappings from cross-situational data, as needed to solve problems such as image captioning and visual question answering. These networks are potentially appealing as cognitive models because they can learn from raw visual and linguistic stimuli, something previous cognitive models have not addressed. In this paper, we examine whether recent machine learning approaches can help explain various behavioral phenomena from the psychological literature on cross-situational word learning. We consider two variants of a multimodal neural network architecture, and look at seven different phenomena associated with cross-situational word learning, and word learning more generally. Our results show that these networks can learn word-referent mappings from a single epoch of training, matching the amount of training found in cross-situational word learning experiments. Additionally, these networks capture some, but not all of the phenomena we studied, with all of the failures related to reasoning via mutual exclusivity. These results provide insight into the kinds of phenomena that arise naturally from relatively generic neural network learning algorithms, and which word learning phenomena require additional inductive biases.

Download Full-text

Fast Pre-Diagnosis of Neoplastic Changes in Cytology Images Using Machine Learning

Applied Sciences ◽

10.3390/app11167181 ◽

2021 ◽

Vol 11 (16) ◽

pp. 7181

Author(s):

Jakub Caputa ◽

Daria Łukasik ◽

Maciej Wielgosz ◽

Michał Karwatowski ◽

Rafał Frączek ◽

...

Keyword(s):

Neural Network ◽

Machine Learning ◽

Network Architecture ◽

State Of The Art ◽

Prototype Model ◽

Model Parameters ◽

Round Cell ◽

Neural Network Architecture ◽

Original Dataset ◽

Automated Tools

We present the experiment results to use the YOLOv3 neural network architecture to automatically detect tumor cells in cytological samples taken from the skin in canines. A rich dataset of 1219 smeared sample images with 28,149 objects was gathered and annotated by the vet doctor to perform the experiments. It covers three types of common round cell neoplasms: mastocytoma, histiocytoma, and lymphoma. The dataset has been thoroughly described in the paper and is publicly available. The YOLOv3 neural network architecture was trained using various schemes involving original dataset modification and the different model parameters. The experiments showed that the prototype model achieved 0.7416 mAP, which outperforms the state-of-the-art machine learning and human estimated results. We also provided a series of analyses that may facilitate ML-based solutions by casting more light on some aspects of its performance. We also presented the main discrepancies between ML-based and human-based diagnoses. This outline may help depict the scenarios and how the automated tools may support the diagnosis process.

Download Full-text

Synthesis of Neural Network Architecture for Recognition of Sea-Going Ship Images

Proceedings of Southwest State University ◽

10.21869/2223-1560-2020-24-1-130-143 ◽

2020 ◽

Vol 24 (1) ◽

pp. 130-143

Author(s):

D. I. Konarev ◽

A. A. Gulamov

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Image Recognition ◽

Network Architecture ◽

Test Sample ◽

Feature Maps ◽

Noticeable Effect ◽

Current Task ◽

Correct Pattern

Purpose of research. The current task is to monitor ships using video surveillance cameras installed along the canal. It is important for information communication support for navigation of the Moscow Canal. The main subtask is direct recognition of ships in an image or video. Implementation of a neural network is perspectively.Methods. Various neural network are described. images of ships are an input data for the network. The learning sample uses CIFAR-10 dataset. The network is built and trained by using Keras and TensorFlow machine learning libraries.Results. Implementation of curving artificial neural networks for problems of image recognition is described. Advantages of such architecture when working with images are also described. The selection of Python language for neural network implementation is justified. The main used libraries of machine learning, such as TensorFlow and Keras are described. An experiment has been conducted to train swirl neural networks with different architectures based on Google collaboratoty service. The effectiveness of different architectures was evaluated as a percentage of correct pattern recognition in the test sample. Conclusions have been drawn about parameters influence of screwing neural network on showing its effectiveness.Conclusion. The network with a single curl layer in each cascade showed insufficient results, so three-stage curls with two and three curl layers in each cascade were used. Feature map extension has the greatest impact on the accuracy of image recognition. The increase in cascades' number has less noticeable effect and the increase in the number of screwdriver layers in each cascade does not always have an increase in the accuracy of the neural network. During the study, a three-frame network with two buckling layers in each cascade and 128 feature maps is defined as an optimal architecture of neural network under described conditions. operability checking of architecture's part under consideration on random images of ships confirmed the correctness of optimal architecture choosing.

Download Full-text

Privacy-Preserving Federated Neural Network Learning for Disease-Associated Cell Classification

10.1101/2022.01.10.475610 ◽

2022 ◽

Author(s):

Sinem Sav ◽

Jean-Philippe Bossuat ◽

Juan R. Troncoso-Pastoriza ◽

Manfred Claassen ◽

Jean-Pierre Hubaux

Keyword(s):

Neural Network ◽

Machine Learning ◽

Neural Networks ◽

Network Architecture ◽

Homomorphic Encryption ◽

Privacy Preserving ◽

Model Parameters ◽

Patient Privacy ◽

Learning Models ◽

Machine Learning Models

Training accurate and robust machine learning models requires a large amount of data that is usually scattered across data-silos. Sharing or centralizing the data of different healthcare institutions is, however, unfeasible or prohibitively difficult due to privacy regulations. In this work, we address this problem by using a novel privacy-preserving federated learning-based approach, PriCell, for complex machine learning models such as convolutional neural networks. PriCell relies on multiparty homomorphic encryption and enables the collaborative training of encrypted neural networks with multiple healthcare institutions. We preserve the confidentiality of each institutions' input data, of any intermediate values, and of the trained model parameters. We efficiently replicate the training of a published state-of-the-art convolutional neural network architecture in a decentralized and privacy-preserving manner. Our solution achieves an accuracy comparable to the one obtained with the centralized solution, with an improvement of at least one-order-of-magnitude in execution time with respect to prior secure solutions. Our work guarantees patient privacy and ensures data utility for efficient multi-center studies involving complex healthcare data.

Download Full-text