scholarly journals Structured Sparsity of Convolutional Neural Networks via Nonconvex Sparse Group Regularization

2021 ◽  
Vol 6 ◽  
Author(s):  
Kevin Bui ◽  
Fredrick Park ◽  
Shuai Zhang ◽  
Yingyong Qi ◽  
Jack Xin
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Group Lasso ◽  
Test Accuracy ◽  
Group Sparsity ◽  
Sparse Group Lasso ◽  
Nonconvex Regularization ◽  
And Performance ◽  
The Individual ◽  
Network Sparsification

Convolutional neural networks (CNN) have been hugely successful recently with superior accuracy and performance in various imaging applications, such as classification, object detection, and segmentation. However, a highly accurate CNN model requires millions of parameters to be trained and utilized. Even to increase its performance slightly would require significantly more parameters due to adding more layers and/or increasing the number of filters per layer. Apparently, many of these weight parameters turn out to be redundant and extraneous, so the original, dense model can be replaced by its compressed version attained by imposing inter- and intra-group sparsity onto the layer weights during training. In this paper, we propose a nonconvex family of sparse group lasso that blends nonconvex regularization (e.g., transformed ℓ1, ℓ1−ℓ2, and ℓ0) that induces sparsity onto the individual weights and ℓ2,1 regularization onto the output channels of a layer. We apply variable splitting onto the proposed regularization to develop an algorithm that consists of two steps per iteration: gradient descent and thresholding. Numerical experiments are demonstrated on various CNN architectures showcasing the effectiveness of the nonconvex family of sparse group lasso in network sparsification and test accuracy on par with the current state of the art.

scholarly journals Road Characteristics Detection Based on Joint Convolutional Neural Networks with Adaptive Squares

2021 ◽  
Vol 10 (6) ◽  
pp. 377
Author(s):  
Chiao-Ling Kuo ◽  
Ming-Hua Tsai
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Autonomous Vehicles ◽  
Detection Accuracy ◽  
Geospatial Information ◽  
Combination Rules ◽  
And Performance ◽  
Road Characteristics ◽  
Machine Readable ◽  
Background Image

The importance of road characteristics has been highlighted, as road characteristics are fundamental structures established to support many transportation-relevant services. However, there is still huge room for improvement in terms of types and performance of road characteristics detection. With the advantage of geographically tiled maps with high update rates, remarkable accessibility, and increasing availability, this paper proposes a novel simple deep-learning-based approach, namely joint convolutional neural networks (CNNs) adopting adaptive squares with combination rules to detect road characteristics from roadmap tiles. The proposed joint CNNs are responsible for the foreground and background image classification and various types of road characteristics classification from previous foreground images, raising detection accuracy. The adaptive squares with combination rules help efficiently focus road characteristics, augmenting the ability to detect them and provide optimal detection results. Five types of road characteristics—crossroads, T-junctions, Y-junctions, corners, and curves—are exploited, and experimental results demonstrate successful outcomes with outstanding performance in reality. The information of exploited road characteristics with location and type is, thus, converted from human-readable to machine-readable, the results will benefit many applications like feature point reminders, road condition reports, or alert detection for users, drivers, and even autonomous vehicles. We believe this approach will also enable a new path for object detection and geospatial information extraction from valuable map tiles.

scholarly journals Identifying Habitat Elements from Bird Images Using Deep Convolutional Neural Networks

Animals ◽  
2021 ◽  
Vol 11 (5) ◽  
pp. 1263
Author(s):  
Zhaojun Wang ◽  
Jiangning Wang ◽  
Congtian Lin ◽  
Yan Han ◽  
Zhaosheng Wang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Digital Technology ◽  
Rapid Development ◽  
Image Data ◽  
Image Database ◽  
Test Accuracy ◽  
Practical Application ◽  
Accuracy Rate ◽  
Deep Convolutional Neural Networks

With the rapid development of digital technology, bird images have become an important part of ornithology research data. However, due to the rapid growth of bird image data, it has become a major challenge to effectively process such a large amount of data. In recent years, deep convolutional neural networks (DCNNs) have shown great potential and effectiveness in a variety of tasks regarding the automatic processing of bird images. However, no research has been conducted on the recognition of habitat elements in bird images, which is of great help when extracting habitat information from bird images. Here, we demonstrate the recognition of habitat elements using four DCNN models trained end-to-end directly based on images. To carry out this research, an image database called Habitat Elements of Bird Images (HEOBs-10) and composed of 10 categories of habitat elements was built, making future benchmarks and evaluations possible. Experiments showed that good results can be obtained by all the tested models. ResNet-152-based models yielded the best test accuracy rate (95.52%); the AlexNet-based model yielded the lowest test accuracy rate (89.48%). We conclude that DCNNs could be efficient and useful for automatically identifying habitat elements from bird images, and we believe that the practical application of this technology will be helpful for studying the relationships between birds and habitat elements.

scholarly journals Simultaneous Channel and Feature Selection of Fused EEG Features Based on Sparse Group Lasso

2015 ◽  
Vol 2015 ◽  
pp. 1-13 ◽  
Author(s):  
Jin-Jia Wang ◽  
Fang Xue ◽  
Hui Li
Keyword(s):  
Feature Selection ◽  
Feature Selection Method ◽  
Group Lasso ◽  
High Dimensional ◽  
Test Accuracy ◽  
Gradient Descent Method ◽  
Feature Subset ◽  
Eeg Signals ◽  
Sparse Group Lasso ◽  
Selection Of

Feature extraction and classification of EEG signals are core parts of brain computer interfaces (BCIs). Due to the high dimension of the EEG feature vector, an effective feature selection algorithm has become an integral part of research studies. In this paper, we present a new method based on a wrapped Sparse Group Lasso for channel and feature selection of fused EEG signals. The high-dimensional fused features are firstly obtained, which include the power spectrum, time-domain statistics, AR model, and the wavelet coefficient features extracted from the preprocessed EEG signals. The wrapped channel and feature selection method is then applied, which uses the logistical regression model with Sparse Group Lasso penalized function. The model is fitted on the training data, and parameter estimation is obtained by modified blockwise coordinate descent and coordinate gradient descent method. The best parameters and feature subset are selected by using a 10-fold cross-validation. Finally, the test data is classified using the trained model. Compared with existing channel and feature selection methods, results show that the proposed method is more suitable, more stable, and faster for high-dimensional feature fusion. It can simultaneously achieve channel and feature selection with a lower error rate. The test accuracy on the data used from international BCI Competition IV reached 84.72%.

scholarly journals Classification and Analysis of Minimally-Processed Data from a Large Magnetoencephalography Dataset using Convolutional Neural Networks

2019 ◽  
Author(s):  
Jon Garry ◽  
Thomas Trappenberg ◽  
Steven Beyea ◽  
Timothy Bardouille
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Optimal Performance ◽  
Sensor Data ◽  
Data Networks ◽  
Test Accuracy ◽  
Data Visualisation ◽  
Minimally Processed ◽  
Important Data

AbstractConvolutional neural networks were used to classify and analyse a large magnetoencephalography (MEG) dataset. Networks were trained to classify between active and baseline intervals of minimally-processed data recorded during cued button pressing. There were two primary objectives for this study: (1) develop networks that can effectively classify MEG data, and (2) identify the important data features that inform classification. Networks with a simple architecture were trained using sensor and source-localised data. Networks trained with sensor data were also trained using varying amounts of data. The important features within the data were identified via saliency and occlusion mapping. An ensemble of networks trained using sensor data performed best (average test accuracy 0.974 ± 0.001). A dataset containing on the order of hundreds of participants was required for optimal performance of this network with these data. Visualisation maps highlighted features known to occur during neuromagnetic recordings of cued button pressing.

scholarly journals Transfer learning with pre-trained deep convolutional neural networks for the automatic assessment of liver steatosis in ultrasound images

2020 ◽  
Author(s):  
Elena Codruta Constantinescu ◽  
Anca-Loredana Udriștoiu ◽  
Ștefan Cristinel Udriștoiu ◽  
Andreea Valentina Iacob ◽  
Lucian Gheorghe Gruionu ◽  
...  
Keyword(s):  
Neural Networks ◽  
Fatty Liver ◽  
Receiver Operating Characteristic ◽  
Convolutional Neural Networks ◽  
Operating Characteristic ◽  
Liver Steatosis ◽  
Normal Liver ◽  
Test Accuracy ◽  
Receiver Operating Characteristic Curves ◽  
Proposed Model

Aim: In this paper we proposed different architectures of convolutional neural network (CNN) to classify fatty liver disease in images using only pixels and diagnosis labels as input. We trained and validated our models using a dataset of 629 images consisting of 2 types of liver images, normal and liver steatosis. Material and methods: We assessed two pre-trained models of convolutional neural networks, Inception-v3 and VGG-16 using fine-tuning. Both models were pre-trained on ImageNet dataset to extract features from B-mode ultrasound liver images. The results obtained through these methods were compared for selecting the predictive model with the best performance metrics. We trained the two models using a dataset of 262 images of liver steatosis and 234 images of normal liver. We assessed the models using a dataset of 70 liver steatosis im-ages and 63 normal liver images. Results. The proposed model that used Inception v3 obtained a 93.23% test accuracy with a sensitivity of 89.9%% and a precision of 96.6%, and areas under each receiver operating characteristic curves (ROC AUC) of 0.93. The other proposed model that used VGG-16, obtained a 90.77% test accuracy with a sensitivity of 88.9% and a precision of 92.85%, and areas under each receiver operating characteristic curves (ROC AUC) of 0.91. Conclusion. The deep learning algorithms that we proposed to detect steatosis and classify the images in normal and fatty liver images, yields an excellent test performance of over 90%. However, future larger studies are required in order to establish how these algorithms can be implemented in a clinical setting.

scholarly journals How Many Features is an Image Worth? Multi-Channel CNN for Steering Angle Prediction in Autonomous Vehicles

2021 ◽  
Author(s):  
Jason Munger ◽  
Carlos W. Morato
Keyword(s):  
Neural Networks ◽  
Autonomous Vehicles ◽  
Deep Neural Networks ◽  
Spatial Information ◽  
Image Data ◽  
Input Image ◽  
Steering Angle ◽  
Rgb Images ◽  
And Performance ◽  
The Individual

This project explores how raw image data obtained from AV cameras can provide a model with more spatial information than can be learned from simple RGB images alone. This paper leverages the advances of deep neural networks to demonstrate steering angle predictions of autonomous vehicles through an end-to-end multi-channel CNN model using only the image data provided from an onboard camera. Image data is processed through existing neural networks to provide pixel segmentation and depth estimates and input to a new neural network along with the raw input image to provide enhanced feature signals from the environment. Various input combinations of Multi-Channel CNNs are evaluated, and their effectiveness is compared to single CNN networks using the individual data inputs. The model with the most accurate steering predictions is identified and performance compared to previous neural networks.

scholarly journals Ensemble of Deep Convolutional Neural Networks for Automatic Pavement Crack Detection and Measurement

Coatings ◽  
2020 ◽  
Vol 10 (2) ◽  
pp. 152 ◽  
Author(s):  
Zhun Fan ◽  
Chong Li ◽  
Ying Chen ◽  
Paola Di Mascio ◽  
Xiaopeng Chen ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Crack Detection ◽  
The Other ◽  
Detection Methods ◽  
Deep Convolutional Neural Networks ◽  
Predicted Probability ◽  
Low Efficiency ◽  
The Individual ◽  
Pavement Crack Detection

Automated pavement crack detection and measurement are important road issues. Agencies have to guarantee the improvement of road safety. Conventional crack detection and measurement algorithms can be extremely time-consuming and low efficiency. Therefore, recently, innovative algorithms have received increased attention from researchers. In this paper, we propose an ensemble of convolutional neural networks (without a pooling layer) based on probability fusion for automated pavement crack detection and measurement. Specifically, an ensemble of convolutional neural networks was employed to identify the structure of small cracks with raw images. Secondly, outputs of the individual convolutional neural network model for the ensemble were averaged to produce the final crack probability value of each pixel, which can obtain a predicted probability map. Finally, the predicted morphological features of the cracks were measured by using the skeleton extraction algorithm. To validate the proposed method, some experiments were performed on two public crack databases (CFD and AigleRN) and the results of the different state-of-the-art methods were compared. To evaluate the efficiency of crack detection methods, three parameters were considered: precision (Pr), recall (Re) and F1 score (F1). For the two public databases of pavement images, the proposed method obtained the highest values of the three evaluation parameters: for the CFD database, Pr = 0.9552, Re = 0.9521 and F1 = 0.9533 (which reach values up to 0.5175 higher than the values obtained on the same database with the other methods), for the AigleRN database, Pr = 0.9302, Re = 0.9166 and F1 = 0.9238 (which reach values up to 0.7313 higher than the values obtained on the same database with the other methods). The experimental results show that the proposed method outperforms the other methods. For crack measurement, the crack length and width can be measure based on different crack types (complex, common, thin, and intersecting cracks.). The results show that the proposed algorithm can be effectively applied for crack measurement.

scholarly journals Apple quality identification and classification by image processing based on convolutional neural networks

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Yanfei Li ◽  
Xianying Feng ◽  
Yandong Liu ◽  
Xingchang Han
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Support Vector ◽  
Svm Classifier ◽  
Test Accuracy ◽  
Training Time ◽  
Proposed Model ◽  
Specific Complex ◽  
Occurrence Matrix ◽  
Apple Quality

AbstractThis work researched apple quality identification and classification from real images containing complicated disturbance information (background was similar to the surface of the apples). This paper proposed a novel model based on convolutional neural networks (CNN) which aimed at accurate and fast grading of apple quality. Specific, complex, and useful image characteristics for detection and classification were captured by the proposed model. Compared with existing methods, the proposed model could better learn high-order features of two adjacent layers that were not in the same channel but were very related. The proposed model was trained and validated, with best training and validation accuracy of 99% and 98.98% at 2590th and 3000th step, respectively. The overall accuracy of the proposed model tested using an independent 300 apple dataset was 95.33%. The results showed that the training accuracy, overall test accuracy and training time of the proposed model were better than Google Inception v3 model and traditional imaging process method based on histogram of oriented gradient (HOG), gray level co-occurrence matrix (GLCM) features merging and support vector machine (SVM) classifier. The proposed model has great potential in Apple’s quality detection and classification.

Diagnosis of Brain Tumor Using Nano Segmentation and Advanced-Convolutional Neural Networks Classification

2021 ◽  
Vol 11 (12) ◽  
pp. 2907-2917
Author(s):  
P. V. Deepa ◽  
S. Joseph Jawhar ◽  
J. Merry Geisa
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Brain Tumours ◽  
Malignant Tumour ◽  
Training Data ◽  
Correct Identification ◽  
Imaging Model ◽  
And Performance ◽  
Percent Accuracy ◽  
High Level

The field of nanotechnology has lately acquired prominence according to the raised level of correct identification and performance in the patients using Computer-Aided Diagnosis (CAD). Nano-scale imaging model enables for a high level of precision and accuracy in determining if a brain tumour is malignant or benign. This contributes to people with brain tumours having a better standard of living. In this study, We present a revolutionary Semantic nano-segmentation methodology for the nanoscale classification of brain tumours. The suggested Advanced-Convolutional Neural Networks-based Semantic Nano-segmentation will aid radiologists in detecting brain tumours even when lesions are minor. ResNet-50 was employed in the suggested Advanced-Convolutional Neural Networks (A-CNN) approach. The tumour image is partitioned using Semantic Nano-segmentation, that has averaged dice and SSIM values of 0.9704 and 0.2133, correspondingly. The input is a nano-image, and the tumour image is segmented using Semantic Nano-segmentation, which has averaged dice and SSIM values of 0.9704 and 0.2133, respectively. The suggested Semantic nano segments achieves 93.2 percent and 92.7 percent accuracy for benign and malignant tumour pictures, correspondingly. For malignant or benign pictures, The accuracy of the A-CNN methodology of correct segmentation is 99.57 percent and 95.7 percent, respectively. This unique nano-method is designed to detect tumour areas in nanometers (nm) and hence accurately assess the illness. The suggested technique’s closeness to with regard to True Positive values, the ROC curve implies that it outperforms earlier approaches. A comparison analysis is conducted on ResNet-50 using testing and training data at rates of 90%–10%, 80%–20%, and 70%–30%, corresponding, indicating the utility of the suggested work.

Sign in / Sign up

Export Citation Format

Share Document