scholarly journals Patent Automatic Classification Based on Symmetric Hierarchical Convolution Neural Network

Symmetry ◽  
2020 ◽  
Vol 12 (2) ◽  
pp. 186 ◽  
Author(s):  
Huiming Zhu ◽  
Chunhui He ◽  
Yang Fang ◽  
Bin Ge ◽  
Meng Xing ◽  
...  
Keyword(s):  
Neural Network ◽  
Input Data ◽  
Large Scale ◽  
Patent Application ◽  
Input Sequence ◽  
Automatic Classification ◽  
Convolution Neural Network ◽  
Coarse Grained ◽  
Short Text ◽  
Patent Classification

With the rapid growth of patent applications, it has become an urgent problem to automatically classify the accepted patent application documents accurately and quickly. Most previous patent automatic classification studies are based on feature engineering and traditional machine learning methods like SVM, and some even rely on the knowledge of domain experts, hence they suffer from low accuracy problem and have poor generalization ability. In this paper, we propose a patent automatic classification method via the symmetric hierarchical convolution neural network (CNN) named PAC-HCNN. We use the title and abstract of the patent as the input data, and then apply the word embedding technique to segment and vectorize the input data. Then we design a symmetric hierarchical CNN framework to classify the patents based on the word embeddings, which is much more efficient than traditional RNN models dealing with texts, meanwhile keeping the history and future information of the input sequence. We also add gated linear units (GLUs) and residual connection to help realize the deep CNN. Additionally, we equip our model with a self attention mechanism to address the long-term dependency problem. Experiments are performed on large-scale datasets for Chinese short text patent classification. Experimental results prove our proposed model’s effectiveness, and it performs better than other state-of-the-art models significantly and consistently on both fine-grained and coarse-grained classification.

scholarly journals Applicability Evaluation of the Hydrological Image and Convolution Neural Network for Prediction of the Biochemical Oxygen Demand and Total Phosphorus Loads in Agricultural Areas

Agriculture ◽  
2020 ◽  
Vol 10 (11) ◽  
pp. 529
Author(s):  
Chul Min Song ◽  
Jin Soo Kim
Keyword(s):  
Neural Network ◽  
Total Phosphorus ◽  
Biochemical Oxygen Demand ◽  
Input Data ◽  
Pearson Correlation ◽  
Oxygen Demand ◽  
Convolution Neural Network ◽  
Stable Model ◽  
Model Learning ◽  
Load Model

This study employed a convolution neural network (CNN) model, hitherto used only for solving classification problems, with two-dimensional input data to predict the pollution loads and evaluate the CNN model’s applicability. A CNN model generally requires two-dimension input data, such as photographs in previous studies. However, this study’s CNN model necessitates the numerical images that reflect hydrological phenomena due to the nature of the study. A hydrological image was used as the input data for the CNN model in this study to address this issue. The last layer of the CNN model was also transformed into a linear function to derive the continuous variable. As a result, the Pearson correlation coefficient, which represents the relationship between the measured and predicted values, demonstrated a Biochemical Oxygen Demand (BOD) load model of 0.94 and a Total Phosphorus (TP) load model of 0.87. Nash–Sutcliffe efficiency was used to evaluate the model performance; the BOD load model was 0.83, while the TP load model was 0.79, respectively, indicating good performance. These results demonstrate that the hydrological images led to stable model learning and generalization, and the proposed CNN model is suitable for predicting the pollution load, with potential future applications in various fields.

scholarly journals Scalable Multi-grained Cross-modal Similarity Query with Interpretability

2021 ◽  
Author(s):  
Mingdong Zhu ◽  
Derong Shen ◽  
Lixin Xu ◽  
Xianfang Wang
Keyword(s):  
Neural Network ◽  
Query Processing ◽  
Large Scale ◽  
Deep Neural Network ◽  
Processing System ◽  
Coarse Grained ◽  
Index Structure ◽  
Neural Network Models ◽  
Similarity Query ◽  
Query Algorithm

AbstractCross-modal similarity query has become a highlighted research topic for managing multimodal datasets such as images and texts. Existing researches generally focus on query accuracy by designing complex deep neural network models and hardly consider query efficiency and interpretability simultaneously, which are vital properties of cross-modal semantic query processing system on large-scale datasets. In this work, we investigate multi-grained common semantic embedding representations of images and texts and integrate interpretable query index into the deep neural network by developing a novel Multi-grained Cross-modal Query with Interpretability (MCQI) framework. The main contributions are as follows: (1) By integrating coarse-grained and fine-grained semantic learning models, a multi-grained cross-modal query processing architecture is proposed to ensure the adaptability and generality of query processing. (2) In order to capture the latent semantic relation between images and texts, the framework combines LSTM and attention mode, which enhances query accuracy for the cross-modal query and constructs the foundation for interpretable query processing. (3) Index structure and corresponding nearest neighbor query algorithm are proposed to boost the efficiency of interpretable queries. (4) A distributed query algorithm is proposed to improve the scalability of our framework. Comparing with state-of-the-art methods on widely used cross-modal datasets, the experimental results show the effectiveness of our MCQI approach.

scholarly journals Using Convolution Neural Network for Defective Image Classification of Industrial Components

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Hao Wu ◽  
Zhi Zhou
Keyword(s):  
Neural Network ◽  
Experimental Study ◽  
Deep Learning ◽  
Large Scale ◽  
Intelligent System ◽  
Convolution Neural Network ◽  
Lens Distortion ◽  
Proposed Model ◽  
Industrial Cameras

Computer vision provides effective solutions in many imaging relation problems, including automatic image segmentation and classification. Artificially trained models can be employed to tag images and identify objects spontaneously. In large-scale manufacturing, industrial cameras are utilized to take constant images of components for several reasons. Due to the limitations caused by motion, lens distortion, and noise, some defective images are captured, which are to be identified and separated. One common way to address this problem is by looking into these images manually. However, this solution is not only very time-consuming but is also inaccurate. The paper proposes a deep learning-based artificially intelligent system that can quickly train and identify faulty images. For this purpose, a pretrained convolution neural network based on the PyTorch framework is employed to extract discriminating features from the dataset, which is then used for the classification task. In order to eliminate the chances of overfitting, the proposed model also employed Dropout technology to adjust the network. The experimental study reveals that the system can precisely classify the normal and defective images with an accuracy of over 91%.

scholarly journals PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices

2020 ◽  
Vol 34 (04) ◽  
pp. 5117-5124 ◽  
Author(s):  
Xiaolong Ma ◽  
Fu-Ming Guo ◽  
Wei Niu ◽  
Xue Lin ◽  
Jian Tang ◽  
...  
Keyword(s):  
Neural Network ◽  
Real Time ◽  
Mobile Devices ◽  
Large Scale ◽  
Coarse Grained ◽  
Convolution Kernel ◽  
Fine Grained ◽  
Model Compression ◽  
Weight Pruning ◽  
Balanced Workload

Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight pruning is a straightforward and effective method. There are currently two mainstreams of pruning methods representing two extremes of pruning regularity: non-structured, fine-grained pruning can achieve high sparsity and accuracy, but is not hardware friendly; structured, coarse-grained pruning exploits hardware-efficient structures in pruning, but suffers from accuracy drop when the pruning rate is high. In this paper, we introduce PCONV, comprising a new sparsity dimension, – fine-grained pruning patterns inside the coarse-grained structures. PCONV comprises two types of sparsities, Sparse Convolution Patterns (SCP) which is generated from intra-convolution kernel pruning and connectivity sparsity generated from inter-convolution kernel pruning. Essentially, SCP enhances accuracy due to its special vision properties, and connectivity sparsity increases pruning rate while maintaining balanced workload on filter computation. To deploy PCONV, we develop a novel compiler-assisted DNN inference framework and execute PCONV models in real-time without accuracy compromise, which cannot be achieved in prior work. Our experimental results show that, PCONV outperforms three state-of-art end-to-end DNN frameworks, TensorFlow-Lite, TVM, and Alibaba Mobile Neural Network with speedup up to 39.2 ×, 11.4 ×, and 6.3 ×, respectively, with no accuracy loss. Mobile devices can achieve real-time inference on large-scale DNNs.

Ship Type Recognition via a Coarse-to-Fine Cascaded Convolution Neural Network

2020 ◽  
Vol 73 (4) ◽  
pp. 813-832 ◽  
Author(s):  
Xinqiang Chen ◽  
Yongsheng Yang ◽  
Shengzheng Wang ◽  
Huafeng Wu ◽  
Jinjun Tang ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Input Data ◽  
Recognition Performance ◽  
Convolution Neural Network ◽  
Bulk Carrier ◽  
Merchant Ship ◽  
General Cargo ◽  
Oil Tanker ◽  
Coarse To Fine

Most previous research has handled the task of ship type recognition by exploring hand-craft ship features, which may fail to distinguish ships with similar visual appearances. This situation motivates us to propose a novel deep learning based ship type recognition framework which we have named coarse-to-fine cascaded convolution neural network (CFCCNN). First, the proposed CFCCNN framework formats the input training ship images and data, and provides trainable input data for the hidden layers of the CFCCNN. Second, the coarse and fine steps are run in a nesting manner to explore discriminative features for different ship types. More specifically, the coarse step is trained in a similar manner to the traditional convolution neural network, while the fine step introduces regularisation mechanisms to extract more intrinsic ship features, and fine tunes parameter settings to obtain better recognition performance. Finally, we evaluate the performance of the CFCCNN model for recognising the most common types of merchant ship (oil tanker, container, LNG tanker, chemical carrier, general cargo, bulk carrier, etc.). The experimental results show that the proposed framework obtains better recognition performance than the conventional methods of ship type recognition.

scholarly journals Hydrological Image Building Using Curve Number and Prediction and Evaluation of Runoff through Convolution Neural Network

Water ◽  
2020 ◽  
Vol 12 (8) ◽  
pp. 2292 ◽  
Author(s):  
Chul Min Song
Keyword(s):  
Neural Network ◽  
Input Data ◽  
Pearson Correlation ◽  
Curve Number ◽  
Convolution Neural Network ◽  
Classification Problems ◽  
Image Building ◽  
Predicted Values ◽  
The Relationship ◽  
Runoff Model

This study developed a runoff model using a convolution neural network (CNN), which had previously only been used for classification problems, to get away from artificial neural networks (ANNs) that have been extensively used for the development of runoff models, and to secure diversity and demonstrate the suitability of the model. For this model’s input data, photographs typically used in the CNN model could not be used; due to the nature of the study, hydrological images reflecting effects such as watershed conditions and rainfall were required, which posed further difficulties. To address this, the method of a generating hydrological image using the curve number (CN) published by the Soil Conservation Service (SCS) was suggested in this study, and the hydrological images using CN were found to be sufficient as input data for the CNN model. Furthermore, this study was able to present a new application for the CN, which had been used only for estimating runoff. The model was trained and generalized stably overall, and R2, which indicates the relationship between the actual and predicted values, was relatively high at 0.82. The Pearson correlation coefficient, Nash–Sutcliffe efficiency (NSE), and root mean square error (RMSE), were 0.87, 0.60, and 16.20 m3/s, respectively, demonstrating a good overall model prediction performance.

Sign in / Sign up

Export Citation Format

Share Document