scholarly journals Multi-Label Patent Categorization with Non-Local Attention-Based Graph Convolutional Network

2020 ◽  
Vol 34 (05) ◽  
pp. 9024-9031
Author(s):  
Pingjie Tang ◽  
Meng Jiang ◽  
Bryan (Ning) Xia ◽  
Jed W. Pitera ◽  
Jeffrey Welser ◽  
...  
Keyword(s):  
Domain Knowledge ◽  
High Performance ◽  
State Of The Art ◽  
Convolutional Network ◽  
Attention Model ◽  
Patent Classification ◽  
International Patent ◽  
Non Local ◽  
Text Content ◽  
Document Representations

Patent categorization, which is to assign multiple International Patent Classification (IPC) codes to a patent document, relies heavily on expert efforts, as it requires substantial domain knowledge. When formulated as a multi-label text classification (MTC) problem, it draws two challenges to existing models: one is to learn effective document representations from text content; the other is to model the cross-section behavior of label set. In this work, we propose a label attention model based on graph convolutional network. It jointly learns the document-word associations and word-word co-occurrences to generate rich semantic embeddings of documents. It employs a non-local attention mechanism to learn label representations in the same space of document representations for multi-label classification. On a large CIRCA patent database, we evaluate the performance of our model and as many as seven competitive baselines. We find that our model outperforms all those prior state of the art by a large margin and achieves high performance on P@k and nDCG@k.

scholarly journals Investigating the impacts of artificial intelligence technology on technological innovation from a patent perspective

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Yi-Hui Qiu ◽  
Xin Yang ◽  
Zong-Ze Li ◽  
Chiyu Zhang ◽  
Shui-Xuan Chen
Keyword(s):  
Artificial Intelligence ◽  
Statistical Analysis ◽  
Technological Innovation ◽  
Systematic Study ◽  
State Of The Art ◽  
The State ◽  
International Patent Classification ◽  
Patent Classification ◽  
International Patent ◽  
Artificial Intelligence Technology

Abstract Artificial intelligence (AI) technology has been widely applied in various fields in recent years. Nevertheless, no systematic study has yet been conducted on the effects of AI technology on different fields. In this study, the impacts of the latest AI technology on technological innovation in different fields were analysed and quantized systematically from a patent perspective. Moreover, trends on AI technological innovation in some fields were analysed thoroughly. We conducted this study on a dataset of patents related to AI technology. Based on the patent dataset, we carried out a statistical analysis on technology fields, which we defined and classified based on international patent classification (IPC) number. Distributions of IPC in different fields were also analysed to determine the trends on AI technological innovation. The research conclusions can provide useful information to investors and enterprises, who are interested in the state of the art concerning AI technology.

scholarly journals EAC-Net: Efficient and Accurate Convolutional Network for Video Recognition

2020 ◽  
Vol 34 (07) ◽  
pp. 11149-11156
Author(s):  
Bowei Jin ◽  
Zhuo Xu
Keyword(s):  
High Performance ◽  
High Efficiency ◽  
Multiple Time Scales ◽  
Multiple Time ◽  
Temporal Relations ◽  
Convolutional Network ◽  
Temporal Modeling ◽  
Video Recognition ◽  
Non Local

Research for computation-efficient video understanding is of great importance to real-world deployment. However, most of high-performance approaches are too computationally expensive for practical application. Though several efficiency oriented works are proposed, they inevitably suffer degradation of performance in terms of accuracy. In this paper, we explore a new architecture EAC-Net, enjoying both high efficiency and high performance. Specifically, we propose Motion Guided Temporal Encode (MGTE) blocks for temporal modeling, which exploits motion information and temporal relations among neighbor frames. EAC-Net is then constructed by inserting multiple MGTE blocks to common 2D CNNs. Furthermore, we proposed Atrous Temporal Encode (ATE) block for capturing long-term temporal relations at multiple time scales for further enhancing representation power of EAC-Net. Through experiments on Kinetics, our EAC-Nets achieved better results than TSM models with fewer FLOPs. With same 2D backbones, EAC-Nets outperformed Non-Local I3D counterparts by achieving higher accuracy with only about 7× fewer FLOPs. On Something-Something-V1 dataset, EAC-Net achieved 47% top-1 accuracy with 70G FLOPs which is 0.9% more accurate and 8× less FLOPs than that of Non-Local I3D+GCN.

scholarly journals Graph Contextualized Self-Attention Network for Session-based Recommendation

2019 ◽  
Author(s):  
Chengfeng Xu ◽  
Pengpeng Zhao ◽  
Yanchi Liu ◽  
Victor S. Sheng ◽  
Jiajie Xu ◽  
...  
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Attention Mechanism ◽  
The Self ◽  
Graph Structure ◽  
Convolutional Network ◽  
Attention Network ◽  
Attention Model ◽  
Sequence Modeling ◽  
Real World Datasets

Session-based recommendation, which aims to predict the user's immediate next action based on anonymous sessions, is a key task in many online services (e.g., e-commerce, media streaming).  Recently, Self-Attention Network (SAN) has achieved significant success in various sequence modeling tasks without using either recurrent or convolutional network. However, SAN lacks local dependencies that exist over adjacent items and limits its capacity for learning contextualized representations of items in sequences.  In this paper, we propose a graph contextualized self-attention model (GC-SAN), which utilizes both graph neural network and self-attention mechanism, for session-based recommendation. In GC-SAN, we dynamically construct a graph structure for session sequences and capture rich local dependencies via graph neural network (GNN).  Then each session learns long-range dependencies by applying the self-attention mechanism. Finally, each session is represented as a linear combination of the global preference and the current interest of that session. Extensive experiments on two real-world datasets show that GC-SAN outperforms state-of-the-art methods consistently.

scholarly journals Efficient End-to-End Sentence-Level Lipreading with Temporal Convolutional Networks

2021 ◽  
Vol 11 (15) ◽  
pp. 6975
Author(s):  
Tao Zhang ◽  
Lun He ◽  
Xudong Li ◽  
Guoqing Feng
Keyword(s):  
Performance Improvement ◽  
State Of The Art ◽  
Error Rates ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Sentence Level ◽  
End To End ◽  
High Level ◽  
Improved Accuracy ◽  
Talking Face

Lipreading aims to recognize sentences being spoken by a talking face. In recent years, the lipreading method has achieved a high level of accuracy on large datasets and made breakthrough progress. However, lipreading is still far from being solved, and existing methods tend to have high error rates on the wild data and have the defects of disappearing training gradient and slow convergence. To overcome these problems, we proposed an efficient end-to-end sentence-level lipreading model, using an encoder based on a 3D convolutional network, ResNet50, Temporal Convolutional Network (TCN), and a CTC objective function as the decoder. More importantly, the proposed architecture incorporates TCN as a feature learner to decode feature. It can partly eliminate the defects of RNN (LSTM, GRU) gradient disappearance and insufficient performance, and this yields notable performance improvement as well as faster convergence. Experiments show that the training and convergence speed are 50% faster than the state-of-the-art method, and improved accuracy by 2.4% on the GRID dataset.

scholarly journals Multiple objects tracking in the UAV system based on hierarchical deep high-resolution network

2021 ◽  
Author(s):  
Wei Huang ◽  
Xiaoshu Zhou ◽  
Mingchao Dong ◽  
Huaiyu Xu
Keyword(s):  
High Resolution ◽  
Object Tracking ◽  
High Performance ◽  
State Of The Art ◽  
Class Imbalance ◽  
Unified Framework ◽  
Multiple Objects ◽  
Tracking Process ◽  
Objects Tracking ◽  
Different Types

AbstractRobust and high-performance visual multi-object tracking is a big challenge in computer vision, especially in a drone scenario. In this paper, an online Multi-Object Tracking (MOT) approach in the UAV system is proposed to handle small target detections and class imbalance challenges, which integrates the merits of deep high-resolution representation network and data association method in a unified framework. Specifically, while applying tracking-by-detection architecture to our tracking framework, a Hierarchical Deep High-resolution network (HDHNet) is proposed, which encourages the model to handle different types and scales of targets, and extract more effective and comprehensive features during online learning. After that, the extracted features are fed into different prediction networks for interesting targets recognition. Besides, an adjustable fusion loss function is proposed by combining focal loss and GIoU loss to solve the problems of class imbalance and hard samples. During the tracking process, these detection results are applied to an improved DeepSORT MOT algorithm in each frame, which is available to make full use of the target appearance features to match one by one on a practical basis. The experimental results on the VisDrone2019 MOT benchmark show that the proposed UAV MOT system achieves the highest accuracy and the best robustness compared with state-of-the-art methods.

scholarly journals Direct laser additive manufacturing of high performance oxide ceramics: a state-of-the-art review

2021 ◽  
Author(s):  
Stefan Pfeiffer ◽  
Kevin Florio ◽  
Dario Puccio ◽  
Marco Grasso ◽  
Bianca Maria Colosimo ◽  
...  
Keyword(s):  
Additive Manufacturing ◽  
High Performance ◽  
State Of The Art ◽  
Oxide Ceramics ◽  
Laser Additive Manufacturing ◽  
Direct Laser

CyberEyes: Cybersecurity Entity Recognition Model Based on Graph Convolutional Network

2020 ◽  
Author(s):  
Yong Fang ◽  
Yuchi Zhang ◽  
Cheng Huang
Keyword(s):  
Complex Structure ◽  
Named Entity Recognition ◽  
Internet Technology ◽  
Entity Recognition ◽  
Local Context ◽  
Knowledge Graph ◽  
Convolutional Network ◽  
Evaluation Standard ◽  
Recognition Model ◽  
Non Local

Abstract Cybersecurity has gradually become the public focus between common people and countries with the high development of Internet technology in daily life. The cybersecurity knowledge analysis methods have achieved high evolution with the help of knowledge graph technology, especially a lot of threat intelligence information could be extracted with fine granularity. But named entity recognition (NER) is the primary task for constructing security knowledge graph. Traditional NER models are difficult to determine entities that have a complex structure in the field of cybersecurity, and it is difficult to capture non-local and non-sequential dependencies. In this paper, we propose a cybersecurity entity recognition model CyberEyes that uses non-local dependencies extracted by graph convolutional neural networks. The model can capture both local context and graph-level non-local dependencies. In the evaluation experiments, our model reached an F1 score of 90.28% on the cybersecurity corpus under the gold evaluation standard for NER, which performed better than the 86.49% obtained by the classic CNN-BiLSTM-CRF model.

scholarly journals Evaluation of recent advances in recommender systems on Arabic content

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Mehdi Srifi ◽  
Ahmed Oussous ◽  
Ayoub Ait Lahcen ◽  
Salma Mouline
Keyword(s):  
Recommender Systems ◽  
High Performance ◽  
Large Scale ◽  
State Of The Art ◽  
Experimental Results ◽  
Recent Advances ◽  
Research Gap ◽  
Text Preprocessing

AbstractVarious recommender systems (RSs) have been developed over recent years, and many of them have concentrated on English content. Thus, the majority of RSs from the literature were compared on English content. However, the research investigations about RSs when using contents in other languages such as Arabic are minimal. The researchers still neglect the field of Arabic RSs. Therefore, we aim through this study to fill this research gap by leveraging the benefit of recent advances in the English RSs field. Our main goal is to investigate recent RSs in an Arabic context. For that, we firstly selected five state-of-the-art RSs devoted originally to English content, and then we empirically evaluated their performance on Arabic content. As a result of this work, we first build four publicly available large-scale Arabic datasets for recommendation purposes. Second, various text preprocessing techniques have been provided for preparing the constructed datasets. Third, our investigation derived well-argued conclusions about the usage of modern RSs in the Arabic context. The experimental results proved that these systems ensure high performance when applied to Arabic content.

scholarly journals Technological Advances to Reduce Apis mellifera Mortality: A Bibliometric Analysis

2021 ◽  
Vol 13 (15) ◽  
pp. 8305
Author(s):  
Cristiano Ziegler ◽  
Tiago Sinigaglia ◽  
Mario Eduardo Santos Martins ◽  
Adriano Mendonça Souza
Keyword(s):  
Apis Mellifera ◽  
The United States ◽  
Mortality Data ◽  
Time Interval ◽  
International Patent Classification ◽  
Patent Classification ◽  
Honey Production ◽  
Technological Advances ◽  
International Patent ◽  
Technology Life Cycle

Bees play a fundamental role in the ecological balance of ecosystems, due to the pollination process they carry out on crops, including the production of honey. However, the mortality of bees is a significant concern; bee mortality can occur for several reasons, such as pesticides, mites, viruses, climate change, pathogens and a reduction in food resources and nests. The honey bee (Apis mellifera) is the most widely used bee for commercial pollination and honey production. Therefore, the main objective is to compare the development of patent families and article publications related to the reduction in A. meliífera mortality. Data on patent families were collected on the Orbit platform, while data on scientific articles were collected on the Scopus database, with a time interval of 1980–2019. Subsequently, the data were analyzed in order to show the main priority countries, main assignees, and main IPC (International Patent Classification) codes, an analysis of the technology life cycle and the correlation between the data of patent families and articles published. The technologies that help to decrease bee mortality showed a technological maturity rate of 27.15% for patent families data and 53.35% for data from articles published in journals. It was noticed that the principal interest regarding the reduction in A. mellifera mortality is focused on universities, mainly in the United States and China.

Sign in / Sign up

Export Citation Format

Share Document