scholarly journals Multiscale Efficient Channel Attention for Fusion Lane Line Segmentation

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Kang Liu ◽  
Xin Gao
Keyword(s):  
Data Structure ◽  
Benchmark Dataset ◽  
Multimodal Fusion ◽  
Fusion Method ◽  
Modal Data ◽  
Structure Level ◽  
Multimodal Sensors ◽  
Lane Line ◽  
Line Segmentation ◽  
Direct Fusion

The use of multimodal sensors for lane line segmentation has become a growing trend. To achieve robust multimodal fusion, we introduced a new multimodal fusion method and proved its effectiveness in an improved fusion network. Specifically, a multiscale fusion module is proposed to extract effective features from data of different modalities, and a channel attention module is used to adaptively calculate the contribution of the fused feature channels. We verified the effect of multimodal fusion on the KITTI benchmark dataset and A2D2 dataset and proved the effectiveness of the proposed method on the enhanced KITTI dataset. Our method achieves robust lane line segmentation, which is 4.53% higher than the direct fusion on the precision index, and obtains the highest F2 score of 79.72%. We believe that our method introduces an optimization idea of modal data structure level for multimodal fusion.

scholarly journals A novel multimodal fusion network based on a joint coding model for lane line segmentation

2021 ◽  
Author(s):  
Zhenhong Zou ◽  
Xinyu Zhang ◽  
Huaping Liu ◽  
Zhiwei Li ◽  
Amir Hussain ◽  
...  
Keyword(s):  
Multimodal Fusion ◽  
Lane Line ◽  
Joint Coding ◽  
Line Segmentation

scholarly journals Intelligent Clustering and Dynamic Incremental Learning to Generate Multi-Codebook Fuzzy Neural Network for Multi-Modal Data Classification

Symmetry ◽  
2020 ◽  
Vol 12 (4) ◽  
pp. 679
Author(s):  
Muhammad Anwar Ma’sum
Keyword(s):  
Neural Networks ◽  
Incremental Learning ◽  
Data Classification ◽  
Fuzzy Neural Networks ◽  
Benchmark Dataset ◽  
Synthetic Dataset ◽  
Special Treatment ◽  
Modal Data ◽  
Fuzzy Neural ◽  
Benchmark Datasets

Classification in multi-modal data is one of the challenges in the machine learning field. The multi-modal data need special treatment as its features are distributed in several areas. This study proposes multi-codebook fuzzy neural networks by using intelligent clustering and dynamic incremental learning for multi-modal data classification. In this study, we utilized intelligent K-means clustering based on anomalous patterns and intelligent K-means clustering based on histogram information. In this study, clustering is used to generate codebook candidates before the training process, while incremental learning is utilized when the condition to generate a new codebook is sufficient. The condition to generate a new codebook in incremental learning is based on the similarity of the winner class and other classes. The proposed method was evaluated in synthetic and benchmark datasets. The experiment results showed that the proposed multi-codebook fuzzy neural networks that use dynamic incremental learning have significant improvements compared to the original fuzzy neural networks. The improvements were 15.65%, 5.31% and 11.42% on the synthetic dataset, the benchmark dataset, and the average of all datasets, respectively, for incremental version 1. The incremental learning version 2 improved by 21.08% 4.63%, and 14.35% on the synthetic dataset, the benchmark dataset, and the average of all datasets, respectively. The multi-codebook fuzzy neural networks that use intelligent clustering also had significant improvements compared to the original fuzzy neural networks, achieving 23.90%, 2.10%, and 15.02% improvements on the synthetic dataset, the benchmark dataset, and the average of all datasets, respectively.

scholarly journals Multimodal Fusion Method Based on Self-Attention Mechanism

2020 ◽  
Vol 2020 ◽  
pp. 1-8
Author(s):  
Hu Zhu ◽  
Ze Wang ◽  
Yu Shi ◽  
Yingying Hua ◽  
Guoxia Xu ◽  
...  
Keyword(s):  
Computational Complexity ◽  
Data Fusion ◽  
Heterogeneous Data ◽  
Attention Mechanism ◽  
Multimodal Fusion ◽  
Fusion Method ◽  
Data Set ◽  
Multimodal Data ◽  
Public Data ◽  
Multimodal Data Fusion

Multimodal fusion is one of the popular research directions of multimodal research, and it is also an emerging research field of artificial intelligence. Multimodal fusion is aimed at taking advantage of the complementarity of heterogeneous data and providing reliable classification for the model. Multimodal data fusion is to transform data from multiple single-mode representations to a compact multimodal representation. In previous multimodal data fusion studies, most of the research in this field used multimodal representations of tensors. As the input is converted into a tensor, the dimensions and computational complexity increase exponentially. In this paper, we propose a low-rank tensor multimodal fusion method with an attention mechanism, which improves efficiency and reduces computational complexity. We evaluate our model through three multimodal fusion tasks, which are based on a public data set: CMU-MOSI, IEMOCAP, and POM. Our model achieves a good performance while flexibly capturing the global and local connections. Compared with other multimodal fusions represented by tensors, experiments show that our model can achieve better results steadily under a series of attention mechanisms.

scholarly journals Multi-modal Data Fusion Method for Human Behavior Recognition Based on Two IA-Net and CHMM

2021 ◽  
Author(s):  
Yinhuan ZHANG ◽  
Qinkun XIAO ◽  
Chaoqin CHU ◽  
Heng XING
Keyword(s):  
Feature Extraction ◽  
Data Fusion ◽  
Human Behavior ◽  
Family Environment ◽  
Fusion Method ◽  
Target Feature ◽  
Behavior Recognition ◽  
Modal Data ◽  
Modal Behavior ◽  
Human Behavior Recognition

The multi-modal data fusion method based on IA-net and CHMM technical proposed is designed to solve the problem that the incompleteness of target behavior information in complex family environment leads to the low accuracy of human behavior recognition.The two improved neural networks(STA-ResNet50、STA-GoogleNet)are combined with LSTM to form two IA-Nets respectively to extract RGB and skeleton modal behavior features in video. The two modal feature sequences are input CHMM to construct the probability fusion model of multi-modal behavior recognition.The experimental results show that the human behavior recognition model proposed in this paper has higher accuracy than the previous fusion methods on HMDB51 and UCF101 datasets. New contributions: attention mechanism is introduced to improve the efficiency of video target feature extraction and utilization. A skeleton based feature extraction framework is proposed, which can be used for human behavior recognition in complex environment. In the field of human behavior recognition, probability theory and neural network are cleverly combined and applied, which provides a new method for multi-modal information fusion.

scholarly journals Clustering versus Incremental Learning Multi-Codebook Fuzzy Neural Network for Multi-Modal Data Classification

Computation ◽  
2020 ◽  
Vol 8 (1) ◽  
pp. 6
Author(s):  
Muhammad Anwar Ma’sum ◽  
Hadaiq Rolis Sanabila ◽  
Petrus Mursanto ◽  
Wisnu Jatmiko
Keyword(s):  
Neural Network ◽  
Incremental Learning ◽  
Fuzzy Neural Network ◽  
Data Classification ◽  
Benchmark Dataset ◽  
Learning Approaches ◽  
Clustering Methods ◽  
Modal Data ◽  
Fuzzy Neural ◽  
Benchmark Datasets

One of the challenges in machine learning is a classification in multi-modal data. The problem needs a customized method as the data has a feature that spreads in several areas. This study proposed a multi-codebook fuzzy neural network classifiers using clustering and incremental learning approaches to deal with multi-modal data classification. The clustering methods used are K-Means and GMM clustering. Experiment result, on a synthetic dataset, the proposed method achieved the highest performance with 84.76% accuracy. Whereas on the benchmark dataset, the proposed method has the highest performance with 79.94% accuracy. The proposed method has 24.9% and 4.7% improvements in synthetic and benchmark datasets respectively compared to the original version. The proposed classifier has better accuracy compared to a popular neural network with 10% and 4.7% margin in synthetic and benchmark dataset respectively.

Sign in / Sign up

Export Citation Format

Share Document