Information measure for assessing pixel-level fusion methods

2004 ◽  
Author(s):  
Vassilis Tsagaris ◽  
Vassilis Anastassopoulos
Keyword(s):  
Information Measure ◽  
Fusion Methods ◽  
Level Fusion

Rank Level Fusion

2013 ◽  
pp. 69-79
Keyword(s):  
Information Aggregation ◽  
Regression Method ◽  
Borda Count ◽  
Fusion Method ◽  
Advantages And Disadvantages ◽  
Current State ◽  
Fusion Methods ◽  
Logistic Regression Method ◽  
Level Fusion ◽  
Voting Method

Rank level fusion is one of the after matching fusion methods used in multibiometric systems. The problem of rank information aggregation has been raised before in various fields. This chapter extensively discusses the rank level fusion methodology, starting with existing literature from the last decade in different application scenarios. Several approaches of existing biometric rank level fusion methods, such as plurality voting method, highest rank method, Borda count method, logistic regression method, and quality-based rank fusion method, are discussed along with their advantages and disadvantages in the context of the current state-of-the-art in the discipline.

A Hybrid Approach for Person Identification Using Palmprint and Face Biometrics

2015 ◽  
Vol 29 (06) ◽  
pp. 1556009 ◽  
Author(s):  
Mina Farmanbar ◽  
Önsen Toygar
Keyword(s):  
Hybrid Approach ◽  
Principal Component ◽  
Local Binary Patterns ◽  
Recognition System ◽  
Extraction Methods ◽  
Score Level Fusion ◽  
Linear Discriminant ◽  
Hybrid Approaches ◽  
Fusion Methods ◽  
Level Fusion

This paper proposes hybrid approaches based on both feature level and score level fusion strategies to provide a robust recognition system against the distortions of individual modalities. In order to compare the proposed schemes, a virtual multimodal database is formed from FERET face and PolyU palmprint databases. The proposed hybrid systems concatenate features extracted by local and global feature extraction methods such as Local Binary Patterns, Log Gabor, Principal Component Analysis and Linear Discriminant Analysis. Match score level fusion is performed in order to show the effectiveness and accuracy of the proposed schemes. The experimental results based on these databases reported a significant improvement of the proposed schemes compared with unimodal systems and other multimodal face–palmprint fusion methods.

Audiovisual Facial Action Unit Recognition using Feature Level Fusion

2016 ◽  
Vol 7 (1) ◽  
pp. 60-76 ◽  
Author(s):  
Zibo Meng ◽  
Shizhong Han ◽  
Min Chen ◽  
Yan Tong
Keyword(s):  
Time Shift ◽  
Visual Features ◽  
Visual Channel ◽  
Facial Action ◽  
Audio Channel ◽  
Feature Level Fusion ◽  
Fusion Methods ◽  
The Difference ◽  
Facial Images ◽  
Level Fusion

Recognizing facial actions is challenging, especially when they are accompanied with speech. Instead of employing information solely from the visual channel, this work aims to exploit information from both visual and audio channels in recognizing speech-related facial action units (AUs). In this work, two feature-level fusion methods are proposed. The first method is based on a kind of human-crafted visual feature. The other method utilizes visual features learned by a deep convolutional neural network (CNN). For both methods, features are independently extracted from visual and audio channels and aligned to handle the difference in time scales and the time shift between the two signals. These temporally aligned features are integrated via feature-level fusion for AU recognition. Experimental results on a new audiovisual AU-coded dataset have demonstrated that both fusion methods outperform their visual counterparts in recognizing speech-related AUs. The improvement is more impressive with occlusions on the facial images, which would not affect the audio channel.

A NEW APPROACH FOR FACE-IRIS MULTIMODAL BIOMETRIC RECOGNITION USING SCORE FUSION

2013 ◽  
Vol 27 (03) ◽  
pp. 1356004 ◽  
Author(s):  
MARYAM ESKANDARI ◽  
ÖNSEN TOYGAR ◽  
HASAN DEMIREL
Keyword(s):  
Recognition Performance ◽  
Recognition System ◽  
Extraction Methods ◽  
Score Level Fusion ◽  
New Approach ◽  
Fusion Methods ◽  
Score Fusion ◽  
Multimodal Biometric Recognition ◽  
Matching Score ◽  
Level Fusion

In this paper, a new approach based on score level fusion is presented to obtain a robust recognition system by concatenating face and iris scores of several standard classifiers. The proposed method concatenates face and iris match scores instead of concatenating features as in feature-level fusion. The features from face and iris are extracted using local and global feature extraction methods such as PCA, subspace LDA, spPCA, mPCA and LBP. Transformation-based score fusion and classifier-based score fusion are then involved in the process to obtain, concatenate and classify the matching scores. Different fusion techniques at matching score level, feature level and decision level are compared with the proposed method to emphasize improvement and effectiveness of the proposed method. In order to validate the proposed scheme, a combined database is formed using ORL and BANCA face databases together with CASIA and UBIRIS iris databases. The results based on recognition performance and ROC analysis demonstrate that the proposed score level fusion achieves a significant improvement over unimodal methods and other multimodal face-iris fusion methods.

PCANet based biometric system with fusion of palmprint and dorsal hand vein

2021 ◽  
pp. 1-9
Author(s):  
Surinder kaur ◽  
Gopal Chaudhary ◽  
Javalkar Dinesh kumar
Keyword(s):  
Recognition System ◽  
Personal Identification ◽  
Identification System ◽  
Biometric System ◽  
Score Level Fusion ◽  
Dorsal Hand ◽  
Hand Vein ◽  
Fusion Methods ◽  
Dorsal Hand Vein ◽  
Level Fusion

Nowadays, Biometric systems are prevalent for personal recognition. But due to pandemic COVID 19, it is difficult to pursue a touch-based biometric system. To encourage a touchless biometric system, a less constrained multimodal personal identification system using palmprint and dorsal hand vein is presented. Hand based Touchless recognition system gives a higher user-friendly system and avoids the spread of coronavirus. A method using Convolution Neural Networks(CNN) to extract discriminative features from the data samples is proposed. A pre-trained function PCANeT is used in the experiments to show the performance of the system in fusion scheme. This method doesn’t require keeping the palm in a specific position or at a certain distance like most other papers. Different patches of ROI are used at two different layers of CNN. Fusion of palmprint and dorsal hand vein is done for final result matching. Both Feature level and score level fusion methods are compared. Results shows the accuracy of upto 98.55% and 98.86% and Equal error rate (EER) of upto 1.22% and 0.93% for score level fusion and feature level fusion, respectively. Our method gives higher accurate results in a less constrained environment.

scholarly journals Two-Level Multimodal Fusion for Sentiment Analysis in Public Security

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Jianguo Sun ◽  
Hanqi Yin ◽  
Ye Tian ◽  
Junpeng Wu ◽  
Linshan Shen ◽  
...  
Keyword(s):  
Sentiment Analysis ◽  
Multimodal Fusion ◽  
Public Security ◽  
Multimodal Data ◽  
Decision Level ◽  
Analysis Task ◽  
Fusion Methods ◽  
Decision Level Fusion ◽  
Level Fusion ◽  
Data Level

Large amounts of data are widely stored in cyberspace. Not only can they bring much convenience to people’s lives and work, but they can also assist the work in the information security field, such as microexpression recognition and sentiment analysis in the criminal investigation. Thus, it is of great significance to recognize and analyze the sentiment information, which is usually described by different modalities. Due to the correlation among different modalities data, multimodal can provide more comprehensive and robust information than unimodal in data analysis tasks. The complementary information from different modalities can be obtained by multimodal fusion methods. These approaches can process multimodal data through fusion algorithms and ensure the accuracy of the information used for subsequent classification or prediction tasks. In this study, a two-level multimodal fusion (TlMF) method with both data-level and decision-level fusion is proposed to achieve the sentiment analysis task. In the data-level fusion stage, a tensor fusion network is utilized to obtain the text-audio and text-video embeddings by fusing the text with audio and video features, respectively. During the decision-level fusion stage, the soft fusion method is adopted to fuse the classification or prediction results of the upstream classifiers, so that the final classification or prediction results can be as accurate as possible. The proposed method is tested on the CMU-MOSI, CMU-MOSEI, and IEMOCAP datasets, and the empirical results and ablation studies confirm the effectiveness of TlMF in capturing useful information from all the test modalities.

scholarly journals Vision-Based Tunnel Lining Health Monitoring via Bi-Temporal Image Comparison and Decision-Level Fusion of Change Maps

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4040
Author(s):  
Leanne Attard ◽  
Carl James Debono ◽  
Gianluca Valentino ◽  
Mario Di Castro
Keyword(s):  
Change Detection ◽  
Health Monitoring ◽  
Detection System ◽  
Automatic Monitoring ◽  
Structural Health ◽  
Decision Level ◽  
Fusion Methods ◽  
Inspection Systems ◽  
Decision Level Fusion ◽  
Level Fusion

Tunnel structural health inspections are predominantly done through periodic visual observations, requiring humans to be physically present on-site, possibly exposing them to hazardous environments. These surveys are subjective (relying on the surveyor experience), time-consuming, and may demand operation shutdown. These issues can be mitigated through accurate automatic monitoring and inspection systems. In this work, we propose a remotely operated machine vision change detection application to improve the structural health monitoring of tunnels. The vision-based sensing system acquires the data from a rig of cameras hosted on a robotic platform that is driven parallel to the tunnel walls. These data are then pre-processed using image processing and deep learning techniques to reduce nuisance changes caused by light variations. Image fusion techniques are then applied to identify the changes occurring in the tunnel structure. Different pixel-based change detection approaches are used to generate temporal change maps. Decision-level fusion methods are then used to combine these change maps to obtain a more reliable detection of the changes that occur between surveys. A quantitative analysis of the results achieved shows that the proposed change detection system achieved a recall value of 81%, a precision value of 93% and an F1-score of 86.7%.

Sign in / Sign up

Export Citation Format

Share Document