AI-Based Multi Sensor Fusion for Smart Decision Making: A Bi-Functional System for Single Sensor Evaluation in a Classification Task

Sensor fusion has gained a great deal of attention in recent years. It is used as an application tool in many different fields, especially the semiconductor, automotive, and medical industries. However, this field of research, regardless of the field of application, still presents different challenges concerning the choice of the sensors to be combined and the fusion architecture to be developed. To decrease application costs and engineering efforts, it is very important to analyze the sensors’ data beforehand once the application target is defined. This pre-analysis is a basic step to establish a working environment with fewer misclassification cases and high safety. One promising approach to do so is to analyze the system using deep neural networks. The disadvantages of this approach are mainly the required huge storage capacity, the big training effort, and that these networks are difficult to interpret. In this paper, we focus on developing a smart and interpretable bi-functional artificial intelligence (AI) system, which has to discriminate the combined data regarding predefined classes. Furthermore, the system can evaluate the single source signals used in the classification task. The evaluation here covers each sensor contribution and robustness. More precisely, we train a smart and interpretable prototype-based neural network, which learns automatically to weight the influence of the sensors for the classification decision. Moreover, the prototype-based classifier is equipped with a reject option to measure classification certainty. To validate our approach’s efficiency, we refer to different industrial sensor fusion applications.

Download Full-text

Gas Detection and Identification Using Multimodal Artificial Intelligence Based Sensor Fusion

Applied System Innovation ◽

10.3390/asi4010003 ◽

2021 ◽

Vol 4 (1) ◽

pp. 3

Author(s):

Parag Narkhede ◽

Rahee Walambe ◽

Shruti Mandaokar ◽

Pulkit Chandel ◽

Ketan Kotecha ◽

...

Keyword(s):

Sensor Fusion ◽

Network Architecture ◽

Cost Effective ◽

Sensor Data ◽

Dense Layer ◽

Gaseous Emissions ◽

Multiple Sensors ◽

Single Output ◽

Novel Approach ◽

Single Sensor

With the rapid industrialization and technological advancements, innovative engineering technologies which are cost effective, faster and easier to implement are essential. One such area of concern is the rising number of accidents happening due to gas leaks at coal mines, chemical industries, home appliances etc. In this paper we propose a novel approach to detect and identify the gaseous emissions using the multimodal AI fusion techniques. Most of the gases and their fumes are colorless, odorless, and tasteless, thereby challenging our normal human senses. Sensing based on a single sensor may not be accurate, and sensor fusion is essential for robust and reliable detection in several real-world applications. We manually collected 6400 gas samples (1600 samples per class for four classes) using two specific sensors: the 7-semiconductor gas sensors array, and a thermal camera. The early fusion method of multimodal AI, is applied The network architecture consists of a feature extraction module for individual modality, which is then fused using a merged layer followed by a dense layer, which provides a single output for identifying the gas. We obtained the testing accuracy of 96% (for fused model) as opposed to individual model accuracies of 82% (based on Gas Sensor data using LSTM) and 93% (based on thermal images data using CNN model). Results demonstrate that the fusion of multiple sensors and modalities outperforms the outcome of a single sensor.

Download Full-text

System on chip generation for multi-sensor and sensor fusion applications

2017 International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS) ◽

10.1109/samos.2017.8344607 ◽

2017 ◽

Cited By ~ 1

Author(s):

Tobias Lieske ◽

Benjamin Pfundt ◽

Steffen Vaas ◽

Marc Reichenbach ◽

Dietmar Fey

Keyword(s):

Sensor Fusion ◽

System On Chip ◽

Fusion Applications ◽

On Chip

Download Full-text

A Robust Cooperative Modulation Classification Scheme with Intra sensor Fusion for the Time correlated Flat Fading Channels

Defence Science Journal ◽

10.14429/dsj.70.14687 ◽

2020 ◽

Vol 70 (1) ◽

pp. 60-65 ◽

Cited By ~ 1

Author(s):

Goran Marković ◽

Vlada Sokolović

Keyword(s):

Fading Channels ◽

Sensor Fusion ◽

Large Scale ◽

Negative Influence ◽

Modulation Classification ◽

Distributed Sensors ◽

Performance Improvements ◽

Flat Fading ◽

Significant Performance ◽

Single Sensor

Networks with distributed sensors, e.g. cognitive radio networks or wireless sensor networks enable large-scale deployments of cooperative automatic modulation classification (AMC). Existing cooperative AMC schemes with centralised fusion offer considerable performance increase in comparison to single sensor reception. Previous studies were generally focused on AMC scenarios in which multipath channel is assumed to be static during a signal reception. However, in practical mobile environments, time-correlated multipath channels occur, which induce large negative influence on the existing cooperative AMC solutions. In this paper, we propose two novel cooperative AMC schemes with the additional intra-sensor fusion, and show that these offer significant performance improvements over the existing ones under given conditions.

Download Full-text

The performance of a real time LAN architecture for sensor fusion applications

[1988] Proceedings. Computer Networking Symposium ◽

10.1109/cns.1988.4988 ◽

2003 ◽

Author(s):

V. Cherkassky ◽

H. Lari-Najaffi ◽

N.L. Lawrie ◽

D. Masson ◽

D.W. Pritty

Keyword(s):

Real Time ◽

Sensor Fusion ◽

Fusion Applications

Download Full-text

Research of Multi-Sensor Integration System for Train Speed and Position Measurement

Applied Mechanics and Materials ◽

10.4028/www.scientific.net/amm.105-107.1920 ◽

2011 ◽

Vol 105-107 ◽

pp. 1920-1925 ◽

Cited By ~ 1

Author(s):

Yu Wei Zhou

Keyword(s):

Sensor Fusion ◽

Simulation Experiment ◽

Doppler Radar ◽

Sensor Integration ◽

Position Measurement ◽

Integration System ◽

Train Control ◽

Federated Kalman Filter ◽

Train Speed ◽

Single Sensor

The speed and position measurement is an important part of the train control system. Accurate measurement of train speed and position ensures safety of train running and improves transportation efficiency. Traditional methods to measure speed and position usually rely on single sensor, which has less accuracy and reliability. A method of measuring train speed and position based on multi-sensor fusion is proposed in this paper. Since trains are running on fixed rail tracks, the position of train can be determined by the traveled distance and speed monitoring is essentially to measure the magnitude of train velocity. According to this characteristic, this measurement system consists of axle odometer, Doppler radar, accelerometer and intermittent inquiry balise. The federated Kalman filter is used to implement multi-sensor fusion. Simulation experiment results prove that this method can improve the accuracy and reliability of train speed and position measurement.

Download Full-text

Fuzzy-Based Sensor Fusion for Cognitive Radio-Based Vehicular Ad Hoc and Sensor Networks

Mathematical Problems in Engineering ◽

10.1155/2015/439272 ◽

2015 ◽

Vol 2015 ◽

pp. 1-9 ◽

Cited By ~ 3

Author(s):

Mohammad Jalil Piran ◽

Amjad Ali ◽

Doug Young Suh

Keyword(s):

Cognitive Radio ◽

Sensor Networks ◽

Sensor Fusion ◽

Ad Hoc ◽

Fuzzy Theory ◽

Wireless Sensor ◽

Mobile Nodes ◽

Level Information ◽

Single Sensor ◽

High Level

In wireless sensor networks, sensor fusion is employed to integrate the acquired data from diverse sensors to provide a unified interpretation. The best and most salient advantage of sensor fusion is to obtain high-level information in both statistical and definitive aspects, which cannot be attained by a single sensor. In this paper, we propose a novel sensor fusion technique based on fuzzy theory for our earlier proposed Cognitive Radio-based Vehicular Ad Hoc and Sensor Networks (CR-VASNET). In the proposed technique, we considered four input sensor readings (antecedents) and one output (consequent). The employed mobile nodes in CR-VASNET are supposed to be equipped with diverse sensors, which cater to our antecedent variables, for example, The Jerk, Collision Intensity, and Temperature and Inclination Degree. Crash_Severity is considered as the consequent variable. The processing and fusion of the diverse sensory signals are carried out by fuzzy logic scenario. Accuracy and reliability of the proposed protocol, demonstrated by the simulation results, introduce it as an applicable system to be employed to reduce the causalities rate of the vehicles’ crashes.

Download Full-text

Sensor Fusion Strategy in the Monitoring of Cutting Tool Wear

Key Engineering Materials ◽

10.4028/www.scientific.net/kem.306-308.727 ◽

2006 ◽

Vol 306-308 ◽

pp. 727-732 ◽

Cited By ~ 1

Author(s):

Muslim Mahardika ◽

Zahari Taha ◽

Djoko Suharto ◽

Kimiyuki Mitsui ◽

Hideki Aoyama

Keyword(s):

Tool Wear ◽

Sensor Fusion ◽

Cutting Tool ◽

Fusion Technology ◽

Machining Processes ◽

Fusion Strategy ◽

Cutting Tool Wear ◽

Single Sensor ◽

Paper Sensor

Cutting tool wear is a major problem in machining processes. It has a great effect on the quality of a workpiece. Thus, monitoring cutting tool wear is very important in order to maintain the workpiece quality as well to reduce production rate and production time. The use of a single sensor in a monitoring system may not be accurate to detect cutting tool wear. In this paper, sensor fusion technology is introduced for monitoring cutting tool wear.

Download Full-text