scholarly journals EvAn: Neuromorphic Event-Based Sparse Anomaly Detection

2021 ◽  
Vol 15 ◽  
Author(s):  
Lakshmi Annamalai ◽  
Anirban Chakraborty ◽  
Chetan Singh Thakur
Keyword(s):  
Deep Learning ◽  
Anomaly Detection ◽  
Dynamic Range ◽  
High Dynamic Range ◽  
Careful Analysis ◽  
Motion Blur ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Unsupervised Deep Learning ◽  
Event Based

Event-based cameras are bio-inspired novel sensors that asynchronously record changes in illumination in the form of events. This principle results in significant advantages over conventional cameras, such as low power utilization, high dynamic range, and no motion blur. Moreover, by design, such cameras encode only the relative motion between the scene and the sensor and not the static background to yield a very sparse data structure. In this paper, we leverage these advantages of an event camera toward a critical vision application—video anomaly detection. We propose an anomaly detection solution in the event domain with a conditional Generative Adversarial Network (cGAN) made up of sparse submanifold convolution layers. Video analytics tasks such as anomaly detection depend on the motion history at each pixel. To enable this, we also put forward a generic unsupervised deep learning solution to learn a novel memory surface known as Deep Learning (DL) memory surface. DL memory surface encodes the temporal information readily available from these sensors while retaining the sparsity of event data. Since there is no existing dataset for anomaly detection in the event domain, we also provide an anomaly detection event dataset with a set of anomalies. We empirically validate our anomaly detection architecture, composed of sparse convolutional layers, on this proposed and online dataset. Careful analysis of the anomaly detection network reveals that the presented method results in a massive reduction in computational complexity with good performance compared to previous state-of-the-art conventional frame-based anomaly detection networks.

scholarly journals Object-based illumination transferring and rendering for applications of mixed reality

2021 ◽  
Author(s):  
Di Xu ◽  
Zhen Li ◽  
Qi Cao
Keyword(s):  
Mixed Reality ◽  
Dynamic Range ◽  
High Dynamic Range ◽  
Virtual Object ◽  
Range Image ◽  
Unified Framework ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Planar Surfaces ◽  
Object Based

AbstractIn applications of augmented reality or mixed reality, rendering virtual objects in real scenes with consistent illumination is crucial for realistic visualization experiences. Prior learning-based methods reported in the literature usually attempt to reconstruct complicated high dynamic range environment maps from limited input, and rely on a separate rendering pipeline to light up the virtual object. In this paper, an object-based illumination transferring and rendering algorithm is proposed to tackle this problem within a unified framework. Given a single low dynamic range image, instead of recovering lighting environment of the entire scene, the proposed algorithm directly infers the relit virtual object. It is achieved by transferring implicit illumination features which are extracted from its nearby planar surfaces. A generative adversarial network is adopted in the proposed algorithm for implicit illumination features extraction and transferring. Compared to previous works in the literature, the proposed algorithm is more robust, as it is able to efficiently recover spatially varying illumination in both indoor and outdoor scene environments. Experiments have been conducted. It is observed that notable experiment results and comparison outcomes have been obtained quantitatively and qualitatively by the proposed algorithm in different environments. It shows the effectiveness and robustness for realistic virtual object insertion and improved realism.

scholarly journals Automatic Discovery of Railway Train Driving Modes Using Unsupervised Deep Learning

2019 ◽  
Vol 8 (7) ◽  
pp. 294
Author(s):  
Han Zheng ◽  
Zanyang Cui ◽  
Xingchen Zhang
Keyword(s):  
Deep Learning ◽  
Unsupervised Learning ◽  
Capacity Utilization ◽  
Learning Models ◽  
Trajectory Data ◽  
Generative Adversarial Network ◽  
Automatic Driving ◽  
Adversarial Network ◽  
Classical Models ◽  
Unsupervised Deep Learning

Driving modes play vital roles in understanding the stochastic nature of a railway system and can support studies of automatic driving and capacity utilization optimization. Integrated trajectory data containing information such as GPS trajectories and gear changes can be good proxies in the study of driving modes. However, in the absence of labeled data, discovering driving modes is challenging. In this paper, instead of classical models (railway-specified feature extraction and classical clustering), we used five deep unsupervised learning models to overcome this difficulty. In these models, adversarial autoencoders and stacked autoencoders are used as feature extractors, along with generative adversarial network-based and Kullback–Leibler (KL) divergence-based networks as clustering models. An experiment based on real and artificial datasets showed the following: (i) The proposed deep learning models outperform the classical models by 27.64% on average. (ii) Integrated trajectory data can improve the accuracy of unsupervised learning by approximately 13.78%. (iii) The different performance rankings of models based on indices with labeled data and indices without labeled data demonstrate the insufficiency of people’s understanding of the existing modes. This study also analyzes the relationship between the discovered modes and railway carrying capacity.

scholarly journals AFOD: An Adaptable Framework for Object Detection in Event-based Vision

2021 ◽  
Author(s):  
Shixiong Zhang ◽  
Wenmin Wang
Keyword(s):  
Computer Vision ◽  
Object Detection ◽  
Temporal Resolution ◽  
Dynamic Range ◽  
High Dynamic Range ◽  
Motion Blur ◽  
High Temporal Resolution ◽  
High Dynamic ◽  
Event Based ◽  
Conventional Computer

<div>Event-based vision is a novel bio-inspired vision that has attracted the interest of many researchers. As a neuromorphic vision, the sensor is different from the traditional frame-based cameras. It has such advantages that conventional frame-based cameras can’t match, e.g., high temporal resolution, high dynamic range(HDR), sparse and minimal motion blur. Recently, a lot of computer vision approaches have been proposed with demonstrated success. However, there is a lack of some general methods to expand the scope of the application of event-based vision. To be able to effectively bridge the gap between conventional computer vision and event-based vision, in this paper, we propose an adaptable framework for object detection in event-based vision.</div>

scholarly journals Range Imaging and Video Generation using Generative Adversarial Network

2021 ◽  
pp. 34-41
Author(s):  
Anderson Stephanie
Keyword(s):  
Dynamic Range ◽  
Journal Article ◽  
High Dynamic Range ◽  
Frame Rate ◽  
Primary Data ◽  
Variable Component ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Camera Systems ◽  
Event Camera

Latency, high temporal pixel density, and dynamic range are just a few of the benefits of event camera systems over conventional camera systems. Methods and algorithms cannot be applied directly because the output data of event camera systems are segments of synchronization events and experiences rather than precise pixel intensities. As a result, generating intensity photographs from occurrences for other functions is difficult. We use occurrence camera-based contingent deep convolutional connections to establish images and videos from a variable component of the occasion stream of data in this journal article. The system is designed to replicate visuals based on spatio-temporal intensity variations using bundles of spatial coordinates of occurrences as input data. The ability of event camera systems to produce High Dynamic Range (HDR) pictures even in exceptional lighting circumstances, as well as non-blurry pictures in rapid motion, is demonstrated. Furthermore, because event cameras have a transient response of about 1 s, the ability to generate very increased frame rate video content has been evidenced, conceivably up to 1 million arrays per second. The implementation of the proposed algorithms are compared to density images recorded onto a similar gridline in the image of events based on the application of accessible primary data obtained and synthesized datasets generated by the occurrence camera simulation model.

scholarly journals Anomaly Detection Neural Network with Dual Auto-Encoders GAN and Its Industrial Inspection Applications

Sensors ◽  
2020 ◽  
Vol 20 (12) ◽  
pp. 3336 ◽  
Author(s):  
Ta-Wei Tang ◽  
Wei-Han Kuo ◽  
Jauh-Hsiang Lan ◽  
Chien-Fang Ding ◽  
Hakiem Hsu ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Anomaly Detection ◽  
Training Data ◽  
Labor Costs ◽  
Generative Adversarial Network ◽  
Detection Ability ◽  
Adversarial Network ◽  
Automated Optical Inspection ◽  
High Level

Recently, researchers have been studying methods to introduce deep learning into automated optical inspection (AOI) systems to reduce labor costs. However, the integration of deep learning in the industry may encounter major challenges such as sample imbalance (defective products that only account for a small proportion). Therefore, in this study, an anomaly detection neural network, dual auto-encoder generative adversarial network (DAGAN), was developed to solve the problem of sample imbalance. With skip-connection and dual auto-encoder architecture, the proposed method exhibited excellent image reconstruction ability and training stability. Three datasets, namely public industrial detection training set, MVTec AD, with mobile phone screen glass and wood defect detection datasets, were used to verify the inspection ability of DAGAN. In addition, training with a limited amount of data was proposed to verify its detection ability. The results demonstrated that the areas under the curve (AUCs) of DAGAN were better than previous generative adversarial network-based anomaly detection models in 13 out of 17 categories in these datasets, especially in categories with high variability or noise. The maximum AUC improvement was 0.250 (toothbrush). Moreover, the proposed method exhibited better detection ability than the U-Net auto-encoder, which indicates the function of discriminator in this application. Furthermore, the proposed method had a high level of AUCs when using only a small amount of training data. DAGAN can significantly reduce the time and cost of collecting and labeling data when it is applied to industrial detection.

scholarly journals AFOD: An Adaptable Framework for Object Detection in Event-based Vision

2021 ◽  
Author(s):  
Shixiong Zhang ◽  
Wenmin Wang
Keyword(s):  
Computer Vision ◽  
Object Detection ◽  
Temporal Resolution ◽  
Dynamic Range ◽  
High Dynamic Range ◽  
Motion Blur ◽  
High Temporal Resolution ◽  
High Dynamic ◽  
Event Based ◽  
Conventional Computer

<div>Event-based vision is a novel bio-inspired vision that has attracted the interest of many researchers. As a neuromorphic vision, the sensor is different from the traditional frame-based cameras. It has such advantages that conventional frame-based cameras can’t match, e.g., high temporal resolution, high dynamic range(HDR), sparse and minimal motion blur. Recently, a lot of computer vision approaches have been proposed with demonstrated success. However, there is a lack of some general methods to expand the scope of the application of event-based vision. To be able to effectively bridge the gap between conventional computer vision and event-based vision, in this paper, we propose an adaptable framework for object detection in event-based vision.</div>

scholarly journals An Imbalanced Image Classification Method for the Cell Cycle Phase

Information ◽  
2021 ◽  
Vol 12 (6) ◽  
pp. 249
Author(s):  
Xin Jin ◽  
Yuanwen Zou ◽  
Zhongbing Huang
Keyword(s):  
Cell Cycle ◽  
Deep Learning ◽  
Image Classification ◽  
Classification Accuracy ◽  
Data Augmentation ◽  
Cycle Phase ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Cellular Life

The cell cycle is an important process in cellular life. In recent years, some image processing methods have been developed to determine the cell cycle stages of individual cells. However, in most of these methods, cells have to be segmented, and their features need to be extracted. During feature extraction, some important information may be lost, resulting in lower classification accuracy. Thus, we used a deep learning method to retain all cell features. In order to solve the problems surrounding insufficient numbers of original images and the imbalanced distribution of original images, we used the Wasserstein generative adversarial network-gradient penalty (WGAN-GP) for data augmentation. At the same time, a residual network (ResNet) was used for image classification. ResNet is one of the most used deep learning classification networks. The classification accuracy of cell cycle images was achieved more effectively with our method, reaching 83.88%. Compared with an accuracy of 79.40% in previous experiments, our accuracy increased by 4.48%. Another dataset was used to verify the effect of our model and, compared with the accuracy from previous results, our accuracy increased by 12.52%. The results showed that our new cell cycle image classification system based on WGAN-GP and ResNet is useful for the classification of imbalanced images. Moreover, our method could potentially solve the low classification accuracy in biomedical images caused by insufficient numbers of original images and the imbalanced distribution of original images.

NM-GAN: Noise-modulated generative adversarial network for video anomaly detection

2021 ◽  
Vol 116 ◽  
pp. 107969
Author(s):  
Dongyue Chen ◽  
Lingyi Yue ◽  
Xingya Chang ◽  
Ming Xu ◽  
Tong Jia
Keyword(s):  
Anomaly Detection ◽  
Generative Adversarial Network ◽  
Adversarial Network

AN AI-BASED APPROACH TO ENHANCED FRACTURE RESOLUTION IN IMAGE LOGS

2021 ◽  
Author(s):  
James Howard ◽  
◽  
Joe Tracey ◽  
Mike Shen ◽  
Shawn Zhang ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Nearest Neighbor ◽  
Rock Fracture ◽  
Short Interval ◽  
Acoustic Properties ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Deep Learning Neural Network ◽  
Borehole Image

Borehole image logs are used to identify the presence and orientation of fractures, both natural and induced, found in reservoir intervals. The contrast in electrical or acoustic properties of the rock matrix and fluid-filled fractures is sufficiently large enough that sub-resolution features can be detected by these image logging tools. The resolution of these image logs is based on the design and operation of the tools, and generally is in the millimeter per pixel range. Hence the quantitative measurement of actual width remains problematic. An artificial intelligence (AI) -based workflow combines the statistical information obtained from a Machine-Learning (ML) segmentation process with a multiple-layer neural network that defines a Deep Learning process that enhances fractures in a borehole image. These new images allow for a more robust analysis of fracture widths, especially those that are sub-resolution. The images from a BHTV log were first segmented into rock and fluid-filled fractures using a ML-segmentation tool that applied multiple image processing filters that captured information to describe patterns in fracture-rock distribution based on nearest-neighbor behavior. The robust ML analysis was trained by users to identify these two components over a short interval in the well, and then the regression model-based coefficients applied to the remaining log. Based on the training, each pixel was assigned a probability value between 1.0 (being a fracture) and 0.0 (pure rock), with most of the pixels assigned one of these two values. Intermediate probabilities represented pixels on the edge of rock-fracture interface or the presence of one or more sub-resolution fractures within the rock. The probability matrix produced a map or image of the distribution of probabilities that determined whether a given pixel in the image was a fracture or partially filled with a fracture. The Deep Learning neural network was based on a Conditional Generative Adversarial Network (cGAN) approach where the probability map was first encoded and combined with a noise vector that acted as a seed for diverse feature generation. This combination was used to generate new images that represented the BHTV response. The second layer of the neural network, the adversarial or discriminator portion, determined whether the generated images were representative of the actual BHTV by comparing the generated images with actual images from the log and producing an output probability of whether it was real or fake. This probability was then used to train the generator and discriminator models that were then applied to the entire log. Several scenarios were run with different probability maps. The enhanced BHTV images brought out fractures observed in the core photos that were less obvious in the original BTHV log through enhanced continuity and improved resolution on fracture widths.

Sign in / Sign up

Export Citation Format

Share Document