Multiple Instance Deep Learning for Weakly Supervised Small-Footprint Audio Event Detection

In training a deep learning system to perform audio transcription, two practical problems may arise. Firstly, most datasets are weakly labelled, having only a list of events present in each recording without any temporal information for training. Secondly, deep neural networks need a very large amount of labelled training data to achieve good quality performance, yet in practice it is difficult to collect enough samples for most classes of interest. In this paper, we propose factorising the final task of audio transcription into multiple intermediate tasks in order to improve the training performance when dealing with this kind of low-resource datasets. We evaluate three data-efficient approaches of training a stacked convolutional and recurrent neural network for the intermediate tasks. Our results show that different methods of training have different advantages and disadvantages.

Download Full-text

Deep Convolutional Neural Network with Structured Prediction for Weakly Supervised Audio Event Detection

Applied Sciences ◽

10.3390/app9112302 ◽

2019 ◽

Vol 9 (11) ◽

pp. 2302 ◽

Cited By ~ 2

Author(s):

Inkyu Choi ◽

Soo Hyun Bae ◽

Nam Soo Kim

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Event Detection ◽

Large Scale ◽

Prediction Method ◽

Deep Convolutional Neural Network ◽

Structured Prediction ◽

Temporal Position ◽

Audio Event ◽

Weakly Supervised

Audio event detection (AED) is a task of recognizing the types of audio events in an audio stream and estimating their temporal positions. AED is typically based on fully supervised approaches, requiring strong labels including both the presence and temporal position of each audio event. However, fully supervised datasets are not easily available due to the heavy cost of human annotation. Recently, weakly supervised approaches for AED have been proposed, utilizing large scale datasets with weak labels including only the occurrence of events in recordings. In this work, we introduce a deep convolutional neural network (CNN) model called DSNet based on densely connected convolution networks (DenseNets) and squeeze-and-excitation networks (SENets) for weakly supervised training of AED. DSNet alleviates the vanishing-gradient problem and strengthens feature propagation and models interdependencies between channels. We also propose a structured prediction method for weakly supervised AED. We apply a recurrent neural network (RNN) based framework and a prediction smoothness cost function to consider long-term contextual information with reduced error propagation. In post-processing, conditional random fields (CRFs) are applied to take into account the dependency between segments and delineate the borders of audio events precisely. We evaluated our proposed models on the DCASE 2017 task 4 dataset and obtained state-of-the-art results on both audio tagging and event detection tasks.

Download Full-text

On the Performance Improvements of Deep Learning Methods for Audio Event Detection and Classification

10.1109/tsp52935.2021.9522625 ◽

2021 ◽

Author(s):

Claudio Eutizi ◽

Francesco Benedetto

Keyword(s):

Deep Learning ◽

Event Detection ◽

Learning Methods ◽

Performance Improvements ◽

Audio Event

Download Full-text

Saliency detection in deep learning era: trends of development

Information and Control Systems ◽

10.31799/1684-8853-2019-3-10-36 ◽

2019 ◽

pp. 10-36 ◽

Cited By ~ 2

Author(s):

M. N. Favorskaya ◽

L. C. Jain

Keyword(s):

Deep Learning ◽

Object Detection ◽

Event Detection ◽

Visual Analysis ◽

Saliency Detection ◽

Salient Object Detection ◽

Public Image ◽

Detection Methods ◽

Salient Object ◽

Salient Event

Introduction:Saliency detection is a fundamental task of computer vision. Its ultimate aim is to localize the objects of interest that grab human visual attention with respect to the rest of the image. A great variety of saliency models based on different approaches was developed since 1990s. In recent years, the saliency detection has become one of actively studied topic in the theory of Convolutional Neural Network (CNN). Many original decisions using CNNs were proposed for salient object detection and, even, event detection.Purpose:A detailed survey of saliency detection methods in deep learning era allows to understand the current possibilities of CNN approach for visual analysis conducted by the human eyes’ tracking and digital image processing.Results:A survey reflects the recent advances in saliency detection using CNNs. Different models available in literature, such as static and dynamic 2D CNNs for salient object detection and 3D CNNs for salient event detection are discussed in the chronological order. It is worth noting that automatic salient event detection in durable videos became possible using the recently appeared 3D CNN combining with 2D CNN for salient audio detection. Also in this article, we have presented a short description of public image and video datasets with annotated salient objects or events, as well as the often used metrics for the results’ evaluation.Practical relevance:This survey is considered as a contribution in the study of rapidly developed deep learning methods with respect to the saliency detection in the images and videos.

Download Full-text