Deep Transfer Learning Based Human Activity Recognition By Transforming IMU Data To Image Domain Using Novel Activity Image Creation Method

Human Activity Recognition is the most popular research area in the pervasive computing field in recent years. Sensor data plays a vital role in identifying several human actions. Convolutional Neural Networks (CNNs) have now become the most recent technique in the computer vision phenomenon, but still it is premature to use CNN for sensor data, particularly in ubiquitous and wearable computing. In this paper, we have proposed the idea of transforming the raw accelerometer and gyroscope sensor data to the visual domain by using our novel activity image creation method (NAICM). Pre-trained CNN (AlexNet) has been used on the converted image domain information. The proposed method is evaluated on several online available human activity recognition dataset. The results show that the proposed novel activity image creation method (NAICM) has successfully created the activity images with a classification accuracy of 98.36% using pre trained CNN.

Music in All Its Beauty

This chapter provides a behind-the-scenes account of the birth of a naturalistic approach to the neuroscience of the musical aesthetic experience. The story starts from a lab talk giving the inspiration to translate the naturalistic paradigm initially applied to neuroimaging studies of the visual domain into music research. The circumstantial co-presence of neuroscientists and computational musicologists at the same center did the trick, permitting the identification of controlled variables for brain signal processing from the automatic extraction of the acoustic features of real music. This approach is now well accepted by the music neuroscience community while still waiting for full exploitation by aesthetic research.

Deep visual domain adaptation and semi-supervised segmentation for understanding wave elevation using wave flume video images

Accurate water surface elevation estimation is essential for understanding nearshore processes, but it is still challenging due to limitations in measuring water level using in-situ acoustic sensors. This paper presents a vision-based water surface elevation estimation approach using multi-view datasets. Specifically, we propose a visual domain adaptation method to build a water level estimator in spite of a situation in which ocean wave height cannot be measured directly. We also implemented a semi-supervised approach to extract wave height information from long-term sequences of wave height observations with minimal supervision. We performed wave flume experiments in a hydraulic laboratory with two cameras with side and top viewpoints to validate the effectiveness of our approach. The performance of the proposed models were evaluated by comparing the estimated time series of water elevation with the ground-truth wave gauge data at three locations along the wave flume. The estimated time series were in good agreement within the averaged correlation coefficient of 0.98 and 0.90 on the measurement and 0.95 and 0.85 on the estimation for regular and irregular waves, respectively.

ViSpa (Vision Spaces): A computer-vision-based representation system for individual images and concept prototypes, with large-scale evaluation

Quantitative, data-driven models for mental representations have long enjoyed popularity and success in psychology (for example, distributional semantic models in the language domain), but have largely been missing for the visual domain. To overcome this, we present ViSpa (Vision Spaces), high-dimensional vector spaces that include vision-based representation for naturalistic images as well as concept prototypes. These vectors are derived directly from visual stimuli through a deep convolutional neural network (DCNN) trained to classify images, and allow us to compute vision-based similarity scores between any pair of images and/or concept prototypes. We successfully evaluate these similarities against human behavioral data in a series of large-scale studies, including off-line judgments – visual similarity judgments for the referents of word pairs (Study 1) and for image pairs (Study 2), and typicality judgments for images given a label (Study 3) – as well as on-line processing times and error rates in a discrimination (Study 4) and priming task (Study 5) with naturalistic image material. ViSpa similarities predict behavioral data across all tasks, which renders ViSpa a theoretically appealing model for vision-based representations and a valuable research tool for data analysis and the construction of experimental material: ViSpa allows for precise control over experimental material consisting of images (also in combination with words), and introduces a specifically vision-based similarity for word pairs. To make ViSpa available to a wide audience, this article a) includes (video) tutorials on how to use ViSpa in R, and b) presents a user-friendly web interface at http://vispa.fritzguenther.de.

Secure Audio-Visual Data Exchange for Android In-Vehicle Ecosystems

Mobile device pairing inside vehicles is a ubiquitous task which requires easy to use and secure solutions. In this work we exploit the audio-video domain for pairing devices inside vehicles. In principle, we rely on the widely used elliptical curve version of the Diffie-Hellman key-exchange protocol and extract the session keys from the acoustic domain as well as from the visual domain by using the head unit display. The need for merging the audio-visual domains first stems from the fact that in-vehicle head units generally do not have a camera so they cannot use visual data from smartphones, however, they are equipped with microphones and can use them to collect audio data. Acoustic channels are less reliable as they are more prone to errors due to environmental noise. However, this noise can be also exploited in a positive way to extract secure seeds from the environment and audio channels are harder to intercept from the outside. On the other hand, visual channels are more reliable but can be more easily spotted by outsiders, so they are more vulnerable for security applications. Fortunately, mixing these two types of channels results in a solution that is both more reliable and secure for performing a key exchange.