Blade-vortex interaction detection and extraction under deep neural network-based scale feature model

Inherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information.

Download Full-text

Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior

10.1101/207530 ◽

2017 ◽

Author(s):

Iris I. A. Groen ◽

Michelle R. Greene ◽

Christopher Baldassano ◽

Li Fei-Fei ◽

Diane M. Beck ◽

...

Keyword(s):

Neural Network ◽

Deep Neural Network ◽

A Priori ◽

Feature Model ◽

Semantic Features ◽

Functional Feature ◽

Scene Representation ◽

Behavioral Assessments ◽

Brain And Behavior ◽

High Level

AbstractInherent correlations between visual and semantic features in real-world scenes make it difficult to determine how different scene properties contribute to neural representations. Here, we assessed the contributions of multiple properties to scene representation by partitioning the variance explained in human behavioral and brain measurements by three feature models whose inter-correlations were minimized a priori through stimulus preselection. Behavioral assessments of scene similarity reflected unique contributions from a functional feature model indicating potential actions in scenes as well as high-level visual features from a deep neural network (DNN). In contrast, similarity of cortical responses in scene-selective areas was uniquely explained by mid- and high-level DNN features only, while an object label model did not contribute uniquely to either domain. The striking dissociation between functional and DNN features in their contribution to behavioral and brain representations of scenes indicates that scene-selective cortex represents only a subset of behaviorally relevant scene information.

Download Full-text

Exploring Methods for Predicting Important Utterances Contributing to Meeting Summarization

Multimodal Technologies and Interaction ◽

10.3390/mti3030050 ◽

2019 ◽

Vol 3 (3) ◽

pp. 50

Author(s):

Nihei ◽

Nakano

Keyword(s):

Neural Network ◽

Deep Learning ◽

Deep Neural Network ◽

User Study ◽

Prediction Models ◽

Network Models ◽

Feature Model ◽

Feature Models ◽

Neural Network Models ◽

F Measure

Meeting minutes are useful, but creating meeting summaries are a time consuming task. Aiming at supporting such task, this paper proposes prediction models for important utterances that should be included in the meeting summary by using multimodal and multiparty features. We will tackle this issue from two approaches: Handcrafted feature models and deep neural network models. The best handcrafted feature model achieved 0.707 in F-measure, and the best deep-learning based verbal and nonverbal model (V-NV model) achieved 0.827 in F-measure. Based on the V-NV model, we implemented a meeting browser, and conducted a user study. The results showed that the proposed meeting browser better contributes to the understanding of the content of the discussion and the participant roles in the discussion than the conventional text-based browser.

Download Full-text