scholarly journals PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging

2021 ◽  
Author(s):  
Yuwei Li ◽  
Minye Wu ◽  
Yuyao Zhang ◽  
Lan Xu ◽  
Jingyi Yu
Keyword(s):  
Parametric Models ◽  
Data Driven ◽  
Resonance Imaging ◽  
Bone Model ◽  
Fine Grained ◽  
Network Layers ◽  
Hand Biomechanics ◽  
Hand Shape ◽  
Rgb Images ◽  
Hand Modeling

Hand modeling is critical for immersive VR/AR, action understanding, or human healthcare. Existing parametric models account only for hand shape, pose, or texture, without modeling the anatomical attributes like bone, which is essential for realistic hand biomechanics analysis. In this paper, we present PIANO, the first parametric bone model of human hands from MRI data. Our PIANO model is biologically correct, simple to animate, and differentiable, achieving more anatomically precise modeling of the inner hand kinematic structure in a data-driven manner than the traditional hand models based on the outer surface only. Furthermore, our PIANO model can be applied in neural network layers to enable training with a fine-grained semantic loss, which opens up the new task of data-driven fine-grained hand bone anatomic and semantic understanding from MRI or even RGB images. We make our model publicly available.

scholarly journals Explaining Neural Networks Using Attentive Knowledge Distillation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1280
Author(s):  
Hyeonseok Lee ◽  
Sungchan Kim
Keyword(s):  
Neural Networks ◽  
Model Prediction ◽  
Saliency Map ◽  
Model Parameters ◽  
Learning Capability ◽  
Fine Grained ◽  
Network Layers ◽  
Saliency Maps ◽  
Novel Approach ◽  
Knowledge Distillation

Explaining the prediction of deep neural networks makes the networks more understandable and trusted, leading to their use in various mission critical tasks. Recent progress in the learning capability of networks has primarily been due to the enormous number of model parameters, so that it is usually hard to interpret their operations, as opposed to classical white-box models. For this purpose, generating saliency maps is a popular approach to identify the important input features used for the model prediction. Existing explanation methods typically only use the output of the last convolution layer of the model to generate a saliency map, lacking the information included in intermediate layers. Thus, the corresponding explanations are coarse and result in limited accuracy. Although the accuracy can be improved by iteratively developing a saliency map, this is too time-consuming and is thus impractical. To address these problems, we proposed a novel approach to explain the model prediction by developing an attentive surrogate network using the knowledge distillation. The surrogate network aims to generate a fine-grained saliency map corresponding to the model prediction using meaningful regional information presented over all network layers. Experiments demonstrated that the saliency maps are the result of spatially attentive features learned from the distillation. Thus, they are useful for fine-grained classification tasks. Moreover, the proposed method runs at the rate of 24.3 frames per second, which is much faster than the existing methods by orders of magnitude.

scholarly journals Multi-Task Driven Feature Models for Thermal Infrared Tracking

2020 ◽  
Vol 34 (07) ◽  
pp. 11604-11611 ◽  
Author(s):  
Qiao Liu ◽  
Xin Li ◽  
Zhenyu He ◽  
Nana Fan ◽  
Di Yuan ◽  
...  
Keyword(s):  
Large Scale ◽  
State Of The Art ◽  
Thermal Infrared ◽  
Training Dataset ◽  
Feature Models ◽  
Relative Gain ◽  
Tir Domain ◽  
Fine Grained ◽  
Rgb Images ◽  
Infrared Tracking

Existing deep Thermal InfraRed (TIR) trackers usually use the feature models of RGB trackers for representation. However, these feature models learned on RGB images are neither effective in representing TIR objects nor taking fine-grained TIR information into consideration. To this end, we develop a multi-task framework to learn the TIR-specific discriminative features and fine-grained correlation features for TIR tracking. Specifically, we first use an auxiliary classification network to guide the generation of TIR-specific discriminative features for distinguishing the TIR objects belonging to different classes. Second, we design a fine-grained aware module to capture more subtle information for distinguishing the TIR objects belonging to the same class. These two kinds of features complement each other and recognize TIR objects in the levels of inter-class and intra-class respectively. These two feature models are learned using a multi-task matching framework and are jointly optimized on the TIR tracking task. In addition, we develop a large-scale TIR training dataset to train the network for adapting the model to the TIR domain. Extensive experimental results on three benchmarks show that the proposed algorithm achieves a relative gain of 10% over the baseline and performs favorably against the state-of-the-art methods. Codes and the proposed TIR dataset are available at https://github.com/QiaoLiuHit/MMNet.

scholarly journals Fine-Grained Recognition: Accounting for Subtle Differences between Similar Classes

2020 ◽  
Vol 34 (07) ◽  
pp. 12047-12054
Author(s):  
Guolei Sun ◽  
Hisham Cholakkal ◽  
Salman Khan ◽  
Fahad Khan ◽  
Ling Shao
Keyword(s):  
Deep Learning ◽  
Loss Function ◽  
Recognition Task ◽  
Data Driven ◽  
Gradient Boosting ◽  
Correct Classification ◽  
Experimental Setting ◽  
Fine Grained ◽  
Feature Representations ◽  
Salient Features

The main requisite for fine-grained recognition task is to focus on subtle discriminative details that make the subordinate classes different from each other. We note that existing methods implicitly address this requirement and leave it to a data-driven pipeline to figure out what makes a subordinate class different from the others. This results in two major limitations: First, the network focuses on the most obvious distinctions between classes and overlooks more subtle inter-class variations. Second, the chance of misclassifying a given sample in any of the negative classes is considered equal, while in fact, confusions generally occur among only the most similar classes. Here, we propose to explicitly force the network to find the subtle differences among closely related classes. In this pursuit, we introduce two key novelties that can be easily plugged into existing end-to-end deep learning pipelines. On one hand, we introduce “diversification block” which masks the most salient features for an input to force the network to use more subtle cues for its correct classification. Concurrently, we introduce a “gradient-boosting” loss function that focuses only on the confusing classes for each sample and therefore moves swiftly along the direction on the loss surface that seeks to resolve these ambiguities. The synergy between these two blocks helps the network to learn more effective feature representations. Comprehensive experiments are performed on five challenging datasets. Our approach outperforms existing methods using similar experimental setting on all five datasets.

scholarly journals Automated movement assessment in stroke rehabilitation

2021 ◽  
Author(s):  
Tamim Ahmed ◽  
Kowshik Thopalli ◽  
Thanassis Rikakis ◽  
Pavan Turaga ◽  
Aisling Kelliher ◽  
...  
Keyword(s):  
Upper Extremity ◽  
Hierarchical Model ◽  
Stroke Rehabilitation ◽  
Expert Knowledge ◽  
Low Cost ◽  
Data Driven ◽  
Task Completion ◽  
Convolutional Network ◽  
Rgb Images ◽  
High Level

We are developing a system for long-term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high-level constraints relating to activity structure (i.e. type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high-level priors to data-driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data-driven techniques. We use a transformer-based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complementary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce robust segmentation and task assessment results on noisy, variable, and limited data, which is characteristic of low-cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification, and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e. lower extremity training for neurological accidents).

scholarly journals Neural Hierarchy of Color Categorization: From Prototype Encoding to Boundary Encoding

2021 ◽  
Vol 15 ◽  
Author(s):  
Mengdan Sun ◽  
Luming Hu ◽  
Xiaoyang Xin ◽  
Xuemin Zhang
Keyword(s):  
Visual Cortex ◽  
Category Boundary ◽  
Functional Magnetic Resonance ◽  
Resonance Imaging ◽  
Fine Grained ◽  
Color Categories ◽  
Color Categorization ◽  
Category Variability ◽  
Candidate Loci ◽  
The Brain

A long-standing debate exists on how our brain assigns the fine-grained perceptual representation of color into discrete color categories. Recent functional magnetic resonance imaging (fMRI) studies have identified several regions as the candidate loci of color categorization, including the visual cortex, language-related areas, and non-language-related frontal regions, but the evidence is mixed. Distinct from most studies that emphasized the representational differences between color categories, the current study focused on the variability among members within a category (e.g., category prototypes and boundaries) to reveal category encoding in the brain. We compared and modeled brain activities evoked by color stimuli with varying distances from the category boundary in an active categorization task. The frontal areas, including the inferior and middle frontal gyri, medial superior frontal cortices, and insular cortices, showed larger responses for colors near the category boundary than those far from the boundary. In addition, the visual cortex encodes both within-category variability and cross-category differences. The left V1 in the calcarine showed greater responses to colors at the category center than to those far from the boundary, and the bilateral V4 showed enhanced responses for colors at the category center as well as colors around the boundary. The additional representational similarity analyses (RSA) revealed that the bilateral insulae and V4a carried information about cross-category differences, as cross-category colors exhibited larger dissimilarities in brain patterns than within-category colors. Our study suggested a hierarchically organized network in the human brain during active color categorization, with frontal (both lateral and medial) areas supporting domain-general decisional processes and the visual cortex encoding category structure and differences, likely due to top-down modulation.

scholarly journals Estimating the size of undetected cases of the SARS-CoV-2 outbreak in Europe: An upper bound estimator

2020 ◽  
Author(s):  
Irene Rocchetti ◽  
Dankmar Boehning ◽  
Heinz Holling ◽  
Antonello Maruotti
Keyword(s):  
Lower Bound ◽  
Population Size ◽  
Upper Bound ◽  
Population Based ◽  
Parametric Models ◽  
European Countries ◽  
Data Driven ◽  
Open Population ◽  
Country Level ◽  
True Number

Background: While the number of detected SARS-CoV-2 infections are widely available, an understanding of the extent of undetected cases is urgently needed for an effective tackling of the pandemic. The aim of this work is to estimate the true number of SARS-CoV-2 (detected and undetected) infections in several European Countries. The question being asked is: How many cases have actually occurred? Methods: We propose an upper bound estimator under cumulative data distributions, in an open population, based on a day-wise estimator that allows for heterogeneity. The estimator is data-driven and can be easily computed from the distributions of daily cases and deaths. Uncertainty surrounding the estimates is obtained using bootstrap methods. Results: We focus on the ratio of the total estimated cases to the observed cases at April 17th. Differences arise at the Country level, and we get estimates ranging from the 3.93 times of Norway to the 7.94 times of France. Accurate estimates are obtained, as bootstrap based intervals are rather narrow. Conclusions: Many parametric or semi-parametric models have been developed to estimate the population size from aggregated counts leading to an approximation of the missed population and/or to the estimate of the threshold under which the number of missed people cannot fall (i.e. a lower bound). Here, we provide a methodological contribution introducing an upper bound estimator and provide reliable estimates on the dark number, i.e. how many undetected cases are going around for several European Countries, where the epidemic spreads differently.

A Two Consequent Multi-layers Deep Discriminative Approach for Classifying fMRI Images

2020 ◽  
Vol 29 (06) ◽  
pp. 2030001
Author(s):  
Abeer M. Mahmoud ◽  
Hanen Karamti ◽  
Fadwa Alrowais
Keyword(s):  
Feature Space ◽  
Machine Learning Algorithms ◽  
Data Driven ◽  
Functional Magnetic Resonance ◽  
Resonance Imaging ◽  
Brain Disorder ◽  
Medical Problems ◽  
Discriminative Approach ◽  
Large Brain ◽  
Data Driven Approach

Functional Magnetic Resonance Imaging (fMRI), for many decades acts as a potential aiding method for diagnosing medical problems. Several successful machine learning algorithms have been proposed in literature to extract valuable knowledge from fMRI. One of these algorithms is the convolutional neural network (CNN) that competent with high capabilities for learning optimal abstractions of fMRI. This is because the CNN learns features similarly to human brain where it preserves local structure and avoids distortion of the global feature space. Focusing on the achievements of using the CNN for the fMRI, and accordingly, the Deep Convolutional Auto-Encoder (DCAE) benefits from the data-driven approach with CNN’s optimal features to strengthen the fMRI classification. In this paper, a new two consequent multi-layers DCAE deep discriminative approach for classifying fMRI Images is proposed. The first DCAE is unsupervised sub-model that is composed of four CNN. It focuses on learning weights to utilize discriminative characteristics of the extracted features for robust reconstruction of fMRI with lower dimensional considering tiny details and refining by its deep multiple layers. Then the second DCAE is a supervised sub-model that focuses on training labels to reach an outperformed results. The proposed approach proved its effectiveness and improved literately reported results on a large brain disorder fMRI dataset.

Sign in / Sign up

Export Citation Format

Share Document