Estimation of in-scanner head pose changes during structural MRI using a convolutional neural network trained on eye tracker video

Autism spectrum disorder (ASD) is a range of neurodevelopmental disorders with behavioral and cognitive impairment and brings huge burdens to the patients’ families and the society. To accurately identify patients with ASD from typical controls is important for early detection and early intervention. However, almost all the current existing classification methods for ASD based on structural MRI (sMRI) mainly utilize the independent local morphological features and do not consider the covariance patterns of these features between regions. In this study, by combining the convolutional neural network (CNN) and individual structural covariance network, we proposed a new framework to classify ASD patients with sMRI data from the ABIDE consortium. Moreover, gradient-weighted class activation mapping (Grad-CAM) was applied to characterize the weight of features contributing to the classification. The experimental results showed that our proposed method outperforms the currently used methods for classifying ASD patients with the ABIDE data and achieves a high classification accuracy of 71.8% across different sites. Furthermore, the discriminative features were found to be mainly located in the prefrontal cortex and cerebellum, which may be the early biomarkers for the diagnosis of ASD. Our study demonstrated that CNN is an effective tool to build the framework for the diagnosis of ASD with individual structural covariance brain network.

Download Full-text

Head pose estimation with soft labels using regularized convolutional neural network

Neurocomputing ◽

10.1016/j.neucom.2018.12.074 ◽

2019 ◽

Vol 337 ◽

pp. 339-353 ◽

Cited By ~ 3

Author(s):

Luhui Xu ◽

Jingying Chen ◽

Yanling Gan

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Pose Estimation ◽

Head Pose Estimation ◽

Head Pose

Download Full-text

Estimation of in-scanner head pose changes during structural MRI using a convolutional neural network trained on eye tracker video

10.1101/2021.03.04.433975 ◽

2021 ◽

Author(s):

Heath R Pardoe ◽

Samantha P Martin ◽

Yijun Zhao ◽

Allan George ◽

Hui Yuan ◽

...

Keyword(s):

Predictive Model ◽

Cortical Thickness ◽

Structural Mri ◽

Video Data ◽

Head Motion ◽

Eye Tracker ◽

Head Pose ◽

Mri Scans ◽

Volume Estimates ◽

Subcortical Volume

Introduction In-scanner head motion is a common cause of reduced image quality in neuroimaging, and causes systematic brain-wide changes in cortical thickness and volumetric estimates derived from structural MRI scans. There are currently no widely available methods for measuring head motion during structural MRI. Here, we train a deep learning predictive model to estimate changes in head pose using video obtained from an in-scanner eye tracker during an EPI-BOLD acquisition with participants undertaking deliberate in-scanner head movements. The predictive model was used to estimate head pose changes during structural MRI scans, and correlated with cortical thickness and subcortical volume estimates. Methods 21 healthy controls (age 32 ± 13 years, 11 female) were studied. Participants carried out a series of stereotyped prompted in-scanner head motions during acquisition of an EPI-BOLD sequence with simultaneous recording of eye tracker video. Motion-affected and motion-free whole brain T1-weighted MRI were also obtained. Image coregistration was used to estimate changes in head pose over the duration of the EPI-BOLD scan, and used to train a predictive model to estimate head pose changes from the video data. Model performance was quantified by assessing the coefficient of determination (R²). We evaluated the utility of our technique by assessing the relationship between video-based head pose changes during structural MRI and (i) vertex-wise cortical thickness and (ii) subcortical volume estimates. Results Video-based head pose estimates were significantly correlated with ground truth head pose changes estimated from EPI-BOLD imaging in a hold-out dataset. We observed a general brain-wide overall reduction in cortical thickness with increased head motion, with some isolated regions showing increased cortical thickness estimates with increased motion. Subcortical volumes were generally reduced in motion affected scans. Conclusions We trained a predictive model to estimate changes in head pose during structural MRI scans using in-scanner eye tracker video. The method is independent of individual image acquisition parameters and does not require markers to be to be fixed to the patient, suggesting it may be well suited to clinical imaging and research environments. Head pose changes estimated using our approach can be used as covariates for morphometric image analyses to improve the neurobiological validity of structural imaging studies of brain development and disease.

Download Full-text

Image Super-Resolution for MRI Images using 3D Faster Super-Resolution Convolutional Neural Network architecture

ITM Web of Conferences ◽

10.1051/itmconf/20203203044 ◽

2020 ◽

Vol 32 ◽

pp. 03044

Author(s):

Vanita Mane ◽

Suchit Jadhav ◽

Praneya Lal

Keyword(s):

Neural Network ◽

High Resolution ◽

Convolutional Neural Network ◽

Network Architecture ◽

Brain Mri ◽

Super Resolution ◽

Structural Mri ◽

Reconstruction Performance ◽

Human Connectome Project ◽

Image Super Resolution

Single image super-resolution using deep learning techniques has shown very high reconstruction performance over the last few years. We propose a novel three-dimensional convolutional neural network called 3D FSRCNN based on FSRCNN, which reinstates the high-resolution quality of structural MRI. The 3D neural network generates output brain images of high-resolution (HR) from a low-resolution (LR) input image. A simple design ensures less time complexity and high reconstruction quality. The network is trained using T1-weighted structural MRI images from the human connectome project dataset which is a large publicly available brain MRI database.

Download Full-text

Head Pose Estimation Based on Robust Convolutional Neural Network

Cybernetics and Information Technologies ◽

10.1515/cait-2016-0083 ◽

2016 ◽

Vol 16 (6) ◽

pp. 133-145 ◽

Cited By ~ 5

Author(s):

Jiao Bao ◽

Mao Ye

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Pose Estimation ◽

Network Architecture ◽

Facial Feature ◽

Head Pose Estimation ◽

Head Pose ◽

Whole Process ◽

Benchmark Datasets ◽

Better Than

Abstract Head pose estimation plays an important role in face recognition. However, it faces vast challenges on account of the initialization, facial feature points’ location accuracy and so on. Inspired by the observation that head pose angles change smoothly and continuously, we present a method based on a robust convolutional neural network for head pose estimation. The proposed network architecture consists of three levels and each level has three convolutional neural networks. The first level is a global one; it predicts the head pose quickly as a preliminary estimation. The following two levels are local ones; they refine the estimation achieved from the previous level step by step. Higher and higher resolution image with different input regions are taken as input in our network. At last, a multi-level regression is employed to combine the estimations from each level. The whole process is conducted in a cascade way to improve the head pose estimation performance directly with three angles together. We perform large experiments on nine challenging benchmark datasets. The experimental results demonstrate that our method performs better than the compared methods.

Download Full-text