scholarly journals Natural Image Reconstruction From fMRI Using Deep Learning: A Survey

2021 ◽  
Vol 15 ◽  
Author(s):  
Zarina Rakhimberdina ◽  
Quentin Jodelet ◽  
Xin Liu ◽  
Tsuyoshi Murata
Keyword(s):  
Deep Learning ◽  
Image Reconstruction ◽  
Visual Information ◽  
Architectural Design ◽  
Computational Models ◽  
Imaging Techniques ◽  
Evaluation Metrics ◽  
Natural Image ◽  
Learning Tools ◽  
Benchmark Datasets

With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics and present a fair performance evaluation across standardized evaluation metrics. Finally, we discuss the strengths and limitations of existing studies and present potential future directions.

scholarly journals Deep image reconstruction from human brain activity

2017 ◽  
Author(s):  
Guohua Shen ◽  
Tomoyasu Horikawa ◽  
Kei Majima ◽  
Yukiyasu Kamitani
Keyword(s):  
Neural Network ◽  
Image Reconstruction ◽  
Human Brain ◽  
Visual Information ◽  
Deep Neural Network ◽  
Brain Activity ◽  
Natural Images ◽  
Perceptual Content ◽  
Natural Image ◽  
Reconstruction Method

AbstractMachine learning-based analysis of human functional magnetic resonance imaging (fMRI) patterns has enabled the visualization of perceptual content. However, it has been limited to the reconstruction with low-level image bases (Miyawaki et al., 2008; Wen et al., 2016) or to the matching to exemplars (Naselaris et al., 2009; Nishimoto et al., 2011). Recent work showed that visual cortical activity can be decoded (translated) into hierarchical features of a deep neural network (DNN) for the same input image, providing a way to make use of the information from hierarchical visual features (Horikawa & Kamitani, 2017). Here, we present a novel image reconstruction method, in which the pixel values of an image are optimized to make its DNN features similar to those decoded from human brain activity at multiple layers. We found that the generated images resembled the stimulus images (both natural images and artificial shapes) and the subjective visual content during imagery. While our model was solely trained with natural images, our method successfully generalized the reconstruction to artificial shapes, indicating that our model indeed ‘reconstructs’ or ‘generates’ images from brain activity, not simply matches to exemplars. A natural image prior introduced by another deep neural network effectively rendered semantically meaningful details to reconstructions by constraining reconstructed images to be similar to natural images. Furthermore, human judgment of reconstructions suggests the effectiveness of combining multiple DNN layers to enhance visual quality of generated images. The results suggest that hierarchical visual information in the brain can be effectively combined to reconstruct perceptual and subjective images.

scholarly journals Image Reconstruction for High-Performance Electrical Capacitance Tomography System Using Deep Learning

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Yanpeng Zhang ◽  
Deyun Chen
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Image Reconstruction ◽  
High Performance ◽  
Imaging Techniques ◽  
Electrical Capacitance Tomography ◽  
Imaging Method ◽  
Electrical Capacitance ◽  
Artificial Neural ◽  
Capacitance Tomography

For great achievements in recent decades, image reconstruction for electrical capacitance tomography (ECT) has been considered in this study. ECT has demonstrated impressive potentials in multiprocess measurement, and the obtained images are of high resolution, which are suitable for advanced procedures in industrial and medical applications and across different tasks and domains. But the ECT system still requires improvements in the quality of image reconstruction given its importance of great significance to obtain the reliability and usefulness of measurement results. The deep neural network is used in this study to extract new features and to update the number of nodes and hidden layers in the system. Recently, deep learning exhibits suitable solutions in many flourishing fields based on different series of artificial neural networks for mapping nonlinear functions. To address the obstacles, this paper proposes an imaging method using an optimizer reconstruction model. An optimization model for imaging is generated as a powerful optimizer for building a computational model to ameliorate the reconstruction accuracy. Based on the deep learning methodology, the previous images reconstructed by using one of the imaging techniques to the required images are abstracted and stored in the deep learning machine, resulting in an error rate of 8.9%, and this is considered good on ECT. Therefore, an artificial neural network of the capacitance (ANNoC) system is introduced to estimate capacitance measurements.

Towards Computational Models of Identifying Protein Ubiquitination Sites

2019 ◽  
Vol 20 (5) ◽  
pp. 565-578 ◽  
Author(s):  
Lidong Wang ◽  
Ruijun Zhang
Keyword(s):  
Computational Methods ◽  
Computational Models ◽  
Feature Representation ◽  
Biological Sequence ◽  
Post Translational Modification ◽  
Test Dataset ◽  
Protein Ubiquitination ◽  
Protein Functions ◽  
Independent Test Dataset ◽  
Benchmark Datasets

Ubiquitination is an important post-translational modification (PTM) process for the regulation of protein functions, which is associated with cancer, cardiovascular and other diseases. Recent initiatives have focused on the detection of potential ubiquitination sites with the aid of physicochemical test approaches in conjunction with the application of computational methods. The identification of ubiquitination sites using laboratory tests is especially susceptible to the temporality and reversibility of the ubiquitination processes, and is also costly and time-consuming. It has been demonstrated that computational methods are effective in extracting potential rules or inferences from biological sequence collections. Up to the present, the computational strategy has been one of the critical research approaches that have been applied for the identification of ubiquitination sites, and currently, there are numerous state-of-the-art computational methods that have been developed from machine learning and statistical analysis to undertake such work. In the present study, the construction of benchmark datasets is summarized, together with feature representation methods, feature selection approaches and the classifiers involved in several previous publications. In an attempt to explore pertinent development trends for the identification of ubiquitination sites, an independent test dataset was constructed and the predicting results obtained from five prediction tools are reported here, together with some related discussions.

An Anatomy of a Hybrid Color Descriptor with a Neural Network Model to Enhance the Retrieval Accuracy of an Image Retrieval System

2019 ◽  
Vol 13 ◽  
Author(s):  
Shikha Bhardwaj ◽  
Gitanjali Pandove ◽  
Pawan Kumar Dahiya
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Image Retrieval ◽  
Hybrid System ◽  
Back Propagation ◽  
Back Propagation Neural Network ◽  
Retrieval Accuracy ◽  
Color Descriptor ◽  
Benchmark Datasets ◽  
Color Moment

Background: In order to retrieve a particular image from vast repository of images, an efficient system is required and such an eminent system is well-known by the name Content-based image retrieval (CBIR) system. Color is indeed an important attribute of an image and the proposed system consist of a hybrid color descriptor which is used for color feature extraction. Deep learning, has gained a prominent importance in the current era. So, the performance of this fusion based color descriptor is also analyzed in the presence of Deep learning classifiers. Method: This paper describes a comparative experimental analysis on various color descriptors and the best two are chosen to form an efficient color based hybrid system denoted as combined color moment-color autocorrelogram (Co-CMCAC). Then, to increase the retrieval accuracy of the hybrid system, a Cascade forward back propagation neural network (CFBPNN) is used. The classification accuracy obtained by using CFBPNN is also compared to Patternnet neural network. Results: The results of the hybrid color descriptor depict that the proposed system has superior results of the order of 95.4%, 88.2%, 84.4% and 96.05% on Corel-1K, Corel-5K, Corel-10K and Oxford flower benchmark datasets respectively as compared to many state-of-the-art related techniques. Conclusion: This paper depict an experimental and analytical analysis on different color feature descriptors namely, Color moment (CM), Color auto-correlogram (CAC), Color histogram (CH), Color coherence vector (CCV) and Dominant color descriptor (DCD). The proposed hybrid color descriptor (Co-CMCAC) is utilized for the withdrawal of color features with Cascade forward back propagation neural network (CFBPNN) is used as a classifier on four benchmark datasets namely Corel-1K, Corel-5K and Corel-10K and Oxford flower.

Deep learning in magnetic resonance image reconstruction

2021 ◽  
Author(s):  
Shekhar S Chandra ◽  
Marlon Bran Lorenzana ◽  
Xinwen Liu ◽  
Siyu Liu ◽  
Steffen Bollmann ◽  
...  
Keyword(s):  
Deep Learning ◽  
Magnetic Resonance ◽  
Image Reconstruction ◽  
Magnetic Resonance Image

scholarly journals Real-Time Environment Monitoring Using a Lightweight Image Super-Resolution Network

2021 ◽  
Vol 18 (11) ◽  
pp. 5890
Author(s):  
Qiang Yu ◽  
Feiqiang Liu ◽  
Long Xiao ◽  
Zitao Liu ◽  
Xiaomin Yang
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Super Resolution ◽  
Model Complexity ◽  
Practical Application ◽  
Single Image ◽  
Feature Maps ◽  
Benchmark Datasets ◽  
Image Super Resolution ◽  
Single Image Super Resolution

Deep-learning (DL)-based methods are of growing importance in the field of single image super-resolution (SISR). The practical application of these DL-based models is a remaining problem due to the requirement of heavy computation and huge storage resources. The powerful feature maps of hidden layers in convolutional neural networks (CNN) help the model learn useful information. However, there exists redundancy among feature maps, which can be further exploited. To address these issues, this paper proposes a lightweight efficient feature generating network (EFGN) for SISR by constructing the efficient feature generating block (EFGB). Specifically, the EFGB can conduct plain operations on the original features to produce more feature maps with parameters slightly increasing. With the help of these extra feature maps, the network can extract more useful information from low resolution (LR) images to reconstruct the desired high resolution (HR) images. Experiments conducted on the benchmark datasets demonstrate that the proposed EFGN can outperform other deep-learning based methods in most cases and possess relatively lower model complexity. Additionally, the running time measurement indicates the feasibility of real-time monitoring.

scholarly journals BioECG: Improving ECG Biometrics with Deep Learning and Enhanced Datasets

2021 ◽  
Vol 11 (13) ◽  
pp. 5880
Author(s):  
Paloma Tirado-Martin ◽  
Raul Sanchez-Reillo
Keyword(s):  
Deep Learning ◽  
Temporal Variations ◽  
Physical Activities ◽  
Learning Tools ◽  
Biometric Recognition ◽  
Learning Network ◽  
Deep Learning Network ◽  
Electrocardiogram Ecg

Nowadays, Deep Learning tools have been widely applied in biometrics. Electrocardiogram (ECG) biometrics is not the exception. However, the algorithm performances rely heavily on a representative dataset for training. ECGs suffer constant temporal variations, and it is even more relevant to collect databases that can represent these conditions. Nonetheless, the restriction in database publications obstructs further research on this topic. This work was developed with the help of a database that represents potential scenarios in biometric recognition as data was acquired in different days, physical activities and positions. The classification was implemented with a Deep Learning network, BioECG, avoiding complex and time-consuming signal transformations. An exhaustive tuning was completed including variations in enrollment length, improving ECG verification for more complex and realistic biometric conditions. Finally, this work studied one-day and two-days enrollments and their effects. Two-days enrollments resulted in huge general improvements even when verification was accomplished with more unstable signals. EER was improved in 63% when including a change of position, up to almost 99% when visits were in a different day and up to 91% if the user experienced a heartbeat increase after exercise.

Named Entity Recognition and Relation Extraction

2021 ◽  
Vol 54 (1) ◽  
pp. 1-39
Author(s):  
Zara Nasar ◽  
Syed Waqar Jaffry ◽  
Muhammad Kamran Malik
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Named Entity Recognition ◽  
Relation Extraction ◽  
The State ◽  
Entity Recognition ◽  
Joint Models ◽  
Named Entity ◽  
Textual Data ◽  
Benchmark Datasets

With the advent of Web 2.0, there exist many online platforms that result in massive textual-data production. With ever-increasing textual data at hand, it is of immense importance to extract information nuggets from this data. One approach towards effective harnessing of this unstructured textual data could be its transformation into structured text. Hence, this study aims to present an overview of approaches that can be applied to extract key insights from textual data in a structured way. For this, Named Entity Recognition and Relation Extraction are being majorly addressed in this review study. The former deals with identification of named entities, and the latter deals with problem of extracting relation between set of entities. This study covers early approaches as well as the developments made up till now using machine learning models. Survey findings conclude that deep-learning-based hybrid and joint models are currently governing the state-of-the-art. It is also observed that annotated benchmark datasets for various textual-data generators such as Twitter and other social forums are not available. This scarcity of dataset has resulted into relatively less progress in these domains. Additionally, the majority of the state-of-the-art techniques are offline and computationally expensive. Last, with increasing focus on deep-learning frameworks, there is need to understand and explain the under-going processes in deep architectures.

Sign in / Sign up

Export Citation Format

Share Document