Unsupervised phenotypic analysis of cellular images with multi-scale convolutional neural networks

AbstractLarge-scale cellular imaging and phenotyping is a widely adopted strategy for understanding biological systems and chemical perturbations. Quantitative analysis of cellular images for identifying phenotypic changes is a key challenge within this strategy, and has recently seen promising progress with approaches based on deep neural networks. However, studies so far require either pre-segmented images as input or manual phenotype annotations for training, or both. To address these limitations, we have developed an unsupervised approach that exploits the inherent groupings within cellular imaging datasets to define surrogate classes that are used to train a multi-scale convolutional neural network. The trained network takes as input full-resolution microscopy images, and, without the need for segmentation, yields as output feature vectors that support phenotypic profiling. Benchmarked on two diverse benchmark datasets, the proposed approach yields accurate phenotypic predictions as well as compound potency estimates comparable to the state-of-the-art. More importantly, we show that the approach identifies novel cellular phenotypes not included in the manual annotation nor detected by previous studies.Author summaryCellular microscopy images provide detailed information about how cells respond to genetic or chemical treatments, and have been widely and successfully used in basic research and drug discovery. The recent breakthrough of deep learning methods for natural imaging recognition tasks has triggered the development and application of deep learning methods to cellular images to understand how cells change upon perturbation. Although successful, deep learning studies so far either can only take images of individual cells as input or require human experts to label a large amount of images. In this paper, we present an unsupervised deep learning approach that, without any human annotation, analyzes directly full-resolution microscopy images displaying typically hundreds of cells. We apply the approach to two benchmark datasets, and show that the approach identifies novel visual phenotypes not detected by previous studies.

Download Full-text

Structure-preserving deep learning

European Journal of Applied Mathematics ◽

10.1017/s0956792521000139 ◽

2021 ◽

pp. 1-49

Author(s):

E. CELLEDONI ◽

M. J. EHRHARDT ◽

C. ETMANN ◽

R. I. MCLACHLAN ◽

B. OWREN ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Large Scale ◽

Computational Effort ◽

Model Complexity ◽

Future Research ◽

Learning Methods ◽

Open Problems ◽

Mathematical Problems ◽

Made In

Over the past few years, deep learning has risen to the foreground as a topic of massive interest, mainly as a result of successes obtained in solving large-scale image processing tasks. There are multiple challenging mathematical problems involved in applying deep learning: most deep learning methods require the solution of hard optimisation problems, and a good understanding of the trade-off between computational effort, amount of data and model complexity is required to successfully design a deep learning approach for a given problem.. A large amount of progress made in deep learning has been based on heuristic explorations, but there is a growing effort to mathematically understand the structure in existing deep learning methods and to systematically design new deep learning methods to preserve certain types of structure in deep learning. In this article, we review a number of these directions: some deep neural networks can be understood as discretisations of dynamical systems, neural networks can be designed to have desirable properties such as invertibility or group equivariance and new algorithmic frameworks based on conformal Hamiltonian systems and Riemannian manifolds to solve the optimisation problems have been proposed. We conclude our review of each of these topics by discussing some open problems that we consider to be interesting directions for future research.

Download Full-text

Is the aspect ratio of cells important in deep learning? A robust comparison of deep learning methods for multi-scale cytopathology cell image classification: From convolutional neural networks to visual transformers

Computers in Biology and Medicine ◽

10.1016/j.compbiomed.2021.105026 ◽

2021 ◽

pp. 105026

Author(s):

Wanli Liu ◽

Chen Li ◽

Md Mamunur Rahaman ◽

Tao Jiang ◽

Hongzan Sun ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Aspect Ratio ◽

Image Classification ◽

Convolutional Neural Networks ◽

Learning Methods ◽

Multi Scale ◽

Cell Image

Download Full-text

Multilabel Text Classification in News Articles Using Long-Term Memory with Word2Vec

Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) ◽

10.29207/resti.v4i2.1655 ◽

2020 ◽

Vol 4 (2) ◽

pp. 276-285

Author(s):

Winda Kurnia Sari ◽

Dian Palupi Rini ◽

Reza Firsandaya Malik ◽

Iman Saladin B. Azhar

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Text Classification ◽

Large Scale ◽

Short Term Memory ◽

Short Term ◽

Learning Methods ◽

Processing Variable ◽

Term Memory ◽

Long Short Term Memory

Multilabel text classification is a task of categorizing text into one or more categories. Like other machine learning, multilabel classification performance is limited to the small labeled data and leads to the difficulty of capturing semantic relationships. It requires a multilabel text classification technique that can group four labels from news articles. Deep Learning is a proposed method for solving problems in multilabel text classification techniques. Some of the deep learning methods used for text classification include Convolutional Neural Networks, Autoencoders, Deep Belief Networks, and Recurrent Neural Networks (RNN). RNN is one of the most popular architectures used in natural language processing (NLP) because the recurrent structure is appropriate for processing variable-length text. One of the deep learning methods proposed in this study is RNN with the application of the Long Short-Term Memory (LSTM) architecture. The models are trained based on trial and error experiments using LSTM and 300-dimensional words embedding features with Word2Vec. By tuning the parameters and comparing the eight proposed Long Short-Term Memory (LSTM) models with a large-scale dataset, to show that LSTM with features Word2Vec can achieve good performance in text classification. The results show that text classification using LSTM with Word2Vec obtain the highest accuracy is in the fifth model with 95.38, the average of precision, recall, and F1-score is 95. Also, LSTM with the Word2Vec feature gets graphic results that are close to good-fit on seventh and eighth models.

Download Full-text

Computational Complexity Reduction of Neural Networks of Brain Tumor Image Segmentation by Introducing Fermi–Dirac Correction Functions

Entropy ◽

10.3390/e23020223 ◽

2021 ◽

Vol 23 (2) ◽

pp. 223

Author(s):

Yen-Ling Tai ◽

Shin-Jhe Huang ◽

Chien-Chang Chen ◽

Henry Horng-Shing Lu

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Computational Complexity ◽

High Performance ◽

Low Cost ◽

Structural Complexity ◽

Correction Function ◽

Computational Time ◽

Learning Methods ◽

Band Theory

Nowadays, deep learning methods with high structural complexity and flexibility inevitably lean on the computational capability of the hardware. A platform with high-performance GPUs and large amounts of memory could support neural networks having large numbers of layers and kernels. However, naively pursuing high-cost hardware would probably drag the technical development of deep learning methods. In the article, we thus establish a new preprocessing method to reduce the computational complexity of the neural networks. Inspired by the band theory of solids in physics, we map the image space into a noninteraction physical system isomorphically and then treat image voxels as particle-like clusters. Then, we reconstruct the Fermi–Dirac distribution to be a correction function for the normalization of the voxel intensity and as a filter of insignificant cluster components. The filtered clusters at the circumstance can delineate the morphological heterogeneity of the image voxels. We used the BraTS 2019 datasets and the dimensional fusion U-net for the algorithmic validation, and the proposed Fermi–Dirac correction function exhibited comparable performance to other employed preprocessing methods. By comparing to the conventional z-score normalization function and the Gamma correction function, the proposed algorithm can save at least 38% of computational time cost under a low-cost hardware architecture. Even though the correction function of global histogram equalization has the lowest computational time among the employed correction functions, the proposed Fermi–Dirac correction function exhibits better capabilities of image augmentation and segmentation.

Download Full-text

Attention Multi-Scale Network for Automatic Layer Extraction of Ice Radar Topological Sequences

Remote Sensing ◽

10.3390/rs13122425 ◽

2021 ◽

Vol 13 (12) ◽

pp. 2425

Author(s):

Yiheng Cai ◽

Dan Liu ◽

Jin Xie ◽

Jingxian Yang ◽

Xiangbin Cui ◽

...

Keyword(s):

Deep Learning ◽

Global Climate ◽

Ice Sheets ◽

Ice Sheet ◽

Sheet Thickness ◽

Learning Methods ◽

Convolutional Network ◽

Multi Scale ◽

Ice Surface ◽

Layer Extraction

Analyzing the surface and bedrock locations in radar imagery enables the computation of ice sheet thickness, which is important for the study of ice sheets, their volume and how they may contribute to global climate change. However, the traditional handcrafted methods cannot quickly provide quantitative, objective and reliable extraction of information from radargrams. Most traditional handcrafted methods, designed to detect ice-surface and ice-bed layers from ice sheet radargrams, require complex human involvement and are difficult to apply to large datasets, while deep learning methods can obtain better results in a generalized way. In this study, an end-to-end multi-scale attention network (MsANet) is proposed to realize the estimation and reconstruction of layers in sequences of ice sheet radar tomographic images. First, we use an improved 3D convolutional network, C3D-M, whose first full connection layer is replaced by a convolution unit to better maintain the spatial relativity of ice layer features, as the backbone. Then, an adjustable multi-scale module uses different scale filters to learn scale information to enhance the feature extraction capabilities of the network. Finally, an attention module extended to 3D space removes a redundant bottleneck unit to better fuse and refine ice layer features. Radar sequential images collected by the Center of Remote Sensing of Ice Sheets in 2014 are used as training and testing data. Compared with state-of-the-art deep learning methods, the MsANet shows a 10% reduction (2.14 pixels) on the measurement of average mean absolute column-wise error for detecting the ice-surface and ice-bottom layers, runs faster and uses approximately 12 million fewer parameters.

Download Full-text

Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽

10.3390/s21082852 ◽

2021 ◽

Vol 21 (8) ◽

pp. 2852

Author(s):

Parvathaneni Naga Srinivasu ◽

Jalluri Gnana SivaSai ◽

Muhammad Fazal Ijaz ◽

Akash Kumar Bhoi ◽

Wonjoon Kim ◽

...

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Skin Disease ◽

Network Architecture ◽

Large Scale ◽

Short Term Memory ◽

Convolutional Networks ◽

Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

Download Full-text

Towards pixel-to-pixel deep nucleus detection in microscopy images

BMC Bioinformatics ◽

10.1186/s12859-019-3037-5 ◽

2019 ◽

Vol 20 (1) ◽

Cited By ~ 3

Author(s):

Fuyong Xing ◽

Yuanpu Xie ◽

Xiaoshuang Shi ◽

Pingjun Chen ◽

Zizhao Zhang ◽

...

Keyword(s):

Neural Networks ◽

Large Scale ◽

Deep Neural Networks ◽

Image Data ◽

Fine Tuning ◽

Cell Detection ◽

Imaging Protocol ◽

Microscopy Image ◽

Microscopy Images ◽

Target Data

Abstract Background Nucleus or cell detection is a fundamental task in microscopy image analysis and supports many other quantitative studies such as object counting, segmentation, tracking, etc. Deep neural networks are emerging as a powerful tool for biomedical image computing; in particular, convolutional neural networks have been widely applied to nucleus/cell detection in microscopy images. However, almost all models are tailored for specific datasets and their applicability to other microscopy image data remains unknown. Some existing studies casually learn and evaluate deep neural networks on multiple microscopy datasets, but there are still several critical, open questions to be addressed. Results We analyze the applicability of deep models specifically for nucleus detection across a wide variety of microscopy image data. More specifically, we present a fully convolutional network-based regression model and extensively evaluate it on large-scale digital pathology and microscopy image datasets, which consist of 23 organs (or cancer diseases) and come from multiple institutions. We demonstrate that for a specific target dataset, training with images from the same types of organs might be usually necessary for nucleus detection. Although the images can be visually similar due to the same staining technique and imaging protocol, deep models learned with images from different organs might not deliver desirable results and would require model fine-tuning to be on a par with those trained with target data. We also observe that training with a mixture of target and other/non-target data does not always mean a higher accuracy of nucleus detection, and it might require proper data manipulation during model training to achieve good performance. Conclusions We conduct a systematic case study on deep models for nucleus detection in a wide variety of microscopy images, aiming to address several important but previously understudied questions. We present and extensively evaluate an end-to-end, pixel-to-pixel fully convolutional regression network and report a few significant findings, some of which might have not been reported in previous studies. The model performance analysis and observations would be helpful to nucleus detection in microscopy images.

Download Full-text

Large-Scale Modeling of Multispecies Acute Toxicity End Points Using Consensus of Multitask Deep Learning Methods

Journal of Chemical Information and Modeling ◽

10.1021/acs.jcim.0c01164 ◽

2021 ◽

Vol 61 (2) ◽

pp. 653-663

Author(s):

Sankalp Jain ◽

Vishal B. Siramshetty ◽

Vinicius M. Alves ◽

Eugene N. Muratov ◽

Nicole Kleinstreuer ◽

...

Keyword(s):

Deep Learning ◽

Acute Toxicity ◽

Large Scale ◽

Learning Methods ◽

End Points ◽

Scale Modeling ◽

Large Scale Modeling

Download Full-text

Human Skin Detection in Color Images Using Deep Learning

International Journal of Computer Vision and Image Processing ◽

10.4018/ijcvip.2015070101 ◽

2015 ◽

Vol 5 (2) ◽

pp. 1-13 ◽

Cited By ~ 1

Author(s):

Mohammadreza Hajiarbabi ◽

Arvin Agah

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Human Skin ◽

Skin Color ◽

Color Image ◽

Gaussian Model ◽

Color Images ◽

Skin Detection ◽

Learning Methods ◽

Rule Based

Human skin detection is an important and challenging problem in computer vision. Skin detection can be used as the first phase in face detection when using color images. The differences in illumination and ranges of skin colors have made skin detection a challenging task. Gaussian model, rule based methods, and artificial neural networks are methods that have been used for human skin color detection. Deep learning methods are new techniques in learning that have shown improved classification power compared to neural networks. In this paper the authors use deep learning methods in order to enhance the capabilities of skin detection algorithms. Several experiments have been performed using auto encoders and different color spaces. The proposed technique is evaluated compare with other available methods in this domain using two color image databases. The results show that skin detection utilizing deep learning has better results compared to other methods such as rule-based, Gaussian model and feed forward neural network.

Download Full-text