Spatio-temporal feature based deep neural network for cell lineage analysis in microscopy images

Background: Time-lapse microscopy has been widely used in biomedical experiments because it can visualize the molecular activities of living cells in real time. However, biomedical researchers are still conducting cell lineage analysis manually. Developing automatic lineage tracing algorithms is a challenging task. In the past two decades, deep neural networks (DNNs) became have shown outstanding performance on computer vision tasks. They can learn complex visual features, capture long-range temporal dependencies, and have the potential to be used for automatic cell lineage analysis. Methods: In this study, we propose a multi-task spatio-temporal feature based deep neural network for cell lineages analysis (Cell-STN). The Cell-STN extracts spatio-temporal features from microscopy image sequences by leveraging our convolutional long short-term memory based core block. And the proposed Cell-STN utilized a task specific network to predict the cell location, the mitosis event, and the apoptosis event in a multi-task manner. Results: We evaluated the Cell-STN on three in-house datasets (MCF7, U2OS, and HCT116) and one public dataset (Fluo-N2DL-HeLa). For cell tracking, we used peak-wise precision, track-wise precision, end-peak precision, and spatial distance as metrics. The overall results showed the Cell-STN models outperform other state-of-the-art cell trackers. For mitosis and apoptosis tasks, we used accuracy, F1-score, temporal distance, and spatial distance as metrics. The Cell-STN models achieved the highest performance on all datasets. Conclusion: This study presented a novel DNNs approach for cell lineage analysis in microscopy images. The Cell-STN showed outstanding performance on the four datasets. Additionally, the Cell-STN required minimal training data and can be adapted to new biological event detection tasks by appending task-specific layers. This algorithm has the potential to be used in real-world biomedical research.