Exploiting the potential of unlabeled endoscopic video data with self-supervised learning

Endoscopic optical imaging technologies for the detection and evaluation of dysplasia and early cancer have made great strides in recent decades. With the capacity of in vivo early detection of subtle lesions, they allow modern endoscopists to provide accurate and effective optical diagnosis in real time. This review mainly analyzes the current status of clinically available endoscopic optical imaging techniques, with emphasis on the latest updates of existing techniques. We summarize current coverage of these technologies in major hospital departments such as gastroenterology, urology, gynecology, otolaryngology, pneumology, and laparoscopic surgery. In order to promote a broader understanding, we further cover the underlying principles of these technologies and analyze their performance. Moreover, we provide a brief overview of future perspectives in related technologies, such as computer-assisted diagnosis (CAD) algorithms dealing with exploring endoscopic video data. We believe all these efforts will benefit the healthcare of the community, help endoscopists improve the accuracy of diagnosis, and relieve patients’ suffering.

Download Full-text

Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks

Sensors ◽

10.3390/s20154133 ◽

2020 ◽

Vol 20 (15) ◽

pp. 4133

Author(s):

Tim Boers ◽

Joost van der Putten ◽

Maarten Struyvenberg ◽

Kiki Fockens ◽

Jelmer Jukema ◽

...

Keyword(s):

Short Term Memory ◽

Temporal Stability ◽

Video Data ◽

Tissue Classification ◽

Endoscopic Video ◽

Spatio Temporal ◽

Averaging Filter ◽

Improved Model ◽

Stability And Accuracy ◽

Fully Connected

Early Barrett’s neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the temporal domain is still open. The temporally stable nature of video data in endoscopic examinations enables to develop a framework that can diagnose the imaged tissue class over time, thereby yielding a more robust and improved model for spatial predictions. We show that the introduction of Recurrent Neural Network nodes offers a more stable and accurate model for tissue classification, compared to classification on individual images. We have developed a customized Resnet18 feature extractor with four types of classifiers: Fully Connected (FC), Fully Connected with an averaging filter (FC Avg (n = 5)), Long Short Term Memory (LSTM) and a Gated Recurrent Unit (GRU). Experimental results are based on 82 pullback videos of the esophagus with 46 high-grade dysplasia patients. Our results demonstrate that the LSTM classifier outperforms the FC, FC Avg (n = 5) and GRU classifier with an average accuracy of 85.9% compared to 82.2%, 83.0% and 85.6%, respectively. The benefit of our novel implementation for endoscopic tissue classification is the inclusion of spatio-temporal information for improved and robust decision making, and it is the first step towards full temporal learning of esophageal cancer detection in endoscopic video.

Download Full-text

Research on behavior recognition based on feature fusion of automatic coder and recurrent neural network

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189290 ◽

2020 ◽

Vol 39 (6) ◽

pp. 8927-8935

Author(s):

Bing Zheng ◽

Dawei Yun ◽

Yan Liang

Keyword(s):

Neural Network ◽

Recurrent Neural Network ◽

Behavior Pattern ◽

Rapid Development ◽

Video Data ◽

Support Vector ◽

Behavior Recognition ◽

Learning Methods ◽

The Impact ◽

Internet Of Things Technology

Under the impact of COVID-19, research on behavior recognition are highly needed. In this paper, we combine the algorithm of self-adaptive coder and recurrent neural network to realize the research of behavior pattern recognition. At present, most of the research of human behavior recognition is focused on the video data, which is based on the video number. At the same time, due to the complexity of video image data, it is easy to violate personal privacy. With the rapid development of Internet of things technology, it has attracted the attention of a large number of experts and scholars. Researchers have tried to use many machine learning methods, such as random forest, support vector machine and other shallow learning methods, which perform well in the laboratory environment, but there is still a long way to go from practical application. In this paper, a recursive neural network algorithm based on long and short term memory (LSTM) is proposed to realize the recognition of behavior patterns, so as to improve the accuracy of human activity behavior recognition.

Download Full-text

Recognition of classroom student state features based on deep learning algorithms and machine learning

Journal of Intelligent & Fuzzy Systems ◽

10.3233/jifs-189232 ◽

2020 ◽

pp. 1-12

Author(s):

Hu Jingchao ◽

Haiying Zhang

Keyword(s):

Deep Learning ◽

Feature Recognition ◽

Subjective Evaluation ◽

Recognition Algorithm ◽

Video Data ◽

State Recognition ◽

Detection Model ◽

State Classification ◽

Intelligent Models ◽

Model Recognition

The difficulty in class student state recognition is how to make feature judgments based on student facial expressions and movement state. At present, some intelligent models are not accurate in class student state recognition. In order to improve the model recognition effect, this study builds a two-level state detection framework based on deep learning and HMM feature recognition algorithm, and expands it as a multi-level detection model through a reasonable state classification method. In addition, this study selects continuous HMM or deep learning to reflect the dynamic generation characteristics of fatigue, and designs random human fatigue recognition experiments to complete the collection and preprocessing of EEG data, facial video data, and subjective evaluation data of classroom students. In addition to this, this study discretizes the feature indicators and builds a student state recognition model. Finally, the performance of the algorithm proposed in this paper is analyzed through experiments. The research results show that the algorithm proposed in this paper has certain advantages over the traditional algorithm in the recognition of classroom student state features.

Download Full-text

Selection of the efficient video data processing strategy based on the analysis of statistical digital images characteristics

Scientific journal of the Ternopil national technical university ◽

10.33108/visnyk_tntu2018.03.107 ◽

2018 ◽

Vol 91 (3) ◽

pp. 107-114

Author(s):

Mykhailo Palamar ◽

Myroslava Yavorska ◽

Mykhailo Strembitskyi ◽

Volodymyr Strembitskyi

Keyword(s):

Data Processing ◽

Digital Images ◽

Video Data ◽

Processing Strategy ◽

Efficient Video ◽

Selection Of

Download Full-text

Video Data Embedding. Part II

Journal of Automation and Information Sciences ◽

10.1615/jautomatinfscien.v45.i4.80 ◽

2013 ◽

Vol 45 (4) ◽

pp. 75-83

Author(s):

Lyubov L. Nikitenko

Keyword(s):

Video Data ◽

Data Embedding

Download Full-text

Detecting “DeepFakes” in H.264 Video Data Using Compression Ghost Artifacts

Electronic Imaging ◽

10.2352/issn.2470-1173.2020.4.mwsf-116 ◽

2020 ◽

Vol 2020 (4) ◽

pp. 116-1-116-7

Author(s):

Raphael Antonius Frick ◽

Sascha Zmudzinski ◽

Martin Steinebach

Keyword(s):

Image Forensics ◽

Video Data ◽

Experimental Results ◽

Video Sequences ◽

The Internet ◽

Video Content ◽

High Quality ◽

The Public

In recent years, the number of forged videos circulating on the Internet has immensely increased. Software and services to create such forgeries have become more and more accessible to the public. In this regard, the risk of malicious use of forged videos has risen. This work proposes an approach based on the Ghost effect knwon from image forensics for detecting forgeries in videos that can replace faces in video sequences or change the mimic of a face. The experimental results show that the proposed approach is able to identify forgery in high-quality encoded video content.

Download Full-text