Inverse Halftoning Methods Based on Deep Learning and Their Evaluation Metrics: A Review

Inverse halftoning is an ill-posed problem that refers to the problem of restoring continuous-tone images from their halftone versions. Although much progress has been achieved over the last decades, the restored images still suffer from detail loss and visual artifacts. Recent studies show that inverse halftoning methods based on deep learning are superior to other traditional methods, and thus this paper aimed to systematically review the inverse halftone methods based on deep learning, so as to provide a reference for the development of inverse halftoning. In this paper, we firstly proposed a classification method for inverse halftoning methods on the basis of the source of halftone images. Then, two types of inverse halftoning methods for digital halftone images and scanned halftone images were investigated in terms of network architecture, loss functions, and training strategies. Furthermore, we studied existing image quality evaluation including subjective and objective evaluation by experiments. The evaluation results demonstrated that methods based on multiple subnetworks and methods based on multi-stage strategies are superior to other methods. In addition, the perceptual loss and the gradient loss are helpful for improving the quality of restored images. Finally, we gave the future research directions by analyzing the shortcomings of existing inverse halftoning methods.

Download Full-text

Deep Learning in Data-Driven Pavement Image Analysis and Automated Distress Detection: A Review

Data ◽

10.3390/data3030028 ◽

2018 ◽

Vol 3 (3) ◽

pp. 28 ◽

Cited By ~ 23

Author(s):

Kasthurirangan Gopalakrishnan

Keyword(s):

Computer Vision ◽

Image Analysis ◽

Deep Learning ◽

Asset Management ◽

Network Architecture ◽

Crack Detection ◽

Future Research ◽

Deep Convolutional Neural Networks ◽

Pavement Distress ◽

Learning Software

Deep learning, more specifically deep convolutional neural networks, is fast becoming a popular choice for computer vision-based automated pavement distress detection. While pavement image analysis has been extensively researched over the past three decades or so, recent ground-breaking achievements of deep learning algorithms in the areas of machine translation, speech recognition, and computer vision has sparked interest in the application of deep learning to automated detection of distresses in pavement images. This paper provides a narrative review of recently published studies in this field, highlighting the current achievements and challenges. A comparison of the deep learning software frameworks, network architecture, hyper-parameters employed by each study, and crack detection performance is provided, which is expected to provide a good foundation for driving further research on this important topic in the context of smart pavement or asset management systems. The review concludes with potential avenues for future research; especially in the application of deep learning to not only detect, but also characterize the type, extent, and severity of distresses from 2D and 3D pavement images.

Download Full-text

Deep Learning for Video Classification: A Review

10.36227/techrxiv.15172920 ◽

2021 ◽

Author(s):

Atiq Rehman ◽

Samir Brahim Belhaouari

Keyword(s):

Deep Learning ◽

Network Architecture ◽

Classification Task ◽

Future Research ◽

Video Classification ◽

Learning Models ◽

Learning Methods ◽

Research Directions ◽

Concise Review ◽

State Of Art

<div><div><div><p>Video classification task has gained a significant success in the recent years. Specifically, the topic has gained more attention after the emergence of deep learning models as a successful tool for automatically classifying videos. In recognition to the importance of video classification task and to summarize the success of deep learning models for this task, this paper presents a very comprehensive and concise review on the topic. There are a number of existing reviews and survey papers related to video classification in the scientific literature. However, the existing review papers are either outdated, and therefore, do not include the recent state-of-art works or they have some limitations. In order to provide an updated and concise review, this paper highlights the key findings based on the existing deep learning models. The key findings are also discussed in a way to provide future research directions. This review mainly focuses on the type of network architecture used, the evaluation criteria to measure the success, and the data sets used. To make the review self- contained, the emergence of deep learning methods towards automatic video classification and the state-of-art deep learning methods are well explained and summarized. Moreover, a clear insight of the newly developed deep learning architectures and the traditional approaches is provided, and the critical challenges based on the benchmarks are highlighted for evaluating the technical progress of these methods. The paper also summarizes the benchmark datasets and the performance evaluation matrices for video classification. Based on the compact, complete, and concise review, the paper proposes new research directions to solve the challenging video classification problem.</p></div></div></div>

Download Full-text

An Efficient Convolutional Neural Network Model Combined with Attention Mechanism for Inverse Halftoning

Electronics ◽

10.3390/electronics10131574 ◽

2021 ◽

Vol 10 (13) ◽

pp. 1574

Author(s):

Linhao Shao ◽

Erhu Zhang ◽

Mei Li

Keyword(s):

Feature Extraction ◽

Image Features ◽

Attention Mechanism ◽

Discriminative Ability ◽

Halftone Image ◽

Multi Stage ◽

Different Types ◽

Inverse Halftoning ◽

Extraction Stage ◽

Halftone Images

Inverse halftoning acting as a special image restoration problem is an ill-posed problem. Although it has been studied in the last several decades, the existing solutions can’t restore fine details and texture accurately from halftone images. Recently, the attention mechanism has shown its powerful effects in many fields, such as image processing, pattern recognition and computer vision. However, it has not yet been used in inverse halftoning. To better solve the problem of detail restoration of inverse halftoning, this paper proposes a simple yet effective deep learning model combined with the attention mechanism, which can better guide the network to remove noise dot-patterns and restore image details, and improve the network adaptation ability. The whole model is designed in an end-to-end manner, including feature extraction stage and reconstruction stage. In the feature extraction stage, halftone image features are extracted and halftone noises are removed. The reconstruction stage is employed to restore continuous-tone images by fusing the feature information extracted in the first stage and the output of the residual channel attention block. In this stage, the attention block is firstly introduced to the field of inverse halftoning, which can make the network focus on informative features and further enhance the discriminative ability of the network. In addition, a multi-stage loss function is proposed to accelerate the network optimization, which is conducive to better reconstruction of the global image. To demonstrate the generalization performance of the network for different types of halftone images, the experiment results confirm that the network can restore six different types of halftone image well. Furthermore, experimental results show that our method outperforms the state-of-the-art methods, especially in the restoration of details and textures.

Download Full-text

An Experimental Analysis of Deep Learning Architectures for Supervised Speech Enhancement

Electronics ◽

10.3390/electronics10010017 ◽

2020 ◽

Vol 10 (1) ◽

pp. 17

Author(s):

Soha A. Nossier ◽

Julie Wall ◽

Mansour Moniri ◽

Cornelius Glackin ◽

Nigel Cannings

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Speech Enhancement ◽

Network Architecture ◽

Deep Neural Networks ◽

Subjective Evaluation ◽

Objective Evaluation ◽

Learning Models ◽

Factors Affecting ◽

General Evaluation

Recent speech enhancement research has shown that deep learning techniques are very effective in removing background noise. Many deep neural networks are being proposed, showing promising results for improving overall speech perception. The Deep Multilayer Perceptron, Convolutional Neural Networks, and the Denoising Autoencoder are well-established architectures for speech enhancement; however, choosing between different deep learning models has been mainly empirical. Consequently, a comparative analysis is needed between these three architecture types in order to show the factors affecting their performance. In this paper, this analysis is presented by comparing seven deep learning models that belong to these three categories. The comparison includes evaluating the performance in terms of the overall quality of the output speech using five objective evaluation metrics and a subjective evaluation with 23 listeners; the ability to deal with challenging noise conditions; generalization ability; complexity; and, processing time. Further analysis is then provided while using two different approaches. The first approach investigates how the performance is affected by changing network hyperparameters and the structure of the data, including the Lombard effect. While the second approach interprets the results by visualizing the spectrogram of the output layer of all the investigated models, and the spectrograms of the hidden layers of the convolutional neural network architecture. Finally, a general evaluation is performed for supervised deep learning-based speech enhancement while using SWOC analysis, to discuss the technique’s Strengths, Weaknesses, Opportunities, and Challenges. The results of this paper contribute to the understanding of how different deep neural networks perform the speech enhancement task, highlight the strengths and weaknesses of each architecture, and provide recommendations for achieving better performance. This work facilitates the development of better deep neural networks for speech enhancement in the future.

Download Full-text

Deep Learning for Video Classification: A Review

10.36227/techrxiv.15172920.v1 ◽

2021 ◽

Author(s):

Atiq Rehman ◽

Samir Brahim Belhaouari

Keyword(s):

Deep Learning ◽

Network Architecture ◽

Classification Task ◽

Future Research ◽

Video Classification ◽

Learning Models ◽

Learning Methods ◽

Research Directions ◽

Concise Review ◽

State Of Art

Download Full-text

Deep Learning Techniques for the Classification of Colorectal Cancer Tissue

Electronics ◽

10.3390/electronics10141662 ◽

2021 ◽

Vol 10 (14) ◽

pp. 1662

Author(s):

Min-Jen Tsai ◽

Yu-Han Tao

Keyword(s):

Colorectal Cancer ◽

Deep Learning ◽

Network Architecture ◽

External Validation ◽

Objective Evaluation ◽

Cancer Tissue ◽

Learning Technology ◽

Histological Image ◽

Colorectal Cancer Tissue ◽

Histological Images

It is very important to make an objective evaluation of colorectal cancer histological images. Current approaches are generally based on the use of different combinations of textual features and classifiers to assess the classification performance, or transfer learning to classify different organizational types. However, since histological images contain multiple tissue types and characteristics, classification is still challenging. In this study, we proposed the best classification methodology based on the selected optimizer and modified the parameters of CNN methods. Then, we used deep learning technology to distinguish between healthy and diseased large intestine tissues. Firstly, we trained a neural network and compared the network architecture optimizers. Secondly, we modified the parameters of the network layer to optimize the superior architecture. Finally, we compared our well-trained deep learning methods on two different histological image open datasets, which comprised 5000 H&E images of colorectal cancer. The other dataset was composed of nine organizational categories of 100,000 images with an external validation of 7180 images. The results showed that the accuracy of the recognition of histopathological images was significantly better than that of existing methods. Therefore, this method is expected to have great potential to assist physicians to make clinical diagnoses and reduce the number of disparate assessments based on the use of artificial intelligence to classify colorectal cancer tissue.

Download Full-text

Android Malware Detection Techniques: A Literature Review

Recent Patents on Engineering ◽

10.2174/1872212114999200710143847 ◽

2020 ◽

Vol 14 ◽

Author(s):

Meghna Dhalaria ◽

Ekta Gandotra

Keyword(s):

Machine Learning ◽

Deep Learning ◽

Malware Detection ◽

Future Research ◽

Android Malware ◽

Detection Techniques ◽

Android Malware Detection ◽

Future Research Directions ◽

To Come ◽

Tools And Techniques

Purpose: This paper provides the basics of Android malware, its evolution and tools and techniques for malware analysis. Its main aim is to present a review of the literature on Android malware detection using machine learning and deep learning and identify the research gaps. It provides the insights obtained through literature and future research directions which could help researchers to come up with robust and accurate techniques for classification of Android malware. Design/Methodology/Approach: This paper provides a review of the basics of Android malware, its evolution timeline and detection techniques. It includes the tools and techniques for analyzing the Android malware statically and dynamically for extracting features and finally classifying these using machine learning and deep learning algorithms. Findings: The number of Android users is expanding very fast due to the popularity of Android devices. As a result, there are more risks to Android users due to the exponential growth of Android malware. On-going research aims to overcome the constraints of earlier approaches for malware detection. As the evolving malware are complex and sophisticated, earlier approaches like signature based and machine learning based are not able to identify these timely and accurately. The findings from the review shows various limitations of earlier techniques i.e. requires more detection time, high false positive and false negative rate, low accuracy in detecting sophisticated malware and less flexible. Originality/value: This paper provides a systematic and comprehensive review on the tools and techniques being employed for analysis, classification and identification of Android malicious applications. It includes the timeline of Android malware evolution, tools and techniques for analyzing these statically and dynamically for the purpose of extracting features and finally using these features for their detection and classification using machine learning and deep learning algorithms. On the basis of the detailed literature review, various research gaps are listed. The paper also provides future research directions and insights which could help researchers to come up with innovative and robust techniques for detecting and classifying the Android malware.

Download Full-text

Application of deep learning in detecting neurological disorders from magnetic resonance images: a survey on the detection of Alzheimer’s disease, Parkinson’s disease and schizophrenia

Brain Informatics ◽

10.1186/s40708-020-00112-2 ◽

2020 ◽

Vol 7 (1) ◽

Cited By ~ 6

Author(s):

Manan Binth Taj Noor ◽

Nusrat Zerin Zenia ◽

M Shamim Kaiser ◽

Shamim Al Mamun ◽

Mufti Mahmud

Keyword(s):

Alzheimer’S Disease ◽

Parkinson’S Disease ◽

Alzheimer's Disease ◽

Parkinson's Disease ◽

Deep Learning ◽

Magnetic Resonance ◽

Neurological Disorders ◽

Magnetic Resonance Images ◽

Future Research ◽

Comparative Performance

Abstract Neuroimaging, in particular magnetic resonance imaging (MRI), has been playing an important role in understanding brain functionalities and its disorders during the last couple of decades. These cutting-edge MRI scans, supported by high-performance computational tools and novel ML techniques, have opened up possibilities to unprecedentedly identify neurological disorders. However, similarities in disease phenotypes make it very difficult to detect such disorders accurately from the acquired neuroimaging data. This article critically examines and compares performances of the existing deep learning (DL)-based methods to detect neurological disorders—focusing on Alzheimer’s disease, Parkinson’s disease and schizophrenia—from MRI data acquired using different modalities including functional and structural MRI. The comparative performance analysis of various DL architectures across different disorders and imaging modalities suggests that the Convolutional Neural Network outperforms other methods in detecting neurological disorders. Towards the end, a number of current research challenges are indicated and some possible future research directions are provided.

Download Full-text

Deep Learning with Neuroimaging and Genomics in Alzheimer’s Disease

International Journal of Molecular Sciences ◽

10.3390/ijms22157911 ◽

2021 ◽

Vol 22 (15) ◽

pp. 7911

Author(s):

Eugene Lin ◽

Chieh-Hsin Lin ◽

Hsien-Yuan Lane

Keyword(s):

Alzheimer’S Disease ◽

Alzheimer's Disease ◽

Deep Learning ◽

Future Research ◽

Learning Approaches ◽

Learning Models ◽

Learning Techniques ◽

Neuroimaging Data ◽

Similarities And Differences ◽

Normal Controls

A growing body of evidence currently proposes that deep learning approaches can serve as an essential cornerstone for the diagnosis and prediction of Alzheimer’s disease (AD). In light of the latest advancements in neuroimaging and genomics, numerous deep learning models are being exploited to distinguish AD from normal controls and/or to distinguish AD from mild cognitive impairment in recent research studies. In this review, we focus on the latest developments for AD prediction using deep learning techniques in cooperation with the principles of neuroimaging and genomics. First, we narrate various investigations that make use of deep learning algorithms to establish AD prediction using genomics or neuroimaging data. Particularly, we delineate relevant integrative neuroimaging genomics investigations that leverage deep learning methods to forecast AD on the basis of incorporating both neuroimaging and genomics data. Moreover, we outline the limitations as regards to the recent AD investigations of deep learning with neuroimaging and genomics. Finally, we depict a discussion of challenges and directions for future research. The main novelty of this work is that we summarize the major points of these investigations and scrutinize the similarities and differences among these investigations.

Download Full-text

Deep learning-based recognition of cell structures in fluorescence microscopy sequences with respect to their morphology on cells infected with Marburg virus

Current Directions in Biomedical Engineering ◽

10.1515/cdbme-2020-3129 ◽

2020 ◽

Vol 6 (3) ◽

pp. 501-504

Author(s):

Dennis Schmidt ◽

Andreas Rausch ◽

Thomas Schanze

Keyword(s):

Neural Network ◽

Deep Learning ◽

Infected Cell ◽

Network Architecture ◽

Image Sequences ◽

Marburg Virus ◽

Microscopic Image ◽

Cell Structures ◽

Fast Recognition ◽

Subviral Particles

AbstractThe Institute of Virology at the Philipps-Universität Marburg is currently researching possible drugs to combat the Marburg virus. This involves classifying cell structures based on fluoroscopic microscopic image sequences. Conventionally, membranes of cells must be marked for better analysis, which is time consuming. In this work, an approach is presented to identify cell structures in images that are marked for subviral particles. It could be shown that there is a correlation between the distribution of subviral particles in an infected cell and the position of the cell’s structures. The segmentation is performed with a "Mask-R-CNN" algorithm, presented in this work. The model (a region-based convolutional neural network) is applied to enable a robust and fast recognition of cell structures. Furthermore, the network architecture is described. The proposed method is tested on data evaluated by experts. The results show a high potential and demonstrate that the method is suitable.

Download Full-text