Toward an Automatic Quality Assessment of Voice-Based Telemedicine Consultations: A Deep Learning Approach

Maintaining a high quality of conversation between doctors and patients is essential in telehealth services, where efficient and competent communication is important to promote patient health. Assessing the quality of medical conversations is often handled based on a human auditory-perceptual evaluation. Typically, trained experts are needed for such tasks, as they follow systematic evaluation criteria. However, the daily rapid increase of consultations makes the evaluation process inefficient and impractical. This paper investigates the automation of the quality assessment process of patient–doctor voice-based conversations in a telehealth service using a deep-learning-based classification model. For this, the data consist of audio recordings obtained from Altibbi. Altibbi is a digital health platform that provides telemedicine and telehealth services in the Middle East and North Africa (MENA). The objective is to assist Altibbi’s operations team in the evaluation of the provided consultations in an automated manner. The proposed model is developed using three sets of features: features extracted from the signal level, the transcript level, and the signal and transcript levels. At the signal level, various statistical and spectral information is calculated to characterize the spectral envelope of the speech recordings. At the transcript level, a pre-trained embedding model is utilized to encompass the semantic and contextual features of the textual information. Additionally, the hybrid of the signal and transcript levels is explored and analyzed. The designed classification model relies on stacked layers of deep neural networks and convolutional neural networks. Evaluation results show that the model achieved a higher level of precision when compared with the manual evaluation approach followed by Altibbi’s operations team.

Download Full-text

Tobacco Leaf Grading Based on Deep Convolutional Neural Networks and Machine Vision

Journal of the ASABE ◽

10.13031/ja.14537 ◽

2021 ◽

Vol 65 (1) ◽

pp. 11-22

Author(s):

Mengyao Lu ◽

Shuwen Jiang ◽

Cong Wang ◽

Dong Chen ◽

Tian’en Chen

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Transfer Learning ◽

Convolutional Neural Networks ◽

Classification Accuracy ◽

Classification Model ◽

List Type ◽

Tobacco Leaves ◽

Tobacco Leaf ◽

Grading Model

HighlightsA classification model for the front and back sides of tobacco leaves was developed for application in industry.A tobacco leaf grading method that combines a CNN with double-branch integration was proposed.The A-ResNet network was proposed and compared with other classic CNN networks.The grading accuracy of eight different grades was 91.30% and the testing time was 82.180 ms, showing a relatively high classification accuracy and efficiency.Abstract. Flue-cured tobacco leaf grading is a key step in the production and processing of Chinese-style cigarette raw materials, directly affecting cigarette blend and quality stability. At present, manual grading of tobacco leaves is dominant in China, resulting in unsatisfactory grading quality and consuming considerable material and financial resources. In this study, for fast, accurate, and non-destructive tobacco leaf grading, 2,791 flue-cured tobacco leaves of eight different grades in south Anhui Province, China, were chosen as the study sample, and a tobacco leaf grading method that combines convolutional neural networks and double-branch integration was proposed. First, a classification model for the front and back sides of tobacco leaves was trained by transfer learning. Second, two processing methods (equal-scaled resizing and cropping) were used to obtain global images and local patches from the front sides of tobacco leaves. A global image-based tobacco leaf grading model was then developed using the proposed A-ResNet-65 network, and a local patch-based tobacco leaf grading model was developed using the ResNet-34 network. These two networks were compared with classic deep learning networks, such as VGGNet, GoogLeNet-V3, and ResNet. Finally, the grading results of the two grading models were integrated to realize tobacco leaf grading. The tobacco leaf classification accuracy of the final model, for eight different grades, was 91.30%, and grading of a single tobacco leaf required 82.180 ms. The proposed method achieved a relatively high grading accuracy and efficiency. It provides a method for industrial implementation of the tobacco leaf grading and offers a new approach for the quality grading of other agricultural products. Keywords: Convolutional neural network, Deep learning, Image classification, Transfer learning, Tobacco leaf grading

Download Full-text

Using Convolutional Neural Networks for the Detection of Spinal Transverse Processes

Inquiry@Queen's Undergraduate Research Conference Proceedings ◽

10.24908/iqurcp.13355 ◽

2019 ◽

Author(s):

Victoria Wu

Keyword(s):

Neural Network ◽

Neural Networks ◽

Deep Learning ◽

Convolutional Neural Network ◽

Transverse Process ◽

Classification Model ◽

Ultrasound Images ◽

Ultrasound Scan ◽

X Ray ◽

Scoliosis Screening

Introduction: Scoliosis, an excessive curvature of the spine, affects approximately 1 in 1,000 individuals. As a result, there have formerly been implementations of mandatory scoliosis screening procedures. Screening programs are no longer widely used as the harms often outweigh the benefits; it causes many adolescents to undergo frequent diagnosis X-ray procedure This makes spinal ultrasounds an ideal substitute for scoliosis screening in patients, as it does not expose them to those levels of radiation. Spinal curvatures can be accurately computed from the location of spinal transverse processes, by measuring the vertebral angle from a reference line [1]. However, ultrasound images are less clear than x-ray images, making it difficult to identify the spinal processes. To overcome this, we employ deep learning using a convolutional neural network, which is a powerful tool for computer vision and image classification [2]. Method: A total of 2,752 ultrasound images were recorded from a spine phantom to train a convolutional neural network. Subsequently, we took another recording of 747 images to be used for testing. All the ultrasound images from the scans were then segmented manually, using the 3D Slicer (www.slicer.org) software. Next, the dataset was fed through a convolutional neural network. The network used was a modified version of GoogLeNet (Inception v1), with 2 linearly stacked inception models. This network was chosen because it provided a balance between accurate performance, and time efficient computations. Results: Deep learning classification using the Inception model achieved an accuracy of 84% for the phantom scan. Conclusion: The classification model performs with considerable accuracy. Better accuracy needs to be achieved, possibly with more available data and improvements in the classification model. Acknowledgements: G. Fichtinger is supported as a Canada Research Chair in Computer-Integrated Surgery. This work was funded, in part, by NIH/NIBIB and NIH/NIGMS (via grant 1R01EB021396-01A1 - Slicer+PLUS: Point-of-Care Ultrasound) and by CANARIE’s Research Software Program. Figure 1: Ultrasound scan containing a transverse process (left), and ultrasound scan containing no transverse process (right). Figure 2: Accuracy of classification for training (red) and validation (blue). References: Ungi T, King F, Kempston M, Keri Z, Lasso A, Mousavi P, Rudan J, Borschneck DP, Fichtinger G. Spinal Curvature Measurement by Tracked Ultrasound Snapshots. Ultrasound in Medicine and Biology, 40(2):447-54, Feb 2014. Krizhevsky A, Sutskeyer I, Hinton GE. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25:1097-1105.

Download Full-text

DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment

Sensors ◽

10.3390/s20226457 ◽

2020 ◽

Vol 20 (22) ◽

pp. 6457

Author(s):

Hayat Ullah ◽

Muhammad Irfan ◽

Kyungjin Han ◽

Jong Weon Lee

Keyword(s):

Deep Learning ◽

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

State Of The Art ◽

Reference Image ◽

Primary Concern ◽

Panoramic Images ◽

Stitching Errors

Due to recent advancements in virtual reality (VR) and augmented reality (AR), the demand for high quality immersive contents is a primary concern for production companies and consumers. Similarly, the topical record-breaking performance of deep learning in various domains of artificial intelligence has extended the attention of researchers to contribute to different fields of computer vision. To ensure the quality of immersive media contents using these advanced deep learning technologies, several learning based Stitched Image Quality Assessment methods have been proposed with reasonable performances. However, these methods are unable to localize, segment, and extract the stitching errors in panoramic images. Further, these methods used computationally complex procedures for quality assessment of panoramic images. With these motivations, in this paper, we propose a novel three-fold Deep Learning based No-Reference Stitched Image Quality Assessment (DLNR-SIQA) approach to evaluate the quality of immersive contents. In the first fold, we fined-tuned the state-of-the-art Mask R-CNN (Regional Convolutional Neural Network) on manually annotated various stitching error-based cropped images from the two publicly available datasets. In the second fold, we segment and localize various stitching errors present in the immersive contents. Finally, based on the distorted regions present in the immersive contents, we measured the overall quality of the stitched images. Unlike existing methods that only measure the quality of the images using deep features, our proposed method can efficiently segment and localize stitching errors and estimate the image quality by investigating segmented regions. We also carried out extensive qualitative and quantitative comparison with full reference image quality assessment (FR-IQA) and no reference image quality assessment (NR-IQA) on two publicly available datasets, where the proposed system outperformed the existing state-of-the-art techniques.

Download Full-text

Face Image Quality Assessment: A Literature Survey

ACM Computing Surveys ◽

10.1145/3507901 ◽

2022 ◽

Author(s):

Torsten Schlett ◽

Christian Rathgeb ◽

Olaf Henniger ◽

Javier Galbally ◽

Julian Fierrez ◽

...

Keyword(s):

Deep Learning ◽

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

Face Image ◽

Quality Data ◽

Learning Approaches ◽

The Face ◽

Future Work

The performance of face analysis and recognition systems depends on the quality of the acquired face data, which is influenced by numerous factors. Automatically assessing the quality of face data in terms of biometric utility can thus be useful to detect low-quality data and make decisions accordingly. This survey provides an overview of the face image quality assessment literature, which predominantly focuses on visible wavelength face image input. A trend towards deep learning based methods is observed, including notable conceptual differences among the recent approaches, such as the integration of quality assessment into face recognition models. Besides image selection, face image quality assessment can also be used in a variety of other application scenarios, which are discussed herein. Open issues and challenges are pointed out, i.a. highlighting the importance of comparability for algorithm evaluations, and the challenge for future work to create deep learning approaches that are interpretable in addition to providing accurate utility predictions.

Download Full-text

Modelling the niches of wild and domesticated Ungulate species using deep learning

10.1101/744441 ◽

2019 ◽

Cited By ~ 1

Author(s):

Mark Rademaker ◽

Laurens Hogeweg ◽

Rutger Vos

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Species Interactions ◽

Drop Out ◽

Classification Model ◽

Strong Increase ◽

Sample Sizes ◽

Environmental Predictors ◽

Suitable Alternative ◽

The Past

AbstractKnowledge of global biodiversity remains limited by geographic and taxonomic sampling biases. The scarcity of species data restricts our understanding of the underlying environmental factors shaping distributions, and the ability to draw comparisons among species. Species distribution models (SDMs) were developed in the early 2000s to address this issue. Although SDMs based on single layered Neural Networks have been experimented with in the past, these performed poorly. However, the past two decades have seen a strong increase in the use of Deep Learning (DL) approaches, such as Deep Neural Networks (DNNs). Despite the large improvement in predictive capacity DNNs provide over shallow networks, to our knowledge these have not yet been applied to SDM. The aim of this research was to provide a proof of concept of a DL-SDM1. We used a pre-existing dataset of the world’s ungulates and abiotic environmental predictors that had recently been used in MaxEnt SDM, to allow for a direct comparison of performance between both methods. Our DL-SDM consisted of a binary classification DNN containing 4 hidden layers and drop-out regularization between each layer. Performance of the DL-SDM was similar to MaxEnt for species with relatively large sample sizes and worse for species with relatively low sample sizes. Increasing the number of occurrences further improved DL-SDM performance for species that already had relatively high sample sizes. We then tried to further improve performance by altering the sampling procedure of negative instances and increasing the number of environmental predictors, including species interactions. This led to a large increase in model performance across the range of sample sizes in the species datasets. We conclude that DL-SDMs provide a suitable alternative to traditional SDMs such as MaxEnt and have the advantage of being both able to directly include species interactions, as well as being able to handle correlated input features. Further improvements to the model would include increasing its scalability by turning it into a multi-classification model, as well as developing a more user friendly DL-SDM Python package.

Download Full-text

CNN-based image quality assessment of consumer photographs

London Imaging Meeting ◽

10.2352/issn.2694-118x.2020.lim-47 ◽

2020 ◽

Vol 2020 (1) ◽

pp. 129-133

Author(s):

Luigi Celona ◽

Raimondo Schettini

Keyword(s):

Neural Networks ◽

Image Quality ◽

Quality Assessment ◽

Image Quality Assessment ◽

Building Blocks ◽

Training Data ◽

Computer Vision Applications ◽

Feasible Solutions ◽

World Image

Convolutional neural networks (CNNs) are used in an increasingly systematic way in a great variety of computer vision applications, including image quality assessment. However, their application to evaluate the perceived quality of images is strongly limited by the lack of adequate and consistent training data. A CNN-based framework for evaluating image quality of consumer photographs is made up of several building blocks that can be implemented in different ways. In this article, we schematically illustrate how these building blocks have been implemented and combined so far to create feasible solutions that take the most positive characteristics of CNNs while mitigating their intrinsic limitations. Some experimental results are reported to show the effectiveness of CNN-based solutions on real-world image quality datasets.

Download Full-text

Recurrence plot and convolutional neural network-based signal quality assessment method for using photoplethysmogram in mobile environment (Preprint)

10.2196/preprints.24054 ◽

2020 ◽

Author(s):

Hangsik Shin

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Quality Assessment ◽

Feature Detection ◽

Layer Structure ◽

Recurrence Plot ◽

Classification Model ◽

Signal Quality ◽

Predictive Values

BACKGROUND In clinical use of photoplethysmogram, waveform distortion due to motion noise or low perfusion may lead to inaccurate analysis and diagnostic results. Therefore, it is necessary to find an appropriate analysis method to evaluate the signal quality of the photoplethysmogram so that its wide use in mobile healthcare can be further increased. OBJECTIVE The purpose of this study was to develop a machine learning model that could accurately evaluate the quality of a photoplethysmogram based on the shape of the photoplethysmogram and the phase relevance in a pulsatile waveform without requiring a complicated pre-processing. Its performance was then verified. METHODS Photoplethysmograms were recorded for 76 participants (5 minutes for each participant). All recorded photoplethysmograms were segmented for each beat to obtain a total of 49,561 pulsatile segments. These pulsatile segments were manually labeled as 'good' and 'bad' classes and converted to a two-dimensional phase space trajectory image with size of 124 × 124 using a recurrence plot. The classification model was implemented using a convolutional neural network with a two-layer structure. It was verified through a five-fold cross validation. RESULTS As a result, the proposed model correctly classified 48,827 segments out of 49,561 segments and misclassified 734 segments, showing a balanced accuracy of 0.975. Sensitivity, specificity, and positive predictive values of the developed model for the test dataset with a ‘bad’ class classification were 0.964, 0.987, and 0.848, respectively. The area under the curve was 0.994. CONCLUSIONS The convolutional neural network model with recurrence plot as input proposed in this study can be used for signal quality assessment as a generalized model with high accuracy through data expansion. It has an advantage in that it does not require a complicated pre-processing or feature detection process. CLINICALTRIAL KCT0002080

Download Full-text

Automated Brain Imaging Diagnosis and Classification Model using Rat Swarm Optimization with Deep Learning based Capsule Network

International Journal of Image and Graphics ◽

10.1142/s0219467822400010 ◽

2021 ◽

pp. 2240001

Author(s):

A. Vasantharaj ◽

Pacha Shoba Rani ◽

Sirajul Huque ◽

K. S. Raghuram ◽

R. Ganeshkumar ◽

...

Keyword(s):

Deep Learning ◽

Classification Model ◽

Swarm Optimization ◽

Diagnosis And Classification ◽

Enhance Efficiency ◽

Magnetic Resonance Imaging Mri ◽

Learning Machine ◽

The Brain

Earlier identification of brain tumor (BT) is essential to increase the survival rate of the patients. The commonly used imaging technique for BT diagnosis is magnetic resonance imaging (MRI). Automated BT classification model is required for assisting the radiologists to save time and enhance efficiency. The classification of BT is difficult owing to the non-uniform shapes of tumors and location of tumors in the brain. Therefore, deep learning (DL) models can be employed for the effective identification, prediction, and diagnosis of diseases. In this view, this paper presents an automated BT diagnosis using rat swarm optimization (RSO) with deep learning based capsule network (DLCN) model, named RSO-DLCN model. The presented RSO-DLCN model involves bilateral filtering (BF) based preprocessing to enhance the quality of the MRI. Besides, non-iterative grabcut based segmentation (NIGCS) technique is applied to detect the affected tumor regions. In addition, DLCN model based feature extractor with RSO algorithm based parameter optimization processes takes place. Finally, extreme learning machine with stacked autoencoder (ELM-SA) based classifier is employed for the effective classification of BT. For validating the BT diagnostic performance of the presented RSO-DLCN model, an extensive set of simulations were carried out and the results are inspected under diverse dimensions. The simulation outcome demonstrated the promising results of the RSO-DLCN model on BT diagnosis with the sensitivity of 98.4%, specificity of 99%, and accuracy of 98.7%.

Download Full-text

Identifying Emotion on Indonesian Tweets using Convolutional Neural Networks

Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) ◽

10.29207/resti.v5i3.3137 ◽

2021 ◽

Vol 5 (3) ◽

pp. 584-593

Author(s):

Naufal Hilmiaji ◽

Kemas Muslim Lhaksmana ◽

Mahendra Dwifebri Purbolaksono

Keyword(s):

Neural Network ◽

Neural Networks ◽

Performance Evaluation ◽

Deep Learning ◽

Convolutional Neural Network ◽

Convolutional Neural Networks ◽

Text Classification ◽

Classification Model ◽

Learning Methods ◽

Expected Performance

especially with the advancement of deep learning methods for text classification. Despite some effort to identify emotion on Indonesian tweets, its performance evaluation results have not achieved acceptable numbers. To solve this problem, this paper implements a classification model using a convolutional neural network (CNN), which has demonstrated expected performance in text classification. To easily compare with the previous research, this classification is performed on the same dataset, which consists of 4,403 tweets in Indonesian that were labeled using five different emotion classes: anger, fear, joy, love, and sadness. The performance evaluation results achieve the precision, recall, and F1-score at respectively 90.1%, 90.3%, and 90.2%, while the highest accuracy achieves 89.8%. These results outperform previous research that classifies the same classification on the same dataset.

Download Full-text

Using Slit-Lamp Images for Deep Learning-Based Identification of Bacterial and Fungal Keratitis: Model Development and Validation with Different Convolutional Neural Networks

Diagnostics ◽

10.3390/diagnostics11071246 ◽

2021 ◽

Vol 11 (7) ◽

pp. 1246

Author(s):

Ning Hung ◽

Andy Kuan-Yu Shih ◽

Chihung Lin ◽

Ming-Tse Kuo ◽

Yih-Shiou Hwang ◽

...

Keyword(s):

Neural Networks ◽

Deep Learning ◽

Diagnostic Accuracy ◽

Convolutional Neural Networks ◽

Learning Algorithm ◽

Model Performance ◽

Fungal Keratitis ◽

Classification Model ◽

Slit Lamp ◽

Deep Learning Algorithm

In this study, we aimed to develop a deep learning model for identifying bacterial keratitis (BK) and fungal keratitis (FK) by using slit-lamp images. We retrospectively collected slit-lamp images of patients with culture-proven microbial keratitis between 1 January 2010 and 31 December 2019 from two medical centers in Taiwan. We constructed a deep learning algorithm consisting of a segmentation model for cropping cornea images and a classification model that applies different convolutional neural networks (CNNs) to differentiate between FK and BK. The CNNs included DenseNet121, DenseNet161, DenseNet169, DenseNet201, EfficientNetB3, InceptionV3, ResNet101, and ResNet50. The model performance was evaluated and presented as the area under the curve (AUC) of the receiver operating characteristic curves. A gradient-weighted class activation mapping technique was used to plot the heat map of the model. By using 1330 images from 580 patients, the deep learning algorithm achieved the highest average accuracy of 80.0%. Using different CNNs, the diagnostic accuracy for BK ranged from 79.6% to 95.9%, and that for FK ranged from 26.3% to 65.8%. The CNN of DenseNet161 showed the best model performance, with an AUC of 0.85 for both BK and FK. The heat maps revealed that the model was able to identify the corneal infiltrations. The model showed a better diagnostic accuracy than the previously reported diagnostic performance of both general ophthalmologists and corneal specialists.

Download Full-text