scholarly journals Multi-label topic classification for COVID-19 literature annotation using an ensemble model based on PubMedBERT

2021 ◽  
Author(s):  
Shubo Tian ◽  
Jinfeng Zhang
Keyword(s):  
Deep Learning ◽  
Loss Function ◽  
Language Model ◽  
Ensemble Model ◽  
Test Results ◽  
Learning Models ◽  
Baseline Model ◽  
Satisfactory Performance ◽  
The Mean ◽  
Better Than

The BioCreative VII Track 5 calls for participants to tackle the multi-label classification task for automated topic annotation of COVID-19 literature. In our participation, we evaluated several deep learning models built on PubMedBERT, a pre-trained language model, with different strategies addressing the challenges of the task. Specifically, multi-instance learning was used to deal with the large variation in the lengths of the articles, and focal loss function was used to address the imbalance in the distribution of different topics. We found that the ensemble model performed the best among all the models we have tested. Test results of our submissions showed that our approach was able to achieve satisfactory performance with an F1 score of 0.9247, which is significantly better than the baseline model (F1 score: 0.8678) and the mean of all the submissions (F1 score: 0.8931).

scholarly journals Cotton Stand Counting from Unmanned Aerial System Imagery Using MobileNet and CenterNet Deep Learning Models

2021 ◽  
Vol 13 (14) ◽  
pp. 2822
Author(s):  
Zhe Lin ◽  
Wenxuan Guo
Keyword(s):  
Deep Learning ◽  
Cotton Plant ◽  
Unmanned Aerial System ◽  
Learning Models ◽  
Training Images ◽  
Testing Dataset ◽  
Cotton Plants ◽  
Detection And Counting ◽  
Different Dimensions ◽  
The Mean

An accurate stand count is a prerequisite to determining the emergence rate, assessing seedling vigor, and facilitating site-specific management for optimal crop production. Traditional manual counting methods in stand assessment are labor intensive and time consuming for large-scale breeding programs or production field operations. This study aimed to apply two deep learning models, the MobileNet and CenterNet, to detect and count cotton plants at the seedling stage with unmanned aerial system (UAS) images. These models were trained with two datasets containing 400 and 900 images with variations in plant size and soil background brightness. The performance of these models was assessed with two testing datasets of different dimensions, testing dataset 1 with 300 by 400 pixels and testing dataset 2 with 250 by 1200 pixels. The model validation results showed that the mean average precision (mAP) and average recall (AR) were 79% and 73% for the CenterNet model, and 86% and 72% for the MobileNet model with 900 training images. The accuracy of cotton plant detection and counting was higher with testing dataset 1 for both CenterNet and MobileNet models. The results showed that the CenterNet model had a better overall performance for cotton plant detection and counting with 900 training images. The results also indicated that more training images are required when applying object detection models on images with different dimensions from training datasets. The mean absolute percentage error (MAPE), coefficient of determination (R2), and the root mean squared error (RMSE) values of the cotton plant counting were 0.07%, 0.98 and 0.37, respectively, with testing dataset 1 for the CenterNet model with 900 training images. Both MobileNet and CenterNet models have the potential to accurately and timely detect and count cotton plants based on high-resolution UAS images at the seedling stage. This study provides valuable information for selecting the right deep learning tools and the appropriate number of training images for object detection projects in agricultural applications.

scholarly journals An Ensemble Energy Consumption Forecasting Model Based on Spatial-Temporal Clustering Analysis in Residential Buildings

Energies ◽  
2021 ◽  
Vol 14 (11) ◽  
pp. 3020
Author(s):  
Anam-Nawaz Khan ◽  
Naeem Iqbal ◽  
Atif Rizwan ◽  
Rashid Ahmad ◽  
Do-Hyeun Kim
Keyword(s):  
Cluster Analysis ◽  
Deep Learning ◽  
Residential Buildings ◽  
Ensemble Forecasting ◽  
Forecasting Model ◽  
Ensemble Model ◽  
Learning Models ◽  
Short Term ◽  
Floor Level ◽  
Consumption Data

Due to the availability of smart metering infrastructure, high-resolution electric consumption data is readily available to study the dynamics of residential electric consumption at finely resolved spatial and temporal scales. Analyzing the electric consumption data enables the policymakers and building owners to understand consumer’s demand-consumption behaviors. Furthermore, analysis and accurate forecasting of electric consumption are substantial for consumer involvement in time-of-use tariffs, critical peak pricing, and consumer-specific demand response initiatives. Alongside its vast economic and sustainability implications, such as energy wastage and decarbonization of the energy sector, accurate consumption forecasting facilitates power system planning and stable grid operations. Energy consumption forecasting is an active research area; despite the abundance of devised models, electric consumption forecasting in residential buildings remains challenging due to high occupant energy use behavior variability. Hence the search for an appropriate model for accurate electric consumption forecasting is ever continuing. To this aim, this paper presents a spatial and temporal ensemble forecasting model for short-term electric consumption forecasting. The proposed work involves exploring electric consumption profiles at the apartment level through cluster analysis based on the k-means algorithm. The ensemble forecasting model consists of two deep learning models; Long Short-Term Memory Unit (LSTM) and Gated Recurrent Unit (GRU). First, the apartment-level historical electric consumption data is clustered. Later the clusters are aggregated based on consumption profiles of consumers. At the building and floor level, the ensemble models are trained using aggregated electric consumption data. The proposed ensemble model forecasts the electric consumption at three spatial scales apartment, building, and floor level for hourly, daily, and weekly forecasting horizon. Furthermore, the impact of spatial-temporal granularity and cluster analysis on the prediction accuracy is analyzed. The dataset used in this study comprises high-resolution electric consumption data acquired through smart meters recorded on an hourly basis over the period of one year. The consumption data belongs to four multifamily residential buildings situated in an urban area of South Korea. To prove the effectiveness of our proposed forecasting model, we compared our model with widely known machine learning models and deep learning variants. The results achieved by our proposed ensemble scheme verify that model has learned the sequential behavior of electric consumption by producing superior performance with the lowest MAPE of 4.182 and 4.54 at building and floor level prediction, respectively. The experimental findings suggest that the model has efficiently captured the dynamic electric consumption characteristics to exploit ensemble model diversities and achieved lower forecasting error. The proposed ensemble forecasting scheme is well suited for predictive modeling and short-term load forecasting.

Parsing of Urban Facades from 3D Point Clouds Based on a Novel Multi-View Domain

2021 ◽  
Vol 87 (4) ◽  
pp. 283-293
Author(s):  
Wei Wang ◽  
Yuan Xu ◽  
Yingchao Ren ◽  
Gang Wang
Keyword(s):  
Deep Learning ◽  
Prior Knowledge ◽  
Performance Improvement ◽  
Data Distribution ◽  
Point Clouds ◽  
Learning Models ◽  
Data Set ◽  
3D Point Clouds ◽  
Segmentation Accuracy ◽  
The Mean

Recently, performance improvement in facade parsing from 3D point clouds has been brought about by designing more complex network structures, which cost huge computing resources and do not take full advantage of prior knowledge of facade structure. Instead, from the perspective of data distribution, we construct a new hierarchical mesh multi-view data domain based on the characteristics of facade objects to achieve fusion of deep-learning models and prior knowledge, thereby significantly improving segmentation accuracy. We comprehensively evaluate the current mainstream method on the RueMonge 2014 data set and demonstrate the superiority of our method. The mean intersection-over-union index on the facade-parsing task reached 76.41%, which is 2.75% higher than the current best result. In addition, through comparative experiments, the reasons for the performance improvement of the proposed method are further analyzed.

scholarly journals Development of Learning Tools by Application of Problem Based Learning Models to Improve Mathematical Communication Capabilities of Sequence and Series Materials

2020 ◽  
Vol 4 (1) ◽  
pp. 62
Author(s):  
Nur Atika ◽  
Yenita Roza ◽  
Atma Murni
Keyword(s):  
Communication Skills ◽  
Problem Based Learning ◽  
Development Research ◽  
Learning Tools ◽  
Test Results ◽  
Design Development ◽  
Learning Models ◽  
Mathematical Communication ◽  
Analysis Design ◽  
Better Than

This research was motivated by the lack of learning tools that can improve students' Mathematical Communication Skills (KKM). Activities that can encourage students to improve KKM have not been designed by the teacher in the learning device properly. This study aims to produce learning tools by applying Problem Based Learning models to improve KKM. This development research used the ADDIE development model with the stages of Analysis, Design, Development, Implementation, and Evaluation. The results of Syllabus, RPP, LKPD and KKM validation results showed highly valid criteria with percentages of 91.67%, 91.30%, 89.09%, and 87.15%, respectively. Practicality results indicate a very practical criterion with the percentage of small group trials at 95.57%, large group trials at 96.07%, and teacher responses at 96.25%. The effectiveness test results state that KKM of students who use learning tools with PBL models is better than KKM of students who use conventional learning.

scholarly journals Efficient Deep Learning Models for DGA Domain Detection

2021 ◽  
Vol 2021 ◽  
pp. 1-15
Author(s):  
Juhong Namgung ◽  
Siwoon Son ◽  
Yang-Sae Moon
Keyword(s):  
Deep Learning ◽  
Short Term Memory ◽  
Ensemble Model ◽  
Learning Models ◽  
Short Term ◽  
Domain Names ◽  
Additional Information ◽  
Domain Sequence ◽  
Long Short Term Memory ◽  
And Control

In recent years, cyberattacks using command and control (C&C) servers have significantly increased. To hide their C&C servers, attackers often use a domain generation algorithm (DGA), which automatically generates domain names for the C&C servers. Accordingly, extensive research on DGA domain detection has been conducted. However, existing methods cannot accurately detect continuously generated DGA domains and can easily be evaded by an attacker. Recently, long short-term memory- (LSTM-) based deep learning models have been introduced to detect DGA domains in real time using only domain names without feature extraction or additional information. In this paper, we propose an efficient DGA domain detection method based on bidirectional LSTM (BiLSTM), which learns bidirectional information as opposed to unidirectional information learned by LSTM. We further maximize the detection performance with a convolutional neural network (CNN) + BiLSTM ensemble model using Attention mechanism, which allows the model to learn both local and global information in a domain sequence. Experimental results show that existing CNN and LSTM models achieved F1-scores of 0.9384 and 0.9597, respectively, while the proposed BiLSTM and ensemble models achieved higher F1-scores of 0.9618 and 0.9666, respectively. In addition, the ensemble model achieved the best performance for most DGA domain classes, enabling more accurate DGA domain detection than existing models.

A Review of Recent Deep Learning Models in COVID-19 Diagnosis

2021 ◽  
Vol 6 (5) ◽  
pp. 10-15
Author(s):  
Ela Bhattacharya ◽  
D. Bhattacharya
Keyword(s):  
Artificial Intelligence ◽  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Test Results ◽  
Learning Models ◽  
Future Directions ◽  
Human Contact ◽  
The World ◽  
Short Span

COVID-19 has emerged as the latest worrisome pandemic, which is reported to have its outbreak in Wuhan, China. The infection spreads by means of human contact, as a result, it has caused massive infections across 200 countries around the world. Artificial intelligence has likewise contributed to managing the COVID-19 pandemic in various aspects within a short span of time. Deep Neural Networks that are explored in this paper have contributed to the detection of COVID-19 from imaging sources. The datasets, pre-processing, segmentation, feature extraction, classification and test results which can be useful for discovering future directions in the domain of automatic diagnosis of the disease, utilizing artificial intelligence-based frameworks, have been investigated in this paper.

scholarly journals AUTOMATED MARINE OIL SPILL DETECTION USING DEEP LEARNING INSTANCE SEGMENTATION MODEL

2020 ◽  
Vol XLIII-B3-2020 ◽  
pp. 1271-1276
Author(s):  
S. T. Yekeen ◽  
A.-L. Balogun
Keyword(s):  
Deep Learning ◽  
Oil Spill ◽  
Semantic Segmentation ◽  
Learning Models ◽  
Marine Oil ◽  
Conventional Machine ◽  
Feature Pyramid ◽  
Model Training ◽  
Instance Segmentation ◽  
Better Than

Abstract. This study developed a novel deep learning oil spill instance segmentation model using Mask-Region-based Convolutional Neural Network (Mask R-CNN) model which is a state-of-the-art computer vision model. A total of 2882 imageries containing oil spill, look-alike, ship, and land area after conducting different pre-processing activities were acquired. These images were subsequently sub-divided into 88% training and 12% for testing, equating to 2530 and 352 images respectively. The model training was conducted using transfer learning on a pre-trained ResNet 101 with COCO data as a backbone in combination with Feature Pyramid Network (FPN) architecture for the extraction of features at 30 epochs with 0.001 learning rate. The model’s performance was evaluated using precision, recall, and F1-measure which shows a higher performance than other existing models with value of 0.964, 0.969 and 0.968 respectively. As a specialized task, the study concluded that the developed deep learning instance segmentation model (Mask R-CNN) performs better than conventional machine learning models and semantic segmentation deep learning models in detection and segmentation of marine oil spill.

scholarly journals Accurate recognition of colorectal cancer with semi-supervised deep learning on pathological images

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Gang Yu ◽  
Kai Sun ◽  
Chao Xu ◽  
Xing-Hua Shi ◽  
Chong Wu ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Colorectal Cancer ◽  
Deep Learning ◽  
Supervised Learning ◽  
Area Under The Curve ◽  
Patient Level ◽  
Significant Difference ◽  
The Mean ◽  
Whole Slide Images ◽  
Better Than

AbstractMachine-assisted pathological recognition has been focused on supervised learning (SL) that suffers from a significant annotation bottleneck. We propose a semi-supervised learning (SSL) method based on the mean teacher architecture using 13,111 whole slide images of colorectal cancer from 8803 subjects from 13 independent centers. SSL (~3150 labeled, ~40,950 unlabeled; ~6300 labeled, ~37,800 unlabeled patches) performs significantly better than the SL. No significant difference is found between SSL (~6300 labeled, ~37,800 unlabeled) and SL (~44,100 labeled) at patch-level diagnoses (area under the curve (AUC): 0.980 ± 0.014 vs. 0.987 ± 0.008, P value = 0.134) and patient-level diagnoses (AUC: 0.974 ± 0.013 vs. 0.980 ± 0.010, P value = 0.117), which is close to human pathologists (average AUC: 0.969). The evaluation on 15,000 lung and 294,912 lymph node images also confirm SSL can achieve similar performance as that of SL with massive annotations. SSL dramatically reduces the annotations, which has great potential to effectively build expert-level pathological artificial intelligence platforms in practice.

scholarly journals A Novel Intelligent System for Detection of Type 2 Diabetes with Modified Loss Function and Regularization

2021 ◽  
Vol 33 (2) ◽  
pp. 93-114
Author(s):  
Mallika G.C. ◽  
Abeer Alsadoon ◽  
Duong Thu Hang Pham ◽  
Salma Hameedi Abdullah ◽  
Ha Thi Mai ◽  
...  
Keyword(s):  
Machine Learning ◽  
Type 2 Diabetes ◽  
Deep Learning ◽  
Intelligent System ◽  
Model Solution ◽  
Majority Voting ◽  
Ensemble Model ◽  
Pima Indians ◽  
Learning Models

Type 2 Diabetes (T2DM) makes up about 90% of diabetes cases, as well as tough restriction on continuous monitoring and detecting become one of key aspects in T2DM. This research aims to develop an ensemble of several machine learning and deep learning models for early detection of T2DM with high accuracy. With high diversity of models, the ensemble will provide more excessive performance than single models. Methodology: The proposed system is modified enhanced ensemble of machine learning models for T2DM prediction. It is composed of Logistic Regression, Random Forest, SVM and Deep Neural Network models to generate a modified ensemble model. Results: The output of each model in the modified ensemble is used to figure out the final output of the system. The datasets being used for these models include Practice Fusion HER, Pima Indians diabetic's data, UCI AIM94 Dataset and CA Diabetes Prevalence 2014. In comparison to the previous solutions, the proposed ensemble model solution exposes the effectiveness of accuracy, sensitivity, and specificity. It provides an accuracy of 87.5% from 83.51% in average, sensitivity of 35.8% from 29.59% as well as specificity of 98.9% from 96.27%. The processing time of the proposed model solution with 96.6ms is faster than the state-of-the-art with 97.5ms. Conclusion: The proposed modified enhanced system in this work improves the overall prediction capability of T2DM using an ensemble of several machine learning and deep learning models. A majority voting scheme utilizes the output from several models to make the final accurate prediction. Regularization function in this work is modified in order to include the regularization of all the models in ensemble, that helps prevent the overfitting and encourages the generalization capacity of the proposed system.

Dissimilarity-Based Regularized Learning of Charts

2021 ◽  
Vol 17 (4) ◽  
pp. 1-23
Author(s):  
Prerna Mishra ◽  
Santosh Kumar ◽  
Mithilesh Kumar Chaube
Keyword(s):  
Deep Learning ◽  
Loss Function ◽  
Structural Variation ◽  
Learning Model ◽  
Discriminative Power ◽  
Learning Models ◽  
Dissimilarity Index ◽  
Regularization Parameters ◽  
Learned Features

Chart images exhibit significant variabilities that make each image different from others even though they belong to the same class or categories. Classification of charts is a major challenge because each chart class has variations in features, structure, and noises. However, due to the lack of affiliation between the dissimilar features and the structure of the chart, it is a challenging task to model these variations for automatic chart recognition. In this article, we present a novel dissimilarity-based learning model for similar structured but diverse chart classification. Our approach jointly learns the features of both dissimilar and similar regions. The model is trained by an improved loss function, which is fused by a structural variation-aware dissimilarity index and incorporated with regularization parameters, making the model more prone toward dissimilar regions. The dissimilarity index enhances the discriminative power of the learned features not only from dissimilar regions but also from similar regions. Extensive comparative evaluations demonstrate that our approach significantly outperforms other benchmark methods, including both traditional and deep learning models, over publicly available datasets.

Sign in / Sign up

Export Citation Format

Share Document