Cross-Entropy Loss for Recommending Efficient Fold-Over Technique

2021 ◽  
Vol 34 (1) ◽  
pp. 402-439
Author(s):  
Lin-Chen Weng ◽  
A. M. Elsawah ◽  
Kai-Tai Fang
Keyword(s):  
Cross Entropy ◽  
Entropy Loss

scholarly journals Improving Neural Response Diversity with Frequency-Aware Cross-Entropy Loss

2019 ◽  
Author(s):  
Shaojie Jiang ◽  
Pengjie Ren ◽  
Christof Monz ◽  
Maarten de Rijke
Keyword(s):  
Neural Response ◽  
Cross Entropy ◽  
Entropy Loss ◽  
Response Diversity

scholarly journals Densely Supervised Hierarchical Policy-Value Network for Image Paragraph Generation

2019 ◽  
Author(s):  
Siying Wu ◽  
Zheng-Jun Zha ◽  
Zilei Wang ◽  
Houqiang Li ◽  
Feng Wu
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
Cross Entropy ◽  
Image Captioning ◽  
Value Network ◽  
Entropy Loss ◽  
Fine Grained ◽  
Performance Improvements ◽  
Single Sentence ◽  
Multiple State

Image paragraph generation aims to describe an image with a paragraph in natural language. Compared to image captioning with a single sentence, paragraph generation provides more expressive and fine-grained description for storytelling. Existing approaches mainly optimize paragraph generator towards minimizing word-wise cross entropy loss, which neglects linguistic hierarchy of paragraph and results in ``sparse" supervision for generator learning. In this paper, we propose a novel Densely Supervised Hierarchical Policy-Value (DHPV) network for effective paragraph generation. We design new hierarchical supervisions consisting of hierarchical rewards and values at both sentence and word levels. The joint exploration of hierarchical rewards and values provides dense supervision cues for learning effective paragraph generator. We propose a new hierarchical policy-value architecture which exploits compositionality at token-to-token and sentence-to-sentence levels simultaneously and can preserve the semantic and syntactic constituent integrity. Extensive experiments on the Stanford image-paragraph benchmark have demonstrated the effectiveness of the proposed DHPV approach with performance improvements over multiple state-of-the-art methods.

scholarly journals Large-Margin Regularized Softmax Cross-Entropy Loss

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 19572-19578 ◽  
Author(s):  
Xiaoxu Li ◽  
Dongliang Chang ◽  
Tao Tian ◽  
Jie Cao
Keyword(s):  
Cross Entropy ◽  
Large Margin ◽  
Entropy Loss

scholarly journals Hybrid Model Structure for Diabetic Retinopathy Classification

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Hao Liu ◽  
Keqiang Yue ◽  
Siyi Cheng ◽  
Chengming Pan ◽  
Jie Sun ◽  
...  
Keyword(s):  
Diabetic Retinopathy ◽  
Hybrid Model ◽  
Network Models ◽  
Classification Performance ◽  
Cross Entropy ◽  
Model Structure ◽  
Training Process ◽  
Neural Network Models ◽  
Entropy Loss ◽  
Model Structures

Diabetic retinopathy (DR) is one of the most common complications of diabetes and the main cause of blindness. The progression of the disease can be prevented by early diagnosis of DR. Due to differences in the distribution of medical conditions and low labor efficiency, the best time for diagnosis and treatment was missed, which results in impaired vision. Using neural network models to classify and diagnose DR can improve efficiency and reduce costs. In this work, an improved loss function and three hybrid model structures Hybrid-a, Hybrid-f, and Hybrid-c were proposed to improve the performance of DR classification models. EfficientNetB4, EfficientNetB5, NASNetLarge, Xception, and InceptionResNetV2 CNNs were chosen as the basic models. These basic models were trained using enhance cross-entropy loss and cross-entropy loss, respectively. The output of the basic models was used to train the hybrid model structures. Experiments showed that enhance cross-entropy loss can effectively accelerate the training process of the basic models and improve the performance of the models under various evaluation metrics. The proposed hybrid model structures can also improve DR classification performance. Compared with the best-performing results in the basic models, the accuracy of DR classification was improved from 85.44% to 86.34%, the sensitivity was improved from 98.48% to 98.77%, the specificity was improved from 71.82% to 74.76%, the precision was improved from 90.27% to 91.37%, and the F1 score was improved from 93.62% to 93.9% by using hybrid model structures.

scholarly journals Show, Recall, and Tell: Image Captioning with Recall Mechanism

2020 ◽  
Vol 34 (07) ◽  
pp. 12176-12183
Author(s):  
Li Wang ◽  
Zechen Bai ◽  
Yonghua Zhang ◽  
Hongtao Lu
Keyword(s):  
State Of The Art ◽  
Text Retrieval ◽  
Text Summarization ◽  
Cross Entropy ◽  
Image Captioning ◽  
Soft Switch ◽  
Entropy Loss ◽  
Human Conduct ◽  
Art Methods ◽  
The Way

Generating natural and accurate descriptions in image captioning has always been a challenge. In this paper, we propose a novel recall mechanism to imitate the way human conduct captioning. There are three parts in our recall mechanism : recall unit, semantic guide (SG) and recalled-word slot (RWS). Recall unit is a text-retrieval module designed to retrieve recalled words for images. SG and RWS are designed for the best use of recalled words. SG branch can generate a recalled context, which can guide the process of generating caption. RWS branch is responsible for copying recalled words to the caption. Inspired by pointing mechanism in text summarization, we adopt a soft switch to balance the generated-word probabilities between SG and RWS. In the CIDEr optimization step, we also introduce an individual recalled-word reward (WR) to boost training. Our proposed methods (SG+RWS+WR) achieve BLEU-4 / CIDEr / SPICE scores of 36.6 / 116.9 / 21.3 with cross-entropy loss and 38.7 / 129.1 / 22.4 with CIDEr optimization on MSCOCO Karpathy test split, which surpass the results of other state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document