On Parameter Adaptation in Softmax-Based Cross-Entropy Loss for Improved Convergence Speed and Accuracy in DNN-Based Speaker Recognition

Image paragraph generation aims to describe an image with a paragraph in natural language. Compared to image captioning with a single sentence, paragraph generation provides more expressive and fine-grained description for storytelling. Existing approaches mainly optimize paragraph generator towards minimizing word-wise cross entropy loss, which neglects linguistic hierarchy of paragraph and results in ``sparse" supervision for generator learning. In this paper, we propose a novel Densely Supervised Hierarchical Policy-Value (DHPV) network for effective paragraph generation. We design new hierarchical supervisions consisting of hierarchical rewards and values at both sentence and word levels. The joint exploration of hierarchical rewards and values provides dense supervision cues for learning effective paragraph generator. We propose a new hierarchical policy-value architecture which exploits compositionality at token-to-token and sentence-to-sentence levels simultaneously and can preserve the semantic and syntactic constituent integrity. Extensive experiments on the Stanford image-paragraph benchmark have demonstrated the effectiveness of the proposed DHPV approach with performance improvements over multiple state-of-the-art methods.

Download Full-text

Glowworm Swarm Optimization and Its Application to Blind Signal Separation

Mathematical Problems in Engineering ◽

10.1155/2016/5481602 ◽

2016 ◽

Vol 2016 ◽

pp. 1-8 ◽

Cited By ~ 2

Author(s):

Zhucheng Li ◽

Xianglin Huang

Keyword(s):

Objective Function ◽

Global Optimum ◽

Convergence Speed ◽

Signal Separation ◽

Blind Signal Separation ◽

Swarm Optimization ◽

Blind Signal ◽

Glowworm Swarm Optimization ◽

Simulation Results ◽

Speed And Accuracy

Traditional optimization algorithms for blind signal separation (BSS) are mainly based on the gradient, which requires the objective function to be continuous and differentiable, so the applications of these algorithms are very limited. Moreover, these algorithms have problems with the convergence speed and accuracy. To overcome these drawbacks, this paper presents a modified glowworm swarm optimization (MGSO) algorithm based on a novel step adjustment rule and then applies MGSO to BSS. Taking kurtosis of the mixed signals as the objective function of BSS, MGSO-BSS succeeds in separating the mixed signals in Matlab environment. The simulation results prove that MGSO is more effective in capturing the global optimum of the objective function of the BSS algorithm and has faster convergence speed and higher accuracy, compared with particle swarm optimization (PSO) and GSO.

Download Full-text

Large-Margin Regularized Softmax Cross-Entropy Loss

IEEE Access ◽

10.1109/access.2019.2897692 ◽

2019 ◽

Vol 7 ◽

pp. 19572-19578 ◽

Cited By ~ 4

Author(s):

Xiaoxu Li ◽

Dongliang Chang ◽

Tao Tian ◽

Jie Cao

Keyword(s):

Cross Entropy ◽

Large Margin ◽

Entropy Loss

Download Full-text

Strong-Background Restrained Cross Entropy Loss for Scene Text Detection

2019 International Joint Conference on Neural Networks (IJCNN) ◽

10.1109/ijcnn.2019.8852198 ◽

2019 ◽

Author(s):

Randong Huang ◽

Bo Xu

Keyword(s):

Text Detection ◽

Cross Entropy ◽

Entropy Loss ◽

Scene Text Detection ◽

Scene Text

Download Full-text

Hybrid Model Structure for Diabetic Retinopathy Classification

Journal of Healthcare Engineering ◽

10.1155/2020/8840174 ◽

2020 ◽

Vol 2020 ◽

pp. 1-9

Author(s):

Hao Liu ◽

Keqiang Yue ◽

Siyi Cheng ◽

Chengming Pan ◽

Jie Sun ◽

...

Keyword(s):

Diabetic Retinopathy ◽

Hybrid Model ◽

Network Models ◽

Classification Performance ◽

Cross Entropy ◽

Model Structure ◽

Training Process ◽

Neural Network Models ◽

Entropy Loss ◽

Model Structures

Diabetic retinopathy (DR) is one of the most common complications of diabetes and the main cause of blindness. The progression of the disease can be prevented by early diagnosis of DR. Due to differences in the distribution of medical conditions and low labor efficiency, the best time for diagnosis and treatment was missed, which results in impaired vision. Using neural network models to classify and diagnose DR can improve efficiency and reduce costs. In this work, an improved loss function and three hybrid model structures Hybrid-a, Hybrid-f, and Hybrid-c were proposed to improve the performance of DR classification models. EfficientNetB4, EfficientNetB5, NASNetLarge, Xception, and InceptionResNetV2 CNNs were chosen as the basic models. These basic models were trained using enhance cross-entropy loss and cross-entropy loss, respectively. The output of the basic models was used to train the hybrid model structures. Experiments showed that enhance cross-entropy loss can effectively accelerate the training process of the basic models and improve the performance of the models under various evaluation metrics. The proposed hybrid model structures can also improve DR classification performance. Compared with the best-performing results in the basic models, the accuracy of DR classification was improved from 85.44% to 86.34%, the sensitivity was improved from 98.48% to 98.77%, the specificity was improved from 71.82% to 74.76%, the precision was improved from 90.27% to 91.37%, and the F1 score was improved from 93.62% to 93.9% by using hybrid model structures.

Download Full-text