On Parameter Adaptation in Softmax-Based Cross-Entropy Loss for Improved Convergence Speed and Accuracy in DNN-Based Speaker Recognition

Author(s):  
Magdalena Rybicka ◽  
Konrad Kowalczyk
2021 ◽  
Vol 34 (1) ◽  
pp. 402-439
Author(s):  
Lin-Chen Weng ◽  
A. M. Elsawah ◽  
Kai-Tai Fang
Keyword(s):  

Author(s):  
Siying Wu ◽  
Zheng-Jun Zha ◽  
Zilei Wang ◽  
Houqiang Li ◽  
Feng Wu

Image paragraph generation aims to describe an image with a paragraph in natural language. Compared to image captioning with a single sentence, paragraph generation provides more expressive and fine-grained description for storytelling. Existing approaches mainly optimize paragraph generator towards minimizing word-wise cross entropy loss, which neglects linguistic hierarchy of paragraph and results in ``sparse" supervision for generator learning. In this paper, we propose a novel Densely Supervised Hierarchical Policy-Value (DHPV) network for effective paragraph generation. We design new hierarchical supervisions consisting of hierarchical rewards and values at both sentence and word levels. The joint exploration of hierarchical rewards and values provides dense supervision cues for learning effective paragraph generator. We propose a new hierarchical policy-value architecture which exploits compositionality at token-to-token and sentence-to-sentence levels simultaneously and can preserve the semantic and syntactic constituent integrity. Extensive experiments on the Stanford image-paragraph benchmark have demonstrated the effectiveness of the proposed DHPV approach with performance improvements over multiple state-of-the-art methods.


2016 ◽  
Vol 2016 ◽  
pp. 1-8 ◽  
Author(s):  
Zhucheng Li ◽  
Xianglin Huang

Traditional optimization algorithms for blind signal separation (BSS) are mainly based on the gradient, which requires the objective function to be continuous and differentiable, so the applications of these algorithms are very limited. Moreover, these algorithms have problems with the convergence speed and accuracy. To overcome these drawbacks, this paper presents a modified glowworm swarm optimization (MGSO) algorithm based on a novel step adjustment rule and then applies MGSO to BSS. Taking kurtosis of the mixed signals as the objective function of BSS, MGSO-BSS succeeds in separating the mixed signals in Matlab environment. The simulation results prove that MGSO is more effective in capturing the global optimum of the objective function of the BSS algorithm and has faster convergence speed and higher accuracy, compared with particle swarm optimization (PSO) and GSO.


IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 19572-19578 ◽  
Author(s):  
Xiaoxu Li ◽  
Dongliang Chang ◽  
Tao Tian ◽  
Jie Cao

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Hao Liu ◽  
Keqiang Yue ◽  
Siyi Cheng ◽  
Chengming Pan ◽  
Jie Sun ◽  
...  

Diabetic retinopathy (DR) is one of the most common complications of diabetes and the main cause of blindness. The progression of the disease can be prevented by early diagnosis of DR. Due to differences in the distribution of medical conditions and low labor efficiency, the best time for diagnosis and treatment was missed, which results in impaired vision. Using neural network models to classify and diagnose DR can improve efficiency and reduce costs. In this work, an improved loss function and three hybrid model structures Hybrid-a, Hybrid-f, and Hybrid-c were proposed to improve the performance of DR classification models. EfficientNetB4, EfficientNetB5, NASNetLarge, Xception, and InceptionResNetV2 CNNs were chosen as the basic models. These basic models were trained using enhance cross-entropy loss and cross-entropy loss, respectively. The output of the basic models was used to train the hybrid model structures. Experiments showed that enhance cross-entropy loss can effectively accelerate the training process of the basic models and improve the performance of the models under various evaluation metrics. The proposed hybrid model structures can also improve DR classification performance. Compared with the best-performing results in the basic models, the accuracy of DR classification was improved from 85.44% to 86.34%, the sensitivity was improved from 98.48% to 98.77%, the specificity was improved from 71.82% to 74.76%, the precision was improved from 90.27% to 91.37%, and the F1 score was improved from 93.62% to 93.9% by using hybrid model structures.


Sign in / Sign up

Export Citation Format

Share Document