scholarly journals An Analysis of the Softmax Cross Entropy Loss for Learning-to-Rank with Binary Relevance

2019 ◽  
Author(s):  
Sebastian Bruch ◽  
Xuanhui Wang ◽  
Michael Bendersky ◽  
Marc Najork
Keyword(s):  
Learning To Rank ◽  
Cross Entropy ◽  
Entropy Loss ◽  
Binary Relevance

Cross-Entropy Loss for Recommending Efficient Fold-Over Technique

2021 ◽  
Vol 34 (1) ◽  
pp. 402-439
Author(s):  
Lin-Chen Weng ◽  
A. M. Elsawah ◽  
Kai-Tai Fang
Keyword(s):  
Cross Entropy ◽  
Entropy Loss

scholarly journals Densely Supervised Hierarchical Policy-Value Network for Image Paragraph Generation

2019 ◽  
Author(s):  
Siying Wu ◽  
Zheng-Jun Zha ◽  
Zilei Wang ◽  
Houqiang Li ◽  
Feng Wu
Keyword(s):  
Natural Language ◽  
State Of The Art ◽  
Cross Entropy ◽  
Image Captioning ◽  
Value Network ◽  
Entropy Loss ◽  
Fine Grained ◽  
Performance Improvements ◽  
Single Sentence ◽  
Multiple State

Image paragraph generation aims to describe an image with a paragraph in natural language. Compared to image captioning with a single sentence, paragraph generation provides more expressive and fine-grained description for storytelling. Existing approaches mainly optimize paragraph generator towards minimizing word-wise cross entropy loss, which neglects linguistic hierarchy of paragraph and results in ``sparse" supervision for generator learning. In this paper, we propose a novel Densely Supervised Hierarchical Policy-Value (DHPV) network for effective paragraph generation. We design new hierarchical supervisions consisting of hierarchical rewards and values at both sentence and word levels. The joint exploration of hierarchical rewards and values provides dense supervision cues for learning effective paragraph generator. We propose a new hierarchical policy-value architecture which exploits compositionality at token-to-token and sentence-to-sentence levels simultaneously and can preserve the semantic and syntactic constituent integrity. Extensive experiments on the Stanford image-paragraph benchmark have demonstrated the effectiveness of the proposed DHPV approach with performance improvements over multiple state-of-the-art methods.

scholarly journals Large-Margin Regularized Softmax Cross-Entropy Loss

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 19572-19578 ◽  
Author(s):  
Xiaoxu Li ◽  
Dongliang Chang ◽  
Tao Tian ◽  
Jie Cao
Keyword(s):  
Cross Entropy ◽  
Large Margin ◽  
Entropy Loss

scholarly journals Hybrid Model Structure for Diabetic Retinopathy Classification

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Hao Liu ◽  
Keqiang Yue ◽  
Siyi Cheng ◽  
Chengming Pan ◽  
Jie Sun ◽  
...  
Keyword(s):  
Diabetic Retinopathy ◽  
Hybrid Model ◽  
Network Models ◽  
Classification Performance ◽  
Cross Entropy ◽  
Model Structure ◽  
Training Process ◽  
Neural Network Models ◽  
Entropy Loss ◽  
Model Structures

Diabetic retinopathy (DR) is one of the most common complications of diabetes and the main cause of blindness. The progression of the disease can be prevented by early diagnosis of DR. Due to differences in the distribution of medical conditions and low labor efficiency, the best time for diagnosis and treatment was missed, which results in impaired vision. Using neural network models to classify and diagnose DR can improve efficiency and reduce costs. In this work, an improved loss function and three hybrid model structures Hybrid-a, Hybrid-f, and Hybrid-c were proposed to improve the performance of DR classification models. EfficientNetB4, EfficientNetB5, NASNetLarge, Xception, and InceptionResNetV2 CNNs were chosen as the basic models. These basic models were trained using enhance cross-entropy loss and cross-entropy loss, respectively. The output of the basic models was used to train the hybrid model structures. Experiments showed that enhance cross-entropy loss can effectively accelerate the training process of the basic models and improve the performance of the models under various evaluation metrics. The proposed hybrid model structures can also improve DR classification performance. Compared with the best-performing results in the basic models, the accuracy of DR classification was improved from 85.44% to 86.34%, the sensitivity was improved from 98.48% to 98.77%, the specificity was improved from 71.82% to 74.76%, the precision was improved from 90.27% to 91.37%, and the F1 score was improved from 93.62% to 93.9% by using hybrid model structures.

Sign in / Sign up

Export Citation Format

Share Document