scholarly journals Function shaping in deep learning

2021 ◽  
Author(s):  
Ēvalds Urtāns
Keyword(s):  
Loss Function ◽  
Value Function ◽  
Metric Learning ◽  
Computer Game ◽  
Loss Functions ◽  
Identification Task ◽  
The Face ◽  
Triplet Loss ◽  
Deep Metric Learning ◽  
The Value Function

This work describes the importance of loss functions and related methods for deep reinforcement learning and deep metric learning. A novel MDQN loss function outperformed DDQN loss function in PLE computer game environments, and a novel Exponential Triplet loss function outperformed the Triplet loss function in the face re-identification task with VGGFace2 dataset reaching 85,7 % accuracy using zero-shot setting. This work also presents a novel UNet-RNN-Skip model to improve the performance of the value function for path planning tasks.

scholarly journals f-Similarity Preservation Loss for Soft Labels: A Demonstration on Cross-Corpus Speech Emotion Recognition

2019 ◽  
Vol 33 ◽  
pp. 5725-5732
Author(s):  
Biqiao Zhang ◽  
Yuqing Kong ◽  
Georg Essl ◽  
Emily Mower Provost
Keyword(s):  
Neural Networks ◽  
Emotion Recognition ◽  
Loss Function ◽  
Deep Neural Networks ◽  
Metric Learning ◽  
Loss Functions ◽  
Speech Emotion Recognition ◽  
Subjective Data ◽  
Dual Form ◽  
Deep Metric Learning

In this paper, we propose a Deep Metric Learning (DML) approach that supports soft labels. DML seeks to learn representations that encode the similarity between examples through deep neural networks. DML generally presupposes that data can be divided into discrete classes using hard labels. However, some tasks, such as our exemplary domain of speech emotion recognition (SER), work with inherently subjective data, data for which it may not be possible to identify a single hard label. We propose a family of loss functions, fSimilarity Preservation Loss (f-SPL), based on the dual form of f-divergence for DML with soft labels. We show that the minimizer of f-SPL preserves the pairwise label similarities in the learned feature embeddings. We demonstrate the efficacy of the proposed loss function on the task of cross-corpus SER with soft labels. Our approach, which combines f-SPL and classification loss, significantly outperforms a baseline SER system with the same structure but trained with only classification loss in most experiments. We show that the presented techniques are more robust to over-training and can learn an embedding space in which the similarity between examples is meaningful.

scholarly journals Enhancing Multi-tissue and Multi-scale Cell Nuclei Segmentation with Deep Metric Learning

2020 ◽  
Vol 10 (2) ◽  
pp. 615 ◽  
Author(s):  
Tomas Iesmantas ◽  
Agne Paulauskaite-Taraseviciene ◽  
Kristina Sutiene
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Metric Learning ◽  
Cell Nuclei ◽  
Similarity Coefficients ◽  
Clinical Practices ◽  
Nuclei Segmentation ◽  
Wide Range ◽  
Triplet Loss ◽  
Deep Metric Learning

(1) Background: The segmentation of cell nuclei is an essential task in a wide range of biomedical studies and clinical practices. The full automation of this process remains a challenge due to intra- and internuclear variations across a wide range of tissue morphologies, differences in staining protocols and imaging procedures. (2) Methods: A deep learning model with metric embeddings such as contrastive loss and triplet loss with semi-hard negative mining is proposed in order to accurately segment cell nuclei in a diverse set of microscopy images. The effectiveness of the proposed model was tested on a large-scale multi-tissue collection of microscopy image sets. (3) Results: The use of deep metric learning increased the overall segmentation prediction by 3.12% in the average value of Dice similarity coefficients as compared to no metric learning. In particular, the largest gain was observed for segmenting cell nuclei in H&E -stained images when deep learning network and triplet loss with semi-hard negative mining were considered for the task. (4) Conclusion: We conclude that deep metric learning gives an additional boost to the overall learning process and consequently improves the segmentation performance. Notably, the improvement ranges approximately between 0.13% and 22.31% for different types of images in the terms of Dice coefficients when compared to no metric deep learning.

scholarly journals Person Re-Identification by Pose Invariant Deep Metric Learning With Improved Triplet Loss

IEEE Access ◽  
2018 ◽  
Vol 6 ◽  
pp. 68089-68095 ◽  
Author(s):  
Min Chen ◽  
Yongxin Ge ◽  
Xin Feng ◽  
Chuanyun Xu ◽  
Dan Yang
Keyword(s):  
Metric Learning ◽  
Triplet Loss ◽  
Deep Metric Learning

scholarly journals Relative Distribution Entropy Loss Function in CNN Image Retrieval

Entropy ◽  
2020 ◽  
Vol 22 (3) ◽  
pp. 321
Author(s):  
Pingping Liu ◽  
Lida Shi ◽  
Zhuang Miao ◽  
Baixin Jin ◽  
Qiuzhan Zhou
Keyword(s):  
Image Retrieval ◽  
Loss Function ◽  
Euclidean Distance ◽  
Metric Learning ◽  
Loss Functions ◽  
Relative Distribution ◽  
Weighted Distance ◽  
Entropy Loss ◽  
Image Descriptors ◽  
Entropy Weighted

Convolutional neural networks (CNN) is the most mainstream solution in the field of image retrieval. Deep metric learning is introduced into the field of image retrieval, focusing on the construction of pair-based loss function. However, most pair-based loss functions of metric learning merely take common vector similarity (such as Euclidean distance) of the final image descriptors into consideration, while neglecting other distribution characters of these descriptors. In this work, we propose relative distribution entropy (RDE) to describe the internal distribution attributes of image descriptors. We combine relative distribution entropy with the Euclidean distance to obtain the relative distribution entropy weighted distance (RDE-distance). Moreover, the RDE-distance is fused with the contrastive loss and triplet loss to build the relative distributed entropy loss functions. The experimental results demonstrate that our method attains the state-of-the-art performance on most image retrieval benchmarks.

scholarly journals Deep metric attention learning for skin lesion classification in dermoscopy images

2022 ◽  
Author(s):  
Xiaoyu He ◽  
Yong Wang ◽  
Shuang Zhao ◽  
Chunli Yao
Keyword(s):  
Skin Lesion ◽  
Metric Learning ◽  
Feature Representation ◽  
Training Procedure ◽  
Training Samples ◽  
Triplet Loss ◽  
Deep Metric Learning ◽  
Localization Ability ◽  
Lesion Localization ◽  
Lesion Classification

AbstractCurrently, convolutional neural networks (CNNs) have made remarkable achievements in skin lesion classification because of their end-to-end feature representation abilities. However, precise skin lesion classification is still challenging because of the following three issues: (1) insufficient training samples, (2) inter-class similarities and intra-class variations, and (3) lack of the ability to focus on discriminative skin lesion parts. To address these issues, we propose a deep metric attention learning CNN (DeMAL-CNN) for skin lesion classification. In DeMAL-CNN, a triplet-based network (TPN) is first designed based on deep metric learning, which consists of three weight-shared embedding extraction networks. TPN adopts a triplet of samples as input and uses the triplet loss to optimize the embeddings, which can not only increase the number of training samples, but also learn the embeddings robust to inter-class similarities and intra-class variations. In addition, a mixed attention mechanism considering both the spatial-wise and channel-wise attention information is designed and integrated into the construction of each embedding extraction network, which can further strengthen the skin lesion localization ability of DeMAL-CNN. After extracting the embeddings, three weight-shared classification layers are used to generate the final predictions. In the training procedure, we combine the triplet loss with the classification loss as a hybrid loss to train DeMAL-CNN. We compare DeMAL-CNN with the baseline method, attention methods, advanced challenge methods, and state-of-the-art skin lesion classification methods on the ISIC 2016 and ISIC 2017 datasets, and test its generalization ability on the PH2 dataset. The results demonstrate its effectiveness.

scholarly journals Deep Metric Learning with Hierarchical Triplet Loss

2018 ◽  
pp. 272-288 ◽  
Author(s):  
Weifeng Ge ◽  
Weilin Huang ◽  
Dengke Dong ◽  
Matthew R. Scott
Keyword(s):  
Metric Learning ◽  
Triplet Loss ◽  
Deep Metric Learning
Sign in / Sign up

Export Citation Format

Share Document