The constrained entropy and cross-entropy functions

2004 ◽  
Vol 334 (3-4) ◽  
pp. 444-458 ◽  
Author(s):  
Robert K. Niven
Keyword(s):  
Cross Entropy ◽  
Entropy Functions

scholarly journals Loss-Based Attention for Deep Multiple Instance Learning

2020 ◽  
Vol 34 (04) ◽  
pp. 5742-5749
Author(s):  
Xiaoshuang Shi ◽  
Fuyong Xing ◽  
Yuanpu Xie ◽  
Zizhao Zhang ◽  
Lei Cui ◽  
...  
Keyword(s):  
State Of The Art ◽  
Classification Performance ◽  
Multiple Instance Learning ◽  
Attention Mechanism ◽  
Cross Entropy ◽  
General Category ◽  
Source Codes ◽  
Model Generalization ◽  
Fully Connected ◽  
Entropy Functions

Although attention mechanisms have been widely used in deep learning for many tasks, they are rarely utilized to solve multiple instance learning (MIL) problems, where only a general category label is given for multiple instances contained in one bag. Additionally, previous deep MIL methods firstly utilize the attention mechanism to learn instance weights and then employ a fully connected layer to predict the bag label, so that the bag prediction is largely determined by the effectiveness of learned instance weights. To alleviate this issue, in this paper, we propose a novel loss based attention mechanism, which simultaneously learns instance weights and predictions, and bag predictions for deep multiple instance learning. Specifically, it calculates instance weights based on the loss function, e.g. softmax+cross-entropy, and shares the parameters with the fully connected layer, which is to predict instance and bag predictions. Additionally, a regularization term consisting of learned weights and cross-entropy functions is utilized to boost the recall of instances, and a consistency cost is used to smooth the training process of neural networks for boosting the model generalization performance. Extensive experiments on multiple types of benchmark databases demonstrate that the proposed attention mechanism is a general, effective and efficient framework, which can achieve superior bag and image classification performance over other state-of-the-art MIL methods, with obtaining higher instance precision and recall than previous attention mechanisms. Source codes are available on https://github.com/xsshi2015/Loss-Attention.

The motion of the particles volume corresponding to invariant solution of rank 2 submodel of hydrodynamic type

2016 ◽  
Vol 11 (2) ◽  
pp. 205-209
Author(s):  
D.T. Siraeva
Keyword(s):  
Equation Of State ◽  
Invariant Solution ◽  
Hydrodynamic Type ◽  
Lagrangian Coordinates ◽  
Hydrodynamic Equations ◽  
Rank 2 ◽  
Invariant Submodel ◽  
Entropy Functions

Invariant submodel of rank 2 on the subalgebra consisting of the sum of transfers for hydrodynamic equations with the equation of state in the form of pressure as the sum of density and entropy functions, is presented. In terms of the Lagrangian coordinates from condition of nonhyperbolic submodel solutions depending on the four essential constants are obtained. For simplicity, we consider the solution depending on two constants. The trajectory of particles motion, the motion of parallelepiped of the same particles are studied using the Maple.

Reduction of partially invariant submodels of rank 3 defect 1 to invariant submodels

2018 ◽  
Vol 13 (3) ◽  
pp. 59-63 ◽  
Author(s):  
D.T. Siraeva
Keyword(s):  
Equation Of State ◽  
Lie Algebra ◽  
Lie Algebras ◽  
Gas Dynamics ◽  
Hydrodynamic Type ◽  
System Of Equations ◽  
Two Dimensional ◽  
Equations Of Gas Dynamics ◽  
Entropy Functions

Equations of hydrodynamic type with the equation of state in the form of pressure separated into a sum of density and entropy functions are considered. Such a system of equations admits a twelve-dimensional Lie algebra. In the case of the equation of state of the general form, the equations of gas dynamics admit an eleven-dimensional Lie algebra. For both Lie algebras the optimal systems of non-similar subalgebras are constructed. In this paper two partially invariant submodels of rank 3 defect 1 are constructed for two-dimensional subalgebras of the twelve-dimensional Lie algebra. The reduction of the constructed submodels to invariant submodels of eleven-dimensional and twelve-dimensional Lie algebras is proved.

Sign in / Sign up

Export Citation Format

Share Document