Efficient method of moment simulation based on higher order bases and CPU/GPU parallelization

2012 ◽  
Author(s):  
Branko Kolundzija ◽  
Dragan Olcan ◽  
Dusan Zoric
Keyword(s):  
Efficient Method ◽  
Higher Order ◽  
Method Of Moment ◽  
Simulation Based

scholarly journals A Simple and Efficient Tensor Calculus

2020 ◽  
Vol 34 (04) ◽  
pp. 4527-4534
Author(s):  
Sören Laue ◽  
Matthias Mitterreiter ◽  
Joachim Giesen
Keyword(s):  
Machine Learning ◽  
Efficient Method ◽  
Automatic Differentiation ◽  
State Of The Art ◽  
Higher Order ◽  
Tensor Representation ◽  
Tensor Calculus ◽  
Correct Algorithm ◽  
Previous State ◽  
Derivatives Of

Computing derivatives of tensor expressions, also known as tensor calculus, is a fundamental task in machine learning. A key concern is the efficiency of evaluating the expressions and their derivatives that hinges on the representation of these expressions. Recently, an algorithm for computing higher order derivatives of tensor expressions like Jacobians or Hessians has been introduced that is a few orders of magnitude faster than previous state-of-the-art approaches. Unfortunately, the approach is based on Ricci notation and hence cannot be incorporated into automatic differentiation frameworks like TensorFlow, PyTorch, autograd, or JAX that use the simpler Einstein notation. This leaves two options, to either change the underlying tensor representation in these frameworks or to develop a new, provably correct algorithm based on Einstein notation. Obviously, the first option is impractical. Hence, we pursue the second option. Here, we show that using Ricci notation is not necessary for an efficient tensor calculus and develop an equally efficient method for the simpler Einstein notation. It turns out that turning to Einstein notation enables further improvements that lead to even better efficiency.

Sign in / Sign up

Export Citation Format

Share Document