Resources

[JohanMabille]

This issue is for tracking resources related to sparse tensors and algorithms.

• Sparse tensor formats

[gouarin]

• Sparse tensor benchmarks
• ExTensor: An accelerator for sparse tensor algebra
• Article about sparse tensor on emerging architectures

[hameerabbasi]

Hello, the first paper is by the TACO team at MIT/Stanford, and they are working on quite a lot of stuff to enhance the performance of sparse tensor algebra, as well as to complete some algorithms that aren't possible in the first paper:

• This one talks about scheduling/SIMD/GPUs and how it can be done in the context of sparse tensors: https://arxiv.org/abs/2001.00532
• This one talks about conversions: https://arxiv.org/abs/2001.02609
• About transpositions: https://arxiv.org/abs/2005.10427
• This one talks about workspaces: https://arxiv.org/abs/1802.10574

A lot of these benchmarks are doing some kind of Apples-Oranges comparison between TACO and other frameworks. If you take the idea of TACO (generalization of iteration to sparse tensors), then you can see the value in TACO, and apply e.g. scheduling GPUs and SIMD to get equivalent performance to other (hand-tweaked) algorithms.

The value of TACO is the flexibility it offers in generalizing to more algorithm types, not performance vs hand-optimized algorithms.

[gouarin]

Thanks @hameerabbasi !
This list of articles will be very useful for our choice in the implementation of several sparse algorithms.

[stellarpower]

This isn't as directly-related, but just sharing as I have found it really interesting:

https://www.thallo-lang.org/

Thallo is a DSL for least-squares optimisation. They JIT-compile a high-level description of the parameters and cost function for a particular problem, and output a custom kernel that is able to run the optimisation just for that exact problem, in a matrix-free manner. I believe it's similar to the sort of approach that TACO will be taking, so just thought to share in case it's of any inspiration for anyone.