Skip to content

Exprimental Implementation of Riemannian manifold calculation

Notifications You must be signed in to change notification settings

overshiki/Riemannian

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Exprimental Implementation of Riemannian manifold calculation

This would be useful in natural gradient and other model comparision technique

current status:

Implemented a small toy method for calculation fisher information matrix. It is slow and, only support small scale neural network since for large scales, the resulting matrix would be too large for either gpu memory or host memory

A toy example is provided for small NeuralNet model with only 11560 parameters.

future development:

Robust method for fisher information matrix combined with natural gradient calculation, hopefully we could use some reduce technique to reduce the memory consumption

About

Exprimental Implementation of Riemannian manifold calculation

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages