Welcome to netket Discussions!

[gcarleo]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[sRb-root]

Hi. I have recently upgraded to Netket 3.0. I had been using the RBMSpin module of the the previous versions to study spin systems with magnetic frustration and I used to get very good convergence with all the optimizers. Since I have upgraded to Netket 3.0, I am finding that the energy doesn't converge to the ground state with RmsProp or Adam but it does with plain SgD. This seems weird as the first two should usually perform better than SgD. I have tried varying the learning parameters but so far havent't been able to achieve convergence. Note that I am using the SR method for finding the ground state.

Any help regarding this would be deeply appreciated. I can send details of my code if necessary.

[PhilipVinc]

Did you check that you are using complex (or real) parameters both in your new code and old code?
The old default was complex, the new is real.

[sRb-root]

Yes I did pass the argument "dtype = jax.numpy.complex128" to the RBM call function.

I should mention though that in the previous versions, the optimizers I had worked with were AdaDelta and AdaMax. I couldn't find these optimizers in version 3.0, but since RmsProp and AdaDelta are fairly similar, I tried to use RmsProp and Adam in the new version.

[gcarleo]

I would maybe add that it is in general not a good idea to use AdaDelta and AdaMax with SR/Natural Gradient, I expect SGD to be more effective when used with SR

[sRb-root]

I would maybe add that it is in general not a good idea to use AdaDelta and AdaMax with SR/Natural Gradient, I expect SGD to be more effective when used with SR

Thanks a lot for the tip!! I am fairly new to RBM, so such tips are very helpful and appreciated!

SgD is actually working fine for me as far as energy convergence is concerned. However, convergence of a few long-range observables are proving to be tricky as they are highly fluctuating and also taking very long times to converge . I was hoping if this could be improved using other optimizers.

[PhilipVinc]

You might also need to decrease the diagonal shift of the SR at some point in the optimisation...