[DO NOT MERGE] add all level buffer support when computing infer_auto_device_map

[SunMarc]

What does this PR do ?

This PR adds all level support when computing infer_auto_device_map. This is especially important for persistant buffer that are in the state_dict. If we don't attribute a device to those buffers, we will get error in transformers when we try to load the state_dict. We might hit edge cases with cpu and disk offload.
Usually buffers are defined in a nn.Module that don't have other children modules. However, in a model such as RecurrentGemmaModel, normalize buffer is defined at the level of the model. So when we have model = RecurrentGemmaForCausalLM, the persistent buffer is defined there at this location model.model.normalize but model.model have children modules such as layers.
cc @ArthurZucker

Another solution would be to just use non persistant buffer all the time for these cases
-> This is the solution that we are going with for now. We can merge this PR if it makes sense later.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[muellerzr]

Thanks, overall this looks fine to me.

Let's make a code comment here potentially, something like:

TODO: May have cpu and disk offload edge-cases.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.