Debug formatter for Tensor is confusing with > 1 GPU

[zackangelo]

When running a model across several processes using NCCL, the debug formatter output will print the same ID for two GPUs:

GPU 0: (dev=Cuda(CudaDevice(DeviceId(1))), shape=[1, 128256], len=128256)
GPU 1: (dev=Cuda(CudaDevice(DeviceId(1))), shape=[1, 128256], len=128256)

It's confusing when looking at logs and trying to figure out which GPU is doing what.

This is because candle uses an atomic counter per-PID to assign a device ID:

candle/candle-core/src/cuda_backend/device.rs

Lines 35 to 39 in 00d8a0c

	impl std::fmt::Debug for CudaDevice {
	fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
	write!(f, "CudaDevice({:?})", self.id)
	}
	}

Would it be a problem to include the CUDA device ordinal in the debug formatter? If not I'll open a PR.

[LaurentMazare]

Yeah feel free to make a PR that would change it to something like CudaDevice(ordinal:id).