Utilities for distributed training

get_rank[source]

get_rank()

Gets the local rank

get_device[source]

get_device()

get_process_index[source]

get_process_index()

Gets the local process index

num_processes[source]

num_processes()

Returns the number of processes in the distributed system