Thanks for the reply!
I wonder why is Distributed Data Parallel any better? I am using 8 gpus in the same machine
Anyway, any tips/ideas on how I could load imagenet faster with pytorch?
Hopefully by changing the HDD to a SSD will help.
The loading time is 5.2 sec average and the batch time is 0.8 sec =/
@eqy