CacheDataset and random transforms clarification

[dianemarquette]

Hi!

I was going once again through the CacheDataset documentation and it states that:

CacheDataset executes non-random transforms and prepares cache content in the main process before the first epoch, then all the subprocesses of DataLoader will read the same cache content in the main process during training.

Does it implies that CacheDataset feeds transformed data which are different for each epoch (if we apply random transforms to our inputs) to DataLoader? In other words, does DataLoader call the _transform method of CacheDataset at each epoch?

Thanks in advance for your help :)

[Nic-Ma]

Hi @dianemarquette ,

Thanks for your interest here.
Usually, we have several non-random transforms + random transforms in the transform-chain, CacheDataset will cache the consistent result of the transforms until the first random transform, then reuse the cache content for every epoch.
You can check the tutorial for more details:
https://github.com/Project-MONAI/tutorials/blob/master/acceleration/dataset_type_performance.ipynb

Thanks.

[dianemarquette]

Thanks for your reply @Nic-Ma.

I get the point that only the result of the transforms up to the first random transform is stored in cache. I just wanted to make sure that the non-random transforms are applied to the cache content at each epoch. As a result, the network "sees" a new dataset at each epoch, correct?

[Nic-Ma]

Yes, you are right.

Thanks.