Offline Distillation Dataloader Generation Utils
-
Preprocessor: A utility to generate offline distillation dataloader, add teacher predict result for each sample of data.
preprocess_labels: Pre-run the teacher predict function on the whole dataset and store the prediction result for replay and distil.create_replay_dataloader: afterpreprocess_labels, preprocessor will onreplaymode,create_replay_dataloaderwill return a dataloader similar to user designed dataloader, the only different is the iteration of it will return additional batched teacher prediction results,(batch,) -> (teacher_results, batch,)
-
_DistilStorage: Save and load the prediction result with keyword format
{uid: result}. NNI hasMemoryStorage,FileStorage,HDF5Storage(not ready), user can customize their own storage likeSqliteStorageby inheriting_DistilStorage. -
_UidDataset: A subclass of
pytorchDataset, it will wrap the original dataset. Assume the original dataset returnssamplein__getitem__, then after wrapped by_UidDataset, it will return(uid, sample). NNi hasIndexedDataset,HashedDataset,AugmentationDataset, user can customize their own uid generation way likeFilePathDatasetby inheriting_UidDataset. -
uid: Users do not need to know what
uidis, if he does not customize_UidDataset. Theuidneeds to have a one-to-one correspondence (or many-to-one) with the sample in the original dataset, and a one-to-one correspondence with the predicted results.
