microsoft
/
archai
зеркало из https://github.com/microsoft/archai.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
2 435 коммитов 9 Ветки 12 Теги 57 MiB
Граф коммитов

196 Коммитов

Автор SHA1 Сообщение Дата
Debadeepta Dey 9be602cc5d Added proxynas natsbench sss main wrapper. 2022-12-16 18:26:04 -03:00
Debadeepta Dey c75543c02b Made DynamicShapeTinyNet compatible with archai trainer. 2022-12-16 18:26:04 -03:00
Debadeepta Dey adc37f14b7 Implemented FEAR with Natsbench SSS space. To be debugged and tested. 2022-12-16 18:26:03 -03:00
Debadeepta Dey 38ce2b23f9 Updated DARTS random search plots. 2022-12-16 18:26:03 -03:00
Debadeepta Dey fe26ec2117 Added analysis scripts for random search with FEAR on DARTS search space. 2022-12-16 18:26:00 -03:00
Debadeepta Dey 3802e31f90 Added darts search space far searcher. 2022-12-16 18:25:53 -03:00
Debadeepta Dey 1cf6843f5d Started testing random search on Darts search space. 2022-12-16 18:25:52 -03:00
Debadeepta Dey 74a36571df Started script to analyse freeze darts space experiments. 2022-12-16 18:25:52 -03:00
Debadeepta Dey 419e4e7b3e Started analysis scripts for darts space + FEAR experiments. 2022-12-16 18:25:52 -03:00
Debadeepta Dey 0911faabb3 Initial implementation of creating a darts search space architecture benchmark for fear. 2022-12-16 18:25:52 -03:00
Debadeepta Dey 0839973e8e Nasbench301 analysis script added. 2022-12-16 18:25:51 -03:00
Debadeepta Dey d06f4ec922 Finished processing local search results on natsbench tss. 2022-12-16 18:25:36 -03:00
Debadeepta Dey 71b96dadd2 Added analysis scripts for local search experiments on Natsbench TSS search space. 2022-12-16 18:25:25 -03:00
Debadeepta Dey f8973b3d8d Updated local search with fear to print out best local minimum. 2022-12-16 18:25:25 -03:00
Debadeepta Dey 6a2a1ea475 Updated fear docs. 2022-12-16 18:25:25 -03:00
Debadeepta Dey 57cc6014af Refactored analysis scripts. 2022-12-16 18:25:02 -03:00
Debadeepta Dey 4c79ba0c78 Last commit before starting to cleanup. 2022-12-16 18:24:52 -03:00
Debadeepta Dey ff1faa3045 Tested local search with FastArchRank on natsbench. 2022-12-16 18:24:46 -03:00
Debadeepta Dey 622b5970ff Fixed the dataloader caching issue which causes OOM error on long discrete searcher jobs. 2022-12-16 18:24:43 -03:00
Debadeepta Dey 8f5199d9c3 Added code for random search with post training of the top ones. 2022-12-16 18:24:33 -03:00
Debadeepta Dey a4e9573bee Added script to analyze zero-cost performance as a function of number of epochs. 2022-12-16 18:24:33 -03:00
Debadeepta Dey e0bf51b45e Generated time to threshold vs. test accuracy plots for paper. 2022-12-16 18:24:29 -03:00
Debadeepta Dey 5ddf9127e5 Generated (hopefully) final versions of pareto curves for FastArchRank. 2022-12-16 18:24:18 -03:00
Debadeepta Dey 3476247efd Processed more random search experiments. 2022-12-16 18:24:18 -03:00
Debadeepta Dey 3d9b54fd7e Various updates related to random search on natsbench. 2022-12-16 18:24:18 -03:00
Debadeepta Dey 99c7c0941f Launched a bunch of new random search experiments. 2022-12-16 18:24:14 -03:00
Debadeepta Dey 41cc82cf07 Added functionality to take run zero-cost measures after every epoch of regular training. 2022-12-16 18:24:03 -03:00
Debadeepta Dey 238c218c9a Analyzed more imagenet16 experiments. 2022-12-16 18:24:03 -03:00
Debadeepta Dey 10232c241a Sanity checked zero cost on synthetic cifar10. 2022-12-16 18:23:59 -03:00
Debadeepta Dey c638a21fc2 Added some more experiments for Imagenet16-120. 2022-12-16 18:23:51 -03:00
Debadeepta Dey 990eae48a0 Added plotting scripts for random search experiments with FastArchRank. 2022-12-16 18:23:51 -03:00
Debadeepta Dey c6ffdab700 Added analysis script for random search with regular evaluation on natsbench tss. 2022-12-16 18:23:51 -03:00
Debadeepta Dey 098a8d7891 Added analysis script for random search via fastarchrank. 2022-12-16 18:23:51 -03:00
Debadeepta Dey 8492392604 Renaming of plots. 2022-12-16 18:23:50 -03:00
Debadeepta Dey a7e87ff6f6 Working on random search using FastArchRank. 2022-12-16 18:23:50 -03:00
Debadeepta Dey 715f04b6bb Updated conditional trainer class everywhere. 2022-12-16 18:23:50 -03:00
Debadeepta Dey 0562f2c09d cifar10 results after finding speed diff on clusters. 2022-12-16 18:23:47 -03:00
Debadeepta Dey 4d3f9376c4 Incorporated stderr reporting in analysis_freeze_natsbench_space_new.py script. 2022-12-16 18:23:36 -03:00
Debadeepta Dey 9310a5ec64 Switching over experimental plots to use a dictionary in configuration for more flexibility. 2022-12-16 18:23:22 -03:00
Debadeepta Dey cd076edf06 Added script that generates benchmark files (arch_id, test_accuracy) for datasets not in natsbench/nasbench101. 2022-12-16 18:22:53 -03:00
Debadeepta Dey 3d03f14ccf Changed natsbench model creation function calls from archai to be num class agnostic. 2022-12-16 18:22:41 -03:00
Debadeepta Dey 1e11e7dcc1 Added intel image classification dataset provider. 2022-12-16 18:22:22 -03:00
Debadeepta Dey bea1a0f789 Added analysis script for freezetrain on natsbench and nb101 that uses the val or train depending on option set. 2022-12-16 18:22:15 -03:00
Debadeepta Dey 407c02871c Changed proxynas natsbench space to optionally use train or val error thresholds. 2022-12-16 18:22:15 -03:00
Debadeepta Dey d7c4b382e6 Analysis script for hog features pipeline experiments is done. 2022-12-16 18:22:15 -03:00
Debadeepta Dey dff428402e Added functionality to plot search space error cdfs. Organized some test scripts. 2022-12-16 18:22:15 -03:00
Debadeepta Dey 195e830cb6 Fixed bug. 2022-12-16 18:22:15 -03:00
Debadeepta Dey 325d44b3d4 Refactored an analysis script. 2022-12-16 18:22:15 -03:00
Debadeepta Dey f598ba3a35 Cleaned up analysis code. 2022-12-16 18:22:15 -03:00
Debadeepta Dey fd9fe4cc36 Updated analysis scripts to take into account non benchmark datasets and zero-cost proxy measures. 2022-12-16 18:22:12 -03:00
Debadeepta Dey f7ed8c886f Added analysis script for comparing synflow with synthetic dataset. 2022-12-16 18:21:59 -03:00
Debadeepta Dey 59e8414e4e Zero cost ranking measures nominally integrated into archai. Requires thorough testing. 2022-12-16 18:21:59 -03:00
Debadeepta Dey e1f335cfb3 Synthetic cifar10 dataset now integrated and tested with natsbench tss architectures. 2022-12-16 18:21:59 -03:00
Debadeepta Dey 8009911fb4 Refactored hog + small neural network classification pipeline. 2022-12-16 18:21:40 -03:00
Debadeepta Dey e5ec907259 Added plotting code to display distribution of architectures. 2022-12-16 18:21:40 -03:00
Debadeepta Dey a652c3bcfc More analysis of experiments on nasbench101. 2022-12-16 18:21:39 -03:00
Debadeepta Dey 16a8b37ef5 Make freezeaddon able to take in different layer activations of nasbench101 as features. 2022-12-16 18:21:39 -03:00
Debadeepta Dey 87c6e61abd Nominal implementation of freezetrain network idea. 2022-12-16 18:21:39 -03:00
Debadeepta Dey ca64dc4156 Modified analysis scripts to deal with nb101 as well. 2022-12-16 18:21:36 -03:00
Debadeepta Dey b09c30de01 Verified that analysis script for regular natsbench also works on nb101 reglar eval. 2022-12-16 18:21:24 -03:00
Debadeepta Dey cea8607763 Added main wrapper for nasbench101 2022-12-16 18:21:24 -03:00
Debadeepta Dey 5186db3e28 Created regular nasbench101 evaluation pipeline. 2022-12-16 18:21:24 -03:00
Debadeepta Dey 1091be9b11 Updated with ImageNet16-120 preliminary results. 2022-12-16 18:21:23 -03:00
Debadeepta Dey b5aac7a6b8 Testing imagenet16 dataloader integration. 2022-12-16 18:21:23 -03:00
Debadeepta Dey e56cce2613 Phased freeze training code is tested now. 2022-12-16 18:21:20 -03:00
Debadeepta Dey 6e47ce94c9 cifar100 now works! 2022-12-16 18:21:10 -03:00
Debadeepta Dey 7603941b16 Getting error with running cifar100. 2022-12-16 18:21:07 -03:00
Debadeepta Dey 6a53ce5b67 Got aggregate plots to be decent. 2022-12-16 18:20:42 -03:00
Debadeepta Dey 5835e0d608 Creating scripts for aggregate analysis of experiments. 2022-12-16 18:20:42 -03:00
Debadeepta Dey ddfbfc2a08 Added dumping timing information to regular natsbench evaluation analysis scripts. 2022-12-16 18:20:42 -03:00
Debadeepta Dey 1a67c72ff0 Made proxynas natsbench space yaml freeze trainer have the same params as conditional training for 256 batch size. 2022-12-16 18:20:39 -03:00
Debadeepta Dey ee3343b5aa Added script for making cross experiment plots for proxynas. 2022-12-16 18:20:25 -03:00
Debadeepta Dey 0a64c62925 Added wrapper for main to clobber more arch ids together. 2022-12-16 18:20:08 -03:00
Debadeepta Dey ca98fe94a8 Implemented phased freeze training method from SVCCA paper. 2022-12-16 18:20:05 -03:00
Debadeepta Dey cd93c0e436 Made proxynas analysis script more robust. 2022-12-16 18:19:44 -03:00
Debadeepta Dey 61bd3222da Investigating best_val_top1() bug. 2022-12-16 18:19:44 -03:00
Debadeepta Dey ef0d9e1e03 Getting ready to run large batch cell13 natsbench without augmentation. 2022-12-16 18:19:42 -03:00
Debadeepta Dey dd7ed42c8a Nasbench101 experiments with various ranking mechanisms are now ready. 2022-12-16 18:19:04 -03:00
Debadeepta Dey b8f80e06c9 Updated freezetrain natsbench space training to have same hyperparams as that for fast natsbench regular training. 2022-12-16 16:53:47 -03:00
Debadeepta Dey 8fbdf48438 Added natsbench regular training with hyperparams closely matching as in the paper. 2022-12-16 16:53:47 -03:00
Debadeepta Dey 7a5bc123f8 Prepared analysis script for conditional naswot training. 2022-12-16 16:53:47 -03:00
Debadeepta Dey 999460c787 Fixed bug where batch size in configuration file for naswot score method was not getting utilized. So scoring mechanism was using 96 batch size which is used for regular training. 2022-12-16 16:53:46 -03:00
Debadeepta Dey ef84386e87 Getting ready to run naswot conditional training. 2022-12-16 16:53:46 -03:00
Debadeepta Dey 8791e1bc38 Found good parameters for cell13 onwards receiving gradients on natsbench. 2022-12-16 16:53:46 -03:00
Debadeepta Dey 6085cdf0fa Modified analysis_aggregate.py to load logs in distributed mode. 2022-12-16 16:53:46 -03:00
Debadeepta Dey 1fd43a76c6 Parameter hunt with cell13 onwards getting gradients. 2022-12-16 16:53:46 -03:00
Debadeepta Dey 993ba13a72 Parallelized analysis script for natsbench experiments. 2022-12-16 16:53:46 -03:00
Debadeepta Dey 6eac185832 Found good parameters for freeze training. 2022-12-16 16:53:41 -03:00
Debadeepta Dey a2f30a6268 Prepared analysis of freeze training experiments. 2022-12-16 16:53:29 -03:00
Debadeepta Dey be44bd1e41 Refactored freeze training code and made things cleaner. 2022-12-16 16:53:29 -03:00
Debadeepta Dey 1a7616dca0 Refactored experiment report scripts to account for new ranking metrics. 2022-12-16 16:53:24 -03:00
Debadeepta Dey 409b778e89 Added natsbench for rapid evaluation of architecture ranking proxies. 2022-12-16 16:53:11 -03:00
Debadeepta Dey 2f2aaf7208 Starting to play with natsbench api. 2022-12-16 16:53:11 -03:00
Debadeepta Dey 2453524ca6 Refactoring experiment analysis script. 2022-12-16 16:52:40 -03:00
Debadeepta Dey 9a54ffb594 Added new algo manual_freeze which can train a handcrafted network with freezetrainer. 2022-12-16 16:52:40 -03:00
Debadeepta Dey e5bf0312e5 Still having the issue with freeze training where training error is going up after freeze! 2022-12-16 16:52:40 -03:00
Debadeepta Dey 3cfc771f46 Fixed exprep script to report results report correctly. 2022-12-16 16:52:40 -03:00
Debadeepta Dey 7e99526376 More progress on freeze training. Testing and debugging underway. 2022-12-16 16:52:40 -03:00
Debadeepta Dey e46f0a48b9 LM1B training is currently broken. 2022-12-16 16:45:54 -03:00
Debadeepta Dey 7f74d16e9a Minor. 2022-12-16 16:45:54 -03:00