зеркало из https://github.com/microsoft/nni.git
Merge pull request #5553 from microsoft/v3.0rc1
Merge v3.0rc1 back to master
This commit is contained in:
liuzhe-lz
GitHub
Коммит
f17385c0ac
|
|
@ -20,7 +20,7 @@ NNI automates feature engineering, neural architecture search, hyperparameter tu
|
|||
|
||||
## What's NEW! <a href="#nni-released-reminder"><img width="48" src="docs/img/release_icon.png"></a>
|
||||
|
||||
* **New release**: [v2.10 is available](https://github.com/microsoft/nni/releases/tag/v2.10) - _released on Nov-14-2022_
|
||||
* **New release**: [v3.0 preview is available](https://github.com/microsoft/nni/releases/tag/v3.0rc1) - _released on May-5-2022_
|
||||
* **New demo available**: [Youtube entry](https://www.youtube.com/channel/UCKcafm6861B2mnYhPbZHavw) | [Bilibili 入口](https://space.bilibili.com/1649051673) - _last updated on June-22-2022_
|
||||
* **New research paper**: [SparTA: Deep-Learning Model Sparsity via Tensor-with-Sparsity-Attribute](https://www.usenix.org/system/files/osdi22-zheng-ningxin.pdf) - _published in OSDI 2022_
|
||||
* **New research paper**: [Privacy-preserving Online AutoML for Domain-Specific Face Detection](https://openaccess.thecvf.com/content/CVPR2022/papers/Yan_Privacy-Preserving_Online_AutoML_for_Domain-Specific_Face_Detection_CVPR_2022_paper.pdf) - _published in CVPR 2022_
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ author = 'Microsoft'
|
|||
version = ''
|
||||
# The full version, including alpha/beta/rc tags
|
||||
# FIXME: this should be written somewhere globally
|
||||
release = 'v2.10'
|
||||
release = 'v3.0rc1'
|
||||
|
||||
# -- General configuration ---------------------------------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -5,6 +5,111 @@
|
|||
Change Log
|
||||
==========
|
||||
|
||||
Release 3.0 Preview - 5/9/2022
|
||||
------------------------------
|
||||
|
||||
Web Portal
|
||||
^^^^^^^^^^
|
||||
|
||||
* New look and feel
|
||||
|
||||
Neural Architecture Search
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
* **Breaking change**: ``nni.retiarii`` is no longer maintained and tested. Please migrate to ``nni.nas``.
|
||||
|
||||
* Inherit ``nni.nas.nn.pytorch.ModelSpace``, rather than use ``@model_wrapper``.
|
||||
* Use ``nni.choice``, rather than ``nni.nas.nn.pytorch.ValueChoice``.
|
||||
* Use ``nni.nas.experiment.NasExperiment`` and ``NasExperimentConfig``, rather than ``RetiariiExperiment``.
|
||||
* Use ``nni.nas.model_context``, rather than ``nni.nas.fixed_arch``.
|
||||
* Please refer to `quickstart <https://nni.readthedocs.io/en/v3.0rc1/tutorials/hello_nas.html>`_ for more changes.
|
||||
|
||||
* A refreshed experience to construct model space.
|
||||
* Enhanced debuggability via ``freeze()`` and ``simplify()`` APIs.
|
||||
* Enhanced expressiveness with ``nni.choice``, ``nni.uniform``, ``nni.normal`` and etc.
|
||||
* Enhanced experience of customization with ``MutableModule``, ``ModelSpace`` and ``ParamterizedModule``.
|
||||
* Search space with constraints is now supported.
|
||||
|
||||
* Improved robustness and stability of strategies.
|
||||
* Supported search space types are now enriched for PolicyBaseRL, ENAS and Proxyless.
|
||||
* Each step of one-shot strategies can be executed alone: model mutation, evaluator mutation and training.
|
||||
* Most multi-trial strategies now supports specifying seed for reproducibility.
|
||||
* Performance of strategies have been verified on a set of benchmarks.
|
||||
|
||||
* Strategy/engine middleware.
|
||||
* Filtering, replicating, deduplicating or retrying models submitted by any strategy.
|
||||
* Merging or transforming models before executing (e.g., CGO).
|
||||
* Arbitrarily-long chains of middlewares.
|
||||
|
||||
* New execution engine.
|
||||
|
||||
* Improved debuggability via SequentialExecutionEngine: trials can run in a single process and breakpoints are effective.
|
||||
* The old execution engine is now decomposed into execution engine and model format.
|
||||
* Enhanced extensibility of execution engines.
|
||||
|
||||
* NAS profiler and hardware-aware NAS.
|
||||
|
||||
* New profilers profile a model space, and quickly compute a profiling result for a sampled architecture or a distribution of architectures (FlopsProfiler, NumParamsProfiler and NnMeterProfiler are officially supported).
|
||||
* Assemble profiler with arbitrary strategies, including both multi-trial and one-shot.
|
||||
* Profiler are extensible. Strategies can be assembled with arbitrary customized profilers.
|
||||
|
||||
Model Compression
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
* Compression framework is refactored, new framework import path is ``nni.contrib.compression``.
|
||||
|
||||
* Configure keys are refactored, support more detailed compression configurations.
|
||||
* Support multi compression methods fusion.
|
||||
* Support distillation as a basic compression component.
|
||||
* Support more compression targets, like ``input``, ``ouptut`` and any registered paramters.
|
||||
* Support compressing any module type by customizing module settings.
|
||||
|
||||
* Pruning
|
||||
|
||||
* Pruner interfaces have fine-tuned for easy to use.
|
||||
* Support configuring ``granularity`` in pruners.
|
||||
* Support different mask ways, multiply zero or add a large negative value.
|
||||
* Support manully setting dependency group and global group.
|
||||
* A new powerful pruning speedup is released, applicability and robustness have been greatly improved.
|
||||
* The end to end transformer compression tutorial has been updated, achieved more extreme compression performance.
|
||||
|
||||
* Quantization
|
||||
|
||||
* Support using ``Evaluator`` to handle training/inferencing.
|
||||
* Support more module fusion combinations.
|
||||
* Support configuring ``granularity`` in quantizers.
|
||||
|
||||
* Distillation
|
||||
|
||||
* DynamicLayerwiseDistiller and Adaptive1dLayerwiseDistiller are supported.
|
||||
|
||||
* Compression documents now updated for the new framework, the old version please view `v2.10 <https://nni.readthedocs.io/en/v2.10/>`_ doc.
|
||||
* New compression examples are under `nni/examples/compression <https://github.com/microsoft/nni/tree/v3.0rc1/examples/compression>`_
|
||||
|
||||
* Create a evaluator: `nni/examples/compression/evaluator <https://github.com/microsoft/nni/tree/v3.0rc1/examples/compression/evaluator>`_
|
||||
* Pruning a model: `nni/examples/compression/pruning <https://github.com/microsoft/nni/tree/v3.0rc1/examples/compression/pruning>`_
|
||||
* Quantize a model: `nni/examples/compression/quantization <https://github.com/microsoft/nni/tree/v3.0rc1/examples/compression/quantization>`_
|
||||
* Fusion compression: `nni/examples/compression/fusion <https://github.com/microsoft/nni/tree/v3.0rc1/examples/compression/fusion>`_
|
||||
|
||||
Training Services
|
||||
^^^^^^^^^^^^^^^^^
|
||||
|
||||
* **Breaking change**: NNI v3.0 cannot resume experiments created by NNI v2.x
|
||||
* Local training service:
|
||||
|
||||
* Reduced latency of creating trials
|
||||
* Fixed "GPU metric not found"
|
||||
* Fixed bugs about resuming trials
|
||||
|
||||
* Remote training service:
|
||||
|
||||
* ``reuse_mode`` now defaults to ``False``; setting it to ``True`` will fallback to v2.x remote training service
|
||||
* Reduced latency of creating trials
|
||||
* Fixed "GPU metric not found"
|
||||
* Fixed bugs about resuming trials
|
||||
* Supported viewing trial logs on the web portal
|
||||
* Supported automatic recover after temporary server failure (network fluctuation, out of memory, etc)
|
||||
|
||||
Release 2.10 - 11/14/2022
|
||||
-------------------------
|
||||
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче