refine README.md
This commit is contained in:
jiahangxu
Родитель
a9749b700f
Коммит
f9237fbc58
119
README.md
119
README.md
|
|
@ -19,7 +19,7 @@ The current supported hardware and inference frameworks:
|
|||
- Those who want to run **hardware-aware NAS with [NNI](https://github.com/microsoft/nni)**.
|
||||
- Those who want to **build latency predictors for their own devices**.
|
||||
|
||||
## Installation
|
||||
# Installation
|
||||
|
||||
To install nn-meter, please first install python3. The test environment uses anaconda python 3.6.10. Install the dependencies via:
|
||||
`pip3 install -r requirements.txt`
|
||||
|
|
@ -27,8 +27,8 @@ Please also check the versions of numpy, scikit_learn. The different versions ma
|
|||
|
||||
If you use nn-meter in NNI, make sure NNI version >= 2.4
|
||||
|
||||
## Usage
|
||||
### Supported input model format
|
||||
# Usage
|
||||
## Supported input model format
|
||||
We have two types of interfaces：
|
||||
- command line `nn-meter` after install the package `nn-meter`
|
||||
- Python binding provided by the module `nn_meter`
|
||||
|
|
@ -37,13 +37,16 @@ Here is a summary of supported inputs of the two methods.
|
|||
|
||||
| Name | Command Support | Python Binding |
|
||||
| :--------------: | :-------------------------------------------------: | :---: |
|
||||
| Tensorflow | Checkpoint file dumped by `tf.save(...)` and endwith `.pb` | ... |
|
||||
| Torch | models in `torchvision.models` | object of `nn.Module` |
|
||||
| Onnx | Checkpoint file dumped by .onnx | model loaded by (`onnx.load()`) |
|
||||
| nn-Meter IR graph | Json file in the format of (...) | `dict` object following ... |
|
||||
| NNI IR graph | - | `dict` object following (nni doc)
|
||||
| Tensorflow | Checkpoint file dumped by `tf.saved_model()` and endwith `.pb` | Checkpoint file dumped by `tf.saved_model` and endwith `.pb` |
|
||||
| Torch | Models in `torchvision.models` | Object of `torch.nn.Module` |
|
||||
| Onnx | Checkpoint file dumped by `onnx.save()` and endwith `.onnx` | Checkpoint file dumped by `onnx.save()` or model loaded by `onnx.load()` |
|
||||
| nn-Meter IR graph | Json file in the format of [nn-Meter IR Graph](./docs/input_models.md#nnmeter-ir-graph) | `dict` object following the format of [nn-Meter IR Graph](./docs/input_models.md#nnmeter-ir-graph) |
|
||||
| NNI IR graph | - | `dict` object following [NNI Doc](https://nni.readthedocs.io/en/stable/Tutorial/InstallationLinux.html#installation) |
|
||||
|
||||
|
||||
## Command line Support
|
||||
### List all predefined predictors
|
||||
|
||||
### To predict a single model: Run nn-Meter demo
|
||||
After installation, a command named `nn-meter` is enabled. Users can get all predefined predictors by running
|
||||
|
||||
```bash
|
||||
|
|
@ -51,43 +54,6 @@ After installation, a command named `nn-meter` is enabled. Users can get all pre
|
|||
nn-meter --list-predictors
|
||||
```
|
||||
|
||||
To predict the latency for a CNN model with a predefined predictor, users can run the following commands
|
||||
|
||||
```bash
|
||||
# for Tensorflow (*.pb) file
|
||||
nn-meter --predictor <hardware> --predictor-version <version> --tensorflow <pb-file_or_folder>
|
||||
|
||||
# for ONNX (*.onnx) file
|
||||
nn-meter --predictor <hardware> --predictor-version <version> --onnx <onnx-file_or_folder>
|
||||
|
||||
# for nn-Meter IR (*.json) file
|
||||
nn-meter --predictor <hardware> --predictor-version <version> --nn-meter-ir <json-file_or_folder>
|
||||
|
||||
# for torch model (str)
|
||||
nn-meter --predictor <hardware> --predictor-version <version> --torchvision <model-name>...
|
||||
```
|
||||
|
||||
`--predictor-version <version>` arguments is optional. When the predictor version is not specified by users, nn-meter will use the latest verison of the predictor.
|
||||
|
||||
nn-Meter can support batch mode prediction. To predict latency for multiple models in the same model type once, user should collect all models in one folder and state the folder after `--<model-type>` liked argument.
|
||||
|
||||
It should also be noted that for PyTorch model, nn-meter can only support existing models in torchvision model zoo. The string followed by `--torchvision` should exactly be one or more string indicating name(s) of some existing torchvision models.
|
||||
|
||||
|
||||
Furthermore, users may be interested to convert tensorflow pb-file or onnx file to nn-meter ir graph. Users could convert nn-meter ir graph and save to `.json` file be running
|
||||
|
||||
```bash
|
||||
# for Tensorflow (*.pb) file
|
||||
nn-meter getir --tensorflow <pb-file> --output <output-name>
|
||||
|
||||
# for ONNX (*.onnx) file
|
||||
nn-meter getir --onnx <onnx-file> --output <output-name>
|
||||
```
|
||||
|
||||
Output name is default to be `/path/to/previous/file/<previous_file_name>_<model-type>_ir.json` if not specified by users.
|
||||
|
||||
|
||||
|
||||
nn-Meter currently supports prediction on the following four config:
|
||||
|
||||
| Predictor (device_inferenceframework) |
|
||||
|
|
@ -95,11 +61,54 @@ nn-Meter currently supports prediction on the following four config:
|
|||
| cortexA76cpu_tflite21 |
|
||||
| adreno640gpu_tflite21 |
|
||||
| adreno630gpu_tflite21 |
|
||||
| myriadvpu_openvino2019r2 |
|
||||
| myriadvpu_openvino2019r2 |
|
||||
|
||||
For the input model file, you can find any example provided under the `data/testmodels`
|
||||
|
||||
#### Import nn-Meter in your python code
|
||||
|
||||
### Predict latency for CNN model
|
||||
|
||||
To predict the latency for a CNN model with a predefined predictor in command line, users can run the following commands
|
||||
|
||||
```bash
|
||||
# for Tensorflow (*.pb) file
|
||||
nn-meter --predictor <hardware> [--predictor-version <version>] --tensorflow <pb-file_or_folder>
|
||||
|
||||
# for ONNX (*.onnx) file
|
||||
nn-meter --predictor <hardware> [--predictor-version <version>] --onnx <onnx-file_or_folder>
|
||||
|
||||
# for torch model from torchvision model zoo (str)
|
||||
nn-meter --predictor <hardware> [--predictor-version <version>] --torchvision <model-name> <model-name>...
|
||||
|
||||
# for nn-Meter IR (*.json) file
|
||||
nn-meter --predictor <hardware> [--predictor-version <version>] --nn-meter-ir <json-file_or_folder>
|
||||
```
|
||||
|
||||
`--predictor-version <version>` arguments is optional. When the predictor version is not specified by users, nn-meter will use the latest verison of the predictor.
|
||||
|
||||
nn-Meter can support batch mode prediction. To predict latency for multiple models in the same model type once, user should collect all models in one folder and state the folder after `--[model-type]` liked argument.
|
||||
|
||||
It should also be noted that for PyTorch model, nn-meter can only support existing models in torchvision model zoo. The string followed by `--torchvision` should be exactly one or more string indicating name(s) of some existing torchvision models.
|
||||
|
||||
|
||||
### Convert to nn-Meter IR Graph
|
||||
|
||||
Furthermore, users may be interested to convert tensorflow pb-file or onnx file to nn-Meter IR graph. Users could convert nn-Meter IR graph and save to `.json` file be running
|
||||
|
||||
```bash
|
||||
# for Tensorflow (*.pb) file
|
||||
nn-meter getir --tensorflow <pb-file> [--output <output-name>]
|
||||
|
||||
# for ONNX (*.onnx) file
|
||||
nn-meter getir --onnx <onnx-file> [--output <output-name>]
|
||||
```
|
||||
|
||||
Output name is default to be `/path/to/input/file/<input_file_name>_<model-type>_ir.json` if not specified by users.
|
||||
|
||||
|
||||
## Import nn-Meter in your python code: Python binding
|
||||
|
||||
After installation, users can import nn-Meter in python code
|
||||
|
||||
```python
|
||||
from nn_meter import load_latency_predictor
|
||||
|
|
@ -116,11 +125,11 @@ By calling `load_latency_predictor`, user selects the target hardware (`Framewor
|
|||
|
||||
Users could view the information all built-in predictors by `list_latency_predictors` or view the config file in `nn_meter/configs/predictors.yaml`.
|
||||
|
||||
Users could get a nn-meter ir graph by applying `model_file_to_graph`.
|
||||
Users could get a nn-Meter IR graph by applying `model_file_to_graph` and `model_to_graph` by calling the model name or model object and specify the model type. The supporting model types of `model_file_to_graph` include "onnx", "pb", "torch", "nnmeter-ir" and "nni-ir", while the supporting model types of `model_to_graph` include "onnx", "torch", "nnmeter-ir" and "nni-ir".
|
||||
|
||||
### Hardware-aware NAS by nn-Meter and NNI
|
||||
## Hardware-aware NAS by nn-Meter and NNI
|
||||
|
||||
#### Run multi-trial SPOS demo
|
||||
### Run multi-trial SPOS demo
|
||||
|
||||
Install NNI by following [NNI Doc](https://nni.readthedocs.io/en/stable/Tutorial/InstallationLinux.html#installation).
|
||||
|
||||
|
|
@ -136,9 +145,9 @@ Then run multi-trail SPOS demo:
|
|||
python ${NNI_ROOT}/examples/nas/oneshot/spos/multi_trial.py
|
||||
```
|
||||
|
||||
#### How the demo works
|
||||
### How the demo works
|
||||
|
||||
Refer to [nni doc ](https://nni.readthedocs.io/en/stable/nas.html)for how to perform NAS by NNI.
|
||||
Refer to [nni doc](https://nni.readthedocs.io/en/stable/nas.html) for how to perform NAS by NNI.
|
||||
|
||||
To support latency-aware NAS, you first need a `Strategy` that supports filtering the models by latency. We provide such a filter named `LatencyFilter` in NNI and initialize a `Random` strategy with the filter:
|
||||
|
||||
|
|
@ -157,7 +166,7 @@ RetiariiExperiment(base_model, trainer, [], simple_strategy, True, example_input
|
|||
|
||||
Here, `parse_shape=True` means extracting shape info from the torch model as it is required by nn-Meter to predict latency. `example_inputs` is required for tracing shape info.
|
||||
|
||||
## Contributing
|
||||
# Contributing
|
||||
|
||||
This project welcomes contributions and suggestions. Most contributions require you to agree to a
|
||||
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
|
||||
|
|
@ -171,7 +180,7 @@ This project has adopted the [Microsoft Open Source Code of Conduct](https://ope
|
|||
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
|
||||
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.
|
||||
|
||||
## License
|
||||
# License
|
||||
|
||||
The entire codebase is under [MIT license](https://github.com/microsoft/nn-Meter/blob/main/LICENSE)
|
||||
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ model = ... # model is instance of torch.nn.Module
|
|||
lat = predictor.predict(model,model_type='torch', input_shape=(3, 224, 224))
|
||||
```
|
||||
|
||||
### nn-Meter IR graph
|
||||
### <span id="nnmeter-ir-graph"> nn-Meter IR graph </span>
|
||||
|
||||
As introduced, nn-Meter will perform a pre-processing step to convert the above model formats into the nn-Meter IR graphs. Now we introduce the defined IR graph.
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,8 @@ from .nn_meter import (
|
|||
nnMeter,
|
||||
load_latency_predictor,
|
||||
list_latency_predictors,
|
||||
model_file_to_graph
|
||||
model_file_to_graph,
|
||||
model_to_graph
|
||||
)
|
||||
from .utils.utils import download_from_url
|
||||
import logging
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче