microsoft
/
nn-Meter
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

change kernel name to sync with nn-meter predictor

This commit is contained in:
jiahangxu 2022-04-04 05:30:38 -04:00
Родитель 9af03d48bc
Коммит 28f6f82ab2
13 изменённых файлов: 266 добавлений и 209 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

217
docs/builder/build_kernel_latency_predictor.md
Просмотреть файл

@ -12,71 +12,86 @@ The second step is generating and profiling kernel model by configurations. Curr
(conv related kernels)
- `"conv_bn_relu"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv_bn_relu6"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv_bn"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv_relu"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv_relu6"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv_hswish"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv_block"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv_bn_hswish"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv-bn-relu"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv-bn-relu6"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv-bn"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv-relu"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv-relu6"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv-hswish"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv-block"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"conv-bn-hswish"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
(dwconv related kernels)
- `"dwconv_bn"`: `HW`, `CIN`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv_relu"`: `HW`, `CIN`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv_relu6"`: `HW`, `CIN`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv_bn_relu"`: `HW`, `CIN`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv_bn_relu6"`: `HW`, `CIN`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv_block"`: `HW`, `CIN`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv_bn_hswish"`: `HW`, `CIN`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv-bn"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv-relu"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv-relu6"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv-bn-relu"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv-bn-relu6"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv-block"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
- `"dwconv-bn-hswish"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `STRIDES`
(other kernels)
- `"maxpool_block"`: `HW`, `CIN`, `KERNEL_SIZE`, `POOL_STRIDES`
- `"avgpool_block"`: `HW`, `CIN`, `KERNEL_SIZE`, `POOL_STRIDES`
- `"fc_block"`: `CIN`, `COUT`
- `"concat_block"`: `HW`, `CIN1`, `CIN2`, `CIN3`, `CIN4`
- `"split_block"`: `HW`, `CIN`
- `"channel_shuffle"`: `HW`, `CIN`
- `"se_block"`: `HW`, `CIN`
- `"globalavgpool_block"`: `HW`, `CIN`
- `"bn_relu"`: `HW`, `CIN`
- `"bn_block"`: `HW`, `CIN`
- `"hswish_block"`: `HW`, `CIN`
- `"relu_block"`: `HW`, `CIN`
- `"add_relu"`: `HW`, `CIN`
- `"add_block"`: `HW`, `CIN`
- `"maxpool"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `POOL_STRIDES`
- `"avgpool"`: `HW`, `CIN`, `COUT`, `KERNEL_SIZE`, `POOL_STRIDES`
- `"fc"`: `CIN`, `COUT`
- `"concat"`: `HW`, `CIN1`, `CIN2`, `CIN3`, `CIN4`
- `"split"`: `HW`, `CIN`
- `"channelshuffle"`: `HW`, `CIN`
- `"se"`: `HW`, `CIN`
- `"global-avgpool"`: `HW`, `CIN`
- `"bnrelu"`: `HW`, `CIN`
- `"bn"`: `HW`, `CIN`
- `"hswish"`: `HW`, `CIN`
- `"relu"`: `HW`, `CIN`
- `"addrelu"`: `HW`, `CIN`
- `"add"`: `HW`, `CIN`
The first and second step are implemented by `nn_meter.builder.nn_meter_builder.sample_and_profile_kernel_data`. Here is an example:
``` python
from nn_meter.builder import builder_config
from nn_meter.builder.nn_meter_builder import sample_and_profile_kernel_data
kernel_type = "conv_bn_relu"
backend = "tflite_cpu"
workspace = "/path/to/workspace/"
builder_config.init(workspace)
# init predictor builder with prior data sampler
kernel_data = sample_and_profile_kernel_data(kernel_type, init_sample_num = 1000, backend, sampling_mode='prior', mark='prior')
kernel_type = "conv-bn-relu"
sample_num = 10
backend = "tflite_cpu"
mark = "test"
kernel_data = sample_and_profile_kernel_data(kernel_type, init_sample_num = sample_num,
backend=backend, sampling_mode='prior', mark=mark)
```
The generated models are saved in `<workspace-path>/predictor_build/models`, and the configuration information and profiled results are dumped in json file to `<workspace-path>/predictor_build/results/<kernel_type>.json` and `<workspace-path>/predictor_build/results/profiled_<kernel_type>.json`.
The method `sample_and_profile_kernel_data` is composed by three main steps, `generate_config_sample`, `convert_models`, `profile_models`. Here is an example as a decomposition of `sample_and_profile_kernel_data`. Users could choose the decomposed interfaces if needed.
``` python
from nn_meter.builder.kernel_predictor_builder import generate_config_sample
from nn_meter.builder import convert_models
# sample configs for kernel and generate models
models = generate_config_sample(kernel_type, sample_num, mark=mark,
sampling_mode=sampling_mode, configs=configs)
The method `sample_and_profile_kernel_data` is composed by three main steps, `generate_config_sample`, `convert_models`, and `profile_models`. Here is an example as a decomposition of `sample_and_profile_kernel_data`. Users could choose the decomposed interfaces if needed.
``` python
from nn_meter.builder.backends import connect_backend
from nn_meter.builder.kernel_predictor_builder import generate_config_sample
from nn_meter.builder import builder_config, convert_models, profile_models
workspace = "/path/to/workspace/"
builder_config.init(workspace)
# connect to backend, run models and get latency
backend = connect_backend(backend_name="tflite-cpu")
kernel_type = "conv-bn-relu"
sample_num = 10
mark = "test"
# sample configs for kernel and generate models
models = generate_config_sample(kernel_type, sample_num, mark=mark,
sampling_mode="prior")
# convert the model to the needed format by backend, in order to increase efficiency when profiling on device.
models = convert_models(backend, saved_name, broken_point_mode=True)
models = convert_models(backend, f"{workspace}/predictor_build/results/{kernel_type}_{mark}.json")
# run models with given backend and return latency of testcase models
profiled_results = profile_models(backend, models, mode='predbuild', save_name="xxx.json", have_converted=True)
profiled_results = profile_models(backend, models, mode='predbuild', have_converted=True,
save_name=f"profiled_{kernel_type}.json")
```
Note: for kernels related to conv and dwconv, our experiment results have shown that all kernels containing one conv layer or one dwconv layer have almost the same latency results. Thus in nn-Meter, all kernels containing one conv or dwconv layer shares the same kernel predictor.
@ -86,23 +101,28 @@ Note: for kernels related to conv and dwconv, our experiment results have shown
After preparing the training data, we construct a random forest regression model as the kernel latency predictor. Here is an example:
```python
import os
from nn_meter.builder import builder_config
from nn_meter.builder.kernel_predictor_builder import build_predictor_by_data
workspace = "/path/to/workspace/"
builder_config.init(workspace)
kernel_type = "conv_bn_relu"
kernel_type = "conv-bn-relu"
mark = "prior"
backend = "tflite_cpu"
error_threshold = 0.1
# extract training feature and target from profiled results
cfgs_path = os.path.join("<workspace-path>", "predictor_build", "results", "conv_bn_relu.json")
lats_path = os.path.join("<workspace-path>", "predictor_build", "results", "profiled_conv_bn_relu.json")
cfgs_path = os.path.join(workspace, "predictor_build", "results", f"{kernel_type}_{mark}.json")
lats_path = os.path.join(workspace, "predictor_build", "results", f"profiled_{kernel_type}.json")
kernel_data = (cfgs_path, lats_path)
# build latency predictor
predictor, acc10, error_configs = build_predictor_by_data(
kernel_type, kernel_data, backend, error_threshold=error_threshold, mark="prior",
save_path=os.path.join("<workspace-path>", "predictor_build", "results")
save_path=os.path.join(workspace, "predictor_build", "results")
)
logging.info(f'Iteration 0: acc10 {acc10}, error_configs number: {len(error_configs)}')
print(f'Iteration 0: acc10 {acc10}, error_configs number: {len(error_configs)}')
```
In the implementation, the tuple of `kernel_data` includes items of `cfgs_path` and `lats_path`, indicating the config information and latency information respectively. `cfgs_path` and `lats_path` accept both json file path string or dictionary of models. In addition, if the config information and latency information are in the same data holder, users could directly specify `kernel_data = cfgs_path`.
@ -131,34 +151,39 @@ The iterative process continues until the predictor accuracy meets user's requir
Here is an example for adaptive data sampling:
```python
for i in range(1, iteration):
from nn_meter.builder.utils import merge_prev_info
finegrained_sample_num = 10
for i in range(1, 5):
# finegrained sampling and profiling for large error data
new_kernel_data = sample_and_profile_kernel_data(
kernel_type, finegrained_sample_num, backend,
sampling_mode = 'finegrained', configs=error_configs, mark=f'finegrained{i}'
)
new_kernel_data = sample_and_profile_kernel_data(kernel_type, finegrained_sample_num, backend,
sampling_mode = 'finegrained', configs=error_configs, mark=f'finegrained{i}')
# merge finegrained data with previous data and build new regression model
kernel_data = merge_prev_info(new_info=new_kernel_data, prev_info=kernel_data)
predictor, acc10, error_configs = build_predictor_by_data(
kernel_type, kernel_data, backend, error_threshold=error_threshold, mark="prior",
save_path=os.path.join("<workspace-path>", "predictor_build", "results")
)
logging.keyinfo(f'Iteration {i}: acc10 {acc10}, error_configs number: {len(error_configs)}')
_, acc10, error_configs = build_predictor_by_data(kernel_type,
kernel_data,
backend,
error_threshold=error_threshold,
mark='finegrained{i}',
save_path=os.path.join(workspace, "results"))
print(f'Iteration {i}: acc10 {acc10}, error_configs number: {len(error_configs)}')
```
## End-to-end Demo
nn-Meter have wrapped the four main steps into one method named `nn_meter.builder.build_predictor_for_kernel`. There is an example to build latency predictor for `"conv_bn_relu"` kernel:
nn-Meter have wrapped the four main steps into one method named `nn_meter.builder.build_predictor_for_kernel`. There is an example to build latency predictor for `"conv-bn-relu"` kernel:
```python
# initialize builder config with workspace
from nn_meter.builder import builder_config
builder_config.init("path/to/workspace/folder")
workspace = "/path/to/workspace/"
builder_config.init(workspace)
# build latency predictor for kernel
from nn_meter.builder import build_predictor_for_kernel
kernel_type = "conv_bn_relu"
kernel_type = "conv-bn-relu"
backend = "tflite_cpu"
predictor, data = build_predictor_for_kernel(
@ -173,7 +198,8 @@ nn-Meter also provided a end-to-end method for users to build a series of genera
``` python
# initialize builder config with workspace
from nn_meter.builder import builder_config
builder_config.init("path/to/workspace/folder") # initialize builder config with workspace
workspace = "/path/to/workspace/"
builder_config.init(workspace)
# build latency predictor for kernel
from nn_meter.builder import build_latency_predictor
@ -192,7 +218,7 @@ nn-Meter provide API for users to customize their own kernel block. In nn-Meter,
- `input_shape` defines the dimension of one model input shape without batch size. Generally, when the input shape is 3D, `input_shape` should be`[config["HW"], config["HW"], config["CIN"]]`, and when the input shape is 1D, `input_shape` should be`[config["CIN"]]`.
- `input_tensor_shape` is a list defining all model inputs. In basic situation, `input_tensor_shape` should be `[input_shape]` if the kernel only has one input. If the kernel has more than one input, such as `add_relu` kernel, `input_tensor_shape` is `[input_shape, input_shape]`.
- `input_tensor_shape` is a list defining all model inputs. In basic situation, `input_tensor_shape` should be `[input_shape]` if the kernel only has one input. If the kernel has more than one input, such as `addrelu` kernel, `input_tensor_shape` is `[input_shape, input_shape]`.
- `get_model` is the implementation of the kernel model and return a instance of `keras.Model` of the kernel.
@ -203,7 +229,7 @@ import tensorflow.keras as keras
from nn_meter.builder.nn_generator import BaseBlock
class MyKernel(BaseBlock):
''' This kernel is built by Conv, BN, and Relu layer, which is the same as the builtin `conv_bn_relu` block.
''' This kernel is built by Conv, BN, and Relu layer, which is the same as the builtin `conv-bn-relu` block.
'''
def __init__(self, config):
self.config = config
@ -305,8 +331,7 @@ Here is an example:
from nn_meter.builder.kernel_predictor_builder import BaseFeatureParser
class MyParser(BaseFeatureParser):
''' This parser utilized config "HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES", as well as the flops and parameter number as feature,
which is the same parser for Conv, Dwconv and FC related kernel.
''' This parser utilized config "HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES", as well as the flops and parameter number as feature.
'''
def __init__(self, kernel_type):
self.kernel_type = kernel_type
@ -372,7 +397,7 @@ Following is an example of the yaml file:
```yaml
builtin_name: mykernel
package_location: /home/USERNAME/working/tftest/kernel_package
package_location: /home/{USERNAME}/working/tftest/kernel_package
class_module: kernel_script
class_name: MyKernel
sampler_module: config_sampler
@ -401,35 +426,35 @@ nn-meter --list-kernels
```
```text
(nn-Meter) Supported kernels: ('*' indicates customized kernels)
(nn-Meter) [Kernel] conv_bn_relu
(nn-Meter) [Kernel] conv_bn_relu6
(nn-Meter) [Kernel] conv_bn
(nn-Meter) [Kernel] conv_relu
(nn-Meter) [Kernel] conv_relu6
(nn-Meter) [Kernel] conv_hswish
(nn-Meter) [Kernel] conv_block
(nn-Meter) [Kernel] conv_bn_hswish
(nn-Meter) [Kernel] dwconv_bn
(nn-Meter) [Kernel] dwconv_relu
(nn-Meter) [Kernel] dwconv_relu6
(nn-Meter) [Kernel] dwconv_bn_relu
(nn-Meter) [Kernel] dwconv_bn_relu6
(nn-Meter) [Kernel] dwconv_block
(nn-Meter) [Kernel] dwconv_bn_hswish
(nn-Meter) [Kernel] maxpool_block
(nn-Meter) [Kernel] avgpool_block
(nn-Meter) [Kernel] fc_block
(nn-Meter) [Kernel] concat_block
(nn-Meter) [Kernel] split_block
(nn-Meter) [Kernel] channel_shuffle
(nn-Meter) [Kernel] se_block
(nn-Meter) [Kernel] globalavgpool_block
(nn-Meter) [Kernel] bn_relu
(nn-Meter) [Kernel] bn_block
(nn-Meter) [Kernel] hswish_block
(nn-Meter) [Kernel] relu_block
(nn-Meter) [Kernel] add_relu
(nn-Meter) [Kernel] add_block
(nn-Meter) [Kernel] conv-bn-relu
(nn-Meter) [Kernel] conv-bn-relu6
(nn-Meter) [Kernel] conv-bn
(nn-Meter) [Kernel] conv-relu
(nn-Meter) [Kernel] conv-relu6
(nn-Meter) [Kernel] conv-hswish
(nn-Meter) [Kernel] conv-block
(nn-Meter) [Kernel] conv-bn-hswish
(nn-Meter) [Kernel] dwconv-bn
(nn-Meter) [Kernel] dwconv-relu
(nn-Meter) [Kernel] dwconv-relu6
(nn-Meter) [Kernel] dwconv-bn-relu
(nn-Meter) [Kernel] dwconv-bn-relu6
(nn-Meter) [Kernel] dwconv-block
(nn-Meter) [Kernel] dwconv-bn-hswish
(nn-Meter) [Kernel] maxpool
(nn-Meter) [Kernel] avgpool
(nn-Meter) [Kernel] fc
(nn-Meter) [Kernel] concat
(nn-Meter) [Kernel] split
(nn-Meter) [Kernel] channelshuffle
(nn-Meter) [Kernel] se
(nn-Meter) [Kernel] global-avgpool
(nn-Meter) [Kernel] bnrelu
(nn-Meter) [Kernel] bn
(nn-Meter) [Kernel] hswish
(nn-Meter) [Kernel] relu
(nn-Meter) [Kernel] addrelu
(nn-Meter) [Kernel] add
(nn-Meter) [Kernel] * mykernel
```

34
docs/builder/customize_predictor.md
Просмотреть файл

@ -2,9 +2,35 @@
nn-Meter supports customized latency predictors, which can be built on users' devices. To utilize customized predictor in nn-Meter, users should provide all the necessary kernel latency predictors and a fusion rule json file. Users could use [nn-Meter builder](../builder/overview.md) to build their own latency predictors.
After preparing kernel latency predictors and fusion rule following guidance [here](../builder/overview.md), users should register the predictor to nn-Meter for reuse. First of all, put all kernel latency predictors and the fusion rule json file into one folder in a predefined location. The kernel latency predictors should be named by the kernel name, such as `"conv-bn-relu.pkl"`. The fusion rule json file should be named as `"fusion_rules.json"`.
After preparing kernel latency predictors and fusion rules following guidance [here](../builder/overview.md), users could register the predictor to nn-Meter for reuse.
### Step 1: Prepare Meta File
### Step 1: Collect predictors and fusion rules
After preparing kernel latency predictors and fusion rules, there will be a folder containing all kernel predictors in `<workspace-path>/predictor_build/results/predictors/`, and a json file containing the fusion rules in `<workspace-path>/fusion_rule_test/results/detected_fusion_rule.json`. The first step is to collect all kernel latency predictors and the fusion rule json file into one folder in a predefined location. The kernel latency predictors should be named by the kernel name with the training mark such as `"prior"` or `"finegrained"` removed. The fusion rule json file should be named as `"fusion_rules.json"`. Here is an example of the folder:
``` text
/home/{USERNAME}/working/customized_predictor
├── fusion_rules.json
├── add.pkl
├── addrelu.pkl
├── avgpool.pkl
├── bn.pkl
├── bnrelu.pkl
├── channelshuffle.pkl
├── concat.pkl
├── conv-bn-relu.pkl
├── dwconv-bn-relu.pkl
├── fc.pkl
├── global-avgpool.pkl
├── hswish.pkl
├── maxpool.pkl
├── meta.yaml
├── relu.pkl
├── se.pkl
└── split.pkl
```
### Step 2: Prepare Meta File
Create a yaml file with following keys as meta file:
@ -24,7 +50,7 @@ Following is an example of the yaml file:
name: my_predictor
version: 1.0
category: cpu
package_location: /home/USERNAME/working/customized_predictor
package_location: /home/{USERNAME}/working/customized_predictor
kernel_predictors:
- conv-bn-relu
- dwconv-bn-relu
@ -44,7 +70,7 @@ kernel_predictors:
- concat
```
### Step 2: Register Customized Predictor into nn-Meter
### Step 3: Register Customized Predictor into nn-Meter
Run the following command to register customized predictor into nn-Meter:

4
docs/builder/prepare_backend.md
Просмотреть файл

@ -229,10 +229,10 @@ Following is an example of the yaml file:
```yaml
builtin_name: my_backend
package_location: /home/USERNAME/working/customized_backend
package_location: /home/{USERNAME}/working/customized_backend
class_module: backend
class_name: MyBackend
defaultConfigFile: /home/USERNAME/working/customized_backend/default_config.yaml
defaultConfigFile: /home/{USERNAME}/working/customized_backend/default_config.yaml
```
### Step 3: Register Customized Backend into nn-Meter

4
docs/builder/test_fusion_rules.md
Просмотреть файл

@ -401,7 +401,7 @@ Following is an example of the yaml file:
```yaml
builtin_name: op1
package_location: /home/USERNAME/working/customized_operator
package_location: /home/{USERNAME}/working/customized_operator
class_module: operator_script
class_name: Op1
```
@ -603,7 +603,7 @@ Following is an example of the yaml file:
```yaml
builtin_name: MyTC
package_location: /home/USERNAME/working/customized_testcase
package_location: /home/{USERNAME}/working/customized_testcase
class_module: testcase_script
class_name: MyTC
```

2
examples/nn-meter_builder_examples/build_kernel_latency_predictor.ipynb
Просмотреть файл

@ -345,7 +345,7 @@
"source": [
"# build latency predictor for kernel\n",
"from nn_meter.builder import build_predictor_for_kernel\n",
"kernel_type = \"conv_bn_relu\"\n",
"kernel_type = \"conv-bn-relu\"\n",
"backend = \"tflite_cpu\"\n",
"\n",
"predictor, data = build_predictor_for_kernel(\n",

58
nn_meter/builder/kernel_predictor_builder/data_sampler/utils.py
Просмотреть файл

@ -11,37 +11,37 @@ logging = logging.getLogger("nn-Meter")
__BUILTIN_KERNELS__ = {
# builtin name: [kernel class name, kernel sampler class name]
"conv_bn_relu": ["ConvBnRelu", "ConvSampler"],
"conv_bn_relu6": ["ConvBnRelu6", "ConvSampler"],
"conv_bn": ["ConvBn", "ConvSampler"],
"conv_relu": ["ConvRelu", "ConvSampler"],
"conv_relu6": ["ConvRelu6", "ConvSampler"],
"conv_hswish": ["ConvHswish", "ConvSampler"],
"conv_block": ["ConvBlock", "ConvSampler"],
"conv_bn_hswish": ["ConvBnHswish", "ConvSampler"],
"conv-bn-relu": ["ConvBnRelu", "ConvSampler"],
"conv-bn-relu6": ["ConvBnRelu6", "ConvSampler"],
"conv-bn": ["ConvBn", "ConvSampler"],
"conv-relu": ["ConvRelu", "ConvSampler"],
"conv-relu6": ["ConvRelu6", "ConvSampler"],
"conv-hswish": ["ConvHswish", "ConvSampler"],
"conv-block": ["ConvBlock", "ConvSampler"],
"conv-bn-hswish": ["ConvBnHswish", "ConvSampler"],
# dwconv
"dwconv_bn": ["DwConvBn", "DwConvSampler"],
"dwconv_relu": ["DwConvRelu", "DwConvSampler"],
"dwconv_relu6": ["DwConvRelu6", "DwConvSampler"],
"dwconv_bn_relu": ["DwConvBnRelu", "DwConvSampler"],
"dwconv_bn_relu6": ["DwConvBnRelu6", "DwConvSampler"],
"dwconv_block": ["DwConvBlock", "DwConvSampler"],
"dwconv_bn_hswish": ["ConvBnHswish", "DwConvSampler"],
"dwconv-bn": ["DwConvBn", "DwConvSampler"],
"dwconv-relu": ["DwConvRelu", "DwConvSampler"],
"dwconv-relu6": ["DwConvRelu6", "DwConvSampler"],
"dwconv-bn-relu": ["DwConvBnRelu", "DwConvSampler"],
"dwconv-bn-relu6": ["DwConvBnRelu6", "DwConvSampler"],
"dwconv-block": ["DwConvBlock", "DwConvSampler"],
"dwconv-bn-hswish": ["ConvBnHswish", "DwConvSampler"],
# others
"maxpool_block": ["MaxPoolBlock", "PoolingSampler"],
"avgpool_block": ["AvgPoolBlock", "PoolingSampler"],
"fc_block": ["FCBlock", "FCSampler"],
"concat_block": ["ConcatBlock", "ConcatSampler"],
"split_block": ["SplitBlock", "CinEvenSampler"],
"channel_shuffle": ["ChannelShuffle", "CinEvenSampler"],
"se_block": ["SEBlock", "CinEvenSampler"],
"globalavgpool_block": ["GlobalAvgPoolBlock", "GlobalAvgPoolSampler"],
"bn_relu": ["BnRelu", "HwCinSampler"],
"bn_block": ["BnBlock", "HwCinSampler"],
"hswish_block": ["HswishBlock", "HwCinSampler"],
"relu_block": ["ReluBlock", "HwCinSampler"],
"add_relu": ["AddRelu", "HwCinSampler"],
"add_block": ["AddBlock", "HwCinSampler"],
"maxpool": ["MaxPoolBlock", "PoolingSampler"],
"avgpool": ["AvgPoolBlock", "PoolingSampler"],
"fc": ["FCBlock", "FCSampler"],
"concat": ["ConcatBlock", "ConcatSampler"],
"split": ["SplitBlock", "CinEvenSampler"],
"channelshuffle": ["ChannelShuffle", "CinEvenSampler"],
"se": ["SEBlock", "CinEvenSampler"],
"global-avgpool": ["GlobalAvgPoolBlock", "GlobalAvgPoolSampler"],
"bnrelu": ["BnRelu", "HwCinSampler"],
"bn": ["BnBlock", "HwCinSampler"],
"hswish": ["HswishBlock", "HwCinSampler"],
"relu": ["ReluBlock", "HwCinSampler"],
"addrelu": ["AddRelu", "HwCinSampler"],
"add": ["AddBlock", "HwCinSampler"],
}

2
nn_meter/builder/kernel_predictor_builder/predictor_builder/__init__.py
Просмотреть файл

@ -1,4 +1,4 @@
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
from .build_predictor import build_predictor_by_data
from .extract_features import get_data_by_profiled_results, BaseFeatureParser
from .extract_feature import get_data_by_profiled_results, BaseFeatureParser

19
nn_meter/builder/kernel_predictor_builder/predictor_builder/build_predictor.py
Просмотреть файл

@ -6,7 +6,7 @@ import logging
from sklearn.model_selection import train_test_split
from .utils import latency_metrics
from .predictor_lib import init_predictor
from .extract_features import get_feature_parser, get_data_by_profiled_results
from .extract_feature import get_feature_parser, get_data_by_profiled_results
logging = logging.getLogger("nn-Meter")
@ -32,7 +32,11 @@ def build_predictor_by_data(kernel_type, kernel_data, backend = None, error_thre
predict_label (str): the predicting label to build kernel predictor
"""
feature_parser = get_feature_parser(kernel_type)
data = get_data_by_profiled_results(kernel_type, feature_parser, kernel_data, save_path=os.path.join(save_path, f'Data_{kernel_type}_{mark}.csv'), predict_label=predict_label)
os.makedirs(os.path.join(save_path, "collection"), exist_ok=True)
os.makedirs(os.path.join(save_path, "predictors"), exist_ok=True)
data = get_data_by_profiled_results(kernel_type, feature_parser, kernel_data,
save_path=os.path.join(save_path, "collection", f'Data_{kernel_type}_{mark}.csv'),
predict_label=predict_label)
# get data for regression
X, Y = data
@ -54,15 +58,16 @@ def build_predictor_by_data(kernel_type, kernel_data, backend = None, error_thre
test_res["True"] = testy
test_res["Pred"] = predicts
test_res["Error"] = pred_error_list
test_res.to_csv(os.path.join(save_path, f"TestResult_{kernel_type}_{mark}.csv"), index=False)
logging.info(f"All test data and predicted results are stored in path {os.path.join(save_path, f'TestResult_{kernel_type}_{mark}.csv')}")
res_save_path = os.path.join(save_path, "collection", f"TestResult_{kernel_type}_{mark}.csv")
test_res.to_csv(res_save_path, index=False)
logging.info(f"All test data and predicted results are stored in path {res_save_path}")
# dump the predictor model
import pickle
save_path = os.path.join(save_path, f"Predictor_{kernel_type}_{mark}.pkl")
with open(save_path, 'wb') as fp:
pred_save_path = os.path.join(save_path, "predictors", f"{kernel_type}_{mark}.pkl")
with open(pred_save_path, 'wb') as fp:
pickle.dump(predictor, fp)
logging.keyinfo(f"Saved the predictor for {kernel_type} in path {save_path}.")
logging.keyinfo(f"Saved the predictor for {kernel_type} in path {pred_save_path}.")
# locate large error data
error_configs = []

67
nn_meter/builder/kernel_predictor_builder/predictor_builder/extract_features.py → nn_meter/builder/kernel_predictor_builder/predictor_builder/extract_feature.py
Просмотреть файл

@ -12,39 +12,40 @@ logging = logging.getLogger("nn-Meter")
feature_for_kernel = {
# conv
"conv_bn_relu": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv_bn_relu6": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv_bn": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv_relu": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv_relu6": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv_hswish": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv_block": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv_bn_hswish": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv-bn-relu": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv-bn-relu6": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv-bn": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv-relu": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv-relu6": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv-hswish": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv-block": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"conv-bn-hswish": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
# dwconv ("COUT" will always be the same as "CIN")
"dwconv_bn": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv_relu": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv_relu6": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv_bn_relu": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv_bn_relu6": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv_block": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv_bn_hswish": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
# others ("COUT" will always be the same as "CIN")
"maxpool_block": ["HW", "CIN", "COUT", "KERNEL_SIZE", "POOL_STRIDES"],
"avgpool_block": ["HW", "CIN", "COUT", "KERNEL_SIZE", "POOL_STRIDES"],
"fc_block": ["CIN", "COUT"],
"concat_block": ["HW", "CIN1", "CIN2", "CIN3", "CIN4"],
"split_block": ["HW", "CIN"],
"channel_shuffle": ["HW", "CIN"],
"se_block": ["HW", "CIN"],
"globalavgpool_block": ["HW", "CIN"],
"bn_relu": ["HW", "CIN"],
"bn_block": ["HW", "CIN"],
"hswish_block": ["HW", "CIN"],
"relu_block": ["HW", "CIN"],
# In "add_relu" block and "add_block", the second feature "CIN" will always be the same as
"dwconv-bn": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv-relu": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv-relu6": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv-bn-relu": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv-bn-relu6": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv-block": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
"dwconv-bn-hswish": ["HW", "CIN", "COUT", "KERNEL_SIZE", "STRIDES"],
# pooling ("COUT" will always be the same as "CIN")
"maxpool": ["HW", "CIN", "COUT", "KERNEL_SIZE", "POOL_STRIDES"],
"avgpool": ["HW", "CIN", "COUT", "KERNEL_SIZE", "POOL_STRIDES"],
# others
"fc": ["CIN", "COUT"],
"concat": ["HW", "CIN1", "CIN2", "CIN3", "CIN4"],
"split": ["HW", "CIN"],
"channelshuffle": ["HW", "CIN"],
"se": ["HW", "CIN"],
"global-avgpool": ["HW", "CIN"],
"bnrelu": ["HW", "CIN"],
"bn": ["HW", "CIN"],
"hswish": ["HW", "CIN"],
"relu": ["HW", "CIN"],
# In "addrelu" block and "add" block, the second feature "CIN" will always be the same as
# the third feature
"add_relu": ["HW", "CIN", "CIN"],
"add_block": ["HW", "CIN", "CIN"],
"addrelu": ["HW", "CIN", "CIN"],
"add": ["HW", "CIN", "CIN"],
}
__user_config_folder__ = os.path.expanduser('~/.nn_meter/config')
@ -117,7 +118,7 @@ def get_data_by_profiled_results(kernel_type, feature_parser, cfgs_path, labs_pa
cfgs_path: path of config information dict, or dict of "origin_kernels.json", such as
{
"conv_bn_relu": {
"conv-bn-relu": {
"id_0": {
"model": "...",
"shapes": [[14, 14, 98]],
@ -134,7 +135,7 @@ def get_data_by_profiled_results(kernel_type, feature_parser, cfgs_path, labs_pa
labs_path: path of profiled label information dict, or dict of "profiled_results", such as
{
"conv_bn_relu": {
"conv-bn-relu": {
"id_0": {
"latency": "42.001 +- 1.0"
}

32
nn_meter/builder/kernel_predictor_builder/predictor_builder/predictor_lib.py
Просмотреть файл

@ -4,7 +4,7 @@ from sklearn.ensemble import RandomForestRegressor
__PREDICTOR_ZOO__ = {
"conv_bn_relu": {
"conv-bn-relu": {
"tflite_cpu": {
"max_depth": 70,
"n_estimators": 320,
@ -35,7 +35,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"dwconv_bn_relu": {
"dwconv-bn-relu": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 240,
@ -65,7 +65,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"fc_block": {
"fc": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 370,
@ -95,7 +95,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"channel_shuffle": {
"channelshuffle": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 370,
@ -115,7 +115,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"se_block": {
"se": {
"tflite_cpu": {
"max_depth": 20,
"n_estimators": 290,
@ -144,7 +144,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"maxpool_block": {
"maxpool": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 210,
@ -173,7 +173,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"globalavgpool_block": {
"global-avgpool": {
"tflite_cpu": {
"max_depth": 70,
"n_estimators": 370,
@ -184,7 +184,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"hswish_block": {
"hswish": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 190,
@ -213,7 +213,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"avgpool_block": {
"avgpool": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 370,
@ -242,7 +242,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"bn_relu": {
"bnrelu": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 370,
@ -271,7 +271,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"relu_block": {
"relu": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 370,
@ -300,7 +300,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"bn_block": {
"bn": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 370,
@ -329,7 +329,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"concat_block": {
"concat": {
"tflite_cpu": {
"max_depth": 100,
"n_estimators": 690,
@ -349,7 +349,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"add_relu": {
"addrelu": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 570,
@ -378,7 +378,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"add_block": {
"add": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 570,
@ -407,7 +407,7 @@ __PREDICTOR_ZOO__ = {
"random_state": 10,
}
},
"split_block": {
"split": {
"tflite_cpu": {
"max_depth": 50,
"n_estimators": 190,

2
nn_meter/builder/nn_generator/interface.py
Просмотреть файл

@ -42,7 +42,7 @@ class BaseBlock:
torch model), and when the input shape is 1D, `input_shape` should be`[config["CIN"]]`.
- input_tensor_shape: a list defining all model inputs. In basic situation, `input_tensor_shape` should be `[input_shape]` if the kernel
only has one input. If the kernel has more than one input, such as `add_relu` kernel, `input_tensor_shape` is `[input_shape, input_shape]`.
only has one input. If the kernel has more than one input, such as `addrelu` kernel, `input_tensor_shape` is `[input_shape, input_shape]`.
- batch_size: the required batch size of the input data
'''

32
nn_meter/configs/builder/predictor_builder/predictorbuild_config.yaml
Просмотреть файл

@ -2,82 +2,82 @@ DETAIL: FALSE
IMPLEMENT: tensorflow
BATCH_SIZE: 1
KERNELS:
conv_bn_relu:
conv-bn-relu:
INIT_SAMPLE_NUM: 10000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
dwconv_bn_relu:
dwconv-bn-relu:
INIT_SAMPLE_NUM: 5000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
maxpool_block:
maxpool:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
avgpool_block:
avgpool:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
fc_block:
fc:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
concat_block:
concat:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
split_block:
split:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
channel_shuffle:
channelshuffle:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
se_block:
se:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
globalavgpool_block:
global-avgpool:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
bn_relu:
bnrelu:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
bn_block:
bn:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
hswish_block:
hswish:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
relu_block:
relu:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
add_relu:
addrelu:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3
ERROR_THRESHOLD: 0.1
add_block:
add:
INIT_SAMPLE_NUM: 3000
FINEGRAINED_SAMPLE_NUM: 20
ITERATION: 3

2
tests/unit_test/test_predictor_builder.py
Просмотреть файл

@ -9,7 +9,7 @@ builder_config.init("/path/to/workspace/") # initialize builder config with work
# build latency predictor for kernel
from nn_meter.builder import build_predictor_for_kernel
kernel_type = "conv_bn_relu"
kernel_type = "conv-bn-relu"
backend = "tflite_cpu"
predictor, data = build_predictor_for_kernel(