**Description**
Add script to generate system config info.
**Major Revision**
- Add script to generate system config info into the dict in superbench/tools.
**Description**
Generate the summarized output files from all nodes. For each metric, do the reduce operation according to the `reduce_op`
**Major Revision**
- Generate the summarized json file per node:
For microbenchmark, the format is `{benchmark_name}/[{run_count}/]{metric_name}[:rank]`
For modelbenchmark, the format is `{benchmark_name}/{sub_benchmark_name}/[{run_count}/]{metric_name}`
`[]` means optional.
```
{
"kernel-launch/overhead_event:0": 0.00583,
"kernel-launch/overhead_event:1": 0.00545,
"kernel-launch/overhead_event:2": 0.00581,
"kernel-launch/overhead_event:3": 0.00572,
"kernel-launch/overhead_event:4": 0.00559,
"kernel-launch/overhead_event:5": 0.00591,
"kernel-launch/overhead_event:6": 0.00562,
"kernel-launch/overhead_event:7": 0.00586,
"resnet_models/pytorch-resnet50/steptime-train-float32": 544.0827468410134,
"resnet_models/pytorch-resnet50/throughput-train-float32": 353.7607016465773,
"resnet_models/pytorch-resnet50/steptime-train-float16": 425.40482617914677,
"resnet_models/pytorch-resnet50/throughput-train-float16": 454.0142363793973,
"pytorch-sharding-matmul/0/allreduce": 10.561786651611328,
"pytorch-sharding-matmul/1/allreduce": 10.561786651611328,
"pytorch-sharding-matmul/0/allgather": 10.088025093078613,
"pytorch-sharding-matmul/1/allgather": 10.088025093078613
}
```
- Generate the summarized jsonl file for all nodes, each line is the result from one node in json format.
* Add `sb deploy` command content.
* Fix inline if-expression syntax in playbook.
* Fix quote escape issue in bash command.
* Add custom env in config.
* Update default config for multi GPU benchmarks.
* Update MANIFEST.in to include jinja2 template.
* Require jinja2 minimum version.
* Fix occasional duplicate output in Ansible runner.
* Fix mixed color from Ansible and Python colorlog.
* Update according to comments.
* Change superbench.env from list to dict in config file.
* add random dataset.
* install pytorch-cpu for test docker.
* fix typo
* add more test cases.
* address comments.
Co-authored-by: Guoshuai Zhao <guzhao@microsoft.com>
Update some lint rules, including:
* change max line length from 79 to 120, following [pytorch]
* add dedent_closing_brackets in yapf
* remove typed def requirements in mypy
Fix return code bug in setup.py, when lint/test command return 1,
`os.system` will return 256 and `sys.exit(256)` will get return code 0.
[pytorch]: https://github.com/pytorch/pytorch/blob/d1dcd5f/.flake8#L3
Initialize setup.py and basic configurations for this project.
Major revisions:
- initialize setup.py for Python package
- add gitignore and dockerignore
- add editorconfig for editors
- configure yapf for auto formating
- configure mypy for type hint
- configure flake8 for lint, including quotes and docstrings
- add pre-commit check for `git commit`
- add spelling check in GitHub Actions
- format existing files according to configured rules
Example usage:
# install dependencies
$ python3 -m pip install -e .[dev,test]
$ pre-commit install
# format code automatically
$ python3 setup.py format
# lint code
$ python3 setup.py lint
# test code
$ python3 setup.py test
Yuting Jiang
guoshzhao
Yifan Xiong
guoshzhao