## Describe your changes
Perf tuning cli.
E.g. for phi2-int4-cpu got from model builder, user can directly tuning
the session parameters in cpu and cuda ep with:
`olive tune-session-params --model model.onnx --device gpu
--providers_list CUDA CPU --hf_model_name microsoft/phi-2`
Also, user can provide their customized data_config:
`olive tune-session-params --model model.onnx --device gpu
--providers_list CUDA CPU --data_config_path data_config.json`
where the data config json looks like
```json
{
"name": "gpu_cpu",
"type": "TransformersTokenDummyDataContainer",
"load_dataset_config": {
"model_name": "microsoft/phi-2"
}
}
```
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
- [ ] Is this PR including examples changes? If yes, please remember to
update [example
documentation](https://github.com/microsoft/Olive/blob/main/docs/source/examples.md)
in a follow-up PR.
## (Optional) Issue link
## Describe your changes
remove sample codes from zipfile packaging
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
- [ ] Is this PR including examples changes? If yes, please remember to
update [example
documentation](https://github.com/microsoft/Olive/blob/main/docs/source/examples.md)
in a follow-up PR.
## (Optional) Issue link
## Describe your changes
This PR adds command line tools for Olive that can be called using
`olive` or `python -m olive` command.
- Previous commands `python -m olive.workflows.run` and `python -m
olive.platform_sdk.qualcomm.configure` are deprecated and will emit a
FutureWarning.
- All cli commands are implemented in the `olive.cli` sub-module. These
must handle the command parsing and, in some cases, the implementation
of the command too.
- Removed `olive.workflows.snpe` sub-module since the code is outdated
and obsolete.
- Some doc fixes where we need to use "``" instead of "`" for code
formatting.
- Documentation is automatic using the [`sphinx-argparse`
extension](https://sphinx-argparse.readthedocs.io/en/stable/index.html):
## Checklist before requesting a review
- [x] Add unit tests for this change.
- [x] Make sure all tests can pass.
- [x] Update documents if necessary.
- [x] Lint and apply fixes to your code by running `lintrunner -a`
- [x] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
- `olive` command line tool has been added.
- Use `olive run` or `python -m olive` instead of `python -m
olive.workflows.run` which has been deprecated.
- [ ] Is this PR including examples changes? If yes, please remember to
update [example
documentation](https://github.com/microsoft/Olive/blob/main/docs/source/examples.md)
in a follow-up PR.
## (Optional) Issue link
## Describe your changes
### [Performance]: Delayed Python module load
Introducing olive_config.json to encapsulates all the different global
environment properties (like pass module configuration) to setup olive.
Also, merged extra_dependencies.json into olive_config.json.
User can provide a alternative configuration file with the command line
argument '--package-config'.
Instead of loading all modules (specifically passes) at launch, delay
the load until after the run config is parsed and load only
the ones that are explicitly required.
Moved a few hard-coded dependency management into the olive config.
### Release Note:
Added support for extending available Olive passes and configuring each
via data file (olive_config.json). The configuration file is provided to
Olive run with command line param `--package-config`.
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [x] Update documents if necessary.
- [x] Lint and apply fixes to your code by running `lintrunner -a`
- [x] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
- [ ] Is this PR including examples changes? If yes, please remember to
update [example
documentation](https://github.com/microsoft/Olive/blob/main/docs/source/examples.md)
in a follow-up PR.
## (Optional) Issue link
## Describe your changes
- Introduce a new `set_tempdir` util that sets the `tempfile.tempdir` if
provided. This helps us remove duplicated code from the repository.
- Add missing package data for python environment system.
- Abstract the engine, workflow creation and run in the multi-ep test.
This logic is same for all systems so there is a lot of duplicate code.
- Clean up aml system unit test
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
1. Enable windows QNN e2e test
2. Added use conda path into windows ci test if it is for qnn. This is a
workaround for this kind of issue only ci pipeline:
https://community.anaconda.cloud/t/cant-install-failed-to-extract-packages-error/62687
3. Update pwsh Join-path to back-compat the version of pwsh <= 5.x,
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
QNN toolkit
CI e2e test for qnn mobile net:
https://github.com/microsoft/Olive/pull/874/checks?check_run_id=20488489220
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
This PR is used to restructure the SNPE code to extract the common
component which can be reusable in following QNN support.
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
---------
Co-authored-by: Mike Guo <myguo@microsoft.com>
## Describe your changes
There are a few issues for current SNPE examples:
1. The libxx.so under customized `python3.6` need to be put visible for
system for SNPE cmd running
- put `str(Path(f"{snpe_root}/python36-env/lib"))` under
`LD_LIBRARY_PATH` can fix.
2. SNPE evaluation extent the logits wrongly,
- fixed in this PR.
3. We did not support tensorflow model evaluation.
- update examples without input model evaluation.
4. Wrong metrics config for inception examples
5. Importing issue for vgg, tell that in README.md to let user update it
manually as a workaround.
6. Python3.6 seems a bit outdated for latest SNPE updates. The vgg
example failed to be run. Updated the python env to 3.8 can fix.
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
Reopen the auto optimization PR with pass flows templates.
Previous PR: https://github.com/microsoft/Olive/pull/610
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
---------
Co-authored-by: Mike Guo <guotuofeng@gmail.com>
## Describe your changes
1. Exclude the test in the final wheel
2. Add the missed dockerfile for docker system
3. Add both docs/examples in the source distribution
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Lint and apply fixes to your code by running `lintrunner -a`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [x] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Format your code by running `pre-commit run --all-files`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
Add missed license header
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Format your code by running `pre-commit run --all-files`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
more lint rules
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Format your code by running `pre-commit run --all-files`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
add lint
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Format your code by running `pre-commit run --all-files`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
Officially drop python 3.7 since it has already reached end of life.
Most olive dependencies have already dropped python 3.7 support so olive
was probably incompatible with it.
`onnxruntime` packages are distributed using different names but they
are all installed under the same `onnxruntime` namespace.
So, we cannot check if the required package is installed using the `try:
__import__` method. This PR uses `importlib.metadata` to check the local
ort installations and log the required steps needed. We don't want to
uninstall packages ourselves so we will just log the command for the
user to run.
Addresses the original issue in
https://github.com/microsoft/Olive/issues/557
Also update Whisper example to provide and use accelerator+ep correctly.
Added FAQ for common issues like
https://github.com/microsoft/Olive/issues/508
Update: Also fixes https://github.com/microsoft/Olive/issues/571
## Checklist before requesting a review
- [ ] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [ ] Update documents if necessary.
- [ ] Format your code by running `pre-commit run --all-files`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
Add Aml sku support for AMLSystem
## Checklist before requesting a review
- [x] Add unit tests for this change.
- [x] Make sure all tests can pass.
- [x] Update documents if necessary.
- [x] Format your code by running `pre-commit run --all-files`
- [ ] Is this a user-facing change? If yes, give a description of this
change to be included in the release notes.
## (Optional) Issue link
## Describe your changes
explicitly close the opened files to prevent file handle leaks
## Checklist before requesting a review
- [x] Add unit tests for this change.
- [ ] Make sure all tests can pass.
- [x] Update documents if necessary.
- [x] Format your code by running `pre-commit run --all-files`
## (Optional) Issue link
This change adds an example of optimizing the SqueezeNet model (a simple
"hello world" style network) for latency with DirectML. It's nice to
have a basic sample that runs quickly and doesn't involve huge models
with external weights.
Other changes:
- Introduces an ONNX pass for basic float-to-float16 conversion outside
of standard model conversion or transformer-specific optimizer pass.
Sometimes it matters where/when this conversion occurs, but for simple
GPU inference on non-transformer models it's nice to have as a
standalone pass.
- Updates OrtPerfTuning pass to filter EPs based on the configured
device. As-is, the pass will create tuning combos for _all_ EPs in the
user's onnxruntime package.
- Adds DML references in docs.
devang-ml
trajep
Xiaoyu
Jambay Kinley
shaahji
Mike Guo
leqiao-1
Justin Stoecker
Justin Stoecker
leqiao