bfb05cefe8
## Describe your changes - The `pair` dataset type was originally added to replicate the original qlora implementation's dataprocessing options. However, it is not used and is has limited use case. Drop support for this dataset type and only keep the common `corpus` type. - Since pair is not a concept anymore, `source_max_len` is renamed to `max_seq_len` to align the parameter name to how it's commonly known as (such as in hf trl, etc). ## Checklist before requesting a review - [ ] Add unit tests for this change. - [ ] Make sure all tests can pass. - [ ] Update documents if necessary. - [ ] Lint and apply fixes to your code by running `lintrunner -a` - [x] Is this a user-facing change? If yes, give a description of this change to be included in the release notes. `pair` dataset type has been removed. `source_max_len` renamed to `max_seq_len`. - [ ] Is this PR including examples changes? If yes, please remember to update [example documentation](https://github.com/microsoft/Olive/blob/main/docs/source/examples.md) in a follow-up PR. ## (Optional) Issue link |
||
|---|---|---|
| .. | ||
| README.md | ||
| config.json | ||
| requirements.txt | ||
README.md
Falcon Optimization
This folder contains a sample use case of Olive to optimize a falcon-7b model using ONNXRuntime tools.
Optimization Workflows
This workflow performs Falcon optimization on CPU with ONNX Runtime. It performs the optimization pipeline:
- PyTorch Model -> Onnx Model -> Transformers Optimized Onnx Model fp16
Config file: config.json
How to run
Pip requirements
Install the necessary python packages:
python -m pip install -r requirements.txt
Run sample using config
The optimization techniques to run are specified in the relevant config json file.
First, install required packages according to passes.
olive run --config config.json --setup
Then, optimize the model
olive run --config config.json
or run simply with python code:
from olive.workflows import run as olive_run
olive_run("config.json")
After running the above command, the model candidates and corresponding config will be saved in the output directory. You can then select the best model and config from the candidates and run the model with the selected config.