1e42f0b2b2
## Describe your changes Added two new passes that split a transformers model into multiple models. The split is done on the transformers layers. - `CaptureSplitInfo`: Used the pytorch/hf model to assign split ids to the transformers layers and saves them as model attributes. The conversion pass or model builder pass adds the split assignments and model metadata. - `SplitModel`: Uses the split_assignments metadata to break the model into splits. - If `include_all_nodes` is `True`: Nodes before first split or outside the splits are assigned to the first split. Nodes after the last split are assigned to the last split. - If `include_all_nodes` is `False`: Such nodes are ignored. This means the embedding, attention mask computation, final norm and lm heads don't appear in the splits. We expect the user to extract them separately. `Cache.save_model` now supports saving composite models. ## Checklist before requesting a review - [x] Add unit tests for this change. - [ ] Make sure all tests can pass. - [x] Update documents if necessary. - [x] Lint and apply fixes to your code by running `lintrunner -a` - [ ] Is this a user-facing change? If yes, give a description of this change to be included in the release notes. - [ ] Is this PR including examples changes? If yes, please remember to update [example documentation](https://github.com/microsoft/Olive/blob/main/docs/source/examples.md) in a follow-up PR. ## (Optional) Issue link |
||
|---|---|---|
| .. | ||
| source | ||
| Makefile | ||
| README.md | ||
| architecture.md | ||
| make.bat | ||
| requirements.txt | ||
README.md
Generating the documentation
To generate the documentation, you first have to build it.
Pre-requisites
Install Olive. At the root of the code repository:
pip install -e .
Install pip requirements. At docs:
pip install -r requirements.txt
Building the documentation
At docs:
make html
make linkcheck
Previewing the documentation
At docs/build/html:
python -m http.server {port-number}
The documentation site will be running at http://localhost:<port-number>