* Move all generic command line launching functionality into a separate module.
* Update the README file to use the new script and the parameters.
* Change the config parameters propagation policy: now values from the parent config override the parameters at the lower levels. This way we can keep the default values in the main config and push the actual values from the globals. E.g., see the `"accessToken"` and `"subscription"` in this diff
Related work items: #501
This PR implements more VM control methods in `azure_services.py` like:
- Stop
- Deprovision
- Reboot
These methods are also updated to be used in the teardowns for `azure_os.py` and `azure_vm.py`.
`main.py` is updated to now teardown the composite environment at the end of the experiment, and we can observe the VM shutting down and being de-provisioned in Azure Portal.
Related work items: #469
This PR includes:
1) Docker folder that runs redis benchmark
2) Test script that 1) builds benchmark docker container, 2) creates redis docker container, 3) links redis and benchmark containers, and 4) runs benchmark docker container with given parameters
Related work items: #471
- include all python files in pylint checks
- improve version handling for pylint and update scripts
- make autopep8 available for vscode to use as a python formatter
- add more cspell words
- address some pylint warnings
- more pylint fixups
make the names of Azure configuration parameters consistent (camel
case) and update the README accordingly. Remove the "*-example.json"
config files - we can use the actual configs as examples now as they
don't contain any sensitive information anymore
Related work items: #497
This PR updates the README.md of `mlos_bench` with the steps needed to run a user's own JSON configuration with `main.py`.
Additionally, a small fix is included in the template `config-example.json` for specifying the corresponding `vmName` when establishing the VM, otherwise we end up always using the default value of `osat-linux-vm` provided in the ARM template.
Related work items: #485
Now we can use the config files as-is and pass the missing parameters via the command line, e.g.
```powershell
python.exe .\mlos_bench\mlos_bench\main.py `
--config .\mlos_bench\config\config.json `
--accessToken "$(az account get-access-token --query accessToken --output tsv)" `
--subscription "$(az account get-access-token --query subscription --output tsv)" `
2>&1 > .\osat.log
```
Related work items: #492
* Store tunables' definitions in a separate config
* Implement the `Tunable` wrapper class
* Implement tunable groups and collections of covariant parameters
Related work items: #457, #458
This is a follow up to !557 (Make conda work on Windows) and addresses some of the lingering comments from that PR.
- document the mamba slow solver workaround
- use the mamba solver if available in Makefile
- add windows jobs to the CI tasks as well
We will do our OS autotuning related work on mlos-bench there.
Main prior to this has been tagged as v0.0.3 for servicing Spark autotune needs if need be.
Plan is to reorganize the source tree after this so we can generate separate `mlos-core` and `mlos-bench` packages.
- ignore some more auto generated files
- don't need regular users to have to build the docs
- tweaks to the docs
- install some modules necessary for vscode extensions to lint check rst
- Add conda env caching for build pipeline speed ups.
- Cache will use items from the last hour (within the same day), but only if the `environment.yml` and `setup.yml` file haven't changed.
- Adds testing for multiple Python versions.
- Mostly as a proof of concept. Right now just 3.8 and 3.9. 3.6 (the current default in Synapse) is broken and disabled.
- We can disable this and just use the `DefaultPython` version again for now as well.
- Also enabled parallel builds as a part of this change.
Related work items: #322
ConfigSpace is used to instantiate the parameter search space for all of our optimizer backends.
Without this I don't think ConfigSpace gets pulled in automatically when installing a whl in Synapse, for instance, and so doesn't work.
Adds basic `register` and `suggest` APIs for a couple of optimizer backends: `Skopt`, `Emukit`, `Random`.
Includes example notebook for starters.
Limited unit testing. Basic function documentation.
Azure DevOps CI pipelines for `pylint` and `pytest` via the `mlos_core` `conda` `environment.yml`.
Related work items: #274, #279
Sergiy Matusevych
Brian Kroth
Eu Jing Chua
Omisa Jinsi
Andreas Mueller
Brian Kroth