e5f879df31
* setup v0.1 * rm \n in comments * QuickStart added for docs.scenarios * experiment result of citi bike update to 1-month version * V0.1 issues installation (#38) * replace pycurl to urllib * rm pycurl in test requirements * fix installation issue, update readme, speed up citibike Co-authored-by: Arthur Jiang <sjian@microsoft.com> * notebook link added for scenarios * fix: add missing licenses, refine long lines (#36) * scenario notebook update (#39) Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> * scenario doc modified according to comments * doc correction * scenario doc modified according to comments * remove examples * rl toolkit refactoring (#42) * rl toolkit refactoring * rename citi_bike * add new line Co-authored-by: Arthur Jiang <sjian@microsoft.com> * fix issue during building playground images (#44) * citibike doc modified according to comments * citibike doc update * scenario experiment result table changed * annotate the empty experiment results * merge master into v0.1 for later merging (#47) * setup v0.1 (#34) Co-authored-by: Arthur Jiang <sjian@microsoft.com> * fix pycurl install issue (#35) Co-authored-by: Arthur Jiang <ArthurSJiang@gmail.com> Co-authored-by: Arthur Jiang <sjian@microsoft.com> * refine readme * fix: recursive copy instead of plain copy (#45) * update maro overall image * typo * add key components * fix deployment issue in multi envs (#49) * typo * refine introduction * fix typo * fix readme * refine readme * refine readme * update readme docs link * fix ~/.maro not exist issue in build (#52) * fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * refine logo * fix toctree * fix & feat: invalid filename & maro grass status (#51) * fix: change invalid filename to valid * feat: add maro grass status * refine * refine * V0.1 test store update (#58) * updateed test_store.py * fixed unpicklable store bug Co-authored-by: ysqyang <v-yangqi@microsoft.com> * rename ecr 2 cim (#62) * correct dependencies * rename ecr to cim for hello, scenario, datalib and tests * remove unused dependency for build * add a new line at the end * a'd * V0.1 rename to cim in example (#63) * renamed ecr to cim and removed unwanted files * renamed ecr to cim in other places * fixed PR conflicts * merged with v0.1 Co-authored-by: ysqyang <v-yangqi@microsoft.com> * fix ut issue under window, now all runpass (#64) * correct dependencies * rename ecr to cim for hello, scenario, datalib and tests * install torch first before test dependencies * missing ecr 2 cim * V0.1 ecr rename (#65) * rename ecr to cim in docs * ecr renamed for notebooks * OOCL removed * rename ecr to cim * title level modified * notebook update Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> * temporarily disable weather, fix station id issue (#66) * fix deployment issue in multi envs * fix typo * fix ~/.maro not exist issue in build * skip deploy when build * update for comments * temporarily disable weather info * replace ecr with cim in setup.py * replace ecr in manifest * remove weather check when read data * fix station id issue * fix format * add TODO in comments * V0.1 ecr rename (#67) * rename ecr to cim in docs * ecr renamed for notebooks * OOCL removed * rename ecr to cim * title level modified * notebook update * topo renamed Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> * change the keys in metrics (#68) * correct dependencies * rename ecr to cim for hello, scenario, datalib and tests * install torch first before test dependencies * missing ecr 2 cim * rename metrics keys * remove transfer cost factor * rename cost to number * update setup * Update build_wheel.yml * Update build_wheel.yml * Update build_wheel.yml * Update build_wheel.yml * rm install from PyPi * V0.1 cim nb (#70) * added cim nb * fixed bugs in nb * fixed bugs in nb * fixed a bug in nb Co-authored-by: ysqyang <v-yangqi@microsoft.com> * the font size of doc diagrams is enlarged (#73) * the font size of doc diagrams is enlarged * disable APIDOC Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> * str() and repr() of DecisionEvent and Action added; notebook output updated (#74) Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> * Update deploy_gh_pages.yml * fix docker build * V0.1 feature docker ci (#77) * Update deploy_docker_image.yml * Update deploy_docker_image.yml * Update deploy_docker_image.yml * Update deploy_docker_image.yml * refine * add contributing Co-authored-by: Arthur Jiang <sjian@microsoft.com> Co-authored-by: Arthur Jiang <sjian@microsoft.com> Co-authored-by: Jinyu Wang <Wang.Jinyu@microsoft.com> Co-authored-by: Romic Huang <romic.hlc@outlook.com> Co-authored-by: Jinyu-W <53509467+Jinyu-W@users.noreply.github.com> Co-authored-by: kaiqli <59279714+kaiqli@users.noreply.github.com> Co-authored-by: Chaos Yu <chaos.you@gmail.com> Co-authored-by: Romic Huang <romic.kid@gmail.com> Co-authored-by: zhanyu wang <pocket_2001@163.com> Co-authored-by: ysqyang <ysqyang@gmail.com> Co-authored-by: ysqyang <v-yangqi@microsoft.com> |
||
---|---|---|
.github/workflows | ||
docker_files | ||
docs | ||
examples | ||
maro | ||
notebooks | ||
scripts | ||
tests | ||
.dockerignore | ||
.gitignore | ||
CODE_OF_CONDUCT.md | ||
CONTRIBUTING.md | ||
LICENSE | ||
MANIFEST.in | ||
README.md | ||
SECURITY.md | ||
playground.md | ||
pyproject.toml | ||
requirements.dev.txt | ||
setup.py |
README.md
Multi-Agent Resource Optimization (MARO) platform is an instance of Reinforcement learning as a Service (RaaS) for real-world resource optimization. It can be applied to many important industrial domains, such as container inventory management in logistics, bike repositioning in transportation, virtual machine provisioning in data centers, and asset management in finance. Besides Reinforcement Learning (RL), it also supports other planning/decision mechanisms, such as Operations Research.
Key Components of MARO:
- Simulation toolkit: it provides some predefined scenarios, and the reusable wheels for building new scenarios.
- RL toolkit: it provides a full-stack abstraction for RL, such as agent manager, agent, RL algorithms, learner, actor, and various shapers.
- Distributed toolkit: it provides distributed communication components, interface of user-defined functions for message auto-handling, cluster provision, and job orchestration.
Contents
File/folder | Description |
---|---|
maro |
MARO source code. |
docs |
MARO docs, it is host on readthedocs. |
examples |
Showcase of MARO. |
notebooks |
MARO quick-start notebooks. |
Prerequisites
Install MARO from Source (editable mode)
-
Prerequisites
- C++ Compiler
- Linux or Mac OS X:
gcc
- Windows: Build Tools for Visual Studio 2017
- Linux or Mac OS X:
- C++ Compiler
-
Enable Virtual Environment
-
Mac OS / Linux
# If your environment is not clean, create a virtual environment firstly. python -m venv maro_venv source ./maro_venv/bin/activate
-
Windows
# If your environment is not clean, create a virtual environment firstly. python -m venv maro_venv .\maro_venv\Scripts\activate # You may need this for SecurityError in PowerShell. Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Unrestricted
-
-
Install MARO
-
Mac OS / Linux
# Install MARO from source. bash scripts/install_maro.sh
-
Windows
# Install MARO from source. .\scripts\install_maro.bat
-
Quick Example
from maro.simulator import Env
env = Env(scenario="cim", topology="toy.5p_ssddd_l0.0", start_tick=0, durations=100)
metrics, decision_event, is_done = env.step(None)
while not is_done:
metrics, decision_event, is_done = env.step(None)
print(f"environment metrics: {env.metrics}")
Run Playground
# Build playground image
docker build -f ./docker_files/cpu.play.df . -t maro/playground:cpu
# Run playground container
# Redis commander (GUI for redis) -> http://127.0.0.1:40009
# Local host docs -> http://127.0.0.1:40010
# Jupyter lab with maro -> http://127.0.0.1:40011
docker run -p 40009:40009 -p 40010:40010 -p 40011:40011 maro/playground:cpu
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
License
Copyright (c) Microsoft Corporation. All rights reserved.
Licensed under the MIT License.