Граф коммитов

1223 Коммитов

Автор SHA1 Сообщение Дата
Jash Shah d782106860
Custom Heirarchical Time series forecasting with Azure Automl (#233)
* Custom Heirarchical Time series forecasting with Azure Automl

* Custom hts refactored

* Custom_Hierarchical_Time_Series_Forecasting_Solution

* Update Readme.md

* AzureML_Custom_Hierarchical_Time_Series_Forecasting 📈

* Heirarchical time series forecasting refactored

* Heirarchical time series forecasting refactored

* Heirarchical time series forecasting refactored
2023-04-30 17:54:37 -04:00
dependabot[bot] c4ba9dbdb3
Bump numpy from 1.16.2 to 1.22.0 in /fclib (#238)
Bumps [numpy](https://github.com/numpy/numpy) from 1.16.2 to 1.22.0.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/HOWTO_RELEASE.rst)
- [Commits](https://github.com/numpy/numpy/compare/v1.16.2...v1.22.0)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-30 17:53:42 -04:00
dependabot[bot] 7e00368f59
Bump tensorflow from 2.0.1 to 2.11.1 in /fclib (#240)
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.0.1 to 2.11.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.0.1...v2.11.1)

---
updated-dependencies:
- dependency-name: tensorflow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-30 17:53:02 -04:00
Daniel Ciborowski 86b421b718
Update autoarima_multi_round.ipynb (#226) 2020-10-20 03:54:53 +11:00
vapaunic 12e2044946
July'20 release (#216)
* pin pmdarima, matplotlib versions (#212)

* pin pmdarima to 1.1.1

* try 1.2.0

* fix envsetup, revert to ==1.1.1

* also update test yml

* fix this script as well

* pin matplotlib

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Testing and refactoring fclib 1 (#214)

* start utils testing

* testing ojdata

* clean

* rm unneeded file

* use conftest.py

* loss fn tests

* tweak R datagen

* feature testing

* feature testing 2

* test dcnn

* fine tune

* modelling tests

* rm unneeded imports

* more pred tests

* set to run code coverage

* add coveragerc

* install pytest-cov

* fixing, removing unneeded cruft

* rm unused energy lag stuff

* more fixes

* yet more fixes

* more tests

* split feature tests up

* use codecov task

* install .net core for cobertura

* don't output html

* no reportDirectory

* tidying up

* linting

Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>
2020-07-16 11:20:25 -07:00
dependabot[bot] bc47741cb1
Bump tensorflow from 2.0.0 to 2.0.1 in /fclib (#210)
* Bump tensorflow from 2.0.0 to 2.0.1 in /fclib

Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.0.0 to 2.0.1.
- [Release notes](https://github.com/tensorflow/tensorflow/releases)
- [Changelog](https://github.com/tensorflow/tensorflow/blob/master/RELEASE.md)
- [Commits](https://github.com/tensorflow/tensorflow/compare/v2.0.0...v2.0.1)

Signed-off-by: dependabot[bot] <support@github.com>

* bump pandas version

* bump pandas to 1.0.5

* Revert "bump pandas to 1.0.5"

This reverts commit 2f7bdaf954.

* Revert "bump pandas version"

This reverts commit f0978965ae.

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>
2020-06-26 13:19:32 -04:00
vapaunic 32d104c5e0
June release (#208)
* Cleanup notice (#206)

* add notice

* reformat

* Version 0.0.2 and ado artifact pipeline (#207)

* First Release of Forecasting Repo (#181)

* Handled edge case where ts_id_col_names is None

* Split long line into separate lines

* Added notebook template

* Added a test yml file

* Added yml file for python unit test pipeline

* Minor update

* Minor update

* Minor update

* Minor update

* Removed triggers

* Removed triggers

* Created a base ts estimator and inherit BaseTSFeaturizer from the BaseTSEstimator.

* Refactored featurizer class hierachy.

* Added week of month method.

* add script to source entire

* formatting

* source only test files

* Inherit temporal featurizers from BaseTSFeaturizer.

* Minor update.

* Replaced max_test_timestamp with max_horizon

* Refactored rolling window featurizers.

* Renamed hour_of_year feature to normalized_hour_of_year

* Inherit all normalizers from base normalizer class.

* address review comments for the PR of contributing

* minor update

* address review comments for PR of r test pipeline

* add a test yml file

* Remove checking target column existence, because testing data may not have the target column.

* Create setter and getter of ts_id_col_names.

* Fixed bug caused by unexpected behavior of pandas.shift

* Some code cleanup.

* Updated some featurizer names.

* Some minor changes in df_config and feature configs.

* Some minor changes in feature names.

* Added usage examples in docstring.

* Computation time update after feature engineering refactoring.

* Removed setting frequency.

* Added docstring to convert_to_tsdf function.

* Removed frequency in convert_to_tsdf call.

* Fixed week_of_month function.

* Added popularity featurizer

* Added utility function for checking Iterable but not string.

* Updated LightGBM feature engineering code to use new feature engineering classes.

* Improved checking whether input column names are Iterable and conver to list.

* Made future_value_available a read-only property.

* Minor docstring update.

* Removed extra space in docstring examples.

* Made some methods staticmethods.

* Minor QRF result update after feature engineering code change.

* Removed calling of validate_file and added catching of the exception

* Update python_unit_tests_base.yml for Azure Pipelines [skip ci]

Updated path of the test results

* Test if the download link is wrong

* Fixed minor format issues.

* Fixed minor format issues.

* Fixed formatting issues.

* Fixed line length.

* Removed data files before downloading and checked dimensions of energy data

* Removed the change made for testing

* Changed folder structure of tests and added table to show build status

* Added missing files

* Updated based on review comments

* new folder structure

* add repo metrics

* remove prototypes folder

* add models placeholder

* adjust featurizers to the new structure of folders

* changes in README and evaluation files

* adjust data download to new folders

* delete unnecessary files

* energy load baseline model with new folders

* delete data files

* fix links in benchmarks file

* fix bug

* adjust GBM, QRF and FNN submissions to the new folder structure

* Replace pd.to_timedelta with pd.offsets.

* Added get_offset_by_frequency helper function.

* fix small bugs

* fix small bugs

* Update TSCVSplitter.

* refactored high-level folders

* added a placeholder folder for PR/issue templates

* added subfolders under notebooks/

* updated tests folder

* renamed notebooks/ to examples/

* Update to CONTRIBUTING instructions (#34)

* style checking and formatting files

* git hook installation guide

* issue and PR templates

* minor change

*  working with github instructions

* added specific issue templates

* addressed PR comments

* addressed Chenhui's comment

* addressing chenhuis comments

* conda environment file (#36)

* conda environment file

* updated environment file

* updated instructions for installing conda env

* Vapaunic/lib (#37)

* initial core for forecasting library

* syncing with new structure

* __init__ files in modules

* renamed lib directory

* Added legal headers and some formatting of py files

* restructured benchmarking directory in lib

* fixed imports, warnings, legal headers

* more import fixes and legal headers

* updated instructions with package installation

* barebones library README

* moved energy benchmark to contrib

* formatting changes plus more legal headers

* Added license to the setup

* moved .swp file to contrib, not sure we need to keep it at all

* added missing headers and a brief snipet to README file

* minor wording change in readme

* Chenhui/cpu unit test pipeline (#38)

* address review comments

* added full conda path

* minor change

* added conda to PATH

* added build status in README

* removed energy data prep placeholder notebook

* moved out data energy explore notebook into contrib

* moved data download script to tools/

* Added getting started section to readme

* Added rbase and rbayesm to conda environment

* modified data download script

* added instructions for data download

* renamed data download script

* fixing issues with test pipeline

* parsing issue in yml file

* cleaning up ci test yaml file for more diagnostic info

* fixed a missing argument in instructions

* removed retail directory under dataset module

* moved feature_engineering.py to the feature engineering module

* moved evaluate.py to evaluation module

* combined benchmark settings into a single file

* moved download sript to the package and modified the tests

* modified instructions

* fixed the build pipeline yml

* fix to the pipeline yml

* fix to the pipeline yml

* moved serve_folds into ojdata.py

* removed data_schema.py file as all content moved to ojdata.py

* fixed split_train_test in ojdata.py

* moved retail_data_schema into ojdata.py

* moved all oj utilities to ojdata.py

* removed paths from benchmark_settings

* fixed up a docstring

* quick fix a typo

* removed benchmark_settings

* parameterized experiment settings

* refactored experiment settings

* Fixed docstrings

* addressed chenhuis comment around round file naming

* renamed experiment to forecast settings

* Chenhui/light gbm quick start (#40)

* initial example notebook for lightgbm

* reduced to one round forecast

* added text

* added text

* added text

* moved week_of_month to feature engineering utils

* moved df_from_cartesian_product to feature utils

* moved functions to feature utils

* moved functions to feature utils

* added lightgbm model utils

* updated plots

* added text and renamed predict function

* reduced print out frequency in model training

* moved data visualization code to utils

* added text

* updated plot function and added docstring

* renamed the notebook

* updated text

* added NOTICE file, currently empty as we're not redistributing any packages

* Chenhui/add scrapbook (#43)

* added scrapbook support

* Added gitpython to environtment.yml file

* added git_repo_path function to utils

* updated notebook

* added test for lightgbm notebook

* included testing of notebooks

* resolve test error

* resolve test error

* added kernel name

* updated kernel name

* trying installing bayesm from cmd

* trying installing bayesm from cmd

* trying installing bayesm from cmd

* excluded notebook test

* excluded notebook test

* added lapack.so link fix

* included notebook tests

* excluded files for notebook test

Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>

* added integration test

* added initial data prep notebook

* updated notebook

* updated notebook

* updated notebook

* updated url

* init

* model parameters

* removed blank quick start notebooks

* removed blank modeling notebooks

* removed blank evaluation notebooks

* Removed blank model selection notebooks

* removed blank o16n notebooks

* removed outdated text from contrib/README

* removed outdated swp file

* updating .gitignore

* removed change log, as we don't plan to maintain this

* Excluding irrelevant directories

* fix settings

* separated out the setup guide

* fix settings

* simplemodel init

* typo

* add rproj file

* Renaming forecasting_lib to fclib (#59)

* renamed forecasting_lib directory

* modified references to forecasting_lib

* Vapaunic/envname (#61)

* renamed conda env

* modified setup instructions

* minor change in contributing guide

* keep top-level gitignore only

* formatting fixes

* Chenhui/add automl example (#62)

* added multiple linear models and example notebook for AutoML

* removed commented code

* address review comments

* minor update to the notebook

* minor update to the notebook

* added text

* changed types in lightgbm to be consistent with the rest of the code

* modified docstrings in multiple_linear_regression.py

* updated ci yaml files

* changed import statement in confest.py

* updated gitpython version to the latest

Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>

* Vapaunic/split bug (#65)

* fixed a yield bug

* removed two blank files

* modified split data function to auto-calculate the splits based on the parameters

* removed forecast_settings module

* removed unused parameter

* modified splitting function to use non-overlapping testing

* tested the split function after the update

* minor fix

* defaults changed in split function

* modified lightgbm example with new split function

* modified automl example (needs verification)

* modified data explore notebook

* quick fix:

* updated data preparation notebook

* changed defaults in split function

* Addressed changes in lightgbm

* addressed issues in automl notebook

* fixed typo in lightgbm plot

* first images of time series split

* updated the pictures

* updated evaluation periods (#66)

* Chenhui/env setup script (#67)

* added a shell script for setting up environment

* changed yaml to yml

* added comments and updated SETUP.md

* modified data preparation notebook with images

* moved r exploration notebook to contrib directory

* modified data explore notebook, updated info about the data, and removed reference to TSPerf

* addressed review feedback and fixed the explore notebook

* Chenhui/multiround lightgbm (#68)

* added initial multiround notebook for lightgbm

* updated data splitting

* updated text

* updated week list

* addressed review comments

* added pyramid-automl to conda file

* first draft of arima notebook

* replace pyramid with pmdarima

* Added a complete function

* minor type

* forecasting across many stores/brands

* complete arima notebook

* renamed data preparation/exploration notebooks

* added git clone to setup

* addressed PR comments

* typo

* Arima to ARIMA

* fixed docstring in plot function

* fixed a bug in MAPE calculation and added plotting

* fixed a bug in predict

* modeling arima on log scale

* Fixing AML Example Notebook (#84)

* Cleaning notebook output, adding get_or_create workspace call, and fixing get_or_create AmlCompute

* Add regression-based models (#64)

* modelling updates

* code tweak

* rebuild

* update mape

* update mape 2

* new forecasting structure

* update eval

* rebuild dataprep

* rebuild with profit

* rm profit

* add plot

* typo

* tidy up

* expand readme

* oops

* clarified setup guide (#94)

* Update SETUP (#95)

minor fix

* Cleaned up unused files and directories (#96)

* removed non-used files

* moved docs into a docs/ dir

* fixed broken links

* Chenhui/dilated cnn example and utils (#76)

* added initial model util file for DCNN

* initial notebook

* added feature utils for DCNN

* upadted evaluation and visualization

* removed plot function

* replaced PRED_HORIZON, PRED_STEPS by HORIZON, GAP

* removed log dir if it exists

* updated model utils

* generalized categorical features in dcnn model util

* generalized network definition

* update training code

* format with blackcellmagic

* address review comments and added README

* Chenhui/add ci tests (#146)

* Update conda env with versions (#99)

* 💥

* revert

* minor changes

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Adding missing Jupyter Extension (#90)

* Update environment.yml

* specified version

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* fix links to examples/ (#104)

* Chenhui/rename notebooks and update automl notebook (#106)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* Arima multi-round notebook (#91)

* working arima model

* final auto arima example

* added tqdm to requirements

* addressed review comments

* Revert "Chenhui/rename notebooks and update automl notebook (#106)" (#107)

This reverts commit 032c91d9bfa389f22ae1f1f2150913a4f063bd18 [formerly 15d25213dc].

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Fixing data download issue (#109)

* removed dependency on __file__ from data download, doesn't work in jupyter

* changed aux to auxdata

* fixe data download function

* fixed path

* auxdata -> auxi

* adding tl;dr directions for setup to README.md (#88)

* adding tl;dr directions for setup to README.md

* added a bit more text

* Cleaned up obsolete (tsperf) code in fclib (#112)

* moved out tsperf files from evaluation module

* moved out tsperf tuning code

* removed more unused files

* Addressing documentation related issues (#111)

* Added conda activate to the setup readme

* added instructions for starting jupyter to setup

* minor

* deleted duplicate instructions

* addressed PR comments

* Chenhui/rename notebooks and updated AutoML example (#108)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* updated pytest file

* address review comments

* reran notebook with blackcellmagic

* adding pylint  (#93)

* adding tl;dr directions for setup to README.md

* removing pylint hook and pylint_junit from the env file

* removed pylint config file

* Chenhui/update example folder (#115)

* restructure examples folder

* updated readme

* added readme

* minor update

* removed R folder

* minor change

* fixed a broken link

* another broken link

* fixing notebook tests

* Chenhui/fix aux file path (#118)

* fixed figure links

* changed to auxi_i.csv

* minor change

* [MINOR] Small changes to Arima notebooks (#121)

* fixed a broken link

* minor text changes

* Documentation (#120)

* added target audience section

* added intro on forecasting

* Added fclib documentation

* improved examples readme

* address comments

* added info about the dataset

* added items to be ignored (#123)

* added items to be ignored

* added *.log and score.py

* Chenhui/toplevel readme (#127)

* added content table

* added references

* added external repo links

* minor update

* Chenhui/tune deploy lgbm (#122)

* added notebook and utils

* updated readme links

* fix data path

* updated text

* group imports

* minor update

* using azureml utils to create workspace and compute (#126)

* using azureml utils to create workspace and compute

* group imports

* Download ojdata directly from github (#128)

* new function to download and load oj data directly from bayesm repo

* removed bayesm

* new R function to only load the data

* removed download R function

* minor fix

* added documentation to load_oj_data.R

* added requests to requirements

* fixed a syntax error (#130)

* fix setup.md link (#129)

* fix setup.md link

* mention related use cases

* Vapaunic/cgbuild (#133)

* added files to generate reqs.txt and the ci yml file

* Added notice generation task

* Checking if notice is there

* Update component_governance.yml for Azure Pipelines

* check in notice file

* Update component_governance.yml for Azure Pipelines

* fixed heading

* Chenhui/windows setup (#131)

* initial test

* added batch script and instructions

* align image to center

* adjust image size

* added text

* adjust image size

* address comments

* Readds R material (#116)

* redo R stuff in new dirs

* dirname fixup

* add Rproj file

* rebuild

* fixups

* roxygenise

* copyright notice

* dataprep

* updated yaml

* more updates

* more tweaks

* reg models

* update reg models

* more updates

* reword

* rendered prophet html

* name fix

* add lintr file

* move stuff

* renamed use case folder (#138)

* renamed use case folder

* dirname change

* updated readme

* added notebooks

* fix ci test

* Vapaunic/featutils (#137)

* moved feature engineering module to contrib

* removed lag submod

* cleaned up feature engineering

* rebuild R notebooks (#139)

* Chenhui/toplevel readme (#140)

* added content table

* added references

* added external repo links

* minor update

* updated setup instructions

* added text

* align text

* removed duplicated Content section

* address review comments

* Chenhui/hyperdrive example update (#142)

* removed blackcellmagic

* removed utils under aml_scripts and updated notebook

* added notebook path

* added ci test of lightgbm multi round example

* make forecast round as parameter

* Make -Agent Name

* resolve duplicated function name

* increased time limit and reduce number of rounds

* increase time limit

* added parameters tag to multiround lightgbm and dilatedcnn

* README change (#147)

* minor change

* hide tags

* hide tags

* added parameters tag

* Revert "Chenhui/add ci tests (#146)" (#149)

This reverts commit de7a19cfa7637476b9ebfc92f5c18a26a8eca4da [formerly f8bd22733c].

* Chenhui/add ci tests (#150)

* Update conda env with versions (#99)

* 💥

* revert

* minor changes

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Adding missing Jupyter Extension (#90)

* Update environment.yml

* specified version

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* fix links to examples/ (#104)

* Chenhui/rename notebooks and update automl notebook (#106)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* Arima multi-round notebook (#91)

* working arima model

* final auto arima example

* added tqdm to requirements

* addressed review comments

* Revert "Chenhui/rename notebooks and update automl notebook (#106)" (#107)

This reverts commit 032c91d9bfa389f22ae1f1f2150913a4f063bd18 [formerly 15d25213dc].

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Fixing data download issue (#109)

* removed dependency on __file__ from data download, doesn't work in jupyter

* changed aux to auxdata

* fixe data download function

* fixed path

* auxdata -> auxi

* adding tl;dr directions for setup to README.md (#88)

* adding tl;dr directions for setup to README.md

* added a bit more text

* Cleaned up obsolete (tsperf) code in fclib (#112)

* moved out tsperf files from evaluation module

* moved out tsperf tuning code

* removed more unused files

* Addressing documentation related issues (#111)

* Added conda activate to the setup readme

* added instructions for starting jupyter to setup

* minor

* deleted duplicate instructions

* addressed PR comments

* Chenhui/rename notebooks and updated AutoML example (#108)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* updated pytest file

* address review comments

* reran notebook with blackcellmagic

* adding pylint  (#93)

* adding tl;dr directions for setup to README.md

* removing pylint hook and pylint_junit from the env file

* removed pylint config file

* Chenhui/update example folder (#115)

* restructure examples folder

* updated readme

* added readme

* minor update

* removed R folder

* minor change

* fixed a broken link

* another broken link

* fixing notebook tests

* Chenhui/fix aux file path (#118)

* fixed figure links

* changed to auxi_i.csv

* minor change

* [MINOR] Small changes to Arima notebooks (#121)

* fixed a broken link

* minor text changes

* Documentation (#120)

* added target audience section

* added intro on forecasting

* Added fclib documentation

* improved examples readme

* address comments

* added info about the dataset

* added items to be ignored (#123)

* added items to be ignored

* added *.log and score.py

* Chenhui/toplevel readme (#127)

* added content table

* added references

* added external repo links

* minor update

* Chenhui/tune deploy lgbm (#122)

* added notebook and utils

* updated readme links

* fix data path

* updated text

* group imports

* minor update

* using azureml utils to create workspace and compute (#126)

* using azureml utils to create workspace and compute

* group imports

* Download ojdata directly from github (#128)

* new function to download and load oj data directly from bayesm repo

* removed bayesm

* new R function to only load the data

* removed download R function

* minor fix

* added documentation to load_oj_data.R

* added requests to requirements

* fixed a syntax error (#130)

* fix setup.md link (#129)

* fix setup.md link

* mention related use cases

* Vapaunic/cgbuild (#133)

* added files to generate reqs.txt and the ci yml file

* Added notice generation task

* Checking if notice is there

* Update component_governance.yml for Azure Pipelines

* check in notice file

* Update component_governance.yml for Azure Pipelines

* fixed heading

* Chenhui/windows setup (#131)

* initial test

* added batch script and instructions

* align image to center

* adjust image size

* added text

* adjust image size

* address comments

* Readds R material (#116)

* redo R stuff in new dirs

* dirname fixup

* add Rproj file

* rebuild

* fixups

* roxygenise

* copyright notice

* dataprep

* updated yaml

* more updates

* more tweaks

* reg models

* update reg models

* more updates

* reword

* rendered prophet html

* name fix

* add lintr file

* move stuff

* renamed use case folder (#138)

* renamed use case folder

* dirname change

* updated readme

* added notebooks

* fix ci test

* Vapaunic/featutils (#137)

* moved feature engineering module to contrib

* removed lag submod

* cleaned up feature engineering

* rebuild R notebooks (#139)

* Chenhui/toplevel readme (#140)

* added content table

* added references

* added external repo links

* minor update

* updated setup instructions

* added text

* align text

* removed duplicated Content section

* address review comments

* Chenhui/hyperdrive example update (#142)

* removed blackcellmagic

* removed utils under aml_scripts and updated notebook

* added notebook path

* added ci test of lightgbm multi round example

* make forecast round as parameter

* Make -Agent Name

* resolve duplicated function name

* increased time limit and reduce number of rounds

* increase time limit

* added parameters tag to multiround lightgbm and dilatedcnn

* README change (#147)

* minor change

* hide tags

* hide tags

* added parameters tag

* Revert "Chenhui/add ci tests (#150)" (#151)

This reverts commit 357453234088f2ebb8453bd8cd77527a1c6c2130 [formerly 21846168a7].

* Chenhui/Add CI tests for notebooks

This reverts commit 8a99549da8b9096b65130fd2f6634e2a217b2dd9 [formerly 89e986fe2c].

* minor update

* Added CI tests for example notebooks

* Update component governance pipeline

* Update component governance pipeline

* add ignored items

* Readds R material (#116)

* Chenhui/windows setup (#131)

* Vapaunic/featutils (#137)

* Chenhui/add CI tests for notebooks

* Vapaunic/arimaint (#154)


* modified conftests to add arima

* added tests

* modified notebooks with parameters

* Chenhui/code improvments (#157)

* updated docstring

* pinged package versions

* minor improvements

* minor improvement

* modified metrics to take any iterable (#158)

* improvement: using Ray to parallelize arima fitting (#159)

* using Ray to parallelize arima fitting

* added ray as dependency

* text about ray, disable warnings, and minor stuff

* scipy 1.4.1 or above

* reverting scipy, azuremlsdk issue

* minor mod

Co-authored-by: Vanja Paunic <15053814+vapaunic@users.noreply.github.com>

* chenhui/improve ray output (#166)

* modified arima multiround to run with ray (#167)

* Chenhui/improve doc (#168)

* minor changes

* remove redundancy

* updated text

* improved text in model tuning and deployment notebook

* clarify the data used

* updated text

* added description of the script

* add explanation of gaps in the curve

* add explanation of gaps in the curve

* updated text

* fix typos

* improve documentation and format

* Addressing a few issues around package dependencies (#169)

* syncronizing utils with other OSS AI repos

* exclude xlrd, leftover from tsperf

* exclude urlib3, leftover from tsperf

* moving tqdm to fclib as only used by lib at the moment

* included fclib dependencies in requirements.txt

* lower bounded package versions that we dont need specific versions of

* lower bound gitpython

* Chenhui/improve checking of run completion (#170)

* Chenhui/added ray dashboard (#171)

* Chenhui/update diagram (#172)

* update multiround training diagram

* minor change

* update diagram and minor change

* Addressing doc related issues (#173)

* taking out inventory optimization link

* pulled contributing out of docs

* Chenhui/ray windows (#177)

* add util to check if module exists

* use ray if available or use sequential training

* updated text

* updated text

* reduce code redundancy

* Chenhui/setup scripts (#178)

* move ray to linux setup script

* remove duplicated azureml-sdk to avoid errors

* add ray to ci yaml files

* update azureml-sdk

* update manual setup instructions

* minor change

* Chenhui/content table (#179)

* update readme

* minor change

* minor update

* Chenhui/multiround arima (#180)

* use ray if it is installed

* update text and reran notebook

* add reference

* Chenhui/dilatedcnn windows (#184)

* resolve format issues

* update log path and tensorboard path

* remove subprocess import

* fix path

* change env name to resolve pipeline failures

* Chenhui/hyperdrive windows (#185)

* resolve format issues

* update log path and tensorboard path

* remove subprocess import

* fetch common utils from chenhui/dilatedcnn_windows

* update notebook

* removed explain module and added notebooks module

* get updated ci yml files

* updated kernel name

* Chenhui/enhancement (#186)

* modified module_path

* updated tensorboard section

* rerun notebook

* only submit local run if python path is found

* minor change and rerun notebook

* updated content section (#187)

* updated content section

* minor change

* address comments

* add links

Co-authored-by: Hong Lu <honglu@microsoft.com>
Co-authored-by: ZhouFang928 <ZhouFang928@users.noreply.github.com>
Co-authored-by: pechyony <pechyony@outlook.com>
Co-authored-by: Ubuntu <chenhui@chhdsvmnc6.hyjxgt1qggauhj0g0g2jh3guwb.bx.internal.cloudapp.net>
Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>
Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Co-authored-by: Daniel Ciborowski <dciborow@microsoft.com>
Co-authored-by: Markus Cozowicz <marcozo@microsoft.com>
Former-commit-id: 6098ecf68c

* Updates to R Markdown notebooks (#189)

* urca package dependency

* Instructions to install fable.prophet

* add urca as requirement

* add urca to all rmds

Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Former-commit-id: 5ca9eeb7b1

* Set theme jekyll-theme-cayman

Former-commit-id: 4559509c7677936b45ef6fbcb7e6fefa12b36c05

* Add retail_turnover example, cleanup contrib folder (#203)

* Removing tsperf files from contrib/ (#199)

* removed tsperf from contrib directory

* modified contrib/readme

* address PR comments

* Hongooi/fable intro (#200)

Adds an introductory example that goes over the basics of time series analysis, using the tsibbledata::aus_retail dataset. Includes discussion on forecasting in general. Closes #57

* Tidyverts update (#202)

Updates the R code in the examples for the latest tidyverts package versions on CRAN.

Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>
Former-commit-id: b98487f42c701e956e0301b33ec87827645b727d

* incremented version number

* tensorflow version to three digits

* tests/ci/ado_artifact_pipeline.yml

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>
Co-authored-by: Hong Lu <honglu@microsoft.com>
Co-authored-by: ZhouFang928 <ZhouFang928@users.noreply.github.com>
Co-authored-by: pechyony <pechyony@outlook.com>
Co-authored-by: Ubuntu <chenhui@chhdsvmnc6.hyjxgt1qggauhj0g0g2jh3guwb.bx.internal.cloudapp.net>
Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Co-authored-by: Daniel Ciborowski <dciborow@microsoft.com>
Co-authored-by: Markus Cozowicz <marcozo@microsoft.com>
Co-authored-by: David Smith <davidsmi@microsoft.com>

* Minor update to README (#209)

Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>
Co-authored-by: Hong Lu <honglu@microsoft.com>
Co-authored-by: ZhouFang928 <ZhouFang928@users.noreply.github.com>
Co-authored-by: pechyony <pechyony@outlook.com>
Co-authored-by: Ubuntu <chenhui@chhdsvmnc6.hyjxgt1qggauhj0g0g2jh3guwb.bx.internal.cloudapp.net>
Co-authored-by: Daniel Ciborowski <dciborow@microsoft.com>
Co-authored-by: Markus Cozowicz <marcozo@microsoft.com>
Co-authored-by: David Smith <davidsmi@microsoft.com>
2020-06-24 09:44:27 -07:00
Chenhui Hu 9e74c16a40 Merge pull request #205 from microsoft/staging
Staging - fix link 

Former-commit-id: 5876b4ee204ac5f2cb52df6f661b07d8317ed22b
2020-06-19 18:19:12 -04:00
Chenhui Hu c33317b3a9 update link (#204)
Former-commit-id: 4ce65ecf27
2020-06-19 17:47:44 -04:00
Hong Ooi b62c56cfc3 Add retail_turnover example, cleanup contrib folder (#203)
* Removing tsperf files from contrib/ (#199)

* removed tsperf from contrib directory

* modified contrib/readme

* address PR comments

* Hongooi/fable intro (#200)

Adds an introductory example that goes over the basics of time series analysis, using the tsibbledata::aus_retail dataset. Includes discussion on forecasting in general. Closes #57

* Tidyverts update (#202)

Updates the R code in the examples for the latest tidyverts package versions on CRAN.

Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>
Former-commit-id: b98487f42c701e956e0301b33ec87827645b727d
2020-06-20 06:58:54 +10:00
Hong Ooi 6e3506738d Tidyverts update (#202)
Updates the R code in the examples for the latest tidyverts package versions on CRAN.

Former-commit-id: 5518583fce
2020-06-20 06:27:50 +10:00
vapaunic 6f8242a0f5 Set theme jekyll-theme-cayman
Former-commit-id: 4559509c7677936b45ef6fbcb7e6fefa12b36c05
2020-06-11 15:32:21 -07:00
Hong Ooi 454caeba34 Hongooi/fable intro (#200)
Adds an introductory example that goes over the basics of time series analysis, using the tsibbledata::aus_retail dataset. Includes discussion on forecasting in general. Closes #57

Former-commit-id: 8affee1a39
2020-06-02 01:15:40 +10:00
vapaunic ed8cafe01e Removing tsperf files from contrib/ (#199)
* removed tsperf from contrib directory

* modified contrib/readme

* address PR comments

Former-commit-id: bd57c9d09d
2020-05-26 13:17:32 -07:00
Chenhui Hu 57e7d1c787 Merge pull request #196 from microsoft/staging
interrupt if an error occurs on environment_setup.sh (#195)

Former-commit-id: d954d55ce89f170292fe140c3f0d9568f653b36e
2020-04-23 14:23:54 -04:00
Taiki Sakamoto 3b203a5bec interrupt if an error occurs on environment_setup.sh (#195)
Former-commit-id: c41fda2308
2020-04-22 22:31:35 -04:00
vapaunic 9ac27a1fe5 Merge pull request #193 from microsoft/staging
Adding html outputs next to R examples (#192)

Former-commit-id: 95f95cc0c55e4dd073882fb11edf70b9c0d06761
2020-04-20 10:37:16 -07:00
vapaunic 4214c0d9c8 html outputs linked in R examples table (#192)
* First Release of Forecasting Repo (#181)

* Handled edge case where ts_id_col_names is None

* Split long line into separate lines

* Added notebook template

* Added a test yml file

* Added yml file for python unit test pipeline

* Minor update

* Minor update

* Minor update

* Minor update

* Removed triggers

* Removed triggers

* Created a base ts estimator and inherit BaseTSFeaturizer from the BaseTSEstimator.

* Refactored featurizer class hierachy.

* Added week of month method.

* add script to source entire

* formatting

* source only test files

* Inherit temporal featurizers from BaseTSFeaturizer.

* Minor update.

* Replaced max_test_timestamp with max_horizon

* Refactored rolling window featurizers.

* Renamed hour_of_year feature to normalized_hour_of_year

* Inherit all normalizers from base normalizer class.

* address review comments for the PR of contributing

* minor update

* address review comments for PR of r test pipeline

* add a test yml file

* Remove checking target column existence, because testing data may not have the target column.

* Create setter and getter of ts_id_col_names.

* Fixed bug caused by unexpected behavior of pandas.shift

* Some code cleanup.

* Updated some featurizer names.

* Some minor changes in df_config and feature configs.

* Some minor changes in feature names.

* Added usage examples in docstring.

* Computation time update after feature engineering refactoring.

* Removed setting frequency.

* Added docstring to convert_to_tsdf function.

* Removed frequency in convert_to_tsdf call.

* Fixed week_of_month function.

* Added popularity featurizer

* Added utility function for checking Iterable but not string.

* Updated LightGBM feature engineering code to use new feature engineering classes.

* Improved checking whether input column names are Iterable and conver to list.

* Made future_value_available a read-only property.

* Minor docstring update.

* Removed extra space in docstring examples.

* Made some methods staticmethods.

* Minor QRF result update after feature engineering code change.

* Removed calling of validate_file and added catching of the exception

* Update python_unit_tests_base.yml for Azure Pipelines [skip ci]

Updated path of the test results

* Test if the download link is wrong

* Fixed minor format issues.

* Fixed minor format issues.

* Fixed formatting issues.

* Fixed line length.

* Removed data files before downloading and checked dimensions of energy data

* Removed the change made for testing

* Changed folder structure of tests and added table to show build status

* Added missing files

* Updated based on review comments

* new folder structure

* add repo metrics

* remove prototypes folder

* add models placeholder

* adjust featurizers to the new structure of folders

* changes in README and evaluation files

* adjust data download to new folders

* delete unnecessary files

* energy load baseline model with new folders

* delete data files

* fix links in benchmarks file

* fix bug

* adjust GBM, QRF and FNN submissions to the new folder structure

* Replace pd.to_timedelta with pd.offsets.

* Added get_offset_by_frequency helper function.

* fix small bugs

* fix small bugs

* Update TSCVSplitter.

* refactored high-level folders

* added a placeholder folder for PR/issue templates

* added subfolders under notebooks/

* updated tests folder

* renamed notebooks/ to examples/

* Update to CONTRIBUTING instructions (#34)

* style checking and formatting files

* git hook installation guide

* issue and PR templates

* minor change

*  working with github instructions

* added specific issue templates

* addressed PR comments

* addressed Chenhui's comment

* addressing chenhuis comments

* conda environment file (#36)

* conda environment file

* updated environment file

* updated instructions for installing conda env

* Vapaunic/lib (#37)

* initial core for forecasting library

* syncing with new structure

* __init__ files in modules

* renamed lib directory

* Added legal headers and some formatting of py files

* restructured benchmarking directory in lib

* fixed imports, warnings, legal headers

* more import fixes and legal headers

* updated instructions with package installation

* barebones library README

* moved energy benchmark to contrib

* formatting changes plus more legal headers

* Added license to the setup

* moved .swp file to contrib, not sure we need to keep it at all

* added missing headers and a brief snipet to README file

* minor wording change in readme

* Chenhui/cpu unit test pipeline (#38)

* address review comments

* added full conda path

* minor change

* added conda to PATH

* added build status in README

* removed energy data prep placeholder notebook

* moved out data energy explore notebook into contrib

* moved data download script to tools/

* Added getting started section to readme

* Added rbase and rbayesm to conda environment

* modified data download script

* added instructions for data download

* renamed data download script

* fixing issues with test pipeline

* parsing issue in yml file

* cleaning up ci test yaml file for more diagnostic info

* fixed a missing argument in instructions

* removed retail directory under dataset module

* moved feature_engineering.py to the feature engineering module

* moved evaluate.py to evaluation module

* combined benchmark settings into a single file

* moved download sript to the package and modified the tests

* modified instructions

* fixed the build pipeline yml

* fix to the pipeline yml

* fix to the pipeline yml

* moved serve_folds into ojdata.py

* removed data_schema.py file as all content moved to ojdata.py

* fixed split_train_test in ojdata.py

* moved retail_data_schema into ojdata.py

* moved all oj utilities to ojdata.py

* removed paths from benchmark_settings

* fixed up a docstring

* quick fix a typo

* removed benchmark_settings

* parameterized experiment settings

* refactored experiment settings

* Fixed docstrings

* addressed chenhuis comment around round file naming

* renamed experiment to forecast settings

* Chenhui/light gbm quick start (#40)

* initial example notebook for lightgbm

* reduced to one round forecast

* added text

* added text

* added text

* moved week_of_month to feature engineering utils

* moved df_from_cartesian_product to feature utils

* moved functions to feature utils

* moved functions to feature utils

* added lightgbm model utils

* updated plots

* added text and renamed predict function

* reduced print out frequency in model training

* moved data visualization code to utils

* added text

* updated plot function and added docstring

* renamed the notebook

* updated text

* added NOTICE file, currently empty as we're not redistributing any packages

* Chenhui/add scrapbook (#43)

* added scrapbook support

* Added gitpython to environtment.yml file

* added git_repo_path function to utils

* updated notebook

* added test for lightgbm notebook

* included testing of notebooks

* resolve test error

* resolve test error

* added kernel name

* updated kernel name

* trying installing bayesm from cmd

* trying installing bayesm from cmd

* trying installing bayesm from cmd

* excluded notebook test

* excluded notebook test

* added lapack.so link fix

* included notebook tests

* excluded files for notebook test

Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>

* added integration test

* added initial data prep notebook

* updated notebook

* updated notebook

* updated notebook

* updated url

* init

* model parameters

* removed blank quick start notebooks

* removed blank modeling notebooks

* removed blank evaluation notebooks

* Removed blank model selection notebooks

* removed blank o16n notebooks

* removed outdated text from contrib/README

* removed outdated swp file

* updating .gitignore

* removed change log, as we don't plan to maintain this

* Excluding irrelevant directories

* fix settings

* separated out the setup guide

* fix settings

* simplemodel init

* typo

* add rproj file

* Renaming forecasting_lib to fclib (#59)

* renamed forecasting_lib directory

* modified references to forecasting_lib

* Vapaunic/envname (#61)

* renamed conda env

* modified setup instructions

* minor change in contributing guide

* keep top-level gitignore only

* formatting fixes

* Chenhui/add automl example (#62)

* added multiple linear models and example notebook for AutoML

* removed commented code

* address review comments

* minor update to the notebook

* minor update to the notebook

* added text

* changed types in lightgbm to be consistent with the rest of the code

* modified docstrings in multiple_linear_regression.py

* updated ci yaml files

* changed import statement in confest.py

* updated gitpython version to the latest

Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>

* Vapaunic/split bug (#65)

* fixed a yield bug

* removed two blank files

* modified split data function to auto-calculate the splits based on the parameters

* removed forecast_settings module

* removed unused parameter

* modified splitting function to use non-overlapping testing

* tested the split function after the update

* minor fix

* defaults changed in split function

* modified lightgbm example with new split function

* modified automl example (needs verification)

* modified data explore notebook

* quick fix:

* updated data preparation notebook

* changed defaults in split function

* Addressed changes in lightgbm

* addressed issues in automl notebook

* fixed typo in lightgbm plot

* first images of time series split

* updated the pictures

* updated evaluation periods (#66)

* Chenhui/env setup script (#67)

* added a shell script for setting up environment

* changed yaml to yml

* added comments and updated SETUP.md

* modified data preparation notebook with images

* moved r exploration notebook to contrib directory

* modified data explore notebook, updated info about the data, and removed reference to TSPerf

* addressed review feedback and fixed the explore notebook

* Chenhui/multiround lightgbm (#68)

* added initial multiround notebook for lightgbm

* updated data splitting

* updated text

* updated week list

* addressed review comments

* added pyramid-automl to conda file

* first draft of arima notebook

* replace pyramid with pmdarima

* Added a complete function

* minor type

* forecasting across many stores/brands

* complete arima notebook

* renamed data preparation/exploration notebooks

* added git clone to setup

* addressed PR comments

* typo

* Arima to ARIMA

* fixed docstring in plot function

* fixed a bug in MAPE calculation and added plotting

* fixed a bug in predict

* modeling arima on log scale

* Fixing AML Example Notebook (#84)

* Cleaning notebook output, adding get_or_create workspace call, and fixing get_or_create AmlCompute

* Add regression-based models (#64)

* modelling updates

* code tweak

* rebuild

* update mape

* update mape 2

* new forecasting structure

* update eval

* rebuild dataprep

* rebuild with profit

* rm profit

* add plot

* typo

* tidy up

* expand readme

* oops

* clarified setup guide (#94)

* Update SETUP (#95)

minor fix

* Cleaned up unused files and directories (#96)

* removed non-used files

* moved docs into a docs/ dir

* fixed broken links

* Chenhui/dilated cnn example and utils (#76)

* added initial model util file for DCNN

* initial notebook

* added feature utils for DCNN

* upadted evaluation and visualization

* removed plot function

* replaced PRED_HORIZON, PRED_STEPS by HORIZON, GAP

* removed log dir if it exists

* updated model utils

* generalized categorical features in dcnn model util

* generalized network definition

* update training code

* format with blackcellmagic

* address review comments and added README

* Chenhui/add ci tests (#146)

* Update conda env with versions (#99)

* 💥

* revert

* minor changes

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Adding missing Jupyter Extension (#90)

* Update environment.yml

* specified version

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* fix links to examples/ (#104)

* Chenhui/rename notebooks and update automl notebook (#106)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* Arima multi-round notebook (#91)

* working arima model

* final auto arima example

* added tqdm to requirements

* addressed review comments

* Revert "Chenhui/rename notebooks and update automl notebook (#106)" (#107)

This reverts commit 032c91d9bfa389f22ae1f1f2150913a4f063bd18 [formerly 15d25213dc].

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Fixing data download issue (#109)

* removed dependency on __file__ from data download, doesn't work in jupyter

* changed aux to auxdata

* fixe data download function

* fixed path

* auxdata -> auxi

* adding tl;dr directions for setup to README.md (#88)

* adding tl;dr directions for setup to README.md

* added a bit more text

* Cleaned up obsolete (tsperf) code in fclib (#112)

* moved out tsperf files from evaluation module

* moved out tsperf tuning code

* removed more unused files

* Addressing documentation related issues (#111)

* Added conda activate to the setup readme

* added instructions for starting jupyter to setup

* minor

* deleted duplicate instructions

* addressed PR comments

* Chenhui/rename notebooks and updated AutoML example (#108)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* updated pytest file

* address review comments

* reran notebook with blackcellmagic

* adding pylint  (#93)

* adding tl;dr directions for setup to README.md

* removing pylint hook and pylint_junit from the env file

* removed pylint config file

* Chenhui/update example folder (#115)

* restructure examples folder

* updated readme

* added readme

* minor update

* removed R folder

* minor change

* fixed a broken link

* another broken link

* fixing notebook tests

* Chenhui/fix aux file path (#118)

* fixed figure links

* changed to auxi_i.csv

* minor change

* [MINOR] Small changes to Arima notebooks (#121)

* fixed a broken link

* minor text changes

* Documentation (#120)

* added target audience section

* added intro on forecasting

* Added fclib documentation

* improved examples readme

* address comments

* added info about the dataset

* added items to be ignored (#123)

* added items to be ignored

* added *.log and score.py

* Chenhui/toplevel readme (#127)

* added content table

* added references

* added external repo links

* minor update

* Chenhui/tune deploy lgbm (#122)

* added notebook and utils

* updated readme links

* fix data path

* updated text

* group imports

* minor update

* using azureml utils to create workspace and compute (#126)

* using azureml utils to create workspace and compute

* group imports

* Download ojdata directly from github (#128)

* new function to download and load oj data directly from bayesm repo

* removed bayesm

* new R function to only load the data

* removed download R function

* minor fix

* added documentation to load_oj_data.R

* added requests to requirements

* fixed a syntax error (#130)

* fix setup.md link (#129)

* fix setup.md link

* mention related use cases

* Vapaunic/cgbuild (#133)

* added files to generate reqs.txt and the ci yml file

* Added notice generation task

* Checking if notice is there

* Update component_governance.yml for Azure Pipelines

* check in notice file

* Update component_governance.yml for Azure Pipelines

* fixed heading

* Chenhui/windows setup (#131)

* initial test

* added batch script and instructions

* align image to center

* adjust image size

* added text

* adjust image size

* address comments

* Readds R material (#116)

* redo R stuff in new dirs

* dirname fixup

* add Rproj file

* rebuild

* fixups

* roxygenise

* copyright notice

* dataprep

* updated yaml

* more updates

* more tweaks

* reg models

* update reg models

* more updates

* reword

* rendered prophet html

* name fix

* add lintr file

* move stuff

* renamed use case folder (#138)

* renamed use case folder

* dirname change

* updated readme

* added notebooks

* fix ci test

* Vapaunic/featutils (#137)

* moved feature engineering module to contrib

* removed lag submod

* cleaned up feature engineering

* rebuild R notebooks (#139)

* Chenhui/toplevel readme (#140)

* added content table

* added references

* added external repo links

* minor update

* updated setup instructions

* added text

* align text

* removed duplicated Content section

* address review comments

* Chenhui/hyperdrive example update (#142)

* removed blackcellmagic

* removed utils under aml_scripts and updated notebook

* added notebook path

* added ci test of lightgbm multi round example

* make forecast round as parameter

* Make -Agent Name

* resolve duplicated function name

* increased time limit and reduce number of rounds

* increase time limit

* added parameters tag to multiround lightgbm and dilatedcnn

* README change (#147)

* minor change

* hide tags

* hide tags

* added parameters tag

* Revert "Chenhui/add ci tests (#146)" (#149)

This reverts commit de7a19cfa7637476b9ebfc92f5c18a26a8eca4da [formerly f8bd22733c].

* Chenhui/add ci tests (#150)

* Update conda env with versions (#99)

* 💥

* revert

* minor changes

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Adding missing Jupyter Extension (#90)

* Update environment.yml

* specified version

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* fix links to examples/ (#104)

* Chenhui/rename notebooks and update automl notebook (#106)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* Arima multi-round notebook (#91)

* working arima model

* final auto arima example

* added tqdm to requirements

* addressed review comments

* Revert "Chenhui/rename notebooks and update automl notebook (#106)" (#107)

This reverts commit 032c91d9bfa389f22ae1f1f2150913a4f063bd18 [formerly 15d25213dc].

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Fixing data download issue (#109)

* removed dependency on __file__ from data download, doesn't work in jupyter

* changed aux to auxdata

* fixe data download function

* fixed path

* auxdata -> auxi

* adding tl;dr directions for setup to README.md (#88)

* adding tl;dr directions for setup to README.md

* added a bit more text

* Cleaned up obsolete (tsperf) code in fclib (#112)

* moved out tsperf files from evaluation module

* moved out tsperf tuning code

* removed more unused files

* Addressing documentation related issues (#111)

* Added conda activate to the setup readme

* added instructions for starting jupyter to setup

* minor

* deleted duplicate instructions

* addressed PR comments

* Chenhui/rename notebooks and updated AutoML example (#108)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* updated pytest file

* address review comments

* reran notebook with blackcellmagic

* adding pylint  (#93)

* adding tl;dr directions for setup to README.md

* removing pylint hook and pylint_junit from the env file

* removed pylint config file

* Chenhui/update example folder (#115)

* restructure examples folder

* updated readme

* added readme

* minor update

* removed R folder

* minor change

* fixed a broken link

* another broken link

* fixing notebook tests

* Chenhui/fix aux file path (#118)

* fixed figure links

* changed to auxi_i.csv

* minor change

* [MINOR] Small changes to Arima notebooks (#121)

* fixed a broken link

* minor text changes

* Documentation (#120)

* added target audience section

* added intro on forecasting

* Added fclib documentation

* improved examples readme

* address comments

* added info about the dataset

* added items to be ignored (#123)

* added items to be ignored

* added *.log and score.py

* Chenhui/toplevel readme (#127)

* added content table

* added references

* added external repo links

* minor update

* Chenhui/tune deploy lgbm (#122)

* added notebook and utils

* updated readme links

* fix data path

* updated text

* group imports

* minor update

* using azureml utils to create workspace and compute (#126)

* using azureml utils to create workspace and compute

* group imports

* Download ojdata directly from github (#128)

* new function to download and load oj data directly from bayesm repo

* removed bayesm

* new R function to only load the data

* removed download R function

* minor fix

* added documentation to load_oj_data.R

* added requests to requirements

* fixed a syntax error (#130)

* fix setup.md link (#129)

* fix setup.md link

* mention related use cases

* Vapaunic/cgbuild (#133)

* added files to generate reqs.txt and the ci yml file

* Added notice generation task

* Checking if notice is there

* Update component_governance.yml for Azure Pipelines

* check in notice file

* Update component_governance.yml for Azure Pipelines

* fixed heading

* Chenhui/windows setup (#131)

* initial test

* added batch script and instructions

* align image to center

* adjust image size

* added text

* adjust image size

* address comments

* Readds R material (#116)

* redo R stuff in new dirs

* dirname fixup

* add Rproj file

* rebuild

* fixups

* roxygenise

* copyright notice

* dataprep

* updated yaml

* more updates

* more tweaks

* reg models

* update reg models

* more updates

* reword

* rendered prophet html

* name fix

* add lintr file

* move stuff

* renamed use case folder (#138)

* renamed use case folder

* dirname change

* updated readme

* added notebooks

* fix ci test

* Vapaunic/featutils (#137)

* moved feature engineering module to contrib

* removed lag submod

* cleaned up feature engineering

* rebuild R notebooks (#139)

* Chenhui/toplevel readme (#140)

* added content table

* added references

* added external repo links

* minor update

* updated setup instructions

* added text

* align text

* removed duplicated Content section

* address review comments

* Chenhui/hyperdrive example update (#142)

* removed blackcellmagic

* removed utils under aml_scripts and updated notebook

* added notebook path

* added ci test of lightgbm multi round example

* make forecast round as parameter

* Make -Agent Name

* resolve duplicated function name

* increased time limit and reduce number of rounds

* increase time limit

* added parameters tag to multiround lightgbm and dilatedcnn

* README change (#147)

* minor change

* hide tags

* hide tags

* added parameters tag

* Revert "Chenhui/add ci tests (#150)" (#151)

This reverts commit 357453234088f2ebb8453bd8cd77527a1c6c2130 [formerly 21846168a7].

* Chenhui/Add CI tests for notebooks

This reverts commit 8a99549da8b9096b65130fd2f6634e2a217b2dd9 [formerly 89e986fe2c].

* minor update

* Added CI tests for example notebooks

* Update component governance pipeline

* Update component governance pipeline

* add ignored items

* Readds R material (#116)

* Chenhui/windows setup (#131)

* Vapaunic/featutils (#137)

* Chenhui/add CI tests for notebooks

* Vapaunic/arimaint (#154)


* modified conftests to add arima

* added tests

* modified notebooks with parameters

* Chenhui/code improvments (#157)

* updated docstring

* pinged package versions

* minor improvements

* minor improvement

* modified metrics to take any iterable (#158)

* improvement: using Ray to parallelize arima fitting (#159)

* using Ray to parallelize arima fitting

* added ray as dependency

* text about ray, disable warnings, and minor stuff

* scipy 1.4.1 or above

* reverting scipy, azuremlsdk issue

* minor mod

Co-authored-by: Vanja Paunic <15053814+vapaunic@users.noreply.github.com>

* chenhui/improve ray output (#166)

* modified arima multiround to run with ray (#167)

* Chenhui/improve doc (#168)

* minor changes

* remove redundancy

* updated text

* improved text in model tuning and deployment notebook

* clarify the data used

* updated text

* added description of the script

* add explanation of gaps in the curve

* add explanation of gaps in the curve

* updated text

* fix typos

* improve documentation and format

* Addressing a few issues around package dependencies (#169)

* syncronizing utils with other OSS AI repos

* exclude xlrd, leftover from tsperf

* exclude urlib3, leftover from tsperf

* moving tqdm to fclib as only used by lib at the moment

* included fclib dependencies in requirements.txt

* lower bounded package versions that we dont need specific versions of

* lower bound gitpython

* Chenhui/improve checking of run completion (#170)

* Chenhui/added ray dashboard (#171)

* Chenhui/update diagram (#172)

* update multiround training diagram

* minor change

* update diagram and minor change

* Addressing doc related issues (#173)

* taking out inventory optimization link

* pulled contributing out of docs

* Chenhui/ray windows (#177)

* add util to check if module exists

* use ray if available or use sequential training

* updated text

* updated text

* reduce code redundancy

* Chenhui/setup scripts (#178)

* move ray to linux setup script

* remove duplicated azureml-sdk to avoid errors

* add ray to ci yaml files

* update azureml-sdk

* update manual setup instructions

* minor change

* Chenhui/content table (#179)

* update readme

* minor change

* minor update

* Chenhui/multiround arima (#180)

* use ray if it is installed

* update text and reran notebook

* add reference

* Chenhui/dilatedcnn windows (#184)

* resolve format issues

* update log path and tensorboard path

* remove subprocess import

* fix path

* change env name to resolve pipeline failures

* Chenhui/hyperdrive windows (#185)

* resolve format issues

* update log path and tensorboard path

* remove subprocess import

* fetch common utils from chenhui/dilatedcnn_windows

* update notebook

* removed explain module and added notebooks module

* get updated ci yml files

* updated kernel name

* Chenhui/enhancement (#186)

* modified module_path

* updated tensorboard section

* rerun notebook

* only submit local run if python path is found

* minor change and rerun notebook

* updated content section (#187)

* updated content section

* minor change

* address comments

* add links

Co-authored-by: Hong Lu <honglu@microsoft.com>
Co-authored-by: ZhouFang928 <ZhouFang928@users.noreply.github.com>
Co-authored-by: pechyony <pechyony@outlook.com>
Co-authored-by: Ubuntu <chenhui@chhdsvmnc6.hyjxgt1qggauhj0g0g2jh3guwb.bx.internal.cloudapp.net>
Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>
Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Co-authored-by: Daniel Ciborowski <dciborow@microsoft.com>
Co-authored-by: Markus Cozowicz <marcozo@microsoft.com>

* Updates to R Markdown notebooks (#189)

* urca package dependency

* Instructions to install fable.prophet

* add urca as requirement

* add urca to all rmds

Co-authored-by: Hong Ooi <hongooi@microsoft.com>

* added html links to examples/R readme

* html links in top level readme

* addressed PR comments

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>
Co-authored-by: Hong Lu <honglu@microsoft.com>
Co-authored-by: ZhouFang928 <ZhouFang928@users.noreply.github.com>
Co-authored-by: pechyony <pechyony@outlook.com>
Co-authored-by: Ubuntu <chenhui@chhdsvmnc6.hyjxgt1qggauhj0g0g2jh3guwb.bx.internal.cloudapp.net>
Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Co-authored-by: Daniel Ciborowski <dciborow@microsoft.com>
Co-authored-by: Markus Cozowicz <marcozo@microsoft.com>
Co-authored-by: David Smith <davidsmi@microsoft.com>
Former-commit-id: 1b0e1c8ece
2020-04-20 10:14:03 -07:00
David Smith be5351bc40 Updates to R Markdown notebooks (#189)
* urca package dependency

* Instructions to install fable.prophet

* add urca as requirement

* add urca to all rmds

Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Former-commit-id: 5ca9eeb7b1
2020-04-14 12:25:00 +10:00
Chenhui Hu 28b5fdb003 Merge pull request #188 from microsoft/staging
First release

Former-commit-id: 10352e07c7
2020-04-06 17:12:59 -04:00
Chenhui Hu 0607fd568f First Release of Forecasting Repo (#181)
* Handled edge case where ts_id_col_names is None

* Split long line into separate lines

* Added notebook template

* Added a test yml file

* Added yml file for python unit test pipeline

* Minor update

* Minor update

* Minor update

* Minor update

* Removed triggers

* Removed triggers

* Created a base ts estimator and inherit BaseTSFeaturizer from the BaseTSEstimator.

* Refactored featurizer class hierachy.

* Added week of month method.

* add script to source entire

* formatting

* source only test files

* Inherit temporal featurizers from BaseTSFeaturizer.

* Minor update.

* Replaced max_test_timestamp with max_horizon

* Refactored rolling window featurizers.

* Renamed hour_of_year feature to normalized_hour_of_year

* Inherit all normalizers from base normalizer class.

* address review comments for the PR of contributing

* minor update

* address review comments for PR of r test pipeline

* add a test yml file

* Remove checking target column existence, because testing data may not have the target column.

* Create setter and getter of ts_id_col_names.

* Fixed bug caused by unexpected behavior of pandas.shift

* Some code cleanup.

* Updated some featurizer names.

* Some minor changes in df_config and feature configs.

* Some minor changes in feature names.

* Added usage examples in docstring.

* Computation time update after feature engineering refactoring.

* Removed setting frequency.

* Added docstring to convert_to_tsdf function.

* Removed frequency in convert_to_tsdf call.

* Fixed week_of_month function.

* Added popularity featurizer

* Added utility function for checking Iterable but not string.

* Updated LightGBM feature engineering code to use new feature engineering classes.

* Improved checking whether input column names are Iterable and conver to list.

* Made future_value_available a read-only property.

* Minor docstring update.

* Removed extra space in docstring examples.

* Made some methods staticmethods.

* Minor QRF result update after feature engineering code change.

* Removed calling of validate_file and added catching of the exception

* Update python_unit_tests_base.yml for Azure Pipelines [skip ci]

Updated path of the test results

* Test if the download link is wrong

* Fixed minor format issues.

* Fixed minor format issues.

* Fixed formatting issues.

* Fixed line length.

* Removed data files before downloading and checked dimensions of energy data

* Removed the change made for testing

* Changed folder structure of tests and added table to show build status

* Added missing files

* Updated based on review comments

* new folder structure

* add repo metrics

* remove prototypes folder

* add models placeholder

* adjust featurizers to the new structure of folders

* changes in README and evaluation files

* adjust data download to new folders

* delete unnecessary files

* energy load baseline model with new folders

* delete data files

* fix links in benchmarks file

* fix bug

* adjust GBM, QRF and FNN submissions to the new folder structure

* Replace pd.to_timedelta with pd.offsets.

* Added get_offset_by_frequency helper function.

* fix small bugs

* fix small bugs

* Update TSCVSplitter.

* refactored high-level folders

* added a placeholder folder for PR/issue templates

* added subfolders under notebooks/

* updated tests folder

* renamed notebooks/ to examples/

* Update to CONTRIBUTING instructions (#34)

* style checking and formatting files

* git hook installation guide

* issue and PR templates

* minor change

*  working with github instructions

* added specific issue templates

* addressed PR comments

* addressed Chenhui's comment

* addressing chenhuis comments

* conda environment file (#36)

* conda environment file

* updated environment file

* updated instructions for installing conda env

* Vapaunic/lib (#37)

* initial core for forecasting library

* syncing with new structure

* __init__ files in modules

* renamed lib directory

* Added legal headers and some formatting of py files

* restructured benchmarking directory in lib

* fixed imports, warnings, legal headers

* more import fixes and legal headers

* updated instructions with package installation

* barebones library README

* moved energy benchmark to contrib

* formatting changes plus more legal headers

* Added license to the setup

* moved .swp file to contrib, not sure we need to keep it at all

* added missing headers and a brief snipet to README file

* minor wording change in readme

* Chenhui/cpu unit test pipeline (#38)

* address review comments

* added full conda path

* minor change

* added conda to PATH

* added build status in README

* removed energy data prep placeholder notebook

* moved out data energy explore notebook into contrib

* moved data download script to tools/

* Added getting started section to readme

* Added rbase and rbayesm to conda environment

* modified data download script

* added instructions for data download

* renamed data download script

* fixing issues with test pipeline

* parsing issue in yml file

* cleaning up ci test yaml file for more diagnostic info

* fixed a missing argument in instructions

* removed retail directory under dataset module

* moved feature_engineering.py to the feature engineering module

* moved evaluate.py to evaluation module

* combined benchmark settings into a single file

* moved download sript to the package and modified the tests

* modified instructions

* fixed the build pipeline yml

* fix to the pipeline yml

* fix to the pipeline yml

* moved serve_folds into ojdata.py

* removed data_schema.py file as all content moved to ojdata.py

* fixed split_train_test in ojdata.py

* moved retail_data_schema into ojdata.py

* moved all oj utilities to ojdata.py

* removed paths from benchmark_settings

* fixed up a docstring

* quick fix a typo

* removed benchmark_settings

* parameterized experiment settings

* refactored experiment settings

* Fixed docstrings

* addressed chenhuis comment around round file naming

* renamed experiment to forecast settings

* Chenhui/light gbm quick start (#40)

* initial example notebook for lightgbm

* reduced to one round forecast

* added text

* added text

* added text

* moved week_of_month to feature engineering utils

* moved df_from_cartesian_product to feature utils

* moved functions to feature utils

* moved functions to feature utils

* added lightgbm model utils

* updated plots

* added text and renamed predict function

* reduced print out frequency in model training

* moved data visualization code to utils

* added text

* updated plot function and added docstring

* renamed the notebook

* updated text

* added NOTICE file, currently empty as we're not redistributing any packages

* Chenhui/add scrapbook (#43)

* added scrapbook support

* Added gitpython to environtment.yml file

* added git_repo_path function to utils

* updated notebook

* added test for lightgbm notebook

* included testing of notebooks

* resolve test error

* resolve test error

* added kernel name

* updated kernel name

* trying installing bayesm from cmd

* trying installing bayesm from cmd

* trying installing bayesm from cmd

* excluded notebook test

* excluded notebook test

* added lapack.so link fix

* included notebook tests

* excluded files for notebook test

Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>

* added integration test

* added initial data prep notebook

* updated notebook

* updated notebook

* updated notebook

* updated url

* init

* model parameters

* removed blank quick start notebooks

* removed blank modeling notebooks

* removed blank evaluation notebooks

* Removed blank model selection notebooks

* removed blank o16n notebooks

* removed outdated text from contrib/README

* removed outdated swp file

* updating .gitignore

* removed change log, as we don't plan to maintain this

* Excluding irrelevant directories

* fix settings

* separated out the setup guide

* fix settings

* simplemodel init

* typo

* add rproj file

* Renaming forecasting_lib to fclib (#59)

* renamed forecasting_lib directory

* modified references to forecasting_lib

* Vapaunic/envname (#61)

* renamed conda env

* modified setup instructions

* minor change in contributing guide

* keep top-level gitignore only

* formatting fixes

* Chenhui/add automl example (#62)

* added multiple linear models and example notebook for AutoML

* removed commented code

* address review comments

* minor update to the notebook

* minor update to the notebook

* added text

* changed types in lightgbm to be consistent with the rest of the code

* modified docstrings in multiple_linear_regression.py

* updated ci yaml files

* changed import statement in confest.py

* updated gitpython version to the latest

Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>

* Vapaunic/split bug (#65)

* fixed a yield bug

* removed two blank files

* modified split data function to auto-calculate the splits based on the parameters

* removed forecast_settings module

* removed unused parameter

* modified splitting function to use non-overlapping testing

* tested the split function after the update

* minor fix

* defaults changed in split function

* modified lightgbm example with new split function

* modified automl example (needs verification)

* modified data explore notebook

* quick fix:

* updated data preparation notebook

* changed defaults in split function

* Addressed changes in lightgbm

* addressed issues in automl notebook

* fixed typo in lightgbm plot

* first images of time series split

* updated the pictures

* updated evaluation periods (#66)

* Chenhui/env setup script (#67)

* added a shell script for setting up environment

* changed yaml to yml

* added comments and updated SETUP.md

* modified data preparation notebook with images

* moved r exploration notebook to contrib directory

* modified data explore notebook, updated info about the data, and removed reference to TSPerf

* addressed review feedback and fixed the explore notebook

* Chenhui/multiround lightgbm (#68)

* added initial multiround notebook for lightgbm

* updated data splitting

* updated text

* updated week list

* addressed review comments

* added pyramid-automl to conda file

* first draft of arima notebook

* replace pyramid with pmdarima

* Added a complete function

* minor type

* forecasting across many stores/brands

* complete arima notebook

* renamed data preparation/exploration notebooks

* added git clone to setup

* addressed PR comments

* typo

* Arima to ARIMA

* fixed docstring in plot function

* fixed a bug in MAPE calculation and added plotting

* fixed a bug in predict

* modeling arima on log scale

* Fixing AML Example Notebook (#84)

* Cleaning notebook output, adding get_or_create workspace call, and fixing get_or_create AmlCompute

* Add regression-based models (#64)

* modelling updates

* code tweak

* rebuild

* update mape

* update mape 2

* new forecasting structure

* update eval

* rebuild dataprep

* rebuild with profit

* rm profit

* add plot

* typo

* tidy up

* expand readme

* oops

* clarified setup guide (#94)

* Update SETUP (#95)

minor fix

* Cleaned up unused files and directories (#96)

* removed non-used files

* moved docs into a docs/ dir

* fixed broken links

* Chenhui/dilated cnn example and utils (#76)

* added initial model util file for DCNN

* initial notebook

* added feature utils for DCNN

* upadted evaluation and visualization

* removed plot function

* replaced PRED_HORIZON, PRED_STEPS by HORIZON, GAP

* removed log dir if it exists

* updated model utils

* generalized categorical features in dcnn model util

* generalized network definition

* update training code

* format with blackcellmagic

* address review comments and added README

* Chenhui/add ci tests (#146)

* Update conda env with versions (#99)

* 💥

* revert

* minor changes

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Adding missing Jupyter Extension (#90)

* Update environment.yml

* specified version

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* fix links to examples/ (#104)

* Chenhui/rename notebooks and update automl notebook (#106)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* Arima multi-round notebook (#91)

* working arima model

* final auto arima example

* added tqdm to requirements

* addressed review comments

* Revert "Chenhui/rename notebooks and update automl notebook (#106)" (#107)

This reverts commit 032c91d9bfa389f22ae1f1f2150913a4f063bd18 [formerly 15d25213dc].

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Fixing data download issue (#109)

* removed dependency on __file__ from data download, doesn't work in jupyter

* changed aux to auxdata

* fixe data download function

* fixed path

* auxdata -> auxi

* adding tl;dr directions for setup to README.md (#88)

* adding tl;dr directions for setup to README.md

* added a bit more text

* Cleaned up obsolete (tsperf) code in fclib (#112)

* moved out tsperf files from evaluation module

* moved out tsperf tuning code

* removed more unused files

* Addressing documentation related issues (#111)

* Added conda activate to the setup readme

* added instructions for starting jupyter to setup

* minor

* deleted duplicate instructions

* addressed PR comments

* Chenhui/rename notebooks and updated AutoML example (#108)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* updated pytest file

* address review comments

* reran notebook with blackcellmagic

* adding pylint  (#93)

* adding tl;dr directions for setup to README.md

* removing pylint hook and pylint_junit from the env file

* removed pylint config file

* Chenhui/update example folder (#115)

* restructure examples folder

* updated readme

* added readme

* minor update

* removed R folder

* minor change

* fixed a broken link

* another broken link

* fixing notebook tests

* Chenhui/fix aux file path (#118)

* fixed figure links

* changed to auxi_i.csv

* minor change

* [MINOR] Small changes to Arima notebooks (#121)

* fixed a broken link

* minor text changes

* Documentation (#120)

* added target audience section

* added intro on forecasting

* Added fclib documentation

* improved examples readme

* address comments

* added info about the dataset

* added items to be ignored (#123)

* added items to be ignored

* added *.log and score.py

* Chenhui/toplevel readme (#127)

* added content table

* added references

* added external repo links

* minor update

* Chenhui/tune deploy lgbm (#122)

* added notebook and utils

* updated readme links

* fix data path

* updated text

* group imports

* minor update

* using azureml utils to create workspace and compute (#126)

* using azureml utils to create workspace and compute

* group imports

* Download ojdata directly from github (#128)

* new function to download and load oj data directly from bayesm repo

* removed bayesm

* new R function to only load the data

* removed download R function

* minor fix

* added documentation to load_oj_data.R

* added requests to requirements

* fixed a syntax error (#130)

* fix setup.md link (#129)

* fix setup.md link

* mention related use cases

* Vapaunic/cgbuild (#133)

* added files to generate reqs.txt and the ci yml file

* Added notice generation task

* Checking if notice is there

* Update component_governance.yml for Azure Pipelines

* check in notice file

* Update component_governance.yml for Azure Pipelines

* fixed heading

* Chenhui/windows setup (#131)

* initial test

* added batch script and instructions

* align image to center

* adjust image size

* added text

* adjust image size

* address comments

* Readds R material (#116)

* redo R stuff in new dirs

* dirname fixup

* add Rproj file

* rebuild

* fixups

* roxygenise

* copyright notice

* dataprep

* updated yaml

* more updates

* more tweaks

* reg models

* update reg models

* more updates

* reword

* rendered prophet html

* name fix

* add lintr file

* move stuff

* renamed use case folder (#138)

* renamed use case folder

* dirname change

* updated readme

* added notebooks

* fix ci test

* Vapaunic/featutils (#137)

* moved feature engineering module to contrib

* removed lag submod

* cleaned up feature engineering

* rebuild R notebooks (#139)

* Chenhui/toplevel readme (#140)

* added content table

* added references

* added external repo links

* minor update

* updated setup instructions

* added text

* align text

* removed duplicated Content section

* address review comments

* Chenhui/hyperdrive example update (#142)

* removed blackcellmagic

* removed utils under aml_scripts and updated notebook

* added notebook path

* added ci test of lightgbm multi round example

* make forecast round as parameter

* Make -Agent Name

* resolve duplicated function name

* increased time limit and reduce number of rounds

* increase time limit

* added parameters tag to multiround lightgbm and dilatedcnn

* README change (#147)

* minor change

* hide tags

* hide tags

* added parameters tag

* Revert "Chenhui/add ci tests (#146)" (#149)

This reverts commit de7a19cfa7637476b9ebfc92f5c18a26a8eca4da [formerly f8bd22733c].

* Chenhui/add ci tests (#150)

* Update conda env with versions (#99)

* 💥

* revert

* minor changes

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Adding missing Jupyter Extension (#90)

* Update environment.yml

* specified version

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* fix links to examples/ (#104)

* Chenhui/rename notebooks and update automl notebook (#106)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* Arima multi-round notebook (#91)

* working arima model

* final auto arima example

* added tqdm to requirements

* addressed review comments

* Revert "Chenhui/rename notebooks and update automl notebook (#106)" (#107)

This reverts commit 032c91d9bfa389f22ae1f1f2150913a4f063bd18 [formerly 15d25213dc].

Co-authored-by: Chenhui Hu <chenhhu@microsoft.com>

* Fixing data download issue (#109)

* removed dependency on __file__ from data download, doesn't work in jupyter

* changed aux to auxdata

* fixe data download function

* fixed path

* auxdata -> auxi

* adding tl;dr directions for setup to README.md (#88)

* adding tl;dr directions for setup to README.md

* added a bit more text

* Cleaned up obsolete (tsperf) code in fclib (#112)

* moved out tsperf files from evaluation module

* moved out tsperf tuning code

* removed more unused files

* Addressing documentation related issues (#111)

* Added conda activate to the setup readme

* added instructions for starting jupyter to setup

* minor

* deleted duplicate instructions

* addressed PR comments

* Chenhui/rename notebooks and updated AutoML example (#108)

* removed unused module

* added outputs in automl notebook

* fixed a notebook name

* updated pytest file

* address review comments

* reran notebook with blackcellmagic

* adding pylint  (#93)

* adding tl;dr directions for setup to README.md

* removing pylint hook and pylint_junit from the env file

* removed pylint config file

* Chenhui/update example folder (#115)

* restructure examples folder

* updated readme

* added readme

* minor update

* removed R folder

* minor change

* fixed a broken link

* another broken link

* fixing notebook tests

* Chenhui/fix aux file path (#118)

* fixed figure links

* changed to auxi_i.csv

* minor change

* [MINOR] Small changes to Arima notebooks (#121)

* fixed a broken link

* minor text changes

* Documentation (#120)

* added target audience section

* added intro on forecasting

* Added fclib documentation

* improved examples readme

* address comments

* added info about the dataset

* added items to be ignored (#123)

* added items to be ignored

* added *.log and score.py

* Chenhui/toplevel readme (#127)

* added content table

* added references

* added external repo links

* minor update

* Chenhui/tune deploy lgbm (#122)

* added notebook and utils

* updated readme links

* fix data path

* updated text

* group imports

* minor update

* using azureml utils to create workspace and compute (#126)

* using azureml utils to create workspace and compute

* group imports

* Download ojdata directly from github (#128)

* new function to download and load oj data directly from bayesm repo

* removed bayesm

* new R function to only load the data

* removed download R function

* minor fix

* added documentation to load_oj_data.R

* added requests to requirements

* fixed a syntax error (#130)

* fix setup.md link (#129)

* fix setup.md link

* mention related use cases

* Vapaunic/cgbuild (#133)

* added files to generate reqs.txt and the ci yml file

* Added notice generation task

* Checking if notice is there

* Update component_governance.yml for Azure Pipelines

* check in notice file

* Update component_governance.yml for Azure Pipelines

* fixed heading

* Chenhui/windows setup (#131)

* initial test

* added batch script and instructions

* align image to center

* adjust image size

* added text

* adjust image size

* address comments

* Readds R material (#116)

* redo R stuff in new dirs

* dirname fixup

* add Rproj file

* rebuild

* fixups

* roxygenise

* copyright notice

* dataprep

* updated yaml

* more updates

* more tweaks

* reg models

* update reg models

* more updates

* reword

* rendered prophet html

* name fix

* add lintr file

* move stuff

* renamed use case folder (#138)

* renamed use case folder

* dirname change

* updated readme

* added notebooks

* fix ci test

* Vapaunic/featutils (#137)

* moved feature engineering module to contrib

* removed lag submod

* cleaned up feature engineering

* rebuild R notebooks (#139)

* Chenhui/toplevel readme (#140)

* added content table

* added references

* added external repo links

* minor update

* updated setup instructions

* added text

* align text

* removed duplicated Content section

* address review comments

* Chenhui/hyperdrive example update (#142)

* removed blackcellmagic

* removed utils under aml_scripts and updated notebook

* added notebook path

* added ci test of lightgbm multi round example

* make forecast round as parameter

* Make -Agent Name

* resolve duplicated function name

* increased time limit and reduce number of rounds

* increase time limit

* added parameters tag to multiround lightgbm and dilatedcnn

* README change (#147)

* minor change

* hide tags

* hide tags

* added parameters tag

* Revert "Chenhui/add ci tests (#150)" (#151)

This reverts commit 357453234088f2ebb8453bd8cd77527a1c6c2130 [formerly 21846168a7].

* Chenhui/Add CI tests for notebooks

This reverts commit 8a99549da8b9096b65130fd2f6634e2a217b2dd9 [formerly 89e986fe2c].

* minor update

* Added CI tests for example notebooks

* Update component governance pipeline

* Update component governance pipeline

* add ignored items

* Readds R material (#116)

* Chenhui/windows setup (#131)

* Vapaunic/featutils (#137)

* Chenhui/add CI tests for notebooks

* Vapaunic/arimaint (#154)


* modified conftests to add arima

* added tests

* modified notebooks with parameters

* Chenhui/code improvments (#157)

* updated docstring

* pinged package versions

* minor improvements

* minor improvement

* modified metrics to take any iterable (#158)

* improvement: using Ray to parallelize arima fitting (#159)

* using Ray to parallelize arima fitting

* added ray as dependency

* text about ray, disable warnings, and minor stuff

* scipy 1.4.1 or above

* reverting scipy, azuremlsdk issue

* minor mod

Co-authored-by: Vanja Paunic <15053814+vapaunic@users.noreply.github.com>

* chenhui/improve ray output (#166)

* modified arima multiround to run with ray (#167)

* Chenhui/improve doc (#168)

* minor changes

* remove redundancy

* updated text

* improved text in model tuning and deployment notebook

* clarify the data used

* updated text

* added description of the script

* add explanation of gaps in the curve

* add explanation of gaps in the curve

* updated text

* fix typos

* improve documentation and format

* Addressing a few issues around package dependencies (#169)

* syncronizing utils with other OSS AI repos

* exclude xlrd, leftover from tsperf

* exclude urlib3, leftover from tsperf

* moving tqdm to fclib as only used by lib at the moment

* included fclib dependencies in requirements.txt

* lower bounded package versions that we dont need specific versions of

* lower bound gitpython

* Chenhui/improve checking of run completion (#170)

* Chenhui/added ray dashboard (#171)

* Chenhui/update diagram (#172)

* update multiround training diagram

* minor change

* update diagram and minor change

* Addressing doc related issues (#173)

* taking out inventory optimization link

* pulled contributing out of docs

* Chenhui/ray windows (#177)

* add util to check if module exists

* use ray if available or use sequential training

* updated text

* updated text

* reduce code redundancy

* Chenhui/setup scripts (#178)

* move ray to linux setup script

* remove duplicated azureml-sdk to avoid errors

* add ray to ci yaml files

* update azureml-sdk

* update manual setup instructions

* minor change

* Chenhui/content table (#179)

* update readme

* minor change

* minor update

* Chenhui/multiround arima (#180)

* use ray if it is installed

* update text and reran notebook

* add reference

* Chenhui/dilatedcnn windows (#184)

* resolve format issues

* update log path and tensorboard path

* remove subprocess import

* fix path

* change env name to resolve pipeline failures

* Chenhui/hyperdrive windows (#185)

* resolve format issues

* update log path and tensorboard path

* remove subprocess import

* fetch common utils from chenhui/dilatedcnn_windows

* update notebook

* removed explain module and added notebooks module

* get updated ci yml files

* updated kernel name

* Chenhui/enhancement (#186)

* modified module_path

* updated tensorboard section

* rerun notebook

* only submit local run if python path is found

* minor change and rerun notebook

* updated content section (#187)

* updated content section

* minor change

* address comments

* add links

Co-authored-by: Hong Lu <honglu@microsoft.com>
Co-authored-by: ZhouFang928 <ZhouFang928@users.noreply.github.com>
Co-authored-by: pechyony <pechyony@outlook.com>
Co-authored-by: Ubuntu <chenhui@chhdsvmnc6.hyjxgt1qggauhj0g0g2jh3guwb.bx.internal.cloudapp.net>
Co-authored-by: vapaunic <15053814+vapaunic@users.noreply.github.com>
Co-authored-by: Hong Ooi <hongooi@microsoft.com>
Co-authored-by: Daniel Ciborowski <dciborow@microsoft.com>
Co-authored-by: Markus Cozowicz <marcozo@microsoft.com>
Former-commit-id: 6098ecf68c
2020-04-06 16:17:18 -04:00
Chenhui Hu f8efa1d54d updated content section (#187)
* updated content section

* minor change

* address comments

* add links

Former-commit-id: ddd0374463
2020-04-06 15:46:20 -04:00
Chenhui Hu 85452929f7 Chenhui/enhancement (#186)
* modified module_path

* updated tensorboard section

* rerun notebook

* only submit local run if python path is found

* minor change and rerun notebook

Former-commit-id: 9f25f3acb8
2020-04-05 22:40:25 -04:00
Chenhui Hu c52fb1c83d Chenhui/hyperdrive windows (#185)
* resolve format issues

* update log path and tensorboard path

* remove subprocess import

* fetch common utils from chenhui/dilatedcnn_windows

* update notebook

* removed explain module and added notebooks module

* get updated ci yml files

* updated kernel name

Former-commit-id: deac0cce96
2020-04-03 11:15:49 -04:00
Chenhui Hu c2c17e4748 Chenhui/dilatedcnn windows (#184)
* resolve format issues

* update log path and tensorboard path

* remove subprocess import

* fix path

* change env name to resolve pipeline failures

Former-commit-id: 56beb18f7d
2020-04-02 17:26:40 -04:00
Chenhui Hu 19df2738e2 Chenhui/multiround arima (#180)
* use ray if it is installed

* update text and reran notebook

* add reference

Former-commit-id: 5668dfe5e2
2020-04-01 11:55:26 -04:00
Chenhui Hu 182b8b8833 Chenhui/content table (#179)
* update readme

* minor change

* minor update

Former-commit-id: 76b0397e41
2020-04-01 11:26:30 -04:00
Chenhui Hu 373d7d797c Chenhui/setup scripts (#178)
* move ray to linux setup script

* remove duplicated azureml-sdk to avoid errors

* add ray to ci yaml files

* update azureml-sdk

* update manual setup instructions

* minor change

Former-commit-id: 88f099759b
2020-03-31 13:42:56 -04:00
Chenhui Hu 74237a7688 Chenhui/ray windows (#177)
* add util to check if module exists

* use ray if available or use sequential training

* updated text

* updated text

* reduce code redundancy

Former-commit-id: eafbbbc7e7
2020-03-30 14:21:58 -04:00
vapaunic 25b9368774 Addressing doc related issues (#173)
* taking out inventory optimization link

* pulled contributing out of docs

Former-commit-id: 2df9231f45
2020-03-30 14:52:51 +01:00
Chenhui Hu d1a070187e Chenhui/update diagram (#172)
* update multiround training diagram

* minor change

* update diagram and minor change

Former-commit-id: db07b00674
2020-03-27 16:01:27 -04:00
Chenhui Hu ce0a41e423 Chenhui/added ray dashboard (#171)
Former-commit-id: 59508a2279
2020-03-27 14:17:12 -04:00
Chenhui Hu cc8a7f7e90 Chenhui/improve checking of run completion (#170)
Former-commit-id: c965fdcc40
2020-03-27 10:52:37 -04:00
vapaunic 622c0c6756 Addressing a few issues around package dependencies (#169)
* syncronizing utils with other OSS AI repos

* exclude xlrd, leftover from tsperf

* exclude urlib3, leftover from tsperf

* moving tqdm to fclib as only used by lib at the moment

* included fclib dependencies in requirements.txt

* lower bounded package versions that we dont need specific versions of

* lower bound gitpython

Former-commit-id: c4ff426a95
2020-03-26 19:57:44 +00:00
Chenhui Hu 2be76e0b80 Chenhui/improve doc (#168)
* minor changes

* remove redundancy

* updated text

* improved text in model tuning and deployment notebook

* clarify the data used

* updated text

* added description of the script

* add explanation of gaps in the curve

* add explanation of gaps in the curve

* updated text

* fix typos

* improve documentation and format

Former-commit-id: 8a21172c06
2020-03-26 10:58:25 -04:00
vapaunic f9a261a89c modified arima multiround to run with ray (#167)
Former-commit-id: 16c00da4d6
2020-03-26 08:13:05 +00:00
Chenhui Hu c49d433426 chenhui/improve ray output (#166)
Former-commit-id: 05dd83ab63
2020-03-25 18:12:09 -04:00
Markus Cozowicz d0c9fd81e1 improvement: using Ray to parallelize arima fitting (#159)
* using Ray to parallelize arima fitting

* added ray as dependency

* text about ray, disable warnings, and minor stuff

* scipy 1.4.1 or above

* reverting scipy, azuremlsdk issue

* minor mod

Co-authored-by: Vanja Paunic <15053814+vapaunic@users.noreply.github.com>
Former-commit-id: 55dabffee2
2020-03-25 21:04:23 +00:00
vapaunic f8cc14a450 modified metrics to take any iterable (#158)
Former-commit-id: bed41898be
2020-03-25 07:10:25 +00:00
Chenhui Hu 21c44cd709 Chenhui/code improvments (#157)
* updated docstring

* pinged package versions

* minor improvements

* minor improvement

Former-commit-id: 43a423eacf
2020-03-24 17:52:47 -04:00
vapaunic 2657c1c7bb Vapaunic/arimaint (#154)
* modified conftests to add arima

* added tests

* modified notebooks with parameters


Former-commit-id: e6d47ee770
2020-03-24 15:31:36 +00:00
Ubuntu 2866b95a25 Chenhui/add CI tests for notebooks
Former-commit-id: 47c6ac5f6f
2020-03-24 03:24:48 +00:00
Ubuntu 3eca65c20e Vapaunic/featutils (#137)
Former-commit-id: 0dff37913b
2020-03-24 03:08:38 +00:00
Ubuntu bac03209b7 Chenhui/windows setup (#131)
Former-commit-id: 57d0252a2b
2020-03-24 02:53:54 +00:00
Ubuntu c0e5ae5165 Readds R material (#116)
Former-commit-id: bb7f71f33d
2020-03-24 02:30:06 +00:00
Ubuntu e10955e5bc add ignored items
Former-commit-id: 4bf358d995
2020-03-24 00:30:52 +00:00
Ubuntu 1fbf16f203 Update component governance pipeline
Former-commit-id: 3a752ed9c2
2020-03-24 00:19:23 +00:00
Ubuntu 58ea7002e1 Update component governance pipeline
Former-commit-id: 21a3ddb148
2020-03-24 00:06:13 +00:00
Ubuntu 0f7bb429e2 Added CI tests for example notebooks
Former-commit-id: 30a69f52fa
2020-03-23 21:59:34 +00:00
Ubuntu ecac9d3f53 minor update
Former-commit-id: b3a8ac5243
2020-03-23 21:50:31 +00:00