Chenhui/add CI tests for notebooks

Former-commit-id: 47c6ac5f6f

examples/README.md

@@ -1,8 +1,8 @@
 # Forecasting examples
 
-This folder contains Python and R examples for building forecasting solutions presented in Python Jupyter notebooks and R Markdown files, respectively. The examples are organized according to forecasting scenarios in different use cases with each subdirectory under `examples/` named after the specific use case. 
+This folder contains Python and R examples for building forecasting solutions presented in Python Jupyter notebooks and R Markdown files, respectively. The examples are organized according to forecasting scenarios in different use cases with each subdirectory under `examples/` named after the specific use case.
 
-At the moment, the repository contains a single retail sales forecasting scenario utilizing [Dominick's OrangeJuice data set](https://www.chicagobooth.edu/research/kilts/datasets/dominicks). The name of the directory is `grocery_sales`.
+At the moment, the repository contains a single retail sales forecasting scenario utilizing [Dominick's OrangeJuice data set](https://www.chicagobooth.edu/research/kilts/datasets/dominicks). The name of the directory is `grocery_sales`. 
 
 
 ## Summary

examples/grocery_sales/R/01_dataprep.Rmd

@@ -4,7 +4,7 @@ output: html_notebook
 ---
 
 _Copyright (c) Microsoft Corporation._<br/>
-_Licensed under the MIT License._
+_Licensed under the MIT License._ 
 
 In this notebook, we generate the datasets that will be used for model training and validating. 
 

examples/grocery_sales/R/02_basic_models.Rmd

@@ -4,7 +4,7 @@ output: html_notebook
 ---
 
 _Copyright (c) Microsoft Corporation._<br/>
-_Licensed under the MIT License._
+_Licensed under the MIT License._ 
 
 ```{r, echo=FALSE, results="hide", message=FALSE}
 library(tidyr)

examples/grocery_sales/R/02a_reg_models.Rmd

@@ -4,7 +4,7 @@ output: html_notebook
 ---
 
 _Copyright (c) Microsoft Corporation._<br/>
-_Licensed under the MIT License._
+_Licensed under the MIT License._ 
 
 ```{r, echo=FALSE, results="hide", message=FALSE}
 library(tidyr)

examples/grocery_sales/R/02b_prophet_models.Rmd

@@ -4,7 +4,7 @@ output: html_notebook
 ---
 
 _Copyright (c) Microsoft Corporation._<br/>
-_Licensed under the MIT License._
+_Licensed under the MIT License._ 
 
 ```{r, echo=FALSE, results="hide", message=FALSE}
 library(tidyr)

examples/grocery_sales/R/README.md

@@ -1,6 +1,6 @@
 # Forecasting examples in R: orange juice retail sales
 
-The Rmarkdown notebooks in this directory are as follows. Each notebook also has a corresponding HTML file, which is the rendered output from running the code.
+The Rmarkdown notebooks in this directory are as follows. Each notebook also has a corresponding HTML file, which is the rendered output from running the code. 
 
 - [`01_dataprep.Rmd`](01_dataprep.Rmd) creates the training and test datasets
 - [`02_basic_models.Rmd`](02_basic_models.Rmd) fits a range of simple time series models to the data, including ARIMA and ETS.

examples/grocery_sales/R/forecast_settings.yaml

@@ -3,4 +3,4 @@ HORIZON: 2
 GAP: 2
 FIRST_WEEK: 40
 LAST_WEEK: 156
-START_DATE: "1989-09-14"
+START_DATE: "1989-09-14"

examples/grocery_sales/README.md

@@ -1,6 +1,6 @@
 # Forecasting examples
 
-This folder contains Python and R examples for building forecasting solutions on the Orange Juice dataset which is part of the [Dominick's dataset](https://www.chicagobooth.edu/research/kilts/datasets/dominicks). The examples are presented in Python Jupyter notebooks and R Markdown files, respectively.
+This folder contains Python and R examples for building forecasting solutions on the Orange Juice dataset which is part of the [Dominick's dataset](https://www.chicagobooth.edu/research/kilts/datasets/dominicks). The examples are presented in Python Jupyter notebooks and R Markdown files, respectively. 
 
 
 ## Orange Juice Dataset
@@ -19,8 +19,8 @@ Note that the week number starts from 40 in this dataset, while the full Dominic
 
 The following summarizes each directory of the forecasting examples.
 
-| Directory | Content | Description |
-| --- | --- | --- |
-| [python](./python)| [00_quick_start/](./python/00_quick_start) <br>[01_prepare_data/](./python/01_prepare_data) <br> [02_model/](./python/02_model) <br> [03_model_tune_deploy/](./python/03_model_tune_deploy/) | <ul> <li> Quick start examples for single-round training </li> <li> Data exploration and preparation notebooks </li> <li> Multi-round training examples </li> <li> Model tuning and deployment example </li> </ul> |
-| [R](./R) | [01_dataprep.Rmd](R/01_dataprep.Rmd) <br> [02_basic_models.Rmd](R/02_basic_models.Rmd) <br> [02a_reg_models.Rmd](R/02a_reg_models.Rmd) <br> [02b_prophet_models.Rmd](R/02b_prophet_models.Rmd) | <ul> <li>Data preparation</li> <li>Basic time series models</li> <li>ARIMA-regression models</li> <li>Prophet models</li> </ul> |
+| Directory          | Content                                                                                                                                                                                        | Description                                                                                                                                                                                                        |
+|--------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [python](./python) | [00_quick_start/](./python/00_quick_start) <br>[01_prepare_data/](./python/01_prepare_data) <br> [02_model/](./python/02_model) <br> [03_model_tune_deploy/](./python/03_model_tune_deploy/)   | <ul> <li> Quick start examples for single-round training </li> <li> Data exploration and preparation notebooks </li> <li> Multi-round training examples </li> <li> Model tuning and deployment example </li> </ul> |
+| [R](./R)           | [01_dataprep.Rmd](R/01_dataprep.Rmd) <br> [02_basic_models.Rmd](R/02_basic_models.Rmd) <br> [02a_reg_models.Rmd](R/02a_reg_models.Rmd) <br> [02b_prophet_models.Rmd](R/02b_prophet_models.Rmd) | <ul> <li>Data preparation</li> <li>Basic time series models</li> <li>ARIMA-regression models</li> <li>Prophet models</li> </ul>                                                                                    |
 

examples/grocery_sales/python/00_quick_start/autoarima_single_round.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i> "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/00_quick_start/azure_automl_single_round.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i>  "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/00_quick_start/lightgbm_single_round.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i>  "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/01_prepare_data/ojdata_exploration.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i> "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/01_prepare_data/ojdata_preparation.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i> "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/02_model/autoarima_multi_round.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i> "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/02_model/dilatedcnn_multi_round.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i>  "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/02_model/lightgbm_multi_round.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i> "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/03_model_tune_deploy/aml_scripts/train_validate.py

@@ -1,5 +1,5 @@
 # Copyright (c) Microsoft Corporation.
-# Licensed under the MIT License.  
+# Licensed under the MIT License. 
 
 """
 Perform cross validation of a LightGBM forecasting model on the training data of the 1st forecast round.

examples/grocery_sales/python/03_model_tune_deploy/azure_hyperdrive_lightgbm.ipynb

@@ -6,7 +6,7 @@
    "source": [
     "<i>Copyright (c) Microsoft Corporation.</i>\n",
     "\n",
-    "<i>Licensed under the MIT License.</i>  "
+    "<i>Licensed under the MIT License.</i>"
    ]
   },
   {

examples/grocery_sales/python/README.md

@@ -1,16 +1,16 @@
 # Forecasting examples in Python
 
-This folder contains Jupyter notebooks with Python examples for building forecasting solutions. To run the notebooks, please ensure your environment is set up with required dependencies by following instructions in the [Setup guide](../../../docs/SETUP.md). 
+This folder contains Jupyter notebooks with Python examples for building forecasting solutions. To run the notebooks, please ensure your environment is set up with required dependencies by following instructions in the [Setup guide](../../../docs/SETUP.md).  
 
 
 ## Summary
 
 The following summarizes each directory of the Python best practice notebooks.
 
-| Directory | Content | Description |
-| --- | --- | --- |
-| [00_quick_start](./00_quick_start)| [autoarima_single_round.ipynb](./00_quick_start/autoarima_single_round.ipynb) <br>[azure_automl_single_round.ipynb](./00_quick_start/azure_automl_single_round.ipynb) <br> [lightgbm_single_round.ipynb](./00_quick_start/lightgbm_single_round.ipynb) | Quick start notebooks that demonstrate workflow of developing a forecasting model using one-round training and testing data|
-| [01_prepare_data](./01_prepare_data) | [ojdata_exploration.ipynb](./01_prepare_data/ojdata_exploration.ipynb) <br> [ojdata_preparation.ipynb](./01_prepare_data/ojdata_preparation.ipynb) | Data exploration and preparation notebooks|
-| [02_model](./02_model) | [dilatedcnn_multi_round.ipynb](./02_model/dilatedcnn_multi_round.ipynb) <br> [lightgbm_multi_round.ipynb](./02_model/lightgbm_multi_round.ipynb) <br> [autoarima_multi_round.ipynb](./02_model/autoarima_multi_round.ipynb) | Deep dive notebooks that perform multi-round training and testing of various classical and deep learning forecast algorithms|
-| [03_model_tune_deploy](./03_model_tune_deploy/) | [azure_hyperdrive_lightgbm.ipynb](./03_model_tune_deploy/azure_hyperdrive_lightgbm.ipynb) <br> [aml_scripts/](./03_model_tune_deploy/aml_scripts) | <ul><li> Example notebook for model tuning using Azure Machine Learning Service and deploying the best model on Azure </ul></li> <ul><li> Scripts for model training and validation </ul></li> |
+| Directory                                       | Content                                                                                                                                                                                                                                                | Description                                                                                                                                                                                    |
+|-------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [00_quick_start](./00_quick_start)              | [autoarima_single_round.ipynb](./00_quick_start/autoarima_single_round.ipynb) <br>[azure_automl_single_round.ipynb](./00_quick_start/azure_automl_single_round.ipynb) <br> [lightgbm_single_round.ipynb](./00_quick_start/lightgbm_single_round.ipynb) | Quick start notebooks that demonstrate workflow of developing a forecasting model using one-round training and testing data                                                                    |
+| [01_prepare_data](./01_prepare_data)            | [ojdata_exploration.ipynb](./01_prepare_data/ojdata_exploration.ipynb) <br> [ojdata_preparation.ipynb](./01_prepare_data/ojdata_preparation.ipynb)                                                                                                     | Data exploration and preparation notebooks                                                                                                                                                     |
+| [02_model](./02_model)                          | [dilatedcnn_multi_round.ipynb](./02_model/dilatedcnn_multi_round.ipynb) <br> [lightgbm_multi_round.ipynb](./02_model/lightgbm_multi_round.ipynb) <br> [autoarima_multi_round.ipynb](./02_model/autoarima_multi_round.ipynb)                            | Deep dive notebooks that perform multi-round training and testing of various classical and deep learning forecast algorithms                                                                   |
+| [03_model_tune_deploy](./03_model_tune_deploy/) | [azure_hyperdrive_lightgbm.ipynb](./03_model_tune_deploy/azure_hyperdrive_lightgbm.ipynb) <br> [aml_scripts/](./03_model_tune_deploy/aml_scripts)                                                                                                      | <ul><li> Example notebook for model tuning using Azure Machine Learning Service and deploying the best model on Azure </ul></li> <ul><li> Scripts for model training and validation </ul></li> |
 

tests/ci/component_governance.yml

@@ -1,5 +1,5 @@
 # Copyright (c) Microsoft Corporation. All rights reserved.
-# Licensed under the MIT License.
+# Licensed under the MIT License. 
 
 # Pull request against these branches will trigger this build
 pr: 

tests/ci/cpu_integration_tests_linux.yml

@@ -1,5 +1,5 @@
 # Copyright (c) Microsoft Corporation.
-# Licensed under the MIT License.
+# Licensed under the MIT License. 
 
 # Pull request against these branches will trigger this build
 pr:

tests/ci/cpu_unit_tests_linux.yml

@@ -1,5 +1,5 @@
 # Copyright (c) Microsoft Corporation.
-# Licensed under the MIT License.
+# Licensed under the MIT License. 
 
 # Pull request against these branches will trigger this build
 pr:

tests/ci/python_unit_tests_base.yml

@@ -1,31 +0,0 @@
-# Pull request against these branches will trigger this build
-#pr:
-#- staging
-#- master
-
-# Any commit to these branches will trigger the build.
-#trigger:
-#- staging
-#- master
-
-pool:
-  name: ForecastingBP
-  vmImage: "forecastingtestmachine"
-
-steps:
-
-- bash: |
-    echo "##vso[task.prependpath]/data/anaconda/bin"
-  displayName: Add conda to PATH
-
-- bash: |
-    . /anaconda/etc/profile.d/conda.sh && \
-    conda activate forecasting_base && \
-    pytest tests/python/unit -m "not notebooks and not spark and not gpu" --junitxml=reports/python-unit-tests-base.xml && \
-    conda deactivate
-  displayName: "Run unit tests"
-
-- task: PublishTestResults@2
-  inputs:
-    testResultsFiles: "reports/python-unit-tests-base.xml"
-    testRunTitle: "Test results of unit tests"

tests/ci/r_unit_tests_base.yml

@@ -1,31 +0,0 @@
-# Pull request against these branches will trigger this build
-#pr:
-#- staging
-#- master
-
-# Any commit to these branches will trigger the build.
-#trigger:
-#- staging
-#- master
-
-pool:
-  name: ForecastingBP
-  vmImage: 'forecastingtestmachine'
-
-steps:
-
-- bash: |
-    echo "##vso[task.prependpath]/data/anaconda/bin"
-  displayName: Add conda to PATH
-
-- bash: |
-    . /anaconda/etc/profile.d/conda.sh && \
-    conda activate forecasting_base && \
-    Rscript tests/unit/source_entire.R -m "not notebooks and not spark and not gpu" --junitxml=reports/test-unit.xml && \
-    conda deactivate
-  displayName: 'Run R unit tests'
-
-- task: PublishTestResults@2
-  inputs:
-    testResultsFiles: '**/test-unitttest.xml'
-    testRunTitle: 'Test results for R Unit Tests'

tests/conftest.py

@@ -1,3 +1,6 @@
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+
 import os
 import pytest
 from fclib.common.utils import git_repo_path

tests/integration/test_notebooks_python.py

@@ -1,5 +1,5 @@
 # Copyright (c) Microsoft Corporation.
-# Licensed under the MIT License.
+# Licensed under the MIT License. 
 
 import os
 import pytest

tests/pytest.ini

@@ -1,4 +1,4 @@
 [pytest]
 markers = 
     notebooks
-    integration
+    integration