Merged PR 4472: Updated CI build to include Windows and check for unit tests

Updated CI build to include Windows and check for unit tests
This commit is contained in:
David Brownell 2019-05-29 17:29:56 +00:00
Родитель c4ef830b65
Коммит 8f29ac010f
6 изменённых файлов: 64 добавлений и 162 удалений

Просмотреть файл

@ -27,8 +27,11 @@ stages:
# In the UX, this display name contains redundant info (the OS and config appear twice). However, it
# needs to be here to ensure that the information appears in the status email messages.
displayName: "Build and Test (${{ parameters.operating_system }} - ${{ parameters.configuration }})"
pool: "${{ parameters.agent_pool }}"
pool:
vmImage: "${{ parameters.agent_pool }}"
workspace:
clean: all
steps:
# These steps are an ugly hack to set Azure DevOps variables conditionally. It really seems like there should be a better way to do this.
- ${{ if eq(parameters.operating_system, 'Windows') }}:
@ -48,7 +51,7 @@ stages:
echo ##vso[task.setvariable variable=azure_agent_temp_directory]%AGENT_TEMPDIRECTORY%
echo ##vso[task.setvariable variable=azure_artifacts_directory]%BUILD_ARTIFACTSTAGINGDIRECTORY%
displayName: "[Implementation] Environment-Specific Variables"
displayName: "[IMPL] Environment-Specific Variables"
- ${{ if eq(parameters.operating_system, 'Linux') }}:
- script: |-
@ -67,7 +70,7 @@ stages:
echo "##vso[task.setvariable variable=azure_agent_temp_directory]${AGENT_TEMPDIRECTORY}"
echo "##vso[task.setvariable variable=azure_artifacts_directory]${BUILD_ARTIFACTSTAGINGDIRECTORY}"
displayName: "[Implementation] Environment-Specific Variables"
displayName: "[IMPL] Environment-Specific Variables"
# Continue with standard processing
- script: |-
@ -88,20 +91,26 @@ stages:
- script: |-
$(azure_bootstrap_command) $(azure_agent_temp_directory) /verbose
displayName: "Bootstrap"
displayName: "<Bootstrap>"
timeoutInMinutes: 180
# TODO: Code formatting
- script: |-
$(azure_activate_script) ${{ parameters.configuration }} && Builder$(azure_script_extension) Execute . "$(azure_artifacts_directory)/Builder" /verbose
displayName: "Builder"
displayName: "<Builder>"
timeoutInMinutes: 180
- script: |-
$(azure_activate_script) ${{ parameters.configuration }} && Tester$(azure_script_extension) MatchAllTests . UnitTests /verbose
displayName: "<Tester (MatchAllTests) - UnitTests>"
timeoutInMinutes: 1800
condition: succeededOrFailed()
- ${{ each test_type in parameters.test_types }}:
- script: |-
$(azure_activate_script) ${{ parameters.configuration }} && Tester$(azure_script_extension) TestAll . "$(azure_artifacts_directory)/Tester/${{ test_type.key }}" ${{ test_type.key }} ${{ test_type.value }} /verbose
displayName: 'Tester - ${{ test_type.key }}'
displayName: '<Tester - ${{ test_type.key }}>'
timeoutInMinutes: 180
condition: succeededOrFailed()

Просмотреть файл

@ -6,26 +6,26 @@ trigger:
- master
stages:
# TODO: - template: BuildAndTest.template.yaml
# TODO: parameters:
# TODO: agent_pool: "Hosted Windows 2019 with VS2019"
# TODO: operating_system: Windows
# TODO: configuration: python
# TODO:
# TODO: - template: BuildAndTest.template.yaml
# TODO: parameters:
# TODO: agent_pool: "Hosted Windows 2019 with VS2019"
# TODO: operating_system: Windows
# TODO: configuration: x64
- template: BuildAndTest.template.yaml
parameters:
agent_pool: vs2015-win2012r2
operating_system: Windows
configuration: python
- template: BuildAndTest.template.yaml
parameters:
agent_pool: "Hosted Ubuntu 1604"
agent_pool: vs2015-win2012r2
operating_system: Windows
configuration: x64
- template: BuildAndTest.template.yaml
parameters:
agent_pool: ubuntu-16.04
operating_system: Linux
configuration: python
- template: BuildAndTest.template.yaml
parameters:
agent_pool: "Hosted Ubuntu 1604"
agent_pool: ubuntu-16.04
operating_system: Linux
configuration: x64
configuration: x64

28
Readme.md Normal file
Просмотреть файл

@ -0,0 +1,28 @@
Data Pipelines
==============
*Data Pipelines* are used to convert from arbitrary data into a matrix/tensor that can be consumed by machine learning algorithms.
Developer Quick Start
---------------------
Note that these commands should be invoked within the root of the repository.
**Linux**
1) Run `sudo bootstrap.sh ..`
This will need to happen once per machine or after a repository has moved to a different directory. Note that `sudo` is not necessary when running within a Docker container.
2) Run `source ./Activate.sh <python|x64>`
This will need to happen each time a new terminal window is opened.
**Windows**
1) Run `bootstrap.cmd ..`
This will need to happen once per machine or after a repository has moved to a different directory.
2) Run `Activate.cmd <python|x64>`
This will need to happen each time a new terminal window is opened.

Просмотреть файл

@ -1,132 +0,0 @@
=======================
<TODO: Repository Name>
=======================
<TODO: Repository Description>. In this template file, all text to be replaced is delimited by ``<TODO: ...>``.
Contents
========
#. `Quick Start`_
#. License_
#. `Supported Platforms`_
#. Definitions_
#. Functionality_
#. `Docker Images`_
#. Dependencies_
#. Support_
Quick Start
===========
Setup_ and Activate_ are required to begin using this repository. Before running these scripts, please make sure that all Dependencies_ have been cloned.
.. _Setup:
Setup
Setup installs/unpacks tools used during development activities and locates its repository dependencies (if any). Setup must be run on your machine after cloning the repository or after changing the file location of repositories that it depends upon (if any).
==================================== =====================================================
Linux ``Setup.sh``
Windows ``Setup.cmd``
Windows (PowerShell) ``Setup.ps1``
==================================== =====================================================
.. _Activate:
Activate
Activate prepares the current environment for development activities and must be run at least once in each terminal window.
==================================== =====================================================
Linux ``Activate.sh <TODO: Update configurations>``
Windows ``Activate.cmd <TODO: Update configurations>``
Windows (PowerShell) ``Activate.ps1 <TODO: Update configurations>``
==================================== =====================================================
License
=======
<TODO: Update for your repo>
This repository is licensed under the `Boost Software License <https://www.boost.org/LICENSE_1_0.txt>`_.
`GitHub <https://github.com>`_ describes this license as:
A simple permissive license only requiring preservation of copyright and license notices for source (and not binary) distribution. Licensed works, modifications, and larger works may be distributed under different terms and without source code.
This repository distributes the following software:
======================================== =========================================
Software License
======================================== =========================================
`Python <https://www.python.org>`_ `PSF License Agreement <https://docs.python.org/3/license.html>`_
======================================== =========================================
Supported Platforms
===================
<TODO: Update for your repo>
This software has been verified on the following platforms.
======================== ====================== =========================================
Platform Scripting Environment Version
======================== ====================== =========================================
Windows Cmd.exe Windows 10 April 2018 Update
Windows PowerShell Windows 10 April 2018 Update
Linux Bash Ubuntu 18.04, 16.04
======================== ====================== =========================================
Definitions
===========
<TODO: Update for your repo>
.. _`Term 1`:
Term 1
Definition
.. _`Term 2`:
Term 2
Definition
Functionality
=============
<TODO: Update for your repo>
.. _`Feature 1`:
Feature 1
Description
.. _`Feature 2`:
Feature 2
Description
Docker Images
=============
Docker images of this repository are generated with updates.
<TODO: Update for your repo>
Dependencies
============
<TODO: Update for your repo>
This repository is dependent upon these repositories.
============================== =================================
Repo Name Description
============================== =================================
Common Environment Common development activities
============================== =================================
Related Repositories
--------------------
============================== =================================
Repo Name Description
============================== =================================
Common Environment Common development activities
============================== =================================
Support
=======
For question or issues, please visit <TODO: Your url>.

Просмотреть файл

@ -15,7 +15,7 @@
@REM ----------------------------------------------------------------------
@echo off
IF "%1"=="" (
IF "%~1"=="" (
echo.
echo This script bootstraps common library enlistment and setup.
echo.
@ -26,7 +26,7 @@ IF "%1"=="" (
exit /B -1
)
set _COMMON_CODE_DIR=%1
set _COMMON_CODE_DIR=%~1
shift /1
if "%DEVELOPMENT_ENVIRONMENT_REPOSITORY_ACTIVATED_FLAG%" NEQ "" (
@ -52,9 +52,9 @@ set _BOOTSTRAP_CLA=
:GetRemainingArgs_Begin
if "%1"=="" goto :GetRemainingArgs_End
if "%~1"=="" goto :GetRemainingArgs_End
set _ARG=%1
set _ARG=%~1
if "%_ARG:~,9%"=="/name_EQ_" goto :GetRemainingArgs_Name
if "%_ARG:~,9%"=="-name_EQ_" goto :GetRemainingArgs_Name

Просмотреть файл

@ -95,11 +95,9 @@ def EntryPoint(
for index, (output_dir, data) in enumerate(enlistment_repositories):
enlist_dm.stream.write("'{}' ({} of {})...".format(data[0], index + 1, len(enlistment_repositories)))
with enlist_dm.stream.DoneManager() as this_dm:
temp_directory = CurrentShell.CreateTempDirectory()
FileSystem.MakeDirs(os.path.dirname(output_dir))
command_line = data[1].format(
output_dir=temp_directory,
)
temp_directory = output_dir + "_tmp"
sink = six.moves.StringIO()
@ -123,7 +121,6 @@ def EntryPoint(
return this_dm.result
FileSystem.MakeDirs(os.path.dirname(output_dir))
shutil.move(temp_directory, output_dir)
dm.stream.write("Setting up {}...".format(inflect.no("repository", len(repo_data))))