* Python 3.8 and pybind11 (#493)

* First steps to enable python 3.7 on windows + use pybind11 instead of boost_python

* Add Python 3.7 for Linux, Mac, remove unnecessary dependencies to boost python

* install pybind11

* fix build

* Fix build on windows

* Update build.cmd

* Append None into list instead of empty objects, translate C++ exception

* fix wrong cast with string

* Fix issues with columns names having unicode characters.

* Update build.cmd

* update build.cmd

* update build.cmd

* Fix one issue with sparse data

* Complete merge

* Update dllmain.cpp

* Update dllmain.cpp

* Quick modifications

* Fix issue with sparse data when switching to pybind11

* Fix one final unit test

* update CI

* update build ci

* fix CI and compilation issues

* Update DataViewInterop.cpp

* Update dllmain.cpp

* add configuration for python 3.7

* fix broken unit test

* Update build.sh

* fix build for Windows

* Linux py3.7 build

* fix pytest version

* upgrade pytest

* fix pytest-cov version

* fix isinstance(., int) for python 2.7

* fix merge issues

* use BOOST_PYTHON for all releases

* fix iteration issue

* Update build.sh

* use custom python

* Update phase-template.yml

* update CI

* Update phase-template.yml

* update CI

* fix CI

* update CI

* Update .vsts-ci.yml

* update python versio

* update CI

* Update .vsts-ci.yml

* Update .vsts-ci.yml

* Update phase-template.yml

* Update phase-template.yml

* Update phase-template.yml

* update CI

* fix paths

* Update build.sh

* fix linux build

* Update phase-template.yml

* Update phase-template.yml

* Update .vsts-ci.yml

* Update build.sh

* Update build.sh

* Update build.sh

* update build

* Update test_estimator_checks.py

* initial commit

* 'fix'

* merge

* Remove boost

* merge

* merge

* pybind11 install

* more pybind11 port

* up version to 1.8.0

* Remove python 2.7
Remove coverage
Fix tests
Upgrade Featurizers lib

* Remove boost

* fix python path

* remove boost libs

* Remove boost & py2.7 for Lin/Mac

* fix Lin build

* fix lin build

* fix mac build

* fix Lin build

* fix libc install

* fix build

* fix linbuild

* remove isnan

* Add python 3.8 build pieces

* fix win build

* fix py 3.8 build

* fix linux build
update setup

* remove platform dependency
use distro instead

* fix python url for linux

* Fix linux python 3.8 build

* linux build

* fix path

* Rollback to preview2

* fix build

* fix build

* build

* fix build

* 'fix'

* 'fix'

* 'sudo'

* 'build'

* 'test'

* 'test'

* 'test'

* 'test'

* 'test'

* fix tests

* fix tests

* fix tests path for 3.8

* fix tests

* fix mac

* fix linux tests

* fix mac

* fix mac tests

* run as root

* fix mac tests

Co-authored-by: xavier dupré <xavier.dupre@gmail.com>
Co-authored-by: Gani Nazirov <ganaziro@microsoft.com>
Co-authored-by: Admin <admin@Admins-MacBook-Pro.local>

* fix comments

* fix build

* fix build

Co-authored-by: xavier dupré <xavier.dupre@gmail.com>
Co-authored-by: Gani Nazirov <ganaziro@microsoft.com>
Co-authored-by: Admin <admin@Admins-MacBook-Pro.local>
This commit is contained in:
Gani Nazirov 2020-06-17 20:25:09 -07:00 коммит произвёл GitHub
Родитель f9218dc5e8
Коммит f6be39ce93
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
45 изменённых файлов: 560 добавлений и 794 удалений

Просмотреть файл

@ -6,12 +6,12 @@ phases:
name: Windows
buildScript: build.cmd
buildMatrix:
Py38:
_configuration: RlsWinPy3.8
Py37:
_configuration: RlsWinPy3.7
Py36:
_configuration: RlsWinPy3.6
Py35:
_configuration: RlsWinPy3.5
buildQueue:
name: Hosted VS2017
testOptions: --includeExtendedTests
@ -22,6 +22,8 @@ phases:
name: Mac
buildScript: ./build.sh
buildMatrix:
Py38:
_configuration: RlsMacPy3.8
Py37:
_configuration: RlsMacPy3.7
buildQueue:
@ -29,7 +31,20 @@ phases:
testOptions: --includeExtendedTests
# Build all configurations for Linux
# Run tests on Ubuntu16
# Run build & tests on Ubuntu16 for Python3.8
- template: /build/ci/phase-template.yml
parameters:
name: Linux_Python38
buildScript: ./build.sh
testDistro: python38
buildMatrix:
Py38:
_configuration: RlsLinPy3.8
buildQueue:
name: Hosted Ubuntu 1604
testOptions: --includeExtendedTests
# Run build & tests on Ubuntu16
- template: /build/ci/phase-template.yml
parameters:
name: Linux_Ubuntu16
@ -43,15 +58,15 @@ phases:
buildQueue:
name: Hosted Ubuntu 1604
testOptions: --includeExtendedTests
# Run tests on CentOS7
# Run build & tests on CentOS7
- template: /build/ci/phase-template.yml
parameters:
name: Linux_CentOS7
buildScript: ./build.sh
testDistro: centos7
buildMatrix:
Py27:
_configuration: RlsLinPy2.7
Py37:
_configuration: RlsLinPy3.7
buildQueue:
name: Hosted Ubuntu 1604

151
build.cmd
Просмотреть файл

@ -8,18 +8,16 @@ set __currentScriptDir=%~dp0
set DependenciesDir=%__currentScriptDir%dependencies\
if not exist "%DependenciesDir%" (md "%DependenciesDir%")
:: Default configuration if no arguents passed to build.cmd (DbgWinPy3.7)
:: Default configuration if no arguents passed to build.cmd (DbgWinPy3.8)
set __BuildArch=x64
set __VCBuildArch=x86_amd64
set Configuration=DbgWinPy3.7
set Configuration=DbgWinPy3.8
set DebugBuild=True
set BuildOutputDir=%__currentScriptDir%x64\
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.7.3-amd64.zip
set PythonRoot=%DependenciesDir%Python3.7
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/debug/windows/Boost-3.7-1.69.0.0.zip
set BoostRoot=%DependenciesDir%BoostDbg3.7
set PythonVersion=3.7
set PythonTag=cp37
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.8.3-amd64.zip
set PythonRoot=%DependenciesDir%Python3.8
set PythonVersion=3.8
set PythonTag=cp38
set RunTests=False
set InstallPythonPackages=False
set RunExtendedTests=False
@ -69,7 +67,7 @@ if /i [%1] == [--azureBuild] (
echo "Usage: build.cmd [--configuration <Configuration>] [--runTests] [--installPythonPackages] [--includeExtendedTests] [--buildDotNetBridgeOnly] [--skipDotNetBridge] [--azureBuild]"
echo ""
echo "Options:"
echo " --configuration <Configuration> Build Configuration (DbgWinPy3.7,DbgWinPy3.6,DbgWinPy3.5,DbgWinPy2.7,RlsWinPy3.7,RlsWinPy3.6,RlsWinPy3.5,RlsWinPy2.7)"
echo " --configuration <Configuration> Build Configuration (DbgWinPy3.8,DbgWinPy3.7,DbgWinPy3.6,RlsWinPy3.8, RlsWinPy3.7,RlsWinPy3.6)"
echo " --runTests Run tests after build"
echo " --installPythonPackages Install python packages after build"
echo " --includeExtendedTests Include the extended tests if the tests are run"
@ -80,13 +78,20 @@ echo " --azureBuild Building in azure devops (adds dotnet
goto :Exit_Success
:Configuration
if /i [%1] == [RlsWinPy3.8] (
set DebugBuild=False
set Configuration=RlsWinPy3.8
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.8.3-amd64.zip
set PythonRoot=%DependenciesDir%Python3.8
set PythonVersion=3.8
set PythonTag=cp38
shift && goto :Arg_Loop
)
if /i [%1] == [RlsWinPy3.7] (
set DebugBuild=False
set Configuration=RlsWinPy3.7
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.7.3-amd64.zip
set PythonRoot=%DependenciesDir%Python3.7
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/windows/Boost-3.7-1.69.0.0.zip
set BoostRoot=%DependenciesDir%BoostRls3.7
set PythonVersion=3.7
set PythonTag=cp37
shift && goto :Arg_Loop
@ -96,32 +101,17 @@ if /i [%1] == [RlsWinPy3.6] (
set Configuration=RlsWinPy3.6
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.6.5-mohoov-amd64.zip
set PythonRoot=%DependenciesDir%Python3.6
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/windows/Boost-3.6-1.64.0.0.zip
set BoostRoot=%DependenciesDir%BoostRls3.6
set PythonVersion=3.6
set PythonTag=cp36
shift && goto :Arg_Loop
)
if /i [%1] == [RlsWinPy3.5] (
set DebugBuild=False
set Configuration=RlsWinPy3.5
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.5.4-mohoov-amd64.zip
set PythonRoot=%DependenciesDir%Python3.5
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/windows/Boost-3.5-1.64.0.0.zip
set BoostRoot=%DependenciesDir%BoostRls3.5
set PythonVersion=3.5
set PythonTag=cp35
shift && goto :Arg_Loop
)
if /i [%1] == [RlsWinPy2.7] (
set DebugBuild=False
set Configuration=RlsWinPy2.7
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-2.7.15-mohoov-amd64.zip
set PythonRoot=%DependenciesDir%Python2.7
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/windows/Boost-2.7-1.64.0.0.zip
set BoostRoot=%DependenciesDir%BoostRls2.7
set PythonVersion=2.7
set PythonTag=cp27
if /i [%1] == [DbgWinPy3.8] (
set DebugBuild=True
set Configuration=DbgWinPy3.8
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.8.3-amd64.zip
set PythonRoot=%DependenciesDir%Python3.8
set PythonVersion=3.8
set PythonTag=cp38
shift && goto :Arg_Loop
)
if /i [%1] == [DbgWinPy3.7] (
@ -129,8 +119,6 @@ if /i [%1] == [DbgWinPy3.7] (
set Configuration=DbgWinPy3.7
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.7.3-amd64.zip
set PythonRoot=%DependenciesDir%Python3.7
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/debug/windows/Boost-3.7-1.69.0.0.zip
set BoostRoot=%DependenciesDir%BoostDbg3.7
set PythonVersion=3.7
set PythonTag=cp37
shift && goto :Arg_Loop
@ -140,34 +128,10 @@ if /i [%1] == [DbgWinPy3.6] (
set Configuration=DbgWinPy3.6
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.6.5-mohoov-amd64.zip
set PythonRoot=%DependenciesDir%Python3.6
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/debug/windows/Boost-3.6-1.64.0.0.zip
set BoostRoot=%DependenciesDir%BoostDbg3.6
set PythonVersion=3.6
set PythonTag=cp36
shift && goto :Arg_Loop
)
if /i [%1] == [DbgWinPy3.5] (
set DebugBuild=True
set Configuration=DbgWinPy3.5
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.5.4-mohoov-amd64.zip
set PythonRoot=%DependenciesDir%Python3.5
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/debug/windows/Boost-3.5-1.64.0.0.zip
set BoostRoot=%DependenciesDir%BoostDbg3.5
set PythonVersion=3.5
set PythonTag=cp35
shift && goto :Arg_Loop
)
if /i [%1] == [DbgWinPy2.7] (
set DebugBuild=True
set Configuration=DbgWinPy2.7
set PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-2.7.15-mohoov-amd64.zip
set PythonRoot=%DependenciesDir%Python2.7
set BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/debug/windows/Boost-2.7-1.64.0.0.zip
set BoostRoot=%DependenciesDir%BoostDbg2.7
set PythonVersion=2.7
set PythonTag=cp27
shift && goto :Arg_Loop
)
:Build
:: Install dotnet SDK version, see https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-install-script
@ -188,7 +152,7 @@ echo "Building Managed code ... "
echo "#################################"
set _dotnet=%_dotnetRoot%\dotnet.exe
if "%Configuration:~-5%" == "Py3.7" set VerifyManifest=True
if "%Configuration:~-5%" == "Py3.6" set VerifyManifest=True
if "%VerifyManifest%" == "True" set BuildManifestGenerator=True
if "%UpdateManifest%" == "True" set BuildManifestGenerator=True
@ -244,16 +208,9 @@ if not exist "%PythonRoot%\.done" (
powershell.exe -nologo -noprofile -command "& { Add-Type -A 'System.IO.Compression.FileSystem'; [IO.Compression.ZipFile]::ExtractToDirectory('%DependenciesDir%python.zip', '%PythonRoot%'); }"
echo.>"%PythonRoot%\.done"
del %DependenciesDir%python.zip
)
:: Download & unzip Boost
if not exist "%BoostRoot%\.done" (
md "%BoostRoot%"
echo Downloading boost zip ...
powershell -command "& {$wc = New-Object System.Net.WebClient; $wc.DownloadFile('%BoostUrl%', '%DependenciesDir%boost.zip');}"
echo Extracting boost zip ...
powershell.exe -nologo -noprofile -command "& { Add-Type -A 'System.IO.Compression.FileSystem'; [IO.Compression.ZipFile]::ExtractToDirectory('%DependenciesDir%boost.zip', '%BoostRoot%'); }"
echo.>"%BoostRoot%\.done"
del %DependenciesDir%boost.zip
echo Python executable: %PythonRoot%\python.exe
echo "Installing pybind11 ..."
call "%PythonRoot%\python.exe" -m pip install pybind11
)
echo ""
@ -276,18 +233,30 @@ if not exist "%_VSCOMNTOOLS%" goto :MissingVersion
set "VSCMD_START_DIR=%__currentScriptDir%"
call "%_VSCOMNTOOLS%\VsDevCmd.bat"
if "%VisualStudioVersion%"=="15.0" (
if "%VisualStudioVersion%"=="16.0" (
goto :VS2019
) else if "%VisualStudioVersion%"=="15.0" (
goto :VS2017
) else if "%VisualStudioVersion%"=="14.0" (
goto :VS2015
) else goto :MissingVersion
:MissingVersion
:: Can't find VS 2015 or 2017
:: Can't find VS 2015 or 2017 or 2019
echo Error: Visual Studio 2015 or 2017 required
echo Please see https://github.com/dotnet/machinelearning/tree/master/Documentation for build instructions.
goto :Exit_Error
:VS2019
:: Setup vars for VS2019
set __PlatformToolset=v142
set __VSVersion=16 2019
if NOT "%__BuildArch%" == "arm64" (
:: Set the environment for the native build
call "%VS150COMNTOOLS%..\..\VC\Auxiliary\Build\vcvarsall.bat" %__VCBuildArch%
)
goto :NativeBridge
:VS2017
:: Setup vars for VS2017
set __PlatformToolset=v141
@ -335,7 +304,7 @@ md %libs%
echo.>"%__currentScriptDir%src\python\nimbusml\internal\libs\__init__.py"
if "%VerifyManifest%" == "True" (
:: Running the check in one python is enough. Entrypoint compiler doesn't run in py2.7.
:: Running the check in one python is enough.
echo Generating low-level Python API from mainifest.json ...
call "%PythonExe%" -m pip install --upgrade autopep8 autoflake isort jinja2
cd "%__currentScriptDir%src\python"
@ -351,21 +320,8 @@ echo Placing binaries in libs dir for wheel packaging
copy "%BuildOutputDir%%Configuration%\DotNetBridge.dll" "%__currentScriptDir%src\python\nimbusml\internal\libs\"
copy "%BuildOutputDir%%Configuration%\pybridge.pyd" "%__currentScriptDir%src\python\nimbusml\internal\libs\"
if %PythonVersion% == 2.7 (
copy "%BuildOutputDir%%Configuration%\Platform\win-x64\publish\*.dll" "%__currentScriptDir%src\python\nimbusml\internal\libs\"
xcopy /S /E /I "%BuildOutputDir%%Configuration%\Platform\win-x64\publish\Data" "%__currentScriptDir%src\python\nimbusml\internal\libs\Data"
:: remove dataprep dlls as its not supported in python 2.7
del "%__currentScriptDir%src\python\nimbusml\internal\libs\Microsoft.DPrep.*"
del "%__currentScriptDir%src\python\nimbusml\internal\libs\Microsoft.Data.*"
del "%__currentScriptDir%src\python\nimbusml\internal\libs\Microsoft.ProgramSynthesis.*"
del "%__currentScriptDir%src\python\nimbusml\internal\libs\Microsoft.DataPrep.dll"
del "%__currentScriptDir%src\python\nimbusml\internal\libs\ExcelDataReader.dll"
del "%__currentScriptDir%src\python\nimbusml\internal\libs\Microsoft.WindowsAzure.Storage.dll"
del "%__currentScriptDir%src\python\nimbusml\internal\libs\Microsoft.Workbench.Messaging.SDK.dll"
) else (
for /F "tokens=*" %%A in (build/libs_win.txt) do copy "%BuildOutputDir%%Configuration%\Platform\win-x64\publish\%%A" "%__currentScriptDir%src\python\nimbusml\internal\libs\"
xcopy /S /E /I "%BuildOutputDir%%Configuration%\Platform\win-x64\publish\Data" "%__currentScriptDir%src\python\nimbusml\internal\libs\Data"
)
for /F "tokens=*" %%A in (build/libs_win.txt) do copy "%BuildOutputDir%%Configuration%\Platform\win-x64\publish\%%A" "%__currentScriptDir%src\python\nimbusml\internal\libs\"
xcopy /S /E /I "%BuildOutputDir%%Configuration%\Platform\win-x64\publish\Data" "%__currentScriptDir%src\python\nimbusml\internal\libs\Data"
if "%DebugBuild%" == "True" (
copy "%BuildOutputDir%%Configuration%\DotNetBridge.pdb" "%__currentScriptDir%src\python\nimbusml\internal\libs\"
@ -393,14 +349,10 @@ if "%InstallPythonPackages%" == "True" (
echo "Installing python packages ... "
echo "#################################"
call "%PythonExe%" -m pip install --upgrade "pip==19.3.1"
call "%PythonExe%" -m pip install --upgrade nose pytest pytest-xdist graphviz imageio pytest-cov "jupyter_client>=4.4.0" "nbconvert>=4.2.0"
call "%PythonExe%" -m pip install --upgrade nose pytest pytest-xdist graphviz imageio "jupyter_client>=4.4.0" "nbconvert>=4.2.0"
if %PythonVersion% == 2.7 (
call "%PythonExe%" -m pip install --upgrade pyzmq
) else (
call "%PythonExe%" -m pip install --upgrade "azureml-dataprep>=1.1.33"
call "%PythonExe%" -m pip install --upgrade onnxruntime
)
call "%PythonExe%" -m pip install --upgrade "azureml-dataprep>=1.1.33"
call "%PythonExe%" -m pip install --upgrade onnxruntime
call "%PythonExe%" -m pip install --upgrade "%__currentScriptDir%target\%WheelFile%"
call "%PythonExe%" -m pip install "scikit-learn==0.19.2"
@ -419,25 +371,24 @@ set PackagePath=%PythonRoot%\Lib\site-packages\nimbusml
set TestsPath1=%PackagePath%\tests
set TestsPath2=%__currentScriptDir%src\python\tests
set TestsPath3=%__currentScriptDir%src\python\tests_extended
set ReportPath=%__currentScriptDir%build\TestCoverageReport
set NumConcurrentTests=%NUMBER_OF_PROCESSORS%
call "%PythonExe%" -m pytest -n %NumConcurrentTests% --verbose --maxfail=1000 --capture=sys "%TestsPath2%" "%TestsPath1%" --cov="%PackagePath%" --cov-report term-missing --cov-report html:"%ReportPath%"
call "%PythonExe%" -m pytest -n %NumConcurrentTests% --verbose --maxfail=1000 --capture=sys "%TestsPath2%" "%TestsPath1%"
if errorlevel 1 (
:: Rerun any failed tests to give them one more
:: chance in case the errors were intermittent.
call "%PythonExe%" -m pytest -n %NumConcurrentTests% --last-failed --verbose --maxfail=1000 --capture=sys "%TestsPath2%" "%TestsPath1%" --cov="%PackagePath%" --cov-report term-missing --cov-report html:"%ReportPath%"
call "%PythonExe%" -m pytest -n %NumConcurrentTests% --last-failed --verbose --maxfail=1000 --capture=sys "%TestsPath2%" "%TestsPath1%"
if errorlevel 1 (
goto :Exit_Error
)
)
if "%RunExtendedTests%" == "True" (
call "%PythonExe%" -m pytest -n %NumConcurrentTests% --verbose --maxfail=1000 --capture=sys "%TestsPath3%" --cov="%PackagePath%" --cov-report term-missing --cov-report html:"%ReportPath%"
call "%PythonExe%" -m pytest -n %NumConcurrentTests% --verbose --maxfail=1000 --capture=sys "%TestsPath3%"
if errorlevel 1 (
:: Rerun any failed tests to give them one more
:: chance in case the errors were intermittent.
call "%PythonExe%" -m pytest -n %NumConcurrentTests% --last-failed --verbose --maxfail=1000 --capture=sys "%TestsPath3%" --cov="%PackagePath%" --cov-report term-missing --cov-report html:"%ReportPath%"
call "%PythonExe%" -m pytest -n %NumConcurrentTests% --last-failed --verbose --maxfail=1000 --capture=sys "%TestsPath3%"
if errorlevel 1 (
goto :Exit_Error
)

218
build.sh
Просмотреть файл

@ -7,13 +7,11 @@ ProductVersion=$(<version.txt)
__currentScriptDir=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
BuildOutputDir=${__currentScriptDir}/x64/
DependenciesDir=${__currentScriptDir}/dependencies
PythonRoot=${DependenciesDir}/Python${PythonVersion}
BoostRoot=${DependenciesDir}/Boost${PythonVersion}
# Platform name for python wheel based on OS
PlatName=manylinux1_x86_64
if [ "$(uname -s)" = "Darwin" ]
then
PlatName=macosx_10_11_x86_64
PlatName=macosx_10_11_x86_64
fi
mkdir -p "${DependenciesDir}"
@ -22,7 +20,7 @@ usage()
echo "Usage: $0 --configuration <Configuration> [--runTests] [--includeExtendedTests] [--installPythonPackages]"
echo ""
echo "Options:"
echo " --configuration <Configuration> Build Configuration (DbgLinPy3.7,DbgLinPy3.6,DbgLinPy3.5,DbgLinPy2.7,RlsLinPy3.7,RlsLinPy3.6,RlsLinPy3.5,RlsLinPy2.7,DbgMacPy3.7,DbgMacPy3.6,DbgMacPy3.5,DbgMacPy2.7,RlsMacPy3.7,RlsMacPy3.6,RlsMacPy3.5,RlsMacPy2.7)"
echo " --configuration <Configuration> Build Configuration (DbgLinPy3.8, DbgLinPy3.7,DbgLinPy3.6,RlsLinPy3.8,RlsLinPy3.7,RlsLinPy3.6,DbgMacPy3.8,DbgMacPy3.7,DbgMacPy3.6,RlsMacPy3.8,RlsMacPy3.7,RlsMacPy3.6)"
echo " --runTests Run tests after build"
echo " --installPythonPackages Install python packages after build"
echo " --runTestsOnly Run tests on a wheel file in default build location (<repo>/target/)"
@ -35,9 +33,9 @@ usage()
# Parameter defaults
if [ "$(uname -s)" = "Darwin" ]
then
__configuration=DbgMacPy3.7
__configuration=DbgMacPy3.8
else
__configuration=DbgLinPy3.7
__configuration=DbgLinPy3.8
fi
__runTests=false
__installPythonPackages=false
@ -85,87 +83,93 @@ while [ "$1" != "" ]; do
done
case $__configuration in
*LinPy3.8)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.8.3-linux64.v2.tar.gz
PythonVersion=3.8
PythonTag=cp38
;;
*LinPy3.7)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/anaconda-full/Anaconda3-Linux-2019.03.v2.tar.gz
BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/linux/Boost-3.7-1.69.0.0.tar.gz
PythonVersion=3.7
PythonTag=cp37
;;
*LinPy3.6)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/anaconda-full/Anaconda3-Linux-5.0.1.v2.tar.gz
BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/linux/Boost-3.6-1.64.0.0.tar.gz
PythonVersion=3.6
PythonTag=cp36
;;
*LinPy3.5)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/anaconda-full/Anaconda3-Linux-4.2.0.v9.tar.gz
BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/linux/Boost-3.5-1.64.0.0.tar.gz
PythonVersion=3.5
PythonTag=cp35
;;
*LinPy2.7)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/anaconda-full/Anaconda2-Linux-5.0.1.v2.tar.gz
BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/linux/Boost-2.7-1.64.0.0.tar.gz
PythonVersion=2.7
PythonTag=cp27
*MacPy3.8)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/python/python-3.8.3-mac64.tar.gz
PythonVersion=3.8
PythonTag=cp38
;;
*MacPy3.7)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/anaconda-full/Anaconda3-Mac-2019.03.v2.tar.gz
BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/mac/Boost-3.7-1.69.0.0.tar.gz
PythonVersion=3.7
PythonTag=cp37
;;
*MacPy3.6)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/anaconda-full/Anaconda3-Mac-5.0.1.tar.gz
BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/mac/Boost-3.6-1.64.0.0.tar.gz
PythonVersion=3.6
PythonTag=cp36
;;
*MacPy3.5)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/anaconda-full/Anaconda3-Mac-4.2.0.tar.gz
BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/mac/Boost-3.5-1.64.0.0.tar.gz
PythonVersion=3.5
PythonTag=cp35
;;
*MacPy2.7)
PythonUrl=https://pythonpkgdeps.blob.core.windows.net/anaconda-full/Anaconda2-Mac-5.0.2.tar.gz
BoostUrl=https://pythonpkgdeps.blob.core.windows.net/boost/release/mac/Boost-2.7-1.64.0.0.tar.gz
PythonVersion=2.7
PythonTag=cp27
;;
*)
echo "Unknown configuration '$__configuration'"; usage; exit 1
esac
PythonRoot=${DependenciesDir}/Python${PythonVersion}
echo "Python root: ${PythonRoot}"
PythonExe="${PythonRoot}/bin/python"
if [ ${PythonVersion} = 3.8 ] && [ "$(uname -s)" != "Darwin" ]
then
PythonExe="python3.8" # use prebuilt version of in docker image on Linux
fi
echo "Python executable: ${PythonExe}"
echo "Downloading Python Dependencies "
# Download & unzip Python
if [ ! -e "${PythonRoot}/.done" ]
then
mkdir -p "${PythonRoot}"
echo "Downloading and extracting Python archive ... "
curl "${PythonUrl}" | tar xz -C "${PythonRoot}"
# Move all binaries out of "anaconda3", "anaconda2", or "anaconda", depending on naming convention for version
mv "${PythonRoot}/anaconda"*/* "${PythonRoot}/"
touch "${PythonRoot}/.done"
mkdir -p "${PythonRoot}"
echo "Downloading and extracting Python archive ... "
curl "${PythonUrl}" | tar xz -C "${PythonRoot}"
if [ ${PythonVersion} != 3.8 ]
then
# Move all binaries out of "anaconda3", "anaconda2", or "anaconda", depending on naming convention for version
mv "${PythonRoot}/anaconda"*/* "${PythonRoot}/"
touch "${PythonRoot}/.done"
echo "Install libc6-dev ... "
if [ "$(uname -s)" != "Darwin" ]
then
{
apt-get update
# Required for Image.py and Image_df.py to run successfully on Ubuntu.
apt-get install libc6-dev -y
#apt-get install libgdiplus -y
# Required for onnxruntime tests
#apt-get install -y locales
#locale-gen en_US.UTF-8
} || {
yum update --skip-broken
# Required for Image.py and Image_df.py to run successfully on CentOS.
yum install glibc-devel -y
# Required for onnxruntime tests
# yum install glibc-all-langpacks
# localedef -v -c -i en_US -f UTF-8 en_US.UTF-8
}
fi
echo "Install pybind11 ... "
"${PythonRoot}/bin/python" -m pip install pybind11
echo "Done installing pybind11 ... "
fi
touch "${PythonRoot}/.done"
fi
PythonExe="${PythonRoot}/bin/python"
echo "Python executable: ${PythonExe}"
if [ ${__buildNativeBridge} = true ]
then
echo "Building Native Bridge ... "
# Download & unzip Boost
if [ ! -e "${BoostRoot}/.done" ]
then
mkdir -p "${BoostRoot}"
echo "Downloading and extracting Boost archive ... "
curl "${BoostUrl}" | tar xz -C "${BoostRoot}"
touch "${BoostRoot}/.done"
fi
bash "${__currentScriptDir}/src/NativeBridge/build.sh" --configuration $__configuration --pythonver "${PythonVersion}" --pythonpath "${PythonRoot}" --boostpath "${BoostRoot}"
echo "Deleting ${BoostRoot} ${__currentScriptDir}/src/NativeBridge/x64"
rm -rf "${BoostRoot}"
rm -rf "${__currentScriptDir}/src/NativeBridge/x64"
bash "${__currentScriptDir}/src/NativeBridge/build.sh" --configuration $__configuration --pythonver "${PythonVersion}" --pythonpath "${PythonRoot}"
rm -rf "${__currentScriptDir}/src/NativeBridge/x64"
fi
if [ ${__buildDotNetBridge} = true ]
@ -206,43 +210,15 @@ then
mv "${BuildOutputDir}/${__configuration}"/pybridge.so "${__currentScriptDir}/src/python/nimbusml/internal/libs/"
# ls -l "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/
if [ ${PythonVersion} = 2.7 ]
then
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/*.dll "${__currentScriptDir}/src/python/nimbusml/internal/libs/"
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/System.Native.a "${__currentScriptDir}/src/python/nimbusml/internal/libs/"
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/createdump "${__currentScriptDir}/src/python/nimbusml/internal/libs/" || :
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/sosdocsunix.txt "${__currentScriptDir}/src/python/nimbusml/internal/libs/"
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/Data "${__currentScriptDir}/src/python/nimbusml/internal/libs/."
ext=*.so
if [ "$(uname -s)" = "Darwin" ]
then
ext=*.dylib
fi
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/${ext} "${__currentScriptDir}/src/python/nimbusml/internal/libs/"
# Obtain "libtensorflow_framework.so.1", which is the upgraded version of "libtensorflow.so". This is required for tests TensorFlowScorer.py to pass in Linux distros with Python 2.7
if [ ! "$(uname -s)" = "Darwin" ]
then
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/libtensorflow_framework.so.1 "${__currentScriptDir}/src/python/nimbusml/internal/libs/"
fi
# remove dataprep dlls as its not supported in python 2.7
rm -f "${__currentScriptDir}/src/python/nimbusml/internal/libs/Microsoft.DPrep.*"
rm -f "${__currentScriptDir}/src/python/nimbusml/internal/libs/Microsoft.Data.*"
rm -f "${__currentScriptDir}/src/python/nimbusml/internal/libs/Microsoft.ProgramSynthesis.*"
rm -f "${__currentScriptDir}/src/python/nimbusml/internal/libs/Microsoft.DataPrep.dll"
rm -f "${__currentScriptDir}/src/python/nimbusml/internal/libs/ExcelDataReader.dll"
rm -f "${__currentScriptDir}/src/python/nimbusml/internal/libs/Microsoft.WindowsAzure.Storage.dll"
rm -f "${__currentScriptDir}/src/python/nimbusml/internal/libs/Microsoft.Workbench.Messaging.SDK.dll"
else
libs_txt=libs_linux.txt
if [ "$(uname -s)" = "Darwin" ]
then
libs_txt=libs_mac.txt
fi
cat build/${libs_txt} | while read i; do
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/$i "${__currentScriptDir}/src/python/nimbusml/internal/libs/"
done
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/Data "${__currentScriptDir}/src/python/nimbusml/internal/libs/."
libs_txt=libs_linux.txt
if [ "$(uname -s)" = "Darwin" ]
then
libs_txt=libs_mac.txt
fi
cat build/${libs_txt} | while read i; do
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/$i "${__currentScriptDir}/src/python/nimbusml/internal/libs/"
done
mv "${BuildOutputDir}/${__configuration}/Platform/${PublishDir}"/publish/Data "${__currentScriptDir}/src/python/nimbusml/internal/libs/."
if [[ $__configuration = Dbg* ]]
then
@ -254,10 +230,15 @@ then
rm -rf "${BuildOutputDir}"
rm -rf "${__currentScriptDir}/cli"
"${PythonExe}" -m pip install --upgrade "wheel>=0.31.0"
cd "${__currentScriptDir}/src/python"
"${PythonExe}" setup.py bdist_wheel --python-tag ${PythonTag} --plat-name ${PlatName}
if [ ${PythonVersion} = 3.8 ]
then
# this is actually python 3.6 preinstalled, it can do 3.8 package
python3 setup.py bdist_wheel --python-tag ${PythonTag} --plat-name ${PlatName}
else
"${PythonExe}" -m pip install "wheel>=0.31.0"
"${PythonExe}" setup.py bdist_wheel --python-tag ${PythonTag} --plat-name ${PlatName}
fi
cd "${__currentScriptDir}"
WheelFile=nimbusml-${ProductVersion}-${PythonTag}-none-${PlatName}.whl
@ -289,22 +270,33 @@ then
echo "Unable to find ${Wheel}"
exit 1
fi
# Review: Adding "--upgrade" to pip install will cause problems when using Anaconda as the python distro because of Anaconda's quirks with pytest.
"${PythonExe}" -m pip install nose "pytest>=4.4.0" pytest-xdist graphviz "pytest-cov>=2.6.1" "jupyter_client>=4.4.0" "nbconvert>=4.2.0"
if [ ${PythonVersion} = 2.7 ]
if [ ${PythonVersion} = 3.8 ] && [ "$(uname -s)" = "Darwin" ]
then
"${PythonExe}" -m pip install --upgrade pyzmq
else
if [ ${PythonVersion} = 3.6 ] && [ "$(uname -s)" = "Darwin" ]
then
"${PythonExe}" -m pip install --upgrade pytest-remotedata
fi
echo "Installing python 3.8 on Mac ... "
curl -O https://www.python.org/ftp/python/3.8.3/python-3.8.3-macosx10.9.pkg
sudo installer -pkg python-3.8.3-macosx10.9.pkg -target /
fi
if [ ${PythonVersion} = 3.8 ] && [ "$(uname -s)" != "Darwin" ]
then
"${PythonExe}" -m pip install --user nose "pytest>=4.4.0" pytest-xdist graphviz
"${PythonExe}" -m pip install --user --upgrade "azureml-dataprep>=1.1.33"
"${PythonExe}" -m pip install --user --upgrade onnxruntime
"${PythonExe}" -m pip install --user --upgrade "${Wheel}"
"${PythonExe}" -m pip install --user scipy "scikit-learn==0.19.2"
else
# Review: Adding "--upgrade" to pip install will cause problems when using Anaconda as the python distro because of Anaconda's quirks with pytest.
"${PythonExe}" -m pip install nose "pytest>=4.4.0" pytest-xdist graphviz
"${PythonExe}" -m pip install --upgrade "azureml-dataprep>=1.1.33"
"${PythonExe}" -m pip install --upgrade onnxruntime
"${PythonExe}" -m pip install --upgrade "${Wheel}"
"${PythonExe}" -m pip install "scikit-learn==0.19.2"
fi
"${PythonExe}" -m pip install --upgrade "${Wheel}"
"${PythonExe}" -m pip install "scikit-learn==0.19.2"
if [ ${PythonVersion} = 3.6 ] && [ "$(uname -s)" = "Darwin" ]
then
"${PythonExe}" -m pip install --upgrade pytest-remotedata
fi
fi
if [ ${__runTests} = true ]
@ -317,14 +309,26 @@ then
TestsPath1=${PackagePath}/tests
TestsPath2=${__currentScriptDir}/src/python/tests
TestsPath3=${__currentScriptDir}/src/python/tests_extended
ReportPath=${__currentScriptDir}/build/TestCoverageReport
"${PythonExe}" -m pytest -n 4 --verbose --maxfail=1000 --capture=sys "${TestsPath2}" "${TestsPath1}" || \
"${PythonExe}" -m pytest -n 4 --last-failed --verbose --maxfail=1000 --capture=sys "${TestsPath2}" "${TestsPath1}"
if [ ${PythonVersion} = 3.8 ]
then
if [ "$(uname -s)" = "Darwin" ]
then
TestsPath1=/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/nimbusml/tests
else
# Linux Python3.8 only here.
TestsPath1=/home/runner/.local/lib/python3.8/site-packages/nimbusml/tests
fi
echo "Test paths: ${TestsPath1} ${TestsPath2} "
"${PythonExe}" -m pytest -n 4 --verbose --maxfail=1000 --capture=sys "${TestsPath2}" "${TestsPath1}"
else
"${PythonExe}" -m pytest -n 4 --verbose --maxfail=1000 --capture=sys "${TestsPath2}" "${TestsPath1}" || \
"${PythonExe}" -m pytest -n 4 --last-failed --verbose --maxfail=1000 --capture=sys "${TestsPath2}" "${TestsPath1}"
fi
if [ ${__runExtendedTests} = true ]
then
echo "Running extended tests ... "
if [ ! "$(uname -s)" = "Darwin" ]
if [ "$(uname -s)" != "Darwin" ]
then
{
apt-get update

Просмотреть файл

@ -27,25 +27,28 @@ phases:
# Mac phases
- ${{ if eq(parameters.name, 'Mac') }}:
# Note: Manual defining of the libomp URL below is needed to avoid error at runtime. Installing using 'brew install libomp' results in "Intel MKL FATAL ERROR: Cannot load libmkl_intel_thread.dylib."
- script: brew update && brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/f5b1ac99a7fba27c19cee0bc4f036775c889b359/Formula/libomp.rb gettext && brew link gettext --force && brew unlink python@2 && brew install mono-libgdiplus
- script: brew update && brew install cmake https://raw.githubusercontent.com/dotnet/machinelearning/master/build/libomp.rb mono-libgdiplus gettext && brew link gettext --force && brew link libomp --force
- ${{ if eq(parameters.testDistro, 'noTests') }}:
- script: chmod 777 $(_buildScript) && $(_buildScript) --configuration $(_configuration)
- ${{ if eq(parameters.testDistro, '') }}:
- script: chmod 777 $(_buildScript) && $(_buildScript) --configuration $(_configuration) --runTests $(_testOptions)
# Linux phases
- ${{ if or(eq(parameters.name, 'Linux_Ubuntu16'), eq(parameters.name, 'Linux_Ubuntu14'), eq(parameters.name, 'Linux_CentOS7')) }}:
- script: $(_dockerRun) mlnet/ubuntu16-nativebuild:0.1 bash -c "source /etc/profile && chmod 777 $(_buildScript) && $(_buildScript) --configuration $(_configuration) --buildNativeBridgeOnly"
- ${{ if or(eq(parameters.name, 'Linux_Ubuntu16'), eq(parameters.name, 'Linux_Ubuntu14'), eq(parameters.name, 'Linux_CentOS7'), eq(parameters.name, 'Linux_Python38')) }}:
- script: $(_dockerRun) mlnet/ubuntu16-general:0.1 bash -c "source /etc/profile && chmod 777 $(_buildScript) && $(_buildScript) --configuration $(_configuration) --buildNativeBridgeOnly"
displayName: Build native bridge
- script: $(_dockerRun) mlnet/ubuntu16-general:0.1 bash -c "source /etc/profile && chmod 777 $(_buildScript) && $(_buildScript) --configuration $(_configuration) --skipNativeBridge"
- script: $(_dockerRun) mlnet/ubuntu16-general:0.1 bash -c "source /etc/profile && $(_buildScript) --configuration $(_configuration) --skipNativeBridge"
displayName: Build python wheel
- ${{ if eq(parameters.testDistro, 'python38') }}:
- script: $(_dockerRun) quay.io/python-devs/ci-image:master bash -c "source /etc/profile && $(_buildScript) --configuration $(_configuration) --runTestsOnly $(_testOptions)"
displayName: Run tests Python3.8
- ${{ if eq(parameters.testDistro, 'ubuntu16') }}:
- script: $(_dockerRun) mlnet/ubuntu16-general:0.1 bash -c "source /etc/profile && chmod 777 $(_buildScript) && $(_buildScript) --configuration $(_configuration) --runTestsOnly $(_testOptions)"
- script: $(_dockerRun) mlnet/ubuntu16-general:0.1 bash -c "source /etc/profile && $(_buildScript) --configuration $(_configuration) --runTestsOnly $(_testOptions)"
displayName: Run tests Ubuntu16
- ${{ if eq(parameters.testDistro, 'ubuntu14') }}:
- script: $(_dockerRun) mlnet/ubuntu14-general:0.1 bash -c "source /etc/profile && chmod 777 $(_buildScript) && $(_buildScript) --configuration $(_configuration) --runTestsOnly $(_testOptions)"
- script: $(_dockerRun) mlnet/ubuntu14-general:0.1 bash -c "source /etc/profile && $(_buildScript) --configuration $(_configuration) --runTestsOnly $(_testOptions)"
displayName: Run tests Ubuntu14
- ${{ if eq(parameters.testDistro, 'centos7') }}:
- script: $(_dockerRun) mlnet/centos7-general:0.1 bash -c "source /root/.bash_profile && source /etc/profile && chmod 777 $(_buildScript) && $(_buildScript) --configuration $(_configuration) --runTestsOnly $(_testOptions)"
- script: $(_dockerRun) mlnet/centos7-general:0.1 bash -c "source /etc/profile && $(_buildScript) --configuration $(_configuration) --runTestsOnly $(_testOptions)"
displayName: Run tests CentOS7
# Publish build artifacts

Просмотреть файл

@ -6,14 +6,12 @@ phases:
name: Windows
buildScript: build.cmd
buildMatrix:
Py38:
_configuration: RlsWinPy3.8
Py37:
_configuration: RlsWinPy3.7
Py36:
_configuration: RlsWinPy3.6
Py35:
_configuration: RlsWinPy3.5
Py27:
_configuration: RlsWinPy2.7
buildQueue:
name: Hosted VS2017
testOptions: --includeExtendedTests
@ -24,14 +22,12 @@ phases:
name: Mac
buildScript: ./build.sh
buildMatrix:
Py38:
_configuration: RlsMacPy3.8
Py37:
_configuration: RlsMacPy3.7
Py36:
_configuration: RlsMacPy3.6
Py35:
_configuration: RlsMacPy3.5
Py27:
_configuration: RlsMacPy2.7
buildQueue:
name: Hosted macOS
testOptions: --includeExtendedTests
@ -44,14 +40,12 @@ phases:
buildScript: ./build.sh
testDistro: ubuntu16
buildMatrix:
Py38:
_configuration: RlsLinPy3.8
Py37:
_configuration: RlsLinPy3.7
Py36:
_configuration: RlsLinPy3.6
Py35:
_configuration: RlsLinPy3.5
Py27:
_configuration: RlsLinPy2.7
buildQueue:
name: Hosted Ubuntu 1604
testOptions: --includeExtendedTests
@ -63,14 +57,12 @@ phases:
buildScript: ./build.sh
testDistro: ubuntu14
buildMatrix:
Py38:
_configuration: RlsLinPy3.8
Py37:
_configuration: RlsLinPy3.7
Py36:
_configuration: RlsLinPy3.6
Py35:
_configuration: RlsLinPy3.5
Py27:
_configuration: RlsLinPy2.7
buildQueue:
name: Hosted Ubuntu 1604
testOptions: --includeExtendedTests
@ -82,14 +74,12 @@ phases:
buildScript: ./build.sh
testDistro: centos7
buildMatrix:
Py38:
_configuration: RlsLinPy3.8
Py37:
_configuration: RlsLinPy3.7
Py36:
_configuration: RlsLinPy3.6
Py35:
_configuration: RlsLinPy3.5
Py27:
_configuration: RlsLinPy2.7
buildQueue:
name: Hosted Ubuntu 1604
testOptions: --includeExtendedTests

Просмотреть файл

@ -9,12 +9,8 @@ phases:
config: RlsWinPy3.6
- template: signed_build_phase.yml
parameters:
name: Build_windows_RlsWinPy3_5
config: RlsWinPy3.5
- template: signed_build_phase.yml
parameters:
name: Build_windows_RlsWinPy2_7
config: RlsWinPy2.7
name: Build_windows_RlsWinPy3_8
config: RlsWinPy3.8
# Build all configurations for Linux
# Run tests on Ubuntu16
@ -28,10 +24,8 @@ phases:
_configuration: RlsMacPy3.7
Py36:
_configuration: RlsMacPy3.6
Py35:
_configuration: RlsMacPy3.5
Py27:
_configuration: RlsMacPy2.7
Py38:
_configuration: RlsMacPy3.8
buildQueue:
name: Hosted macOS
- template: /build/ci/phase-template.yml
@ -44,9 +38,7 @@ phases:
_configuration: RlsLinPy3.7
Py36:
_configuration: RlsLinPy3.6
Py35:
_configuration: RlsLinPy3.5
Py27:
_configuration: RlsLinPy2.7
Py38:
_configuration: RlsLinPy3.8
buildQueue:
name: Hosted Ubuntu 1604

Просмотреть файл

@ -1,7 +1,7 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 15
VisualStudioVersion = 15.0.27004.2005
# Visual Studio Version 16
VisualStudioVersion = 16.0.30104.148
MinimumVisualStudioVersion = 10.0.40219.1
Project("{888888A0-9F3D-457C-B088-3A5042F75D52}") = "nimbusml", "src\python\nimbusml.pyproj", "{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}"
EndProject
@ -24,142 +24,121 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "ManifestGenerator", "src\Ma
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
DbgLinPy2.7|x64 = DbgLinPy2.7|x64
DbgLinPy3.5|x64 = DbgLinPy3.5|x64
DbgLinPy3.6|x64 = DbgLinPy3.6|x64
DbgLinPy3.7|x64 = DbgLinPy3.7|x64
DbgWinPy2.7|x64 = DbgWinPy2.7|x64
DbgWinPy3.5|x64 = DbgWinPy3.5|x64
DbgLinPy3.8|x64 = DbgLinPy3.8|x64
DbgWinPy3.6|x64 = DbgWinPy3.6|x64
DbgWinPy3.7|x64 = DbgWinPy3.7|x64
RlsLinPy2.7|x64 = RlsLinPy2.7|x64
RlsLinPy3.5|x64 = RlsLinPy3.5|x64
DbgWinPy3.8|x64 = DbgWinPy3.8|x64
RlsLinPy3.6|x64 = RlsLinPy3.6|x64
RlsLinPy3.7|x64 = RlsLinPy3.7|x64
RlsLinPy3.8|x64 = RlsLinPy3.8|x64
RlsMacPy3.6|x64 = RlsMacPy3.6|x64
RlsMacPy3.7|x64 = RlsMacPy3.7|x64
RlsWinPy2.7|x64 = RlsWinPy2.7|x64
RlsWinPy3.5|x64 = RlsWinPy3.5|x64
RlsMacPy3.8|x64 = RlsMacPy3.8|x64
RlsWinPy3.6|x64 = RlsWinPy3.6|x64
RlsWinPy3.7|x64 = RlsWinPy3.7|x64
RlsWinPy3.8|x64 = RlsWinPy3.8|x64
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgLinPy2.7|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgLinPy3.5|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgLinPy3.6|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgLinPy3.7|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgWinPy2.7|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgWinPy3.5|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgLinPy3.8|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgWinPy3.6|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgWinPy3.7|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsLinPy2.7|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsLinPy3.5|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.DbgWinPy3.8|x64.ActiveCfg = Debug|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsLinPy3.6|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsLinPy3.7|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsLinPy3.8|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsMacPy3.6|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsMacPy3.7|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsWinPy2.7|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsWinPy3.5|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsMacPy3.8|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsWinPy3.6|x64.ActiveCfg = Release|Any CPU
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsWinPy3.7|x64.ActiveCfg = Release|Any CPU
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy2.7|x64.ActiveCfg = DbgLinPy2.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy2.7|x64.Build.0 = DbgLinPy2.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy3.5|x64.ActiveCfg = DbgLinPy3.5|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy3.5|x64.Build.0 = DbgLinPy3.5|x64
{8DFF150B-E1C6-4EB4-90C7-9D69E9E7CCA1}.RlsWinPy3.8|x64.ActiveCfg = Release|Any CPU
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy3.6|x64.ActiveCfg = DbgLinPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy3.6|x64.Build.0 = DbgLinPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy3.7|x64.ActiveCfg = DbgLinPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy3.7|x64.Build.0 = DbgLinPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy2.7|x64.ActiveCfg = DbgWinPy2.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy2.7|x64.Build.0 = DbgWinPy2.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy3.5|x64.ActiveCfg = DbgWinPy3.5|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy3.5|x64.Build.0 = DbgWinPy3.5|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy3.8|x64.ActiveCfg = DbgLinPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgLinPy3.8|x64.Build.0 = DbgLinPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy3.6|x64.ActiveCfg = DbgWinPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy3.6|x64.Build.0 = DbgWinPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy3.7|x64.ActiveCfg = DbgWinPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy3.7|x64.Build.0 = DbgWinPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy2.7|x64.ActiveCfg = RlsLinPy2.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy2.7|x64.Build.0 = RlsLinPy2.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy3.5|x64.ActiveCfg = RlsLinPy3.5|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy3.5|x64.Build.0 = RlsLinPy3.5|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy3.8|x64.ActiveCfg = DbgWinPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.DbgWinPy3.8|x64.Build.0 = DbgWinPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy3.6|x64.ActiveCfg = RlsLinPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy3.6|x64.Build.0 = RlsLinPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy3.7|x64.ActiveCfg = RlsLinPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy3.7|x64.Build.0 = RlsLinPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy3.8|x64.ActiveCfg = RlsLinPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsLinPy3.8|x64.Build.0 = RlsLinPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsMacPy3.6|x64.ActiveCfg = RlsMacPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsMacPy3.6|x64.Build.0 = RlsMacPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsMacPy3.7|x64.ActiveCfg = RlsMacPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsMacPy3.7|x64.Build.0 = RlsMacPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy2.7|x64.ActiveCfg = RlsWinPy2.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy2.7|x64.Build.0 = RlsWinPy2.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy3.5|x64.ActiveCfg = RlsWinPy3.5|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy3.5|x64.Build.0 = RlsWinPy3.5|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsMacPy3.8|x64.ActiveCfg = RlsMacPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsMacPy3.8|x64.Build.0 = RlsMacPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy3.6|x64.ActiveCfg = RlsWinPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy3.6|x64.Build.0 = RlsWinPy3.6|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy3.7|x64.ActiveCfg = RlsWinPy3.7|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy3.7|x64.Build.0 = RlsWinPy3.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgLinPy2.7|x64.ActiveCfg = DbgWinPy2.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgLinPy3.5|x64.ActiveCfg = DbgWinPy3.5|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy3.8|x64.ActiveCfg = RlsWinPy3.8|x64
{EC58F2CF-A1D5-4E28-97F9-69B1E46F6F63}.RlsWinPy3.8|x64.Build.0 = RlsWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgLinPy3.6|x64.ActiveCfg = DbgWinPy3.6|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgLinPy3.7|x64.ActiveCfg = DbgWinPy3.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy2.7|x64.ActiveCfg = DbgWinPy2.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy2.7|x64.Build.0 = DbgWinPy2.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy3.5|x64.ActiveCfg = DbgWinPy3.5|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy3.5|x64.Build.0 = DbgWinPy3.5|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgLinPy3.8|x64.ActiveCfg = DbgWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy3.6|x64.ActiveCfg = DbgWinPy3.6|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy3.6|x64.Build.0 = DbgWinPy3.6|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy3.7|x64.ActiveCfg = DbgWinPy3.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy3.7|x64.Build.0 = DbgWinPy3.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsLinPy2.7|x64.ActiveCfg = RlsWinPy2.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsLinPy3.5|x64.ActiveCfg = RlsWinPy3.5|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy3.8|x64.ActiveCfg = DbgWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.DbgWinPy3.8|x64.Build.0 = DbgWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsLinPy3.6|x64.ActiveCfg = RlsWinPy3.6|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsLinPy3.7|x64.ActiveCfg = RlsWinPy3.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsLinPy3.8|x64.ActiveCfg = RlsWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsLinPy3.8|x64.Build.0 = RlsWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsMacPy3.6|x64.ActiveCfg = DbgWinPy3.6|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsMacPy3.7|x64.ActiveCfg = DbgWinPy3.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy2.7|x64.ActiveCfg = RlsWinPy2.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy2.7|x64.Build.0 = RlsWinPy2.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy3.5|x64.ActiveCfg = RlsWinPy3.5|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy3.5|x64.Build.0 = RlsWinPy3.5|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsMacPy3.8|x64.ActiveCfg = RlsWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsMacPy3.8|x64.Build.0 = RlsWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy3.6|x64.ActiveCfg = RlsWinPy3.6|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy3.6|x64.Build.0 = RlsWinPy3.6|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy3.7|x64.ActiveCfg = RlsWinPy3.7|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy3.7|x64.Build.0 = RlsWinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy2.7|x64.ActiveCfg = DbgLinPy2.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy2.7|x64.Build.0 = DbgLinPy2.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy3.5|x64.ActiveCfg = DbgLinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy3.5|x64.Build.0 = DbgLinPy3.5|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy3.8|x64.ActiveCfg = RlsWinPy3.8|x64
{3DA0AF32-A05B-4ECF-8010-83B14612FBB3}.RlsWinPy3.8|x64.Build.0 = RlsWinPy3.8|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy3.6|x64.ActiveCfg = DbgLinPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy3.6|x64.Build.0 = DbgLinPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy3.7|x64.ActiveCfg = DbgLinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy3.7|x64.Build.0 = DbgLinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy2.7|x64.ActiveCfg = DbgWinPy2.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy2.7|x64.Build.0 = DbgWinPy2.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy3.5|x64.ActiveCfg = DbgWinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy3.5|x64.Build.0 = DbgWinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy3.8|x64.ActiveCfg = DbgLinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgLinPy3.8|x64.Build.0 = DbgLinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy3.6|x64.ActiveCfg = DbgWinPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy3.6|x64.Build.0 = DbgWinPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy3.7|x64.ActiveCfg = DbgWinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy3.7|x64.Build.0 = DbgWinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy2.7|x64.ActiveCfg = RlsLinPy2.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy2.7|x64.Build.0 = RlsLinPy2.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy3.5|x64.ActiveCfg = RlsLinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy3.5|x64.Build.0 = RlsLinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy3.8|x64.ActiveCfg = DbgWinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.DbgWinPy3.8|x64.Build.0 = DbgWinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy3.6|x64.ActiveCfg = RlsLinPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy3.6|x64.Build.0 = RlsLinPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy3.7|x64.ActiveCfg = RlsLinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy3.7|x64.Build.0 = RlsLinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy3.8|x64.ActiveCfg = RlsLinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsLinPy3.8|x64.Build.0 = RlsLinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsMacPy3.6|x64.ActiveCfg = RlsMacPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsMacPy3.6|x64.Build.0 = RlsMacPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsMacPy3.7|x64.ActiveCfg = RlsMacPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsMacPy3.7|x64.Build.0 = RlsMacPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy2.7|x64.ActiveCfg = RlsWinPy2.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy2.7|x64.Build.0 = RlsWinPy2.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy3.5|x64.ActiveCfg = RlsWinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy3.5|x64.Build.0 = RlsWinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsMacPy3.8|x64.ActiveCfg = RlsMacPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsMacPy3.8|x64.Build.0 = RlsMacPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy3.6|x64.ActiveCfg = RlsWinPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy3.6|x64.Build.0 = RlsWinPy3.6|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy3.7|x64.ActiveCfg = RlsWinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy3.7|x64.Build.0 = RlsWinPy3.7|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy3.8|x64.ActiveCfg = RlsWinPy3.5|x64
{D3AED287-722F-4243-966E-77AD0652B38E}.RlsWinPy3.8|x64.Build.0 = RlsWinPy3.5|x64
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE

Просмотреть файл

@ -28,38 +28,24 @@
<LinkIncremental>false</LinkIncremental>
<WholeProgramOptimization>true</WholeProgramOptimization>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'RlsWinPy2.7'">
<BoostRoot>$(MSBuildThisFileDirectory)..\dependencies\BoostRls2.7</BoostRoot>
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python2.7</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'DbgWinPy2.7'">
<BoostRoot>$(MSBuildThisFileDirectory)..\dependencies\BoostDbg2.7</BoostRoot>
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python2.7</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'RlsWinPy3.5'">
<BoostRoot>$(MSBuildThisFileDirectory)..\dependencies\BoostRls3.5</BoostRoot>
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python3.5</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'DbgWinPy3.5'">
<BoostRoot>$(MSBuildThisFileDirectory)..\dependencies\BoostDbg3.5</BoostRoot>
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python3.5</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'RlsWinPy3.6'">
<BoostRoot>$(MSBuildThisFileDirectory)..\dependencies\BoostRls3.6</BoostRoot>
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python3.6</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'RlsWinPy3.7'">
<BoostRoot>$(MSBuildThisFileDirectory)..\dependencies\BoostRls3.7</BoostRoot>
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python3.7</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'RlsWinPy3.8'">
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python3.8</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'DbgWinPy3.6'">
<BoostRoot>$(MSBuildThisFileDirectory)..\dependencies\BoostDbg3.6</BoostRoot>
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python3.6</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'DbgWinPy3.7'">
<BoostRoot>$(MSBuildThisFileDirectory)..\dependencies\BoostDbg3.7</BoostRoot>
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python3.7</PythonRoot>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)' == 'DbgWinPy3.8'">
<PythonRoot>$(MSBuildThisFileDirectory)..\dependencies\Python3.8</PythonRoot>
</PropertyGroup>
<PropertyGroup>
<ConfigurationOutputPath>$(SolutionDir)\bin\$(Configuration)\Win\</ConfigurationOutputPath>
</PropertyGroup>

Просмотреть файл

@ -8,7 +8,7 @@
<RootNamespace>DotNetBridge</RootNamespace>
<AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
<OutputPath>..\$(Platform)\$(Configuration)\</OutputPath>
<Configurations>DbgWinPy3.7;DbgWinPy3.6;DbgWinPy3.5;DbgWinPy2.7;RlsWinPy3.7;RlsWinPy3.6;RlsWinPy3.5;RlsWinPy2.7;DbgLinPy3.7;DbgLinPy3.6;DbgLinPy3.5;DbgLinPy2.7;RlsLinPy3.7;RlsLinPy3.6;RlsLinPy3.5;RlsLinPy2.7;RlsMacPy3.7;RlsMacPy3.6</Configurations>
<Configurations>DbgWinPy3.7;DbgWinPy3.6;DbgWinPy3.8;RlsWinPy3.7;RlsWinPy3.6;RlsWinPy3.8;DbgLinPy3.7;DbgLinPy3.6;DbgLinPy3.8;RlsLinPy3.7;RlsLinPy3.6;RlsLinPy3.8;RlsMacPy3.7;RlsMacPy3.6;RlsMacPy3.8</Configurations>
<Version>0.10.0</Version>
<Company>Microsoft Corporation</Company>
<Copyright>(c) Microsoft Corporation. All rights reserved.</Copyright>
@ -39,8 +39,8 @@
<PackageReference Include="Microsoft.ML.ImageAnalytics" Version="1.5.0-preview2" />
<PackageReference Include="Microsoft.ML.LightGBM" Version="1.5.0-preview2" />
<PackageReference Include="Microsoft.ML.OnnxConverter" Version="0.17.0-preview2" />
<PackageReference Include="Microsoft.ML.OnnxTransformer" Version="1.5.0-preview2" />
<PackageReference Include="Microsoft.ML.OnnxRuntime" Version="1.2.0" />
<PackageReference Include="Microsoft.ML.OnnxTransformer" Version="1.5.0-preview2" />
<PackageReference Include="Microsoft.ML.TensorFlow" Version="1.5.0-preview2" />
<PackageReference Include="Microsoft.ML.Vision" Version="1.5.0-preview2" />
<PackageReference Include="Microsoft.ML.Ensemble" Version="0.17.0-preview2" />

Просмотреть файл

@ -43,15 +43,14 @@ set(CMAKE_INSTALL_DIR "${CMAKE_BINARY_DIR}/../")
MESSAGE( STATUS "CMAKE_INSTALL_DIR: " ${CMAKE_INSTALL_DIR})
MESSAGE( STATUS "CMAKE_BINARY_DIR: " ${CMAKE_BINARY_DIR} )
MESSAGE( STATUS "CMAKE_SOURCE_DIR: " ${CMAKE_SOURCE_DIR} )
MESSAGE( STATUS "BOOST_DIR: " ${BOOST_DIR} )
MESSAGE( STATUS "PYTHON_DIR: " ${PYTHON_DIR} )
MESSAGE( STATUS "CMAKE_SYSTEM_NAME: " ${CMAKE_SYSTEM_NAME} )
include_directories(${PYTHON_DIR}/include/python2.7 ${PYTHON_DIR}/include/python3.5m ${PYTHON_DIR}/include/python3.6m ${PYTHON_DIR}/include/python3.7m ${BOOST_DIR}/Include)
include_directories(${PYTHON_DIR}/include ${PYTHON_DIR}/include/python3.6m ${PYTHON_DIR}/include/python3.7m)
#set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror -Wno-error=unused-variable -Wno-error=attributes -Wno-error=unused-value")
if (CMAKE_SYSTEM_NAME STREQUAL "Darwin")
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,-rpath,'/Library/Frameworks/Python.framework/Versions/3.7/lib'" )
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,-rpath,'/Library/Frameworks/Python.framework/Versions/${PYTHON_VER}/lib'" )
else ()
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,-rpath,'\${ORIGIN}'" )
endif()
@ -70,24 +69,15 @@ set (PYBRIDGE_SOURCE_FILES
add_library(PyBridge_objects OBJECT ${PYBRIDGE_SOURCE_FILES})
add_library(PyBridge SHARED $<TARGET_OBJECTS:PyBridge_objects>)
# Define lib dependencies to be used as target link libraries
if (${PYTHON_VER} STREQUAL "2.7")
set(BOOST_LIBS ${BOOST_DIR}/lib/libboost_python.a ${BOOST_DIR}/lib/libboost_numpy.a)
elseif (${PYTHON_VER} STREQUAL "3.7")
set(BOOST_LIBS ${BOOST_DIR}/lib/libboost_python37.a ${BOOST_DIR}/lib/libboost_numpy37.a)
else()
set(BOOST_LIBS ${BOOST_DIR}/lib/libboost_python3.a ${BOOST_DIR}/lib/libboost_numpy3.a)
endif()
if (CMAKE_SYSTEM_NAME STREQUAL "Linux")
if (${PYTHON_VER} STREQUAL "2.7")
set(LIB_PYTHON ${PYTHON_DIR}/lib/libpython2.7.so)
else()
set(LIB_PYTHON ${PYTHON_DIR}/lib/libpython${PYTHON_VER}m.so)
endif()
if (PYTHON_VER STREQUAL "3.8")
set(LIB_PYTHON ${PYTHON_DIR}/libpython${PYTHON_VER}.so)
else ()
set(LIB_PYTHON ${PYTHON_DIR}/lib/libpython${PYTHON_VER}m.so)
endif()
endif()
set(PYBRIDGE_LIB_DEPS ${BOOST_LIBS} ${LIB_PYTHON})
set(PYBRIDGE_LIB_DEPS ${LIB_PYTHON})
set_target_properties(PyBridge PROPERTIES PREFIX "")
set_target_properties(PyBridge PROPERTIES OUTPUT_NAME "pybridge")

Просмотреть файл

@ -5,33 +5,39 @@
#include "DataViewInterop.h"
#include "ManagedInterop.h"
DataSourceBlock::DataSourceBlock(bp::dict& data)
DataSourceBlock::DataSourceBlock(pb::dict& data)
{
// Assert that this class doesn't have a vtable.
assert(offsetof(DataSourceBlock, ccol) == 0);
CxInt64 llTotalNumRows = -1;
assert(data.contains(PYTHON_DATA_KEY_INFO));
bp::dict varInfo = bp::extract<bp::dict>(data[PYTHON_DATA_KEY_INFO]);
pb::dict varInfo = pb::cast<pb::dict>(data[PYTHON_DATA_KEY_INFO]);
assert(data.contains(PYTHON_DATA_COL_TYPES));
bp::list colTypes = bp::extract<bp::list>(data[PYTHON_DATA_COL_TYPES]);
bp::stl_input_iterator<bp::object> keys(data.keys()), end1;
bp::stl_input_iterator<bp::object> values(data.values());
pb::list colTypes = pb::cast<pb::list>(data[PYTHON_DATA_COL_TYPES]);
CxInt64 dataframeColCount = -1;
for (; keys != end1; keys++)
for (auto it = data.begin(); it != data.end(); ++it)
{
bp::object key = *keys;
char* name = bp::extract<char*>(key);
bp::object value = *values++;
if (strcmp(name, PYTHON_DATA_KEY_INFO) == 0 || strcmp(name, PYTHON_DATA_COL_TYPES) == 0)
if (!pb::isinstance<pb::str>(it->first))
continue;
// Function PyUnicode_AS_DATA is deprecated as getting the expected unicode
// requires the creation of a new object. We keep a copy of the string.
// See https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AS_DATA.
// pybind11 uses a bytes representation of the string and then call
// PyUnicode_AsEncodedString which creates a new reference.
pb::str skey = pb::cast<pb::str>(it->first.ptr());
std::string name = (std::string)skey;
if (name == PYTHON_DATA_KEY_INFO || name == PYTHON_DATA_COL_TYPES)
continue;
pb::object value = pb::cast<pb::object>(it->second);
// now it should be a column names
std::string colName = bp::extract<std::string>(key);
std::string colName = pb::cast<std::string>(it->first);
dataframeColCount++;
auto tp = bp::extract<const char*>(colTypes[dataframeColCount]);
const char* tp = (char*)PyUnicode_DATA(colTypes[dataframeColCount].ptr());
ML_PY_TYPE_MAP_ENUM colType = static_cast<ML_PY_TYPE_MAP_ENUM>(tp[0]);
BYTE kind;
@ -41,10 +47,10 @@ DataSourceBlock::DataSourceBlock(bp::dict& data)
bool isText = false;
CxInt64 vecCard = -1;
// Numeric or bool values.
if (bp::extract<np::ndarray>(value).check())
if (pb::isinstance<pb::array>(value))
{
isNumeric = true;
np::ndarray val = bp::extract<np::ndarray>(value);
pb::array val = pb::cast<pb::array>(value);
switch (colType)
{
case (ML_PY_BOOL):
@ -105,7 +111,7 @@ DataSourceBlock::DataSourceBlock(bp::dict& data)
default:
throw std::invalid_argument("column " + colName + " has unsupported type");
}
const char *data = val.get_data();
const char *data = (const char*)val.data();
this->_vdata.push_back(data);
assert(this->_mpnum.size() == dataframeColCount);
@ -116,19 +122,19 @@ DataSourceBlock::DataSourceBlock(bp::dict& data)
assert(llTotalNumRows == val.shape(0));
}
// Text or key values.
else if (bp::extract<bp::list>(value).check())
else if (pb::isinstance<pb::list>(value))
{
bp::list list = bp::extract<bp::list>(value);
pb::list list = pb::cast<pb::list>(value);
// Key values.
switch (colType)
{
case (ML_PY_CAT):
if (varInfo.contains(colName))
if (varInfo.contains(pb::str(colName)))
{
isKey = true;
assert(bp::extract<bp::list>(varInfo[colName]).check());
bp::list keyNames = bp::extract<bp::list>(varInfo[colName]);
assert(pb::isinstance<pb::list>(varInfo[pb::str(colName)]));
pb::list keyNames = pb::cast<pb::list>(varInfo[pb::str(colName)]);
kind = U4;
pgetter = (void*)GetKeyInt;
@ -176,16 +182,16 @@ DataSourceBlock::DataSourceBlock(bp::dict& data)
}
}
// A sparse vector.
else if (bp::extract<bp::dict>(value).check())
else if (pb::isinstance<pb::dict>(value))
{
bp::dict sparse = bp::extract<bp::dict>(value);
np::ndarray indices = bp::extract<np::ndarray>(sparse["indices"]);
_sparseIndices = (int*)indices.get_data();
np::ndarray indptr = bp::extract<np::ndarray>(sparse["indptr"]);
_indPtr = (int*)indptr.get_data();
pb::dict sparse = pb::cast<pb::dict>(value);
pb::array indices = pb::cast<pb::array>(sparse["indices"]);
_sparseIndices = (int*)indices.data();
pb::array indptr = pb::cast<pb::array>(sparse["indptr"]);
_indPtr = (int*)indptr.data();
np::ndarray values = bp::extract<np::ndarray>(sparse["values"]);
_sparseValues = values.get_data();
pb::array values = pb::cast<pb::array>(sparse["values"]);
_sparseValues = (void*)values.data();
switch (colType)
{
case (ML_PY_BOOL):
@ -237,7 +243,7 @@ DataSourceBlock::DataSourceBlock(bp::dict& data)
default:
throw std::invalid_argument("column " + colName + " has unsupported type");
}
vecCard = bp::extract<int>(sparse["colCount"]);
vecCard = pb::cast<int>(sparse["colCount"]);
name = (char*)"Data";
if (llTotalNumRows == -1)
@ -250,7 +256,7 @@ DataSourceBlock::DataSourceBlock(bp::dict& data)
this->_vgetter.push_back(pgetter);
this->_vname.push_back(name);
this->_vkind.push_back(kind);
this->_vkind.push_back(kind);
_vvecCard.push_back(vecCard);
if (!isNumeric)
@ -283,7 +289,11 @@ DataSourceBlock::DataSourceBlock(bp::dict& data)
if (this->ccol > 0)
{
this->names = &this->_vname[0];
for (const std::string& string : this->_vname)
this->_cname.push_back(const_cast<char*>(string.c_str()));
this->_cname.push_back(nullptr);
this->names = (const char**)(this->_cname.data());
this->kinds = &this->_vkind[0];
this->keyCards = &this->_vkeyCard[0];
this->vecCards = &this->_vvecCard[0];

Просмотреть файл

@ -14,10 +14,6 @@ class DataSourceBlock;
// Callback function for getting labels for key-type columns. Returns success.
typedef MANAGED_CALLBACK_PTR(bool, GETLABELS)(DataSourceBlock *source, int col, int count, const char **buffer);
// REVIEW: boost_python is not updated at the same speed as swig or pybind11.
// Both have a larger audience now, see about pybind11 https://github.com/davisking/dlib/issues/293
// It handles csr_matrix: https://pybind11-rtdtest.readthedocs.io/en/stable/advanced.html#transparent-conversion-of-dense-and-sparse-eigen-data-types.
using namespace boost::python;
// The data source wrapper used for managed interop. Some of the fields of this are visible to managed code.
// As such, it is critical that this class NOT have a vtable, so virtual functions are illegal!
@ -62,7 +58,8 @@ private:
// The vectors below here are parallel.
// Column names.
std::vector<const char*> _vname;
std::vector<std::string> _vname;
std::vector<const char*> _cname;
// Column DataKind values.
std::vector<BYTE> _vkind;
// Column key type cardinalities. Zero for unbounded, -1 for non-key-types.
@ -73,10 +70,10 @@ private:
std::vector<const void *> _vgetter;
std::vector<const void*> _vdata;
std::vector<bp::list> _vtextdata;
std::vector<pb::list> _vtextdata;
std::vector<char*> _vtextdata_cache;
std::vector<bp::list> _vkeydata;
std::vector<bp::list> _vkeynames;
std::vector<pb::list> _vkeydata;
std::vector<pb::list> _vkeynames;
// Stores the sparse data.
// REVIEW: need better documentatoin here - is this a pointer, or buffer ? If buffer, why this is not a vector ? Where do we store type of values ? What is indptr ?
@ -85,18 +82,18 @@ private:
int* _indPtr;
public:
DataSourceBlock(bp::dict& data);
DataSourceBlock(pb::dict& data);
~DataSourceBlock();
private:
bp::object SelectItemForType(bp::list& container)
pb::object SelectItemForType(pb::list& container)
{
auto length = len(container);
for (auto index = 0; index < length; index++)
{
bp::object item = container[index];
pb::object item = container[index];
if (!item.is_none())
{
@ -104,7 +101,7 @@ private:
}
}
return bp::object();
return pb::object();
}
// Callback methods. These are only needed from managed code via the embedded function pointers above,
@ -121,10 +118,7 @@ private:
CxInt64 numCol = pdata->_mpnum[col];
assert(0 <= numCol && numCol < (CxInt64)pdata->_vdata.size());
const double *charData = reinterpret_cast<const double*>(pdata->_vdata[numCol]);
if (boost::math::isnan(charData[index]))
dst = -1;
else
dst = (signed char)charData[index];
dst = (signed char)charData[index];
}
static MANAGED_CALLBACK(void) GetU1(DataSourceBlock *pdata, int col, long index, /*out*/ unsigned char &dst)
{
@ -202,13 +196,13 @@ private:
{
CxInt64 txCol = pdata->_mptxt[col];
assert(0 <= txCol && txCol < (CxInt64)pdata->_vtextdata.size());
bp::object s = pdata->_vtextdata[txCol][index];
pb::object s = pdata->_vtextdata[txCol][index];
if (bp::extract<const char*>(s).check())
if (pb::isinstance<pb::str>(s))
{
size = -1;
missing = -1;
pch = bp::extract<const char*>(s);
pch = (char*)PyUnicode_DATA(s.ptr());
if (s.is_none())
{
size = 0;
@ -225,7 +219,7 @@ private:
else
{
// Missing values in Python are float.NaN.
assert(bp::extract<float>(s).check());
assert(pb::cast<float>(s) != NULL);
missing = 1;
}
}
@ -238,11 +232,11 @@ private:
assert(0 <= txCol && txCol < (CxInt64)pdata->_vtextdata.size());
auto s = pdata->_vtextdata[txCol][index];
if (bp::extract<const char*>(str(s).encode("utf_8")).check())
if (pb::isinstance<pb::str>(s))
{
size = -1;
missing = -1;
pch = bp::extract<const char*>(str(s).encode("utf_8"));
pch = (char*)PyUnicode_DATA(s.ptr());
#if _MSC_VER
Utf8ToUtf16le(pch, pch, size);
#endif
@ -251,7 +245,7 @@ private:
else
{
// Missing values in Python are float.NaN.
assert(bp::extract<float>(s).check());
assert(pb::cast<float>(s) != NULL);
missing = 1;
}
}
@ -294,9 +288,9 @@ private:
assert(0 <= keyCol && keyCol < (CxInt64)pdata->_vkeydata.size());
auto & list = pdata->_vkeydata[keyCol];
bp::object obj = pdata->SelectItemForType(list);
pb::object obj = pdata->SelectItemForType(list);
assert(strcmp(obj.ptr()->ob_type->tp_name, "int") == 0);
dst = bp::extract<int>(list[index]);
dst = pb::cast<int>(list[index]);
}
// Callback function for getting labels for key-type columns. Returns success.
@ -325,7 +319,7 @@ private:
}
CxInt64 keyCol = pdata->_mpkey[col];
bp::list & names = pdata->_vkeynames[keyCol];
pb::list & names = pdata->_vkeynames[keyCol];
if (len(names) != count)
{
// No labels for this column. This is not a logic error.
@ -333,7 +327,7 @@ private:
}
for (int i = 0; i < count; ++i, ++buffer)
*buffer = bp::extract<const char*>(names[i]);
*buffer = (char*)PyUnicode_DATA(names[i].ptr());
return true;
}

Просмотреть файл

@ -176,11 +176,11 @@ STATIC MANAGED_CALLBACK(bool) EnvironmentBlock::CheckCancel()
return false;
}
bp::dict EnvironmentBlock::GetData()
pb::dict EnvironmentBlock::GetData()
{
if (_columns.size() == 0)
{
return bp::dict();
return pb::dict();
}
size_t maxRows = 0;
@ -191,7 +191,7 @@ bp::dict EnvironmentBlock::GetData()
}
CxInt64 numKeys = 0;
bp::dict dict = bp::dict();
pb::dict dict = pb::dict();
for (size_t i = 0; i < _columns.size(); i++)
{
PyColumnBase* column = _columns[i];
@ -205,21 +205,22 @@ bp::dict EnvironmentBlock::GetData()
{
PyColumnSingle<signed char>* col = dynamic_cast<PyColumnSingle<signed char>*>(column);
auto shrd = col->GetData();
bp::list list;
pb::list list;
for (size_t i = 0; i < shrd->size(); i++)
{
bp::object obj;
pb::object obj;
signed char value = shrd->at(i);
if (value < 0)
obj = bp::object(NAN);
obj = pb::cast<double>(NAN);
else if (value == 0)
obj = bp::object(false);
obj = pb::cast<bool>(false);
else
obj = bp::object(true);
obj = pb::cast<bool>(true);
list.append(obj);
}
dict[_names[i]] = list;
dict[pb::str(_names[i])] = list;
}
break;
case BL:

Просмотреть файл

@ -82,7 +82,7 @@ public:
EnvironmentBlock(int verbosity = 0, int maxSlots = -1, int seed = 42, const char* pythonPath = NULL);
~EnvironmentBlock();
std::string GetErrorMessage() { return _errMessage; }
bp::dict GetData();
pb::dict GetData();
private:
static MANAGED_CALLBACK(void) DataSink(EnvironmentBlock *penv, const DataViewBlock *pdata, void **&setters, void *&keyValueSetter);

Просмотреть файл

@ -2,14 +2,6 @@
<Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="DbgWinPy2.7|x64">
<Configuration>DbgWinPy2.7</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="DbgWinPy3.5|x64">
<Configuration>DbgWinPy3.5</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="DbgWinPy3.6|x64">
<Configuration>DbgWinPy3.6</Configuration>
<Platform>x64</Platform>
@ -18,12 +10,8 @@
<Configuration>DbgWinPy3.7</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="RlsWinPy2.7|x64">
<Configuration>RlsWinPy2.7</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="RlsWinPy3.5|x64">
<Configuration>RlsWinPy3.5</Configuration>
<ProjectConfiguration Include="DbgWinPy3.8|x64">
<Configuration>DbgWinPy3.8</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="RlsWinPy3.6|x64">
@ -34,6 +22,10 @@
<Configuration>RlsWinPy3.7</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="RlsWinPy3.8|x64">
<Configuration>RlsWinPy3.8</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<Import Project="..\CommonCpp.props" />
<PropertyGroup Label="Globals">
@ -42,71 +34,53 @@
<RootNamespace>NativeBridge</RootNamespace>
<ProjectName>NativeBridge</ProjectName>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy2.7|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy2.7|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.7|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.8|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.6|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.5|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.7|x64'" Label="Configuration">
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.8|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.6|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.5|x64'" Label="Configuration">
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.7|x64'" Label="Configuration">
<ConfigurationType>DynamicLibrary</ConfigurationType>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy2.7|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy2.7|x64'" Label="PropertySheets">
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.6|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.7|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.6|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.5|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.7|x64'" Label="PropertySheets">
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.8|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.6|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.5|x64'" Label="PropertySheets">
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.7|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.8|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy2.7|x64'">
<LinkIncremental>true</LinkIncremental>
<TargetExt>.pyd</TargetExt>
<TargetName>pybridge</TargetName>
<OutDir>..\..\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy2.7|x64'">
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.7|x64'">
<LinkIncremental>false</LinkIncremental>
<TargetExt>.pyd</TargetExt>
<TargetName>pybridge</TargetName>
<OutDir>..\..\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.7|x64'">
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.8|x64'">
<LinkIncremental>false</LinkIncremental>
<TargetExt>.pyd</TargetExt>
<TargetName>pybridge</TargetName>
@ -118,13 +92,7 @@
<TargetName>pybridge</TargetName>
<OutDir>..\..\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.5|x64'">
<LinkIncremental>false</LinkIncremental>
<TargetExt>.pyd</TargetExt>
<TargetName>pybridge</TargetName>
<OutDir>..\..\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.7|x64'">
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.8|x64'">
<LinkIncremental>false</LinkIncremental>
<TargetExt>.pyd</TargetExt>
<TargetName>pybridge</TargetName>
@ -136,40 +104,21 @@
<TargetName>pybridge</TargetName>
<OutDir>..\..\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.5|x64'">
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.7|x64'">
<LinkIncremental>false</LinkIncremental>
<TargetExt>.pyd</TargetExt>
<TargetName>pybridge</TargetName>
<OutDir>..\..\$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy2.7|x64'">
<ClCompile>
<PrecompiledHeader>Use</PrecompiledHeader>
<WarningLevel>Level3</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>CORECLR;_DEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;BOOST_USE_STATIC_LIBS;BOOST_PYTHON_STATIC_LIB;BOOST_ALL_NO_LIB;BOOST_NUMPY_STATIC_LIB;_HAS_ITERATOR_DEBUGGING;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(BoostRoot)\Include;$(PythonRoot)\include</AdditionalIncludeDirectories>
<MinimalRebuild>true</MinimalRebuild>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
<FunctionLevelLinking>true</FunctionLevelLinking>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<SetChecksum>true</SetChecksum>
<AdditionalDependencies>libboost_numpy-vc140-mt-gd-1_64.lib;libboost_python-vc140-mt-gd-1_64.lib</AdditionalDependencies>
<AdditionalLibraryDirectories>$(Boostroot)\lib;$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.5|x64'">
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.6|x64'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>Use</PrecompiledHeader>
<Optimization>Disabled</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>false</IntrinsicFunctions>
<PreprocessorDefinitions>CORECLR;_DEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;BOOST_USE_STATIC_LIBS;BOOST_PYTHON_STATIC_LIB;BOOST_ALL_NO_LIB;BOOST_NUMPY_STATIC_LIB;_HAS_ITERATOR_DEBUGGING;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(BoostRoot)\Include;$(PythonRoot)\include</AdditionalIncludeDirectories>
<PreprocessorDefinitions>CORECLR;_DEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;_HAS_ITERATOR_DEBUGGING;NOMINMAX;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(PythonRoot)\include</AdditionalIncludeDirectories>
<MinimalRebuild>true</MinimalRebuild>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
</ClCompile>
@ -179,8 +128,7 @@
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<SetChecksum>true</SetChecksum>
<AdditionalDependencies>libboost_numpy3-vc140-mt-gd-1_64.lib;libboost_python3-vc140-mt-gd-1_64.lib</AdditionalDependencies>
<AdditionalLibraryDirectories>$(Boostroot)\lib;$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
<AdditionalLibraryDirectories>$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.7|x64'">
@ -190,8 +138,8 @@
<Optimization>Disabled</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>false</IntrinsicFunctions>
<PreprocessorDefinitions>CORECLR;_DEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;BOOST_USE_STATIC_LIBS;BOOST_PYTHON_STATIC_LIB;BOOST_ALL_NO_LIB;BOOST_NUMPY_STATIC_LIB;_HAS_ITERATOR_DEBUGGING;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(BoostRoot)\Include;$(PythonRoot)\include</AdditionalIncludeDirectories>
<PreprocessorDefinitions>CORECLR;_DEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;_HAS_ITERATOR_DEBUGGING;NOMINMAX;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(PythonRoot)\include</AdditionalIncludeDirectories>
<MinimalRebuild>true</MinimalRebuild>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
</ClCompile>
@ -201,19 +149,18 @@
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<SetChecksum>true</SetChecksum>
<AdditionalDependencies>libboost_numpy37-vc140-mt-gd-1_69.lib;libboost_python37-vc140-mt-gd-1_69.lib</AdditionalDependencies>
<AdditionalLibraryDirectories>$(Boostroot)\lib;$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
<AdditionalLibraryDirectories>$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.6|x64'">
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.8|x64'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>Use</PrecompiledHeader>
<Optimization>Disabled</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>false</IntrinsicFunctions>
<PreprocessorDefinitions>CORECLR;_DEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;BOOST_USE_STATIC_LIBS;BOOST_PYTHON_STATIC_LIB;BOOST_ALL_NO_LIB;BOOST_NUMPY_STATIC_LIB;_HAS_ITERATOR_DEBUGGING;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(BoostRoot)\Include;$(PythonRoot)\include</AdditionalIncludeDirectories>
<PreprocessorDefinitions>CORECLR;_DEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;_HAS_ITERATOR_DEBUGGING;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(PythonRoot)\include</AdditionalIncludeDirectories>
<MinimalRebuild>true</MinimalRebuild>
<RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
</ClCompile>
@ -223,48 +170,7 @@
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<SetChecksum>true</SetChecksum>
<AdditionalDependencies>libboost_numpy3-vc140-mt-gd-1_64.lib;libboost_python3-vc140-mt-gd-1_64.lib</AdditionalDependencies>
<AdditionalLibraryDirectories>$(Boostroot)\lib;$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy2.7|x64'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>Use</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>CORECLR;NDEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;BOOST_USE_STATIC_LIBS;BOOST_PYTHON_STATIC_LIB;BOOST_ALL_NO_LIB;BOOST_NUMPY_STATIC_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(BoostRoot)\Include;$(PythonRoot)\include</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<SetChecksum>true</SetChecksum>
<AdditionalDependencies>libboost_python-vc140-mt-1_64.lib;libboost_numpy-vc140-mt-1_64.lib</AdditionalDependencies>
<AdditionalLibraryDirectories>$(Boostroot)\lib;$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.5|x64'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>Use</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>CORECLR;NDEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;BOOST_USE_STATIC_LIBS;BOOST_PYTHON_STATIC_LIB;BOOST_ALL_NO_LIB;BOOST_NUMPY_STATIC_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(BoostRoot)\Include;$(PythonRoot)\include</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<SetChecksum>true</SetChecksum>
<AdditionalDependencies>libboost_python3-vc140-mt-1_64.lib;libboost_numpy3-vc140-mt-1_64.lib</AdditionalDependencies>
<AdditionalLibraryDirectories>$(Boostroot)\lib;$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
<AdditionalLibraryDirectories>$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.6|x64'">
@ -274,8 +180,8 @@
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>CORECLR;NDEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;BOOST_USE_STATIC_LIBS;BOOST_PYTHON_STATIC_LIB;BOOST_ALL_NO_LIB;BOOST_NUMPY_STATIC_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(BoostRoot)\Include;$(PythonRoot)\include</AdditionalIncludeDirectories>
<PreprocessorDefinitions>CORECLR;NDEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;NOMINMAX;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(PythonRoot)\include</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
@ -283,8 +189,7 @@
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<SetChecksum>true</SetChecksum>
<AdditionalDependencies>libboost_python3-vc140-mt-1_64.lib;libboost_numpy3-vc140-mt-1_64.lib</AdditionalDependencies>
<AdditionalLibraryDirectories>$(Boostroot)\lib;$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
<AdditionalLibraryDirectories>$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.7|x64'">
@ -294,8 +199,8 @@
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>CORECLR;NDEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;BOOST_USE_STATIC_LIBS;BOOST_PYTHON_STATIC_LIB;BOOST_ALL_NO_LIB;BOOST_NUMPY_STATIC_LIB;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(BoostRoot)\Include;$(PythonRoot)\include</AdditionalIncludeDirectories>
<PreprocessorDefinitions>CORECLR;NDEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(PythonRoot)\include</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
@ -303,8 +208,26 @@
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<SetChecksum>true</SetChecksum>
<AdditionalDependencies>libboost_python37-vc140-mt-1_69.lib;libboost_numpy37-vc140-mt-1_69.lib</AdditionalDependencies>
<AdditionalLibraryDirectories>$(Boostroot)\lib;$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
<AdditionalLibraryDirectories>$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.8|x64'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>Use</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>CORECLR;NDEBUG;_WINDOWS;_USRDLL;PYBRIDGE_EXPORTS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>$(PythonRoot)\include</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<SetChecksum>true</SetChecksum>
<AdditionalLibraryDirectories>$(PythonRoot)\libs;$(OutDir)</AdditionalLibraryDirectories>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
@ -322,14 +245,12 @@
<ClCompile Include="ManagedInterop.cpp" />
<ClCompile Include="PythonInterop.cpp" />
<ClCompile Include="stdafx.cpp">
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='DbgWinPy2.7|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='RlsWinPy2.7|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.8|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.7|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.6|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='DbgWinPy3.5|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.8|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.7|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.6|x64'">Create</PrecompiledHeader>
<PrecompiledHeader Condition="'$(Configuration)|$(Platform)'=='RlsWinPy3.5|x64'">Create</PrecompiledHeader>
</ClCompile>
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />

Просмотреть файл

@ -94,7 +94,7 @@ template <class T, class T2> PyColumnBase* PyColumnBase::CreateVariable(const in
}
template <class T>
void PyColumnSingle<T>::AddToDict(bp::dict& dict,
void PyColumnSingle<T>::AddToDict(pb::dict& dict,
const std::string& name,
const std::vector<std::string>* keyNames,
const size_t expectedRows)
@ -105,44 +105,36 @@ void PyColumnSingle<T>::AddToDict(bp::dict& dict,
{
case DataKind::BL:
{
bp::handle<> h(::PyCapsule_New((void*)this, NULL, (PyCapsule_Destructor)&destroyManagerCObject));
dict[name] = np::from_data(
data,
np::dtype::get_builtin<bool>(),
bp::make_tuple(_pData->size()),
bp::make_tuple(sizeof(bool)), bp::object(h));
pb::handle h(::PyCapsule_New((void*)this, NULL, (PyCapsule_Destructor)&destroyManagerCObject));
dict[pb::str(name)] = pb::array(pb::dtype("bool"), _pData->size(), data, h);
}
break;
case DataKind::I1:
case DataKind::I2:
case DataKind::I4:
{
bp::handle<> h(::PyCapsule_New((void*)this, NULL, (PyCapsule_Destructor)&destroyManagerCObject));
np::ndarray npdata = np::from_data(
data,
np::dtype::get_builtin<T>(),
bp::make_tuple(_pData->size()),
bp::make_tuple(sizeof(float)), bp::object(h));
pb::handle h(::PyCapsule_New((void*)this, NULL, (PyCapsule_Destructor)&destroyManagerCObject));
pb::array npdata = pb::array(_pData->size(), data, h);
if (keyNames == nullptr)
{
dict[name] = npdata;
dict[pb::str(name)] = npdata;
}
else
{
dict[name] = bp::dict();
dict[name]["..Data"] = npdata;
bp::list list;
dict[pb::str(name)] = pb::dict();
dict[pb::str(name)]["..Data"] = npdata;
pb::list list;
for (int j = 0; j < keyNames->size(); j++)
{
bp::object obj;
pb::object obj;
const std::string& value = keyNames->at(j);
if (!value.empty())
{
obj = bp::object(value);
obj = pb::str(value);
}
list.append(obj);
}
dict[name]["..KeyValues"] = list;
dict[pb::str(name)]["..KeyValues"] = list;
}
}
break;
@ -154,48 +146,37 @@ void PyColumnSingle<T>::AddToDict(bp::dict& dict,
case DataKind::R4:
case DataKind::R8:
{
bp::handle<> h(::PyCapsule_New((void*)this, NULL, (PyCapsule_Destructor)&destroyManagerCObject));
dict[name] = np::from_data(
data,
np::dtype::get_builtin<T>(),
bp::make_tuple(_pData->size()),
bp::make_tuple(sizeof(T)), bp::object(h));
pb::handle h(::PyCapsule_New((void*)this, NULL, (PyCapsule_Destructor)&destroyManagerCObject));
dict[pb::str(name)] = pb::array(_pData->size(), data, h);
}
break;
case DataKind::DT:
{
bp::handle<> h(::PyCapsule_New((void*)this, NULL, (PyCapsule_Destructor)&destroyManagerCObject));
np::ndarray npdata = np::from_data(
data,
np::dtype::get_builtin<T>(),
bp::make_tuple(_pData->size()),
bp::make_tuple(sizeof(T)), bp::object(h));
pb::capsule h(::PyCapsule_New((void*)this, NULL, (PyCapsule_Destructor)&destroyManagerCObject));
pb::array npdata = pb::array(_pData->size(), data, h);
dict[name] = bp::dict();
dict[name]["..DateTime"] = npdata;
dict[pb::str(name)] = pb::dict();
dict[pb::str(name)]["..DateTime"] = npdata;
}
break;
}
}
template <>
void PyColumnSingle<std::string>::AddToDict(bp::dict& dict,
void PyColumnSingle<std::string>::AddToDict(pb::dict& dict,
const std::string& name,
const std::vector<std::string>* keyNames,
const size_t expectedRows)
{
bp::list list;
pb::list list;
for (size_t i = 0; i < _pData->size(); i++)
{
bp::object obj;
pb::object obj;
const std::string& value = _pData->at(i);
if (!value.empty())
{
obj = bp::object(value);
}
obj = pb::str(value);
list.append(obj);
}
dict[name] = list;
dict[pb::str(name)] = list;
}
template <class T, class T2>
@ -252,7 +233,7 @@ void PyColumnVariable<T, T2>::Deleter(PyObject* obj)
}
template<class T, class T2>
void PyColumnVariable<T, T2>::AddToDict(bp::dict& dict,
void PyColumnVariable<T, T2>::AddToDict(pb::dict& dict,
const std::string& name,
const std::vector<std::string>* keyNames,
const size_t expectedRows)
@ -295,7 +276,7 @@ void PyColumnVariable<T, T2>::AddToDict(bp::dict& dict,
}
template<class T, class T2>
void PyColumnVariable<T, T2>::AddColumnToDict(bp::dict& dict,
void PyColumnVariable<T, T2>::AddColumnToDict(pb::dict& dict,
const std::string& name,
size_t index)
{
@ -305,35 +286,31 @@ void PyColumnVariable<T, T2>::AddColumnToDict(bp::dict& dict,
deleteData->instance = this;
deleteData->column = index;
bp::handle<> h(::PyCapsule_New((void*)deleteData, NULL, (PyCapsule_Destructor)&Deleter));
dict[name] = np::from_data(
data,
np::dtype::get_builtin<T2>(),
bp::make_tuple(_data[index]->size()),
bp::make_tuple(sizeof(T2)), bp::object(h));
pb::handle h(::PyCapsule_New((void*)deleteData, NULL, (PyCapsule_Destructor)&Deleter));
dict[pb::str(name)] = pb::array(_data[index]->size(), data, h);
}
template<>
void PyColumnVariable<std::string, NullableString>::AddColumnToDict(bp::dict& dict,
void PyColumnVariable<std::string, NullableString>::AddColumnToDict(pb::dict& dict,
const std::string& name,
size_t index)
{
bp::list list;
pb::list list;
std::vector<NullableString>* pColData = _data[index];
size_t numRows = pColData->size();
for (size_t i = 0; i < numRows; i++)
{
bp::object obj;
pb::object obj;
NullableString value = pColData->at(i);
if (value)
{
obj = bp::object(*value);
obj = value;
}
list.append(obj);
}
dict[name] = list;
dict[pb::str(name)] = list;
}

Просмотреть файл

@ -4,7 +4,6 @@
#pragma once
#include <map>
#include <vector>
#include <boost/optional.hpp>
// Taken from ML.NET source code. These values should be stable.
@ -70,7 +69,7 @@ public:
PyColumn(const int& kind) : PyColumnBase(kind) {}
virtual ~PyColumn() {}
virtual void SetAt(size_t nRow, size_t nCol, const T& value) = 0;
virtual void AddToDict(bp::dict& dict,
virtual void AddToDict(pb::dict& dict,
const std::string& name,
const std::vector<std::string>* keyNames,
const size_t expectedRows) = 0;
@ -90,7 +89,7 @@ public:
PyColumnSingle(const int& kind, size_t numRows = 0);
virtual ~PyColumnSingle();
virtual void SetAt(size_t nRow, size_t nCol, const T& value);
virtual void AddToDict(bp::dict& dict,
virtual void AddToDict(pb::dict& dict,
const std::string& name,
const std::vector<std::string>* keyNames,
const size_t expectedRows);
@ -135,8 +134,7 @@ inline size_t PyColumnSingle<T>::GetNumCols()
return 1;
}
typedef boost::optional<std::string> NullableString;
typedef pb::object NullableString;
/*
* Handles the variable value case.
@ -154,7 +152,7 @@ public:
PyColumnVariable(const int& kind, size_t numRows = 0);
virtual ~PyColumnVariable();
virtual void SetAt(size_t nRow, size_t nCol, const T& value);
virtual void AddToDict(bp::dict& dict,
virtual void AddToDict(pb::dict& dict,
const std::string& name,
const std::vector<std::string>* keyNames,
const size_t expectedRows);
@ -164,7 +162,7 @@ public:
T2 GetMissingValue();
T2 GetConvertedValue(const T& value);
void AddColumnToDict(bp::dict& dict, const std::string& name, size_t index);
void AddColumnToDict(pb::dict& dict, const std::string& name, size_t index);
public:
typedef struct
@ -220,11 +218,11 @@ inline T2 PyColumnVariable<T, T2>::GetConvertedValue(const T& value)
template <>
inline NullableString PyColumnVariable<std::string, NullableString>::GetMissingValue()
{
return boost::none;
return pb::cast<pb::none>(Py_None);;
}
template <>
inline NullableString PyColumnVariable<std::string, NullableString>::GetConvertedValue(const std::string& value)
{
return value;
return pb::str(value);
}

Просмотреть файл

@ -7,10 +7,9 @@ usage()
echo "Usage: $0 --configuration <Configuration> "
echo ""
echo "Options:"
echo " --configuration <Configuration> Build Configuration (DbgLinPy3.7,DbgLinPy3.6,DbgLinPy3.5,DbgLinPy2.7,RlsLinPy3.7,RlsLinPy3.6,RlsLinPy3.5,RlsLinPy2.7,DbgMacPy3.7,DbgMacPy3.6,DbgMacPy3.5,DbgMacPy2.7,RlsMacPy3.7,RlsMacPy3.6,RlsMacPy3.5,RlsMacPy2.7)"
echo " --pythonver <Python version> Python version number (3.7, 3.6, 3.5, 2.7)"
echo " --configuration <Configuration> Build Configuration (DbgLinPy3.8,DbgLinPy3.7,DbgLinPy3.6,RlsLinPy3.8,RlsLinPy3.7,RlsLinPy3.6,DbgMacPy3.8,DbgMacPy3.7,DbgMacPy3.6,RlsMacPy3.8,RlsMacPy3.7,RlsMacPy3.6)"
echo " --pythonver <Python version> Python version number (3.8, 3.7, 3.6)"
echo " --pythonpath <Python path> Path to python library."
echo " --boostpath <Boost path> Path to boost library."
exit 1
}
@ -27,7 +26,6 @@ __configuration=DbgLinPy3.7
__pythonver=3.7
__rootBinPath="$RootRepo/x64"
__pythonpath=""
__boostpath=""
while [ "$1" != "" ]; do
lowerI="$(echo $1 | awk '{print tolower($0)}')"
@ -52,10 +50,6 @@ while [ "$1" != "" ]; do
shift
__pythonpath=$1
;;
--boostpath)
shift
__boostpath=$1
;;
*)
echo "Unknown argument to build.sh $1"; usage; exit 1
esac
@ -81,7 +75,7 @@ else
echo "Using default system compiler ... "
fi
__cmake_defines="-DCMAKE_BUILD_TYPE=${__configuration} -DPYTHON_VER=${__pythonver} -DPYTHON_DIR=${__pythonpath} -DBOOST_DIR=${__boostpath}"
__cmake_defines="-DCMAKE_BUILD_TYPE=${__configuration} -DPYTHON_VER=${__pythonver} -DPYTHON_DIR=${__pythonpath}"
__IntermediatesDir="$__rootBinPath/$__configuration/obj"
rm -rf "$__IntermediatesDir"

Просмотреть файл

@ -67,17 +67,18 @@ void translate_mlnet_exception(MlNetExecutionError const& exc)
::PyErr_SetString(::PyExc_RuntimeError, exc.what());
}
bp::dict pxCall(bp::dict& params)
pb::dict pxCall(pb::dict& params)
{
bp::dict res = bp::dict();
pb::dict res = pb::dict();
try
{
bp::extract<std::string> graph(params[PARAM_GRAPH]);
bp::extract<std::string> mlnetPath(params[PARAM_MLNET_PATH]);
bp::extract<std::string> dotnetClrPath(params[PARAM_DOTNETCLR_PATH]);
bp::extract<std::string> dprepPath(params[PARAM_DPREP_PATH]);
bp::extract<std::string> pythonPath(params[PARAM_PYTHON_PATH]);
bp::extract<std::int32_t> verbose(params[PARAM_VERBOSE]);
auto graph = pb::cast<std::string>(params[PARAM_GRAPH]);
auto mlnetPath = pb::cast<std::string>(params[PARAM_MLNET_PATH]);
auto dotnetClrPath = pb::cast<std::string>(params[PARAM_DOTNETCLR_PATH]);
auto dprepPath = pb::cast<std::string>(params[PARAM_DPREP_PATH]);
auto pythonPath = pb::cast<std::string>(params[PARAM_PYTHON_PATH]);
auto verbose = pb::cast<std::int32_t>(params[PARAM_VERBOSE]);
std::int32_t i_verbose = std::int32_t(verbose);
std::string s_mlnetPath = std::string(mlnetPath);
std::string s_dotnetClrPath = std::string(dotnetClrPath);
@ -94,18 +95,19 @@ bp::dict pxCall(bp::dict& params)
+ s_mlnetPath + " and " + s_dotnetClrPath);
int seed = 42;
if (params.has_key(PARAM_SEED))
seed = bp::extract<int>(params[PARAM_SEED]);
if (params.contains(PARAM_SEED))
seed = pb::cast<int>(params[PARAM_SEED]);
int maxSlots = -1;
if (params.has_key(PARAM_MAX_SLOTS))
maxSlots = bp::extract<int>(params[PARAM_MAX_SLOTS]);
if (params.contains(PARAM_MAX_SLOTS))
maxSlots = pb::cast<int>(params[PARAM_MAX_SLOTS]);
EnvironmentBlock env(i_verbose, maxSlots, seed, s_pythonPath.c_str());
int retCode;
if (params.has_key(PARAM_DATA) && bp::extract<bp::dict>(params[PARAM_DATA]).check())
if (params.contains(PARAM_DATA) && pb::isinstance<pb::dict>(params[PARAM_DATA]))
{
bp::dict d = bp::extract<bp::dict>(params[PARAM_DATA]);
pb::dict d = pb::cast<pb::dict>(params[PARAM_DATA]);
DataSourceBlock data(d);
const DataSourceBlock *datas[1];
datas[0] = &data;
@ -123,7 +125,7 @@ bp::dict pxCall(bp::dict& params)
{
throw MlNetExecutionError(e.what());
}
catch (bp::error_already_set const&)
catch (pb::error_already_set const&)
{
PyErr_Print();
}
@ -131,7 +133,7 @@ bp::dict pxCall(bp::dict& params)
return res;
}
BOOST_PYTHON_MODULE(pybridge)
PYBIND11_MODULE(pybridge, m)
{
//The managed code assumes that each pointer occupies 8 bytes.
assert(sizeof(void*) == 8);
@ -141,11 +143,19 @@ BOOST_PYTHON_MODULE(pybridge)
//
Py_Initialize();
//
// initialize numpy types
//
np::initialize();
static pb::exception<MlNetExecutionError> exc(m, "MlNetExecutionError");
pb::register_exception_translator([](std::exception_ptr p) {
try {
if (p)
std::rethrow_exception(p);
}
catch (const MlNetExecutionError &em) {
::PyErr_SetString(::PyExc_RuntimeError, em.what());
}
catch (const std::exception &e) {
exc(e.what());
}
});
bp::register_exception_translator<MlNetExecutionError>(&translate_mlnet_exception);
def("px_call", pxCall);
m.def("px_call", pxCall);
}

Просмотреть файл

@ -35,81 +35,19 @@ typedef unsigned char BYTE;
#define MANAGED_CALLBACK_PTR(ret, name) ret ( *name)
#define CLASS_ALIGN __attribute__((aligned(8)))
#ifdef BOOST_NO_CXX11_NULLPTR
#define nullptr 0
#endif //BOOST_NO_CXX11_NULLPTR
typedef long long CxInt64;
typedef unsigned long long CxUInt64;
typedef unsigned char BYTE;
#endif //_MSC_VER
#include <boost/python.hpp>
#include <pybind11/pybind11.h>
#include <string>
#include <exception>
#include <boost/python/numpy.hpp>
#include <boost/python/scope.hpp>
#include <pybind11/numpy.h>
#include <sysmodule.h>
// #define MEASURE_PERFORMANCE 1
#if defined(MEASURE_PERFORMANCE)
#include <chrono>
#include <functional>
#include <iostream>
#include <sstream>
class StopWatch
{
public:
typedef std::function<void(const char *)> PrintFunction;
StopWatch(const char* description, PrintFunction function = nullptr) :
m_description(description),
m_printFunction(function)
{
m_startTime = std::chrono::high_resolution_clock::now();
if (m_printFunction == nullptr)
{
m_printFunction = [](const char * message) { std::cout << message; };
}
}
~StopWatch()
{
auto endTime = std::chrono::high_resolution_clock::now();
std::stringstream buffer;
buffer << m_description << ":" << ((endTime - m_startTime).count() / 1000000) << " msecs" << std::endl;
m_printFunction(buffer.str().c_str());
}
private:
//
// data members
//
std::chrono::steady_clock::time_point m_startTime;
PrintFunction m_printFunction;
const char* m_description;
};
#define TOKENPASTE(x, y) x ## y
#define TOKENPASTE2(x, y) TOKENPASTE(x, y)
#define MEASURE_ELAPSED_TIME(description) StopWatch TOKENPASTE2(_stopWatch_, __LINE__)(description)
#else
#define MEASURE_ELAPSED_TIME(description)
#endif
//
// frequently used namespace aliases
//
namespace bp = boost::python;
namespace np = boost::python::numpy;
namespace pb = pybind11;

Просмотреть файл

@ -5,7 +5,7 @@
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.1</TargetFramework>
<Platforms>x64</Platforms>
<Configurations>DbgWinPy3.7;DbgWinPy3.6;DbgWinPy3.5;DbgWinPy2.7;RlsWinPy3.7;RlsWinPy3.6;RlsWinPy3.5;RlsWinPy2.7;DbgLinPy3.7;DbgLinPy3.6;DbgLinPy3.5;DbgLinPy2.7;RlsLinPy3.7;RlsLinPy3.6;RlsLinPy3.5;RlsLinPy2.7;RlsMacPy3.7;RlsMacPy3.6</Configurations>
<Configurations>DbgWinPy3.7;DbgWinPy3.6;DbgWinPy3.8;RlsWinPy3.7;RlsWinPy3.6;RlsWinPy3.8;DbgLinPy3.7;DbgLinPy3.6;DbgLinPy3.8;RlsLinPy3.7;RlsLinPy3.6;RlsLinPy3.8;RlsMacPy3.7;RlsMacPy3.6;RlsMacPy3.8</Configurations>
<OutputPath>$(ProjectDir)..\..\x64\$(Configuration)\Platform\</OutputPath>
<AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
</PropertyGroup>

Просмотреть файл

@ -2,7 +2,7 @@
Microsoft Machine Learning for Python
"""
__version__ = '1.7.1'
__version__ = '1.8.0'
# CoreCLR version of MicrosoftML is built on Windows.
# But file permissions are not preserved when it's copied to Linux.

Просмотреть файл

@ -4,7 +4,6 @@
# --------------------------------------------------------------------------------------------
# - Generated by tools/entrypoint_compiler.py: do not edit by hand
class Exp:
"""
Some of the trainers accept a loss parameter that will be used for

Просмотреть файл

@ -166,4 +166,5 @@ class TestNumericDataType(unittest.TestCase):
if __name__ == '__main__':
TestNumericDataType().test_check_datatype_single_sparse_list_sparse()
unittest.main()

Просмотреть файл

@ -39,7 +39,6 @@ class TestNGramFeaturizer(unittest.TestCase):
vector_normalizer='None') << 'SentimentText'
X_train = texttransform.fit_transform(X_train[:100])
sum = X_train.iloc[:].sum().sum()
print(sum)
assert_equal(sum, 30513, "sum of all features is incorrect!")

Просмотреть файл

@ -4,7 +4,7 @@
# --------------------------------------------------------------------------------------------
import os
import platform
import distro
import unittest
import pandas as pd
@ -35,8 +35,8 @@ class TestWordEmbedding(unittest.TestCase):
# (path? access rights?)
@unittest.skipIf(
os.name != "nt" and (
platform.linux_distribution()[0] != "Ubuntu" or
platform.linux_distribution()[1] != "16.04"),
distro.linux_distribution(full_distribution_name=False)[0] != "Ubuntu" or
distro.linux_distribution(full_distribution_name=False)[1] != "16.04"),
"not supported on this platform")
def test_ssweembedding(self):
wordvectors = pd.DataFrame(data=dict(w1=["like", "hate", "okay"],
@ -69,8 +69,8 @@ class TestWordEmbedding(unittest.TestCase):
# (path? access rights?)
@unittest.skipIf(
os.name != "nt" and (
platform.linux_distribution()[0] != "Ubuntu" or
platform.linux_distribution()[1] != "16.04"),
distro.linux_distribution(full_distribution_name=False)[0] != "Ubuntu" or
distro.linux_distribution(full_distribution_name=False)[1] != "16.04"),
"not supported on this platform")
def test_word_embedding_example(self):
path = get_dataset('infert').as_filepath()
@ -110,8 +110,8 @@ class TestWordEmbedding(unittest.TestCase):
# (path? access rights?)
@unittest.skipIf(
os.name != "nt" and (
platform.linux_distribution()[0] != "Ubuntu" or
platform.linux_distribution()[1] != "16.04"),
distro.linux_distribution(full_distribution_name=False)[0] != "Ubuntu" or
distro.linux_distribution(full_distribution_name=False)[1] != "16.04"),
"not supported on this platform")
def test_word_embedding_example2(self):
path = get_dataset('infert').as_filepath()
@ -148,8 +148,8 @@ class TestWordEmbedding(unittest.TestCase):
# (path? access rights?)
@unittest.skipIf(
os.name != "nt" and (
platform.linux_distribution()[0] != "Ubuntu" or
platform.linux_distribution()[1] != "16.04"),
distro.linux_distribution(full_distribution_name=False)[0] != "Ubuntu" or
distro.linux_distribution(full_distribution_name=False)[1] != "16.04"),
"not supported on this platform")
def test_word_embedding_example_dict_same_name(self):
path = get_dataset('infert').as_filepath()

Просмотреть файл

@ -4,7 +4,7 @@
# --------------------------------------------------------------------------------------------
import os
import platform
import distro
import unittest
import numpy as np
@ -135,8 +135,8 @@ class TestSweep(unittest.TestCase):
# machines to download resources for wordembedding transform
@unittest.skipIf(
os.name != "nt" and (
platform.linux_distribution()[0] != "Ubuntu" or
platform.linux_distribution()[1] != "16.04"),
distro.linux_distribution(full_distribution_name=False)[0] != "Ubuntu" or
distro.linux_distribution(full_distribution_name=False)[1] != "16.04"),
"not supported on this platform")
def test_NGramFeaturizer_sweep(self):
# grid search over number_of_trees and then confirm the best number_of_trees by
@ -188,8 +188,8 @@ class TestSweep(unittest.TestCase):
# machines to download resources for wordembedding transform
@unittest.skipIf(
os.name != "nt" and (
platform.linux_distribution()[0] != "Ubuntu" or
platform.linux_distribution()[1] != "16.04"),
distro.linux_distribution(full_distribution_name=False)[0] != "Ubuntu" or
distro.linux_distribution(full_distribution_name=False)[1] != "16.04"),
"not supported on this platform")
def test_NGramFeaturizer_glove(self):
# grid search over number_of_trees and then confirm the best number_of_trees by

Просмотреть файл

@ -34,6 +34,8 @@ class TestNaiveBayesClassifier(unittest.TestCase):
X_train = texttransform.fit_transform(X_train)
X_test = texttransform.transform(X_test)
unique = set(X_train.columns)
assert len(unique) == X_train.shape[1]
mymodel = NaiveBayesClassifier()
mymodel.fit(X_train, y_train)

Просмотреть файл

@ -114,7 +114,7 @@ class TestPredictProba(unittest.TestCase):
def test_pass_predict_proba_multiclass_3class(self):
clf = FastLinearClassifier(number_of_threads=1)
clf.fit(X_train_3class, y_train_3class)
clf.fit(X_train_3class, y_train_3class, verbose=0)
s = clf.predict_proba(X_test_3class).sum()
assert_almost_equal(
s,
@ -228,7 +228,7 @@ class TestDecisionFunction(unittest.TestCase):
def test_pass_decision_function_multiclass_3class(self):
clf = FastLinearClassifier(number_of_threads=1)
clf.fit(X_train_3class, y_train_3class)
clf.fit(X_train_3class, y_train_3class, verbose=0)
s = clf.decision_function(X_test_3class).sum()
assert_almost_equal(
s,

Просмотреть файл

@ -3,7 +3,7 @@
# Licensed under the MIT License.
# --------------------------------------------------------------------------------------------
import platform
import distro
import unittest
import numpy as np
@ -162,7 +162,7 @@ class TestDataWithMissing(unittest.TestCase):
assert_equal(result.dtypes['f1'], np.object)
assert_equal(result.dtypes['f2.f2'], np.float32)
@unittest.skipIf('centos' in platform.linux_distribution()[0].lower(), "centos is not supported")
@unittest.skipIf('centos' in distro.linux_distribution(full_distribution_name=False)[0].lower(), "centos is not supported")
def test_category_imputation(self):
data={'f0': [4, 4, np.nan, 9],
'f1': [4, 4, np.nan, np.nan]}

Просмотреть файл

@ -2,7 +2,7 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
# --------------------------------------------------------------------------------------------
import platform
import distro
import unittest
import pandas
@ -10,7 +10,7 @@ from nimbusml import Pipeline
from nimbusml.preprocessing.normalization import RobustScaler
@unittest.skipIf('centos' in platform.linux_distribution()[0].lower(), "centos is not supported")
@unittest.skipIf('centos' in distro.linux_distribution(full_distribution_name=False)[0].lower(), "centos is not supported")
class TestRobustScaler(unittest.TestCase):
def test_with_integer_inputs(self):

Просмотреть файл

@ -3,7 +3,7 @@
# Licensed under the MIT License.
# --------------------------------------------------------------------------------------------
import platform
import distro
import unittest
import pandas
@ -13,7 +13,7 @@ from nimbusml.preprocessing.schema import ColumnSelector
from sklearn.utils.testing import assert_equal
@unittest.skipIf('centos' in platform.linux_distribution()[0].lower(), "centos is not supported")
@unittest.skipIf('centos' in distro.linux_distribution(full_distribution_name=False)[0].lower(), "centos is not supported")
class TestDateTimeSplitter(unittest.TestCase):
def test_check_estimator_DateTimeSplitter(self):

Просмотреть файл

@ -3,7 +3,7 @@
# Licensed under the MIT License.
# --------------------------------------------------------------------------------------------
import platform
import os
import unittest
import numpy as np
@ -11,7 +11,7 @@ import pandas as pd
from nimbusml.preprocessing import ToKeyImputer
@unittest.skipIf('centos' in platform.linux_distribution()[0].lower(), "centos is not supported")
@unittest.skipIf(os.name != "nt", "not supported on this platform")
class TestToKeyImputer(unittest.TestCase):
def test_tokeyimputer(self):

Просмотреть файл

@ -3,7 +3,7 @@
# Licensed under the MIT License.
# --------------------------------------------------------------------------------------------
import platform
import distro
import unittest
import numpy as np
@ -12,7 +12,7 @@ from nimbusml.preprocessing import ToString
from sklearn.utils.testing import assert_equal
@unittest.skipIf('centos' in platform.linux_distribution()[0].lower(), "centos is not supported")
@unittest.skipIf('centos' in distro.linux_distribution(full_distribution_name=False)[0].lower(), "centos is not supported")
class TestToString(unittest.TestCase):
def test_tostring(self):

Просмотреть файл

@ -9,6 +9,7 @@ import unittest
import numpy as np
import pandas as pd
from scipy.sparse.csr import csr_matrix
from nimbusml import Pipeline as nimbusmlPipeline
from nimbusml.datasets import get_dataset
from nimbusml.ensemble import FastTreesBinaryClassifier

Просмотреть файл

@ -6,7 +6,7 @@
test sparse
"""
import unittest
import warnings
import numpy as np
import pandas as pd
from nimbusml.linear_model import FastLinearBinaryClassifier
@ -135,7 +135,9 @@ def test_dtype(xtype=None, ytype=None, dense=False):
class TestDTypes(unittest.TestCase):
def test_data_types(self):
types = [
# float16 is not supported, move it in first position to fail faster
None,
np.float16,
np.bool,
np.int8,
np.int16,
@ -147,10 +149,9 @@ class TestDTypes(unittest.TestCase):
np.uint64,
np.double,
np.float,
np.float16
]
for xtype in types:
for ytype in types:
for ytype in types:
for xtype in types:
print(
"================ Testing sparse xtype %s, ytype %s "
"================" %

Просмотреть файл

@ -39,7 +39,7 @@ class TestSyntax(unittest.TestCase):
OneHotVectorizer(),
FastLinearBinaryClassifier(maximum_number_of_iterations=1)
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -59,7 +59,7 @@ class TestSyntax(unittest.TestCase):
OneHotVectorizer(max_num_terms=2) << 'workclass',
FastLinearBinaryClassifier(maximum_number_of_iterations=1)
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -79,7 +79,7 @@ class TestSyntax(unittest.TestCase):
OneHotVectorizer(max_num_terms=2) << 'workclass',
FastLinearBinaryClassifier(maximum_number_of_iterations=1)
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -105,7 +105,7 @@ class TestSyntax(unittest.TestCase):
# every available column at one step.
FastLinearBinaryClassifier(maximum_number_of_iterations=1)
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -127,7 +127,7 @@ class TestSyntax(unittest.TestCase):
Concat() << {'Inputs': ['edu1', 'edu2', 'wki']},
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << 'Inputs'
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -149,7 +149,7 @@ class TestSyntax(unittest.TestCase):
Concat() << {'Inputs': ['edu1', 'edu2', 'wki']},
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << 'Inputs'
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -171,7 +171,7 @@ class TestSyntax(unittest.TestCase):
Concat() << {'Inputs': ['edu1', 'edu2', 'wki']},
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << 'Inputs'
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -193,7 +193,7 @@ class TestSyntax(unittest.TestCase):
Concat(columns={'Inputs': ['edu1', 'edu2', 'wki']}),
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << 'Inputs'
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -218,7 +218,7 @@ class TestSyntax(unittest.TestCase):
'wki']
])
try:
exp.fit(X, y)
exp.fit(X, y, verbose=0)
assert False
except RuntimeError as e:
assert "ConcatTransform() << {'Input': ['edu1', 'edu2', 'wki']}" \
@ -242,7 +242,7 @@ class TestSyntax(unittest.TestCase):
'wki']
])
try:
exp.fit(X, y)
exp.fit(X, y, verbose=0)
raise AssertionError("The test should not reach this line.")
except Exception as e:
assert "Feature column 'edu4' not found" in str(e)
@ -261,7 +261,7 @@ class TestSyntax(unittest.TestCase):
Concat() << {'Features': ['f%d' % i for i in range(1, 4)]},
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << 'Features'
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -289,7 +289,7 @@ class TestSyntax(unittest.TestCase):
Concat() << {'Features': 'f[0-9]+'},
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << 'Features'
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -312,7 +312,7 @@ class TestSyntax(unittest.TestCase):
Drop() << ['education', 'workclass', 'f1', 'f2', 'f3'],
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << ['Features']
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -335,7 +335,7 @@ class TestSyntax(unittest.TestCase):
Drop() << ['education', 'workclass', 'f1', 'f2', 'f3'],
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << 'FeaturesCustom'
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -364,7 +364,7 @@ class TestSyntax(unittest.TestCase):
Drop() << ['education', 'workclass', 'f1', 'f2', 'f3'],
FastLinearBinaryClassifier(maximum_number_of_iterations=1) << ['Features']
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -388,7 +388,7 @@ class TestSyntax(unittest.TestCase):
Drop() << '~Features',
FastLinearBinaryClassifier(maximum_number_of_iterations=1)
])
exp.fit(X, y)
exp.fit(X, y, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -458,7 +458,7 @@ class TestSyntax(unittest.TestCase):
transform_1 = NGramFeaturizer(word_feature_extractor=n_gram())
transform_2 = MutualInformationSelector()
exp = Pipeline([transform_1, transform_2])
res = exp.fit_transform(X, y)
res = exp.fit_transform(X, y, verbose=0)
assert res is not None
# Scikit compatibility (Compose transforms inside Scikit Pipeline).
@ -466,7 +466,7 @@ class TestSyntax(unittest.TestCase):
transform_1 = NGramFeaturizer(word_feature_extractor=n_gram())
transform_2 = MutualInformationSelector(slots_in_output=2)
pipe = Pipeline([transform_1, transform_2])
res = pipe.fit_transform(X, y)
res = pipe.fit_transform(X, y, verbose=0)
assert res is not None
def test_syntax10_multi_output1(self):
@ -482,7 +482,7 @@ class TestSyntax(unittest.TestCase):
normed = LogMeanVarianceScaler() << {
'Petal_Normed': 'Petal_Length',
'Sepal_Normed': 'Sepal_Width'}
out_df = normed.fit_transform(in_df)
out_df = normed.fit_transform(in_df, verbose=0)
self.assertEqual(sorted(list(out_df.columns)),
['Petal_Length', 'Petal_Normed', 'Sepal_Length',
'Sepal_Normed', 'Sepal_Width', 'Species'])
@ -501,7 +501,7 @@ class TestSyntax(unittest.TestCase):
'Petal_Normed': 'Petal_Length',
'Sepal_Normed': 'Sepal_Width'}
normed = LogMeanVarianceScaler() << newcols
out_df = normed.fit_transform(in_df)
out_df = normed.fit_transform(in_df, verbose=0)
self.assertEqual(sorted(list(out_df.columns)),
['Petal_Length', 'Petal_Normed', 'Sepal_Length',
'Sepal_Normed', 'Sepal_Width', 'Species'])
@ -520,7 +520,7 @@ class TestSyntax(unittest.TestCase):
'edu2': 'education'}, FastLinearBinaryClassifier(
maximum_number_of_iterations=1) << {
'Features': ['edu1', 'edu2'], Role.Label: 'y'}])
exp.fit(df)
exp.fit(df, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [
@ -551,7 +551,7 @@ class TestSyntax(unittest.TestCase):
del exp[-1]
assert len(exp) == 3
exp.fit(df)
exp.fit(df, verbose=0)
prediction = exp.predict(X)
assert isinstance(prediction, pandas.DataFrame)
assert sorted(list(prediction.columns)) == [

Просмотреть файл

@ -3,7 +3,7 @@
# Licensed under the MIT License.
# --------------------------------------------------------------------------------------------
import platform
import distro
import unittest
import numpy as np
@ -11,7 +11,7 @@ import pandas as pd
from nimbusml.timeseries import TimeSeriesImputer
@unittest.skipIf('centos' in platform.linux_distribution()[0].lower(), "centos is not supported")
@unittest.skipIf('centos' in distro.linux_distribution(full_distribution_name=False)[0].lower(), "centos is not supported")
class TestTimeSeriesImputer(unittest.TestCase):
def test_timeseriesimputer_adds_new_row(self):

Просмотреть файл

@ -45,7 +45,7 @@ setup(
# Versions should comply with PEP440. For a discussion on
# single-sourcing the version across setup.py and the project code, see
# https://packaging.python.org/en/latest/single_source_version.html
version='1.7.1',
version='1.8.0',
description='NimbusML',
long_description=long_description,
@ -138,7 +138,7 @@ setup(
'onnxruntime',
],
python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <3.8.*',
python_requires='>3.5.*, <3.9.*',
# If there are data files included in your packages that need to be
# installed, specify them here. If using Python 2.6 or less,

Просмотреть файл

@ -138,7 +138,7 @@ setup(
'onnxruntime'
],
python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, <3.8.*',
python_requires='>3.5.*, <3.9.*',
# If there are data files included in your packages that need to be
# installed, specify them here. If using Python 2.6 or less,

Просмотреть файл

@ -7,7 +7,7 @@ run check_estimator tests
"""
import json
import os
import platform
import distro
import unittest
from nimbusml.cluster import KMeansPlusPlus
@ -284,7 +284,7 @@ skip_epoints = set([
'TimeSeriesImputer'
])
if 'centos' in platform.linux_distribution()[0].lower():
if 'centos' in distro.linux_distribution(full_distribution_name=False)[0].lower():
skip_epoints |= set([
'DateTimeSplitter',
'RobustScaler',

Просмотреть файл

@ -3,7 +3,7 @@
# Licensed under the MIT License.
# -------------------------------------------------------------------------
import os
import platform
import distro
import subprocess
import sys
import unittest
@ -87,8 +87,14 @@ def get_examples():
for folder, name in folder_files:
if name in ['__init__.py',]:
continue
# skip for all linux & mac tests, windows is ok
if os.name != "nt":
if name in [
'ToKeyImputer.py',
'ToKeyImputer_df.py']:
continue
# skip for all linux tests, mac is ok
if os.name == "posix" and platform.linux_distribution()[0] != '':
if os.name == "posix" and distro.linux_distribution(full_distribution_name=False)[0] != '':
if name in [
# SymSgdNative fails to load on linux
'SymSgdBinaryClassifier.py',

Просмотреть файл

@ -7,7 +7,7 @@ test ipython notebooks in nimbusml
"""
import json
import os
import platform
import distro
import sys
import time
import unittest
@ -50,8 +50,8 @@ class TestDocsNotebooks(unittest.TestCase):
os.name == 'nt',
"Not supported on this platform without using Anaconda.")
# REVIEW: Figure out how to spin off kernels on linux with no Anaconda
@unittest.skipIf(platform.linux_distribution()[0] == "Ubuntu" and
platform.linux_distribution()[
@unittest.skipIf(distro.linux_distribution(full_distribution_name=False)[0] == "Ubuntu" and
distro.linux_distribution(full_distribution_name=False)[
1] == "16.04", "not supported on this platform")
def test_notebooks(self):
# These tests are blocking PRs b/c they fail in linux build. Disabling

Просмотреть файл

@ -157,7 +157,11 @@ SKIP = {
'CountSelector',
'KMeansPlusPlus',
'ToKey',
'ColumnSelector'
'ColumnSelector',
# below are fixed in ML.NET 1.5, broken 1.5.preview2
'FastLinearClassifier',
'LogisticRegressionClassifier',
'OneHotVectorizer'
}
INSTANCES = {

Просмотреть файл

@ -1 +1 @@
1.7.1
1.8.0