Add v1.0.0 of the Python Extension (#6)

* Add python extension

* update README and add newlines

* Update Logger.h

* modify README to point each file reference to its file

.gitignore

@@ -328,3 +328,6 @@ ASALocalRun/
 
 # MFractors (Xamarin productivity tool) working folder 
 .mfractor/
+
+# Visual Studio build files
+*.vcxproj

language-extensions/python/LICENSE_1_0.txt

@@ -0,0 +1,23 @@
+Boost Software License - Version 1.0 - August 17th, 2003
+
+Permission is hereby granted, free of charge, to any person or organization
+obtaining a copy of the software and accompanying documentation covered by
+this license (the "Software") to use, reproduce, display, distribute,
+execute, and transmit the Software, and to prepare derivative works of the
+Software, and to permit third-parties to whom the Software is furnished to
+do so, all subject to the following:
+
+The copyright notices in the Software and this entire statement, including
+the above license grant, this restriction and the following disclaimer,
+must be included in all copies of the Software, in whole or in part, and
+all derivative works of the Software, unless such copies or derivative
+works are solely in the form of machine-executable object code generated by
+a source language processor.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
+SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
+FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+DEALINGS IN THE SOFTWARE.

language-extensions/python/README.md

@@ -0,0 +1,56 @@
+# Python Language Extension 
+
+## Getting Started
+Language Extensions is a feature of SQL Server used for executing external code. The relational data can be used in the external code using the extensibility framework.
+
+For more information about SQL Server Language Extensions, refer to this [documentation](https://docs.microsoft.com/en-us/sql/language-extensions/language-extensions-overview?view=sql-server-ver15).
+
+The Python extension version in this repository is compatible with SQL Server 2019 CU3 onwards. 
+
+Note that the Python Extension released in the current repository works with Python 3.7.x - to use the released package, follow (this tutorial)[https://docs.microsoft.com/en-us/sql/machine-learning/install/custom-runtime-py?view=sql-server-ver15]. For any other version of Python (3.6, 3.8, etc) you must modify and rebuild the Python Extension binaries using the following instructions.
+
+## Building
+
+### Windows
+1. Install [CMake for Windows](https://cmake.org/download/) and [Python](https://www.python.org/downloads/release/python-379/).
+
+1. Install the package numpy into your python. This is needed to build boost_numpy in the next step.
+
+1. Download and build [Boost Python](https://www.boost.org/doc/libs/1_74_0/libs/python/doc/html/building/no_install_quickstart.html) with your version of Python. Make sure to build the static libraries.
+
+1. Install C++ Tools for CMake from the [Build Tools for Visual Studio 2017](https://my.visualstudio.com/Downloads?q=visual%20studio%202017&wt.mc_id=o~msft~vscom~older-downloads). 
+		Download the Visual Studio 2017 Build Tools installer and check the **Visual C++ build tools** option under Workloads. In the sidebar on the right, make sure **Visual C++ tools for CMake** is checked, then install.
+		
+1. Set CMAKE_ROOT, PYTHONHOME, and BOOST_ROOT pointing to their respective installation and build folders.
+
+1. Modify [**language-extensions\python\src\CMakeLists.txt**](.\src\CMakeLists.txt). Change each `find_library` call to point to your custom python and boost libraries. 
+
+1. Run [**build-python-extension.cmd**](.\build\windows\build-python-extension.cmd) which will generate: \
+		- PATH\TO\ENLISTMENT\build-output\pythonextension\windows\release\release\pythonextension.dll 
+		
+1. Run [**create-python-extension-zip.cmd**](.\build\windows\create-python-extension-zip.cmd) which will generate: \
+		- PATH\TO\ENLISTMENT\build-output\pythonextension\windows\release\release\packages\python-lang-extension.zip \
+		This zip can be used in CREATE EXTERNAL LANGUAGE, as detailed in the tutorial in the Usage section below.
+
+### Linux
+1. Install [CMake for Linux](https://cmake.org/download/) and [Python](https://www.python.org/downloads/source/)
+
+1. Install the package numpy into your python. This is needed to build boost_numpy in the next step.
+
+1. Download and build [Boost Python](https://www.boost.org/doc/libs/1_74_0/libs/python/doc/html/building/no_install_quickstart.html) with your version of Python. Make sure to build the static libraries.
+
+1. Set PYTHONHOME and BOOST_ROOT to point to your python installation and boost build folder respectively.
+
+1. Modify [**language-extensions\python\src\CMakeLists.txt**](.\src\CMakeLists.txt). Change each `find_library` call to point to your custom python and boost libraries. 
+
+1. Run [**build-python-extension.sh**](.\build\linux\build-python-extension.sh) which will generate: \
+		- PATH/TO/ENLISTMENT/build-output/pythonextension/linux/release/libPythonExtension.so.1.0 
+
+1. Run [**create-python-extension-zip.sh**](.\build\linux\create-python-extension-zip.sh) which will generate: \
+		- PATH/TO/ENLISTMENT/build-output/pythonextension/linux/release/packages/python-lang-extension.zip \
+		This zip can be used in CREATE EXTERNAL LANGUAGE, as detailed in the tutorial in the Usage section below.
+
+## Usage
+After downloading or building the Python extension zip, use CREATE EXTERNAL LANGUAGE to create the language on the SQL Server. 
+
+This [tutorial](https://docs.microsoft.com/en-us/sql/machine-learning/install/custom-runtime-py?view=sql-server-ver15) will walk you through an end to end sample using the Python language extension. 

language-extensions/python/build/linux/build-python-extension.sh

@@ -0,0 +1,126 @@
+#!/bin/bash
+
+function check_exit_code {
+	EXIT_CODE=$?
+	if [ ${EXIT_CODE} -eq 0 ]; then
+		echo $1
+	else
+		echo $2
+		exit ${EXIT_CODE}
+	fi
+}
+
+function build {
+	# Set cmake config to first arg
+	#
+	CMAKE_CONFIGURATION=$1
+
+	# Output directory and output lib name
+	#
+	TARGET=${ENL_ROOT}/build-output/pythonextension/target/${CMAKE_CONFIGURATION}
+
+	# Create the output directories
+	#
+	mkdir -p ${TARGET}
+
+	if ! [[ -d ${PYTHONEXTENSION_WORKING_DIR} ]]; then
+		mkdir -p ${PYTHONEXTENSION_WORKING_DIR}
+	fi
+
+	cd ${PYTHONEXTENSION_WORKING_DIR}
+
+	# Compile
+	#
+	cmake -DPLATFORM=Linux \
+		-DENL_ROOT=${ENL_ROOT} \
+		-DCMAKE_BUILD_TYPE=${CMAKE_CONFIGURATION} \
+		-DPYTHONHOME=${PYTHONHOME} \
+		-DBOOST_PYTHON_ROOT=${BOOST_PYTHON_ROOT} \
+		-DINCLUDE_ROOT=${INCLUDE_ROOT} \
+		${PYTHONEXTENSION_HOME}/src
+	cmake --build ${PYTHONEXTENSION_WORKING_DIR} --config ${CMAKE_CONFIGURATION} --target install
+
+	# Check the exit code of the compiler and exit appropriately so that build will fail.
+	#
+	check_exit_code "Success: Built libPythonExtension.so.1.0" "Error: Failed to build python extension"
+
+	# Move the generated libs to configuration folder
+	#
+	mkdir -p ${CMAKE_CONFIGURATION}
+	mv libPythonExtension.so* ${CMAKE_CONFIGURATION}/
+
+	cd ${CMAKE_CONFIGURATION}/
+
+	# This will create the python extension package with unsigned binaries, this is used for local development and non-release builds. release
+	# builds will call create-python-extension-zip.sh after the binaries have been signed and this will be included in the zip
+	#
+	zip ${TARGET}/python-lang-extension libPythonExtension.so.1.0
+
+	check_exit_code "Success: Created python-lang-extension.zip" "Error: Failed to create zip for python extension"
+}
+
+# Enlistment root and location of pythonextension
+#
+SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+ENL_ROOT=${SCRIPTDIR}/../../../..
+
+# Set environment variables required in Cmake
+#
+PYTHONEXTENSION_HOME=${ENL_ROOT}/language-extensions/python
+PYTHONEXTENSION_WORKING_DIR=${ENL_ROOT}/build-output/pythonextension/linux
+
+INCLUDE_ROOT=/usr/include
+
+DEFAULT_PYTHONHOME=/usr
+DEFAULT_BOOST_ROOT=/usr/lib/boost_1_69_0
+
+# Find PYTHONHOME from user, or set to default for tests.
+# Error code 1 is generic bash error.
+#
+if [ -z "${PYTHONHOME}" ]; then
+	if [ -x "${DEFAULT_PYTHONHOME}" ]; then
+		PYTHONHOME=${DEFAULT_PYTHONHOME}
+	else
+		echo "PYTHONHOME is empty but needs to be set to build the python extension"
+		exit 1
+	fi
+fi
+
+# Find BOOST_ROOT from user, or set to default for tests.
+# Error code 1 is generic bash error.
+#
+if [ -z "${BOOST_ROOT}" ]; then
+	if [ -x "${DEFAULT_BOOST_ROOT}" ]; then
+		BOOST_ROOT=${DEFAULT_BOOST_ROOT}
+	else
+		echo "BOOST_ROOT is empty but needs to be set to build the python extension"
+		exit 1
+	fi
+fi
+
+DEFAULT_BOOST_PYTHON_ROOT=${BOOST_ROOT}/stage/lib
+
+# Find BOOST_PYTHON_ROOT from user, or set to default for tests.
+# Error code 1 is generic bash error.
+#
+if [ -z "${BOOST_PYTHON_ROOT}" ]; then
+	if [ -x "${DEFAULT_BOOST_PYTHON_ROOT}" ]; then
+		BOOST_PYTHON_ROOT=${DEFAULT_BOOST_PYTHON_ROOT}
+	else
+		echo "BOOST_PYTHON_ROOT is empty but needs to be set to build the python extension"
+		exit 1
+	fi
+fi
+
+# Build in release mode if nothing is specified
+#
+if [ "$1" == "" ]; then
+	set -- release
+fi
+
+while [ "$1" != "" ]; do
+	# Advance arg passed to build.cmd
+	#
+	build $1
+	shift
+done;

language-extensions/python/build/linux/create-python-extension-zip.sh

@@ -0,0 +1,41 @@
+#!/bin/bash
+
+function check_exit_code {
+	EXIT_CODE=$?
+	if [ ${EXIT_CODE} -eq 0 ]; then
+		echo "Success: Created zip for $1 config"
+	else
+		echo "Error: Failed to create zip for $1 config"
+		exit ${EXIT_CODE}
+	fi
+}
+
+
+function build {
+	BUILD_CONFIGURATION=$1
+	
+	SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+	ENL_ROOT=${SCRIPTDIR}/../../../..
+
+	BUILD_OUTPUT=${ENL_ROOT}/build-output/pythonextension/linux/${BUILD_CONFIGURATION}
+
+	mkdir -p ${BUILD_OUTPUT}/packages
+	cd ${BUILD_OUTPUT}
+	zip packages/python-lang-extension libPythonExtension.so.1.0
+	check_exit_code  ${BUILD_CONFIGURATION}
+}
+
+# Build in release mode if nothing is specified
+#
+if [ "$1" == "" ]; then
+	set -- release
+fi
+
+while [ "$1" != "" ]; do
+	# Advance arg passed to this script
+	#
+	build $1
+	shift
+done;
+
+exit $?

language-extensions/python/build/windows/build-python-extension.cmd

@@ -0,0 +1,150 @@
+@ECHO off
+SETLOCAL
+
+REM Nuget packages directory and location of python libs
+REM
+SET ENL_ROOT=%~dp0..\..\..\..
+SET PACKAGES_ROOT=%ENL_ROOT%\packages
+SET PYTHONEXTENSION_HOME=%ENL_ROOT%\language-extensions\python
+SET PYTHONEXTENSION_WORKING_DIR=%ENL_ROOT%\build-output\pythonextension\windows
+
+IF EXIST %PYTHONEXTENSION_WORKING_DIR% (RMDIR /s /q %PYTHONEXTENSION_WORKING_DIR%)
+MKDIR %PYTHONEXTENSION_WORKING_DIR%
+
+SET DEFAULT_BOOST_ROOT=%PACKAGES_ROOT%\External-Boost.master.Boost.1.69.0.1020
+SET DEFAULT_BOOST_PYTHON_ROOT=%DEFAULT_BOOST_ROOT%\windows\lib
+SET DEFAULT_PYTHONHOME=%PACKAGES_ROOT%\python
+SET DEFAULT_CMAKE_ROOT=%PACKAGES_ROOT%\CMake-win64.3.15.5
+
+REM Find boost, python, and cmake paths from user, or set to default for tests.
+REM 
+SET ENVVAR_NOT_FOUND=203
+
+IF "%BOOST_ROOT%" == "" (
+	IF EXIST %DEFAULT_BOOST_ROOT% (
+		SET BOOST_ROOT=%DEFAULT_BOOST_ROOT%
+	) ELSE (
+		CALL :CHECKERROR %ENVVAR_NOT_FOUND% "Error: BOOST_ROOT variable must be set to build the python extension" || EXIT /b %ENVVAR_NOT_FOUND%
+	)
+)
+
+IF "%BOOST_PYTHON_ROOT%" == "" (
+	IF EXIST %DEFAULT_BOOST_PYTHON_ROOT% (
+		SET BOOST_PYTHON_ROOT=%DEFAULT_BOOST_PYTHON_ROOT%
+	) ELSE (
+		CALL :CHECKERROR %ENVVAR_NOT_FOUND% "Error: BOOST_PYTHON_ROOT variable must be set to build the python extension" || EXIT /b %ENVVAR_NOT_FOUND%
+	)
+)
+
+IF "%PYTHONHOME%" == "" (
+	IF EXIST %DEFAULT_PYTHONHOME% (
+		SET PYTHONHOME=%DEFAULT_PYTHONHOME%
+	) ELSE (
+		CALL :CHECKERROR %ENVVAR_NOT_FOUND% "Error: PYTHONHOME variable must be set to build the python extension" || EXIT /b %ENVVAR_NOT_FOUND%
+	)
+)
+
+IF "%CMAKE_ROOT%" == "" (
+	IF EXIST %DEFAULT_CMAKE_ROOT% (
+		SET CMAKE_ROOT=%DEFAULT_CMAKE_ROOT%
+	) ELSE (
+		CALL :CHECKERROR %ENVVAR_NOT_FOUND% "Error: CMAKE_ROOT variable must be set to build the python extension" || EXIT /b %ENVVAR_NOT_FOUND%
+	)
+)
+
+:LOOP
+
+REM Set cmake config to first arg
+REM
+SET CMAKE_CONFIGURATION=%1
+
+REM *Setting CMAKE_CONFIGURATION to anything but "debug" will set CMAKE_CONFIGURATION to "release".
+REM The string comparison for CMAKE_CONFIGURATION is case-insensitive.
+REM
+IF NOT DEFINED CMAKE_CONFIGURATION (SET CMAKE_CONFIGURATION=release)
+IF /I NOT %CMAKE_CONFIGURATION%==debug (SET CMAKE_CONFIGURATION=release)
+
+REM Output directory and output dll name
+REM
+SET TARGET="%ENL_ROOT%\build-output\pythonextension\target\%CMAKE_CONFIGURATION%"
+
+REM Remove existing output files
+REM
+IF EXIST %TARGET% (RMDIR /s /q %TARGET%)
+
+REM Create the output directories
+REM
+mkdir %TARGET%
+
+REM VSCMD_START_DIR set the working directory to this variable after calling VsDevCmd.bat
+REM otherwise, it will default to %USERPROFILE%\Source
+REM
+SET VSCMD_START_DIR=%ENL_ROOT%
+
+REM Do not call VsDevCmd if the environment is already set. Otherwise, it will keep appending
+REM to the PATH environment variable and it will be too long for windows to handle.
+REM
+if not defined DevEnvDir (
+	call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\Tools\VsDevCmd.bat" -arch=amd64 -host_arch=amd64
+)
+
+ECHO "[INFO] Generating Python extension project build files using CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%"
+
+SET BUILD_OUTPUT=%PYTHONEXTENSION_WORKING_DIR%\%CMAKE_CONFIGURATION%
+MKDIR %BUILD_OUTPUT%
+PUSHD %BUILD_OUTPUT%
+
+REM Call cmake
+REM
+CALL "%CMAKE_ROOT%\bin\cmake.exe" ^
+	-G "Visual Studio 15 2017 Win64" ^
+	-DPLATFORM=Windows ^
+	-DENL_ROOT="%ENL_ROOT%" ^
+	-DCMAKE_BUILD_TYPE=%CMAKE_CONFIGURATION% ^
+	-DPYTHONHOME="%PYTHONHOME%" ^
+	-DBOOST_ROOT="%BOOST_ROOT%" ^
+	-DBOOST_PYTHON_ROOT="%BOOST_PYTHON_ROOT%" ^
+	%PYTHONEXTENSION_HOME%/src
+
+CALL :CHECKERROR %ERRORLEVEL% "Error: Failed to generate make files for CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%" || EXIT /b %ERRORLEVEL%
+
+ECHO "[INFO] Building Python extension project using CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%"
+
+REM Call cmake build
+REM
+CALL "%CMAKE_ROOT%\bin\cmake.exe" --build . --config %CMAKE_CONFIGURATION% --target INSTALL
+
+CALL :CHECKERROR %ERRORLEVEL% "Error: Failed to build Python extension for CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%" || EXIT /b %ERRORLEVEL%
+
+REM Copy DLL, LIB, etc files out of debug/debug and release/release into the build output folder
+REM
+copy %BUILD_OUTPUT%\%CMAKE_CONFIGURATION%\* %BUILD_OUTPUT%\
+
+REM This will create the Python extension package with unsigned binaries, this is used for local development and non-release builds. 
+REM Release builds will call create-python-extension-zip.cmd after the binaries have been signed and this will be included in the zip
+REM
+IF /I %CMAKE_CONFIGURATION%==debug (
+	powershell -NoProfile -ExecutionPolicy Unrestricted -Command "Compress-Archive -Force -Path %BUILD_OUTPUT%\pythonextension.dll, %BUILD_OUTPUT%\pythonextension.pdb -DestinationPath %TARGET%\python-lang-extension.zip"
+) ELSE (
+	powershell -NoProfile -ExecutionPolicy Unrestricted -Command "Compress-Archive -Force -Path %BUILD_OUTPUT%\pythonextension.dll -DestinationPath %TARGET%\python-lang-extension.zip"
+)
+
+CALL :CHECKERROR %ERRORLEVEL% "Error: Failed to create zip for Python extension for CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%" || EXIT /b %ERRORLEVEL%
+
+REM Advance arg passed to build-pythonextension.cmd
+REM
+SHIFT
+
+REM Continue building using more configs until argv has been exhausted
+REM
+IF NOT "%~1"=="" GOTO LOOP
+
+EXIT /b %ERRORLEVEL%
+
+:CHECKERROR
+	IF %1 NEQ 0 (
+		ECHO %2
+		EXIT /b %1
+	)
+
+	EXIT /b 0

language-extensions/python/build/windows/create-python-extension-zip.cmd

@@ -0,0 +1,42 @@
+@ECHO off
+SETLOCAL
+
+SET ENL_ROOT=%~dp0..\..\..\..
+
+:LOOP
+
+REM Set cmake config to first arg
+REM
+SET BUILD_CONFIGURATION=%1
+
+REM Setting BUILD_CONFIGURATION to anything but "debug" will set BUILD_CONFIGURATION to "release".
+REM The string comparison for BUILD_CONFIGURATION is case-insensitive.
+REM
+IF NOT DEFINED BUILD_CONFIGURATION (SET BUILD_CONFIGURATION=release)
+IF /I NOT %BUILD_CONFIGURATION%==debug (SET BUILD_CONFIGURATION=release)
+
+SET BUILD_OUTPUT=%ENL_ROOT%\build-output\pythonextension\windows\%BUILD_CONFIGURATION%
+
+mkdir %BUILD_OUTPUT%\packages
+
+powershell -NoProfile -ExecutionPolicy Unrestricted -Command "Compress-Archive -Path %BUILD_OUTPUT%\pythonextension.dll -DestinationPath %BUILD_OUTPUT%\packages\python-lang-extension.zip -Force"
+
+CALL :CHECK_BUILD_ERROR %ERRORLEVEL% %BUILD_CONFIGURATION%
+
+REM Advance arg passed to create-python-extension.cmd
+REM
+SHIFT
+
+REM Continue building using more configs until argv has been exhausted
+REM
+IF NOT "%~1"=="" GOTO LOOP
+
+EXIT /b %ERRORLEVEL%
+
+:CHECK_BUILD_ERROR
+	IF %1 == 0 (
+		ECHO Success: Created zip for %2 config
+	) ELSE (
+		ECHO Error: Failed to create zip for %2 config
+		EXIT /b %1
+	)

language-extensions/python/include/Common.h

@@ -0,0 +1,30 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: Common.h
+//
+// Purpose:
+//  Common headers for extension
+//
+//*************************************************************************************************
+
+#pragma once
+
+#ifdef _WIN64
+#include <windows.h>
+#endif
+
+#include <string>
+#include <iostream>
+#include <exception>
+#include <stdio.h>
+#include <vector>
+#include <sql.h>
+#include <sqlext.h>
+#include <sqltypes.h>
+
+#include <boost/python.hpp>
+#include <boost/python/numpy.hpp>

language-extensions/python/include/Logger.h

@@ -0,0 +1,45 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: Logger.h
+//
+// Purpose:
+//  Logging functions for extension
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+
+#define LOG(msg) Logger::Log(msg)
+#define LOG_ERROR(msg) Logger::LogError(msg)
+#define LOG_EXCEPTION(e) Logger::LogException(e)
+
+class Logger
+{
+public:
+	// Log an error to stderr
+	//
+	static void LogError(const std::string &errorMsg);
+
+	// Log a extension exception to stderr
+	//
+	static void LogException(const std::exception &e);
+
+	// Log a message to stdout
+	//
+	static void Log(const std::string &msg);
+
+private:
+	// Get a string of the current timestamp in the same format
+	// of SQL format
+	//
+	static const std::string GetCurrentTimestamp();
+
+	// Buffer to hold the timestamp string.
+	//
+	static char sm_timestampBuffer[];
+};

language-extensions/python/include/PythonColumn.h

@@ -0,0 +1,75 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonColumn.h
+//
+// Purpose:
+//  Encapsulate dataset column attributes
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+
+// Encapsulate column information
+//
+class PythonColumn
+{
+public:
+	PythonColumn(
+		const SQLCHAR *columnName,
+		SQLSMALLINT   columnNameLength,
+		SQLSMALLINT   dataType,
+		SQLULEN       columnSize,
+		SQLSMALLINT   decimalDigits,
+		SQLSMALLINT   nullable);
+
+	const std::string& Name() const
+	{
+		return m_name;
+	}
+
+	SQLSMALLINT DataType() const
+	{
+		return m_dataType;
+	}
+
+	SQLULEN Size() const
+	{
+		return m_size;
+	}
+
+	SQLSMALLINT DecimalDigits() const
+	{
+		return m_decimalDigits;
+	}
+
+	SQLSMALLINT Nullable() const
+	{
+		return m_nullable;
+	}
+
+protected:
+	// Name of the column.
+	//
+	std::string m_name;
+
+	// Data Type of the column.
+	//
+	SQLSMALLINT m_dataType;
+
+	// Size of the column.
+	//
+	SQLULEN m_size;
+
+	// Decimal digits of the column.
+	//
+	SQLSMALLINT m_decimalDigits;
+
+	// The column is nullable or not.
+	//
+	SQLSMALLINT m_nullable;
+};

language-extensions/python/include/PythonDataSet.h

@@ -0,0 +1,370 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonDataSet.h
+//
+// Purpose:
+// Classes handling loading and retrieving data from a DataFrame.
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+#include "PythonColumn.h"
+#include "PythonExtensionUtils.h"
+
+#include <unordered_map>
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonDataSet
+//
+// Description:
+//  Base class storing information about the PythonExtension DataSet.
+//
+class PythonDataSet
+{
+public:
+
+	// Initialize the PythonDataSet.
+	//
+	void Init(
+		const SQLCHAR         *dataName,
+		SQLUSMALLINT          dataNameLength,
+		SQLUSMALLINT          schemaColumnsNumber,
+		boost::python::object mainNamespace);
+
+	// Check whether there are any SQL_NULL_DATA in the strLen_or_Ind
+	//
+	bool HasNulls(
+		SQLULEN    rowsNumber,
+		SQLINTEGER *strLen_or_Ind) const;
+
+	// Clean up the dataset from the namespace
+	//
+	void Cleanup();
+
+	// Returns the number of columns in the PythonDataSet from the vector of PythonColumns
+	// at the time this is called.
+	// Note: Once Init() resizes the vector to schemaColumnsNumber, its size
+	// doesn't increase when InitColumn is done for each column.
+	//
+	SQLUSMALLINT GetVectorColumnsNumber() const
+	{
+		return m_columns.size();
+	}
+
+	// Getter for m_columns.
+	//
+	const std::vector<std::unique_ptr<PythonColumn>>& Columns() const
+	{
+		return m_columns;
+	}
+
+	// Get the underlying pointer of m_columnNullMap.
+	//
+	SQLINTEGER** GetColumnNullMap()
+	{
+		return m_columnNullMap.data();
+	}
+
+protected:
+
+	// A protected constructor to stop instantiation of PythonDataSet
+	// but allow derived classes to be instantiated.
+	//
+	PythonDataSet()
+	{
+	}
+
+	// data type of simple python objects
+	//
+	const boost::python::numpy::dtype m_ObjType =
+		boost::python::numpy::array(boost::python::object()).get_dtype();
+
+	// Maps the ODBC C type to python type
+	//
+	static const std::unordered_map<std::string, SQLSMALLINT> sm_pythonToOdbcTypeMap;
+	typedef std::unordered_map<std::string, SQLSMALLINT> pythonToOdbcTypeMap;
+
+	// The underlying boost::python namespace, which contains all the python variables.
+	// We execute any python scripts on this namespace.
+	//
+	boost::python::object m_mainNamespace;
+
+	// Name of the dataset in python.
+	//
+	std::string m_name;
+
+	// A vector of PythonColumn objects representing the column info.
+	//
+	std::vector<std::unique_ptr<PythonColumn>> m_columns;
+
+	// A vector of pointers to strlen_or_Ind info of each column.
+	// For a column, strLen_or_Ind is specified for each row.
+	//
+	std::vector<SQLINTEGER*> m_columnNullMap;
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonInputDataSet
+//
+// Description:
+//  Class representing an input PythonDataSet for data load
+//  from PythonExtension to the namespace environment.
+//
+class PythonInputDataSet : public PythonDataSet
+{
+public:
+
+	// Initializes each Column of the member vector m_columns.
+	//
+	void InitColumn(
+		SQLUSMALLINT   columnNumber,
+		const SQLCHAR  *columnName,
+		SQLSMALLINT    columnNameLength,
+		SQLSMALLINT    dataType,
+		SQLULEN        columnSize,
+		SQLSMALLINT    decimalDigits,
+		SQLSMALLINT    nullable);
+
+	// Returns the number of columns in the PythonDataSet that have been added to
+	// to the underlying DataFrame at the time this is called.
+	// Note: The length returned increases as new columns are added to the DataFrame.
+	//
+	SQLUSMALLINT GetDataFrameColumnsNumber() const
+	{
+		return boost::python::len(m_dataDict.keys());
+	}
+
+	// Add columns to the underlying boost python dictionary with the given rowsNumber and data.
+	//
+	void AddColumnsToDictionary(
+		SQLULEN    rowsNumber,
+		SQLPOINTER *data,
+		SQLINTEGER **strLen_or_Ind);
+
+	// Adds the underlying boost::python dictionary to namespace as a pandas DataFrame.
+	//
+	void AddDictionaryToNamespace();
+
+private:
+	// Adds a column of values into the python dictionary
+	// Valid for integer, simple numeric, and boolean dataTypes.
+	//
+	template<class SQLType>
+	void AddColumnToDictionary(
+		SQLSMALLINT columnNumber,
+		SQLULEN     rowsNumber,
+		SQLPOINTER  data,
+		SQLINTEGER  *strLen_or_Ind);
+
+	// Adds a column of boolean values into the python dictionary
+	//
+	void AddBooleanColumnToDictionary(
+		SQLSMALLINT columnNumber,
+		SQLULEN     rowsNumber,
+		SQLPOINTER  data,
+		SQLINTEGER  *strLen_or_Ind);
+
+	// Adds a column of string values into the python dictionary
+	//
+	template<class CharType>
+	void AddStringColumnToDictionary(
+		SQLSMALLINT columnNumber,
+		SQLULEN     rowsNumber,
+		SQLPOINTER  data,
+		SQLINTEGER  *strLen_or_Ind);
+
+	// Adds a column of raw values into the python dictionary
+	//
+	void AddRawColumnToDictionary(
+		SQLSMALLINT columnNumber,
+		SQLULEN     rowsNumber,
+		SQLPOINTER  data,
+		SQLINTEGER  *strLen_or_Ind);
+
+	// Adds a column of date/datetime values into the python dictionary
+	//
+	template<class DateTimeStruct>
+	void AddDateTimeColumnToDictionary(
+		SQLSMALLINT columnNumber,
+		SQLULEN     rowsNumber,
+		SQLPOINTER  data,
+		SQLINTEGER  *strLen_or_Ind);
+
+	// Add column function pointer definition
+	//
+	using fnAddColumn = void (PythonInputDataSet::*)(
+		SQLSMALLINT columnNumber,
+		SQLULEN     rowsNumber,
+		SQLPOINTER  data,
+		SQLINTEGER  *strLen_or_Ind);
+
+	// Function map to add columns to the data frame and its typedef
+	//
+	static const std::unordered_map<SQLSMALLINT, fnAddColumn> sm_FnAddColumnMap;
+	typedef std::unordered_map<SQLSMALLINT, fnAddColumn> AddColumnFnMap;
+
+	// The underlying boost::python dictionary.
+	//
+	boost::python::dict m_dataDict;
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonOutputDataSet
+//
+// Description:
+//  Class representing an output PythonDataSet for data retrieval from namespace to PythonExtension.
+//
+class PythonOutputDataSet : public PythonDataSet
+{
+public:
+
+	// Initialize a DataFrame for OutputDataSet to the python namespace.
+	//
+	void InitializeDataFrameInNamespace();
+
+	// Get the number of columns in the underlying DataFrame
+	//
+	SQLUSMALLINT GetDataFrameColumnsNumber();
+
+	// Returns the list of names of the columns of the DataFrame
+	//
+	boost::python::list GetColumnNames();
+
+	// Gets column data from the DataFrame and stores it in the instance.
+	//
+	void RetrieveColumnsFromDataFrame();
+
+	// Get one of the columns from the underlying pandas DataFrame
+	//
+	boost::python::numpy::ndarray ExtractArrayFromDataFrame(const std::string columnName);
+
+	// Extract all the time stamp data from a Date / DateTime PyObject and return a TIMESTAMP_STRUCT.
+	//
+	SQL_TIMESTAMP_STRUCT ExtractTimestampFromPyObject(const PyObject *dateObject);
+
+	// Finds the data type of all columns in the DataFrame.
+	//
+	void PopulateColumnsDataType();
+
+	// Populate the m_rowsNumber field from the DataFrame
+	//
+	void PopulateNumberOfRows();
+
+	// Getter for numberOfRows.
+	//
+	SQLULEN RowsNumber() const
+	{
+		return m_rowsNumber;
+	}
+
+	// Get the underlying pointer of m_data.
+	//
+	SQLPOINTER* GetData()
+	{
+		return m_data.data();
+	}
+
+	// Call CleanupColumn on each column.
+	//
+	void CleanupColumns();
+
+private:
+	// Gets the column information, adds data to m_data and nullmap to m_columnNullMap
+	//
+	template<class SQLType, class NullType, SQLSMALLINT DataType>
+	void RetrieveColumnFromDataFrame(
+		std::string columnName,
+		SQLULEN     &columnSize,
+		SQLSMALLINT &decimalDigits,
+		SQLSMALLINT &nullable);
+
+	// Gets the boolean column information, adds data to m_data and nullmap to m_columnNullMap
+	//
+	void RetrieveBooleanColumnFromDataFrame(
+		std::string columnName,
+		SQLULEN     &columnSize,
+		SQLSMALLINT &decimalDigits,
+		SQLSMALLINT &nullable);
+
+	// Gets the string column information, adds data to m_data and nullmap to m_columnNullMap
+	//
+	void RetrieveStringColumnFromDataFrame(
+		std::string columnName,
+		SQLULEN     &columnSize,
+		SQLSMALLINT &decimalDigits,
+		SQLSMALLINT &nullable);
+
+	// Gets the raw column information, adds data to m_data and nullmap to m_columnNullMap
+	//
+	void RetrieveRawColumnFromDataFrame(
+		std::string columnName,
+		SQLULEN     &columnSize,
+		SQLSMALLINT &decimalDigits,
+		SQLSMALLINT &nullable);
+
+	// Gets the datetime column information, adds data to m_data and nullmap to m_columnNullMap
+	//
+	template<class DateTimeStruct>
+	void RetrieveDateTimeColumnFromDataFrame(
+		std::string columnName,
+		SQLULEN     &columnSize,
+		SQLSMALLINT &decimalDigits,
+		SQLSMALLINT &nullable);
+
+	// Determine the data type of the given columnNumber.
+	//
+	SQLSMALLINT PopulateColumnDataType(SQLUSMALLINT columnNumber) const;
+
+	// Cleanup data buffer and nullmap.
+	//
+	template<class SQLType>
+	void CleanupColumn(SQLUSMALLINT columnNumber);
+
+	// GetColumn function pointer definition
+	//
+	using fnRetrieveColumn = void (PythonOutputDataSet::*)(
+		std::string columnName,
+		SQLULEN     &columnSize,
+		SQLSMALLINT &decimalDigits,
+		SQLSMALLINT &nullable);
+
+	// CleanupColumn function pointer definition.
+	//
+	using fnCleanupColumn = void (PythonOutputDataSet::*)(SQLUSMALLINT columnNumber);
+
+	// Function map for getting column from DataSet
+	//
+	static const std::unordered_map<SQLSMALLINT, fnRetrieveColumn> sm_FnRetrieveColumnMap;
+
+	// Function map for Cleanup
+	//
+	static const std::unordered_map<SQLSMALLINT, fnCleanupColumn> sm_FnCleanupColumnMap;
+
+	// Function map typedefs.
+	//
+	typedef std::unordered_map<SQLSMALLINT, fnRetrieveColumn> GetColumnFnMap;
+	typedef std::unordered_map<SQLSMALLINT, fnCleanupColumn> CleanupColumnFnMap;
+
+	// Vector of pointers to data from all columns to be sent back to ExtHost.
+	//
+	std::vector<SQLPOINTER> m_data;
+
+	// List of column names
+	//
+	boost::python::list m_columnNames;
+
+	SQLULEN m_columnsNumber = 0;
+
+	// Number of rows in the DataSet.
+	//
+	SQLULEN m_rowsNumber = 0;
+
+	// A vector of ODBC C data type of all columns.
+	//
+	std::vector<SQLSMALLINT> m_columnsDataType;
+};

language-extensions/python/include/PythonExtensionUtils.h

@@ -0,0 +1,59 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonExtensionUtils.h
+//
+// Purpose:
+//  Platform specific utility functions for Python Extension
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+#include <unordered_map>
+
+class PythonExtensionUtils
+{
+public:
+
+	// Parses the value of the active python exception
+	// Type, value, and traceback are in separate pointers
+	//
+	static std::string ParsePythonException();
+
+	// Extract the string from a boost::python object or PyObject*
+	//
+	static std::string ExtractString(PyObject *pObj);
+	static std::string ExtractString(boost::python::object handle);
+
+	// Get the value of an environment variable
+	//
+	static std::string GetEnvVariable(const std::string &envVarName);
+
+	// Normalize path strings from \ to /
+	//
+	static std::string NormalizePathString(std::string pathString);
+
+	// Check if bitValue is True or not
+	//
+	static bool IsBitTrue(SQLCHAR bitValue);
+
+	// Converts a SQLGUID to a string
+	//
+	static std::string ConvertGuidToString(const SQLGUID *guid);
+
+	// Close an open dll handle
+	//
+	static void FreeDLL(void *pDll);
+
+	// Get the path to the python executable
+	//
+	static std::string GetPathToPython();
+
+	// Map to store the ODBC C type to null value mapping
+	//
+	static const std::unordered_map<SQLSMALLINT, const void *> sm_DataTypeToNullMap;
+};

language-extensions/python/include/PythonLibrarySession.h

@@ -0,0 +1,64 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonLibrarySession.h
+//
+// Purpose:
+//  Class encapsulating operations performed in a library management session
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+#include <experimental/filesystem>
+
+class PythonLibrarySession
+{
+public:
+
+	//  Initializes the Python library session, initialize the main namespace.
+	//
+	void Init(const SQLGUID *sessionId);
+
+	// Install the specified library.
+	//
+	SQLRETURN InstallLibrary(
+		std::string tempFolder,
+		SQLCHAR     *libraryName,
+		SQLINTEGER  libraryNameLength,
+		SQLCHAR     *libraryFile,
+		SQLINTEGER  libraryFileLength,
+		SQLCHAR     *libraryInstallDirectory,
+		SQLINTEGER  libraryInstallDirectoryLength);
+
+	// Uninstall the specified library.
+	//
+	SQLRETURN UninstallLibrary(
+		SQLCHAR    *libraryName,
+		SQLINTEGER libraryNameLength,
+		SQLCHAR    *libraryInstallDirectory,
+		SQLINTEGER libraryInstallDirectoryLength);
+
+	// Get top level directory/ies for a package
+	//
+	std::vector<std::experimental::filesystem::directory_entry> GetTopLevel(
+		std::string libName,
+		std::string installDir);
+
+	// Get all the artifacts we can find of a package that are in the path
+	//
+	std::vector<std::experimental::filesystem::directory_entry> GetAllArtifacts(
+		std::string libName,
+		std::string path);
+
+private:
+	SQLGUID m_sessionId{ 0, 0, 0, {0} };
+
+	// The underlying boost::python namespace, which contains all the python variables.
+	// We execute any python scripts on this namespace.
+	//
+	boost::python::object m_mainNamespace;
+};

language-extensions/python/include/PythonNamespace.h

@@ -0,0 +1,53 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonNamespace.h
+//
+// Purpose:
+//  Global class to keep the global python namespace
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+
+//-------------------------------------------------------------------------------------------------
+// Description:
+//  Global class storing the python namespace
+//
+class PythonNamespace
+{
+public:
+	// Initialize this global class
+	//
+	static void Init();
+
+	// Cleanup this global class
+	//
+	static void Cleanup();
+
+	// Get the main python module
+	//
+	static boost::python::object &MainModule() { return sm_mainModule; }
+
+	// Get the main python namespace
+	//
+	static boost::python::object& MainNamespace() { return sm_mainNamespace; }
+
+	// Get the original path list
+	//
+	static boost::python::object& OriginalPath() { return sm_originalPath; }
+
+private:
+	static boost::python::object sm_mainModule; // The boost python module, contains the namespace.
+
+	// The underlying boost::python namespace, which contains all the python variables.
+	// We execute any python scripts on this namespace.
+	//
+	static boost::python::object sm_mainNamespace;
+	
+	static boost::python::object sm_originalPath;  // The original system python path
+};

language-extensions/python/include/PythonParam.h

@@ -0,0 +1,402 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonParam.h
+//
+// Purpose:
+//  Classes storing information about an PythonExtension input/output parameter.
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonParam
+//
+// Description:
+// Base class storing information about the PythonExtension input/output parameter.
+//
+class PythonParam
+{
+public:
+	// Get m_name
+	//
+	const std::string& Name() const
+	{
+		return m_name;
+	}
+
+	// Get m_pyObject
+	//
+	const boost::python::object& PythonObject() const
+	{
+		return m_pyObject;
+	}
+
+	// Get m_strLenOrInd
+	//
+	SQLINTEGER StrLenOrInd() const
+	{
+		return m_strLenOrInd;
+	}
+
+	// Get m_inputOutputType
+	//
+	SQLSMALLINT InputOutputType() const
+	{
+		return m_inputOutputType;
+	}
+
+	// Get m_value
+	//
+	virtual SQLPOINTER Value() const = 0;
+
+	// Retrieve data from python namespace, fill it in m_value
+	// and set m_strLenOrInd accordingly.
+	//
+	virtual void RetrieveValueAndStrLenInd(boost::python::object mainNamespace) = 0;
+
+protected:
+	// Protected constructor to initialize the members.
+	// Do not allow other classes to create objects of this class - it is used as a base class.
+	//
+	PythonParam(
+		SQLUSMALLINT  id,
+		const SQLCHAR *paramName,
+		SQLSMALLINT   paramNameLength,
+		SQLSMALLINT   type,
+		SQLULEN       paramSize,
+		SQLSMALLINT   decimalDigits,
+		SQLINTEGER    strLen_or_Ind,
+		SQLSMALLINT   inputOutputType);
+
+	// Verifies if the input paramSize is equal to the size of the template type T.
+	//
+	template<class T>
+	void CheckParamSize();
+
+	// The boost::python object with the value to be placed in the nameSpace
+	//
+	boost::python::object m_pyObject;
+
+	// Id of the parameter.
+	//
+	SQLUSMALLINT m_id;
+
+	// Name of the parameter.
+	//
+	std::string m_name;
+
+	// Data type of the parameter.
+	//
+	SQLSMALLINT m_type;
+
+	// Size of the parameter.
+	//
+	SQLULEN m_size;
+
+	// Decimal digits of the parameter.
+	//
+	SQLSMALLINT m_decimalDigits;
+
+	// Indicates string length or SQL_NULL_DATA for null.
+	// Note about expected m_strLenOrInd, m_size according to type:
+	//
+	// For fixed non-char non-binary types,
+	//     value   |  m_strLenorInd                                |  m_size
+	//----------------------------------------------------------------------------
+	//     NULL    |  SQL_NULL_DATA                                |  sizeof<type>
+	//     Non-NULL|  0                                            |  sizeof<type>
+	//
+	// For char(n), binary(n) types,
+	//     value   |  m_strLenOrInd                                |  m_size
+	//----------------------------------------------------------------------------
+	//     NULL    |  SQL_NULL_DATA                                |  n
+	//     Non-NULL|  n                                            |  n
+	//
+	// For varchar(n), varbinary(n) types,
+	//     value   |  m_strLenOrInd                                |  m_size
+	//----------------------------------------------------------------------------
+	//     NULL    |  SQL_NULL_DATA                                |  n
+	//     Non-NULL|  actualNumberOfBytes(same as length)          |  n
+	//
+	// For nchar(n) type,
+	//     value   |  m_strLenOrInd                                |  m_size
+	//----------------------------------------------------------------------------
+	//     NULL    |  SQL_NULL_DATA                                |  n
+	//     Non-NULL|  n*sizeof(char16_t))                          |  n
+	//
+	// For nvarchar(n) type,
+	//     value   |  m_strLenOrInd                                |  m_size
+	//----------------------------------------------------------------------------
+	//     NULL    |  SQL_NULL_DATA                                |  n
+	//     Non-NULL|  actualNumberOfBytes(length*sizeof(char16_t)) |  n
+	//
+	SQLINTEGER m_strLenOrInd;
+
+	// Parameter type (Input/Output)
+	//
+	SQLSMALLINT m_inputOutputType;
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonParamTemplate
+//
+// Description:
+// Template class representing numeric, integer parameters
+//
+template<class SQLType>
+class PythonParamTemplate : public PythonParam
+{
+
+public:
+	// Constructor to initialize the members
+	//
+	PythonParamTemplate(
+		SQLUSMALLINT  id,
+		const SQLCHAR *paramName,
+		SQLSMALLINT   paramNameLength,
+		SQLSMALLINT   type,
+		SQLULEN       paramSize,
+		SQLSMALLINT   decimalDigits,
+		SQLPOINTER    paramValue,
+		SQLINTEGER    strLen_or_Ind,
+		SQLSMALLINT   inputOutputType);
+
+	// Retrieve data from python namespace, fill it in m_value
+	// and set m_strLenOrInd accordingly
+	//
+	void RetrieveValueAndStrLenInd(boost::python::object mainNamespace) override;
+
+	// Get the data underlying m_value vector
+	//
+	SQLPOINTER Value() const override
+	{
+		if (m_value.size() > 0)
+		{
+			return static_cast<SQLPOINTER>(
+				const_cast<SQLType*>(m_value.data()));
+		}
+		else
+		{
+			return nullptr;
+		}
+	}
+
+private:
+	// Vector holding the value of the parameter as retrieved from python namespace,
+	// holding the contents before sending them back to ExtHost
+	// Only useful for output parameter types.
+	//
+	std::vector<SQLType> m_value;
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonBooleanParam
+//
+// Description:
+// Class representing a boolean parameter.
+// Corresponds to ODBC C type SQL_C_BIT.
+//
+class PythonBooleanParam : public PythonParam
+{
+public:
+
+	// Constructor to initialize the members
+	//
+	PythonBooleanParam(
+		SQLUSMALLINT  id,
+		const SQLCHAR *paramName,
+		SQLSMALLINT   paramNameLength,
+		SQLSMALLINT   type,
+		SQLULEN       paramSize,
+		SQLSMALLINT   decimalDigits,
+		SQLPOINTER    paramValue,
+		SQLINTEGER    strLen_or_Ind,
+		SQLSMALLINT   inputOutputType);
+
+	// Retrieve data from python namespace, fill it in m_value
+	// and set m_strLenOrInd accordingly
+	//
+	void RetrieveValueAndStrLenInd(boost::python::object mainNamespace) override;
+
+	// Get the data underlying m_value vector
+	//
+	SQLPOINTER Value() const override
+	{
+		if (m_value.size() > 0)
+		{
+			return static_cast<SQLPOINTER>(
+				const_cast<SQLCHAR*>(m_value.data()));
+		}
+		else
+		{
+			return nullptr;
+		}
+	}
+
+private:
+	// Character vector holding the contents before sending them back to ExtHost.
+	// Useful for output parameter types.
+	//
+	std::vector<SQLCHAR> m_value;
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonStringParam
+//
+// Description:
+// Class representing a string parameter.
+// Corresponds to ODBC C type SQL_C_CHAR and SQL_C_WCHAR.
+//
+template<class CharType>
+class PythonStringParam : public PythonParam
+{
+public:
+
+	// Constructor to initialize the members
+	//
+	PythonStringParam(
+		SQLUSMALLINT  id,
+		const SQLCHAR *paramName,
+		SQLSMALLINT   paramNameLength,
+		SQLSMALLINT   type,
+		SQLULEN       paramSize,
+		SQLSMALLINT   decimalDigits,
+		SQLPOINTER    paramValue,
+		SQLINTEGER    strLen_or_Ind,
+		SQLSMALLINT   inputOutputType);
+
+	// Retrieve data from python namespace, fill it in m_value
+	// and set m_strLenOrInd accordingly
+	//
+	void RetrieveValueAndStrLenInd(boost::python::object mainNamespace) override;
+
+	// Get the data underlying m_value vector
+	//
+	SQLPOINTER Value() const override
+	{
+		if (m_value.size() > 0)
+		{
+			return static_cast<SQLPOINTER>(
+				const_cast<CharType*>(m_value.data()));
+		}
+		else
+		{
+			return nullptr;
+		}
+	}
+
+private:
+	// Character vector holding the contents before sending them back to ExtHost.
+	// Useful for output parameter types.
+	//
+	std::vector<CharType> m_value;
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonRawParam
+//
+// Description:
+// Class representing a raw parameter.
+// Corresponds to ODBC C type SQL_C_BINARY.
+//
+class PythonRawParam : public PythonParam
+{
+public:
+
+	// Constructor to initialize the members
+	//
+	PythonRawParam(
+		SQLUSMALLINT  id,
+		const SQLCHAR *paramName,
+		SQLSMALLINT   paramNameLength,
+		SQLSMALLINT   type,
+		SQLULEN       paramSize,
+		SQLSMALLINT   decimalDigits,
+		SQLPOINTER    paramValue,
+		SQLINTEGER    strLen_or_Ind,
+		SQLSMALLINT   inputOutputType);
+
+	// Retrieve data from python namespace, fill it in m_value
+	// and set m_strLenOrInd accordingly
+	//
+	void RetrieveValueAndStrLenInd(boost::python::object mainNamespace) override;
+
+	// Get the data underlying m_value vector
+	//
+	SQLPOINTER Value() const override
+	{
+		if (m_value.size() > 0)
+		{
+			return static_cast<SQLPOINTER>(
+				const_cast<SQLCHAR *>(m_value.data()));
+		}
+		else
+		{
+			return nullptr;
+		}
+	}
+
+private:
+	// Character vector holding the contents before sending them back to ExtHost.
+	// Useful for output parameter types.
+	//
+	std::vector<SQLCHAR> m_value;
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonDateTimeParam
+//
+// Description:
+//  Class representing a Date/DateTime parameter.
+//  Corresponds to ODBC C type SQL_C_TYPE_DATE and SQL_C_TYPE_TIMESTAMP.
+//
+template<SQLSMALLINT SQLType>
+class PythonDateTimeParam : public PythonParam
+{
+public:
+
+	// Constructor to initialize the members
+	//
+	PythonDateTimeParam(
+		SQLUSMALLINT  id,
+		const SQLCHAR *paramName,
+		SQLSMALLINT   paramNameLength,
+		SQLSMALLINT   type,
+		SQLULEN       paramSize,
+		SQLSMALLINT   decimalDigits,
+		SQLPOINTER    paramValue,
+		SQLINTEGER    strLen_or_Ind,
+		SQLSMALLINT   inputOutputType);
+
+	// Retrieve data from python namespace, fill it in m_value
+	// and set m_strLenOrInd accordingly
+	//
+	void RetrieveValueAndStrLenInd(boost::python::object mainNamespace) override;
+
+	// Get the data underlying m_value vector
+	//
+	SQLPOINTER Value() const override
+	{
+		if (m_value.size() > 0)
+		{
+			return static_cast<SQLPOINTER>(
+				const_cast<SQL_TIMESTAMP_STRUCT *>(m_value.data()));
+		}
+		else
+		{
+			return nullptr;
+		}
+	}
+
+private:
+	// Timestamp vector holding the contents before sending them back to ExtHost.
+	// Useful for output parameter types.
+	//
+	std::vector<SQL_TIMESTAMP_STRUCT> m_value;
+};

language-extensions/python/include/PythonParamContainer.h

@@ -0,0 +1,104 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonParamContainer.h
+//
+// Purpose:
+//  A container that stores the input and output parameters passed to the python script.
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+
+#include <unordered_map>
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonParamContainer
+//
+// Description:
+// Container for PythonExtension parameters.
+//
+class PythonParamContainer
+{
+public:
+	// Initialize the container with the number of parameters.
+	//
+	void Init(SQLSMALLINT paramsNumber);
+
+	// Creates an PythonParam object, adds the parameter with paramValue for given dataType
+	// to the boost::python namespace and stores it in m_params for future use.
+	//
+	void AddParamToNamespace(
+		boost::python::object nameSpace,
+		SQLUSMALLINT          paramNumber,
+		const SQLCHAR         *paramName,
+		SQLSMALLINT           paramNameLength,
+		SQLSMALLINT           dataType,
+		SQLULEN               paramSize,
+		SQLSMALLINT           decimalDigits,
+		SQLPOINTER            paramValue,
+		SQLINTEGER            strLen_or_Ind,
+		SQLSMALLINT           inputOutputType);
+
+	// Returns the number of parameters in the container.
+	// Note: The container is initialized with the desired number of parameters,
+	// so the size returned here does not change even after adding parameters to
+	// the boost::python namespace.
+	//
+	SQLUSMALLINT GetSize() const
+	{
+		return static_cast<SQLUSMALLINT>(m_params.size());
+	}
+
+	// For the given paramNumber, retrieve and return paramValue and strLen_or_Ind.
+	//
+	void GetParamValueAndStrLenInd(
+		boost::python::object mainNamespace,
+		SQLUSMALLINT          paramNumber,
+		SQLPOINTER            *paramValue,
+		SQLINTEGER            *strLen_or_Ind);
+
+private:
+
+	// Template to create a parameter and add it to the python namespace
+	//
+	template<class ParamType>
+	void CreateParam(
+		boost::python::object nameSpace,
+		SQLUSMALLINT          paramNumber,
+		const SQLCHAR         *paramName,
+		SQLSMALLINT           paramNameLength,
+		SQLSMALLINT           dataType,
+		SQLULEN               paramSize,
+		SQLSMALLINT           decimalDigits,
+		SQLPOINTER            paramValue,
+		SQLINTEGER            strLen_or_Ind,
+		SQLSMALLINT           inputOutputType);
+
+	// Create parameter function pointer definition
+	//
+	using fnCreateParam = void (PythonParamContainer::*)(
+		boost::python::object nameSpace,
+		SQLUSMALLINT          paramNumber,
+		const SQLCHAR         *paramName,
+		SQLSMALLINT           paramNameLength,
+		SQLSMALLINT           dataType,
+		SQLULEN               paramSize,
+		SQLSMALLINT           decimalDigits,
+		SQLPOINTER            paramValue,
+		SQLINTEGER            strLen_or_Ind,
+		SQLSMALLINT           inputOutputType);
+
+	// Vector to store created parameters
+	//
+	std::vector<std::unique_ptr<PythonParam>> m_params;
+
+	// Function map to add parameters to the namespace and its typedef
+	//
+	static std::unordered_map<SQLSMALLINT, fnCreateParam> sm_FnCreateParamMap;
+	typedef std::unordered_map<SQLSMALLINT, fnCreateParam> CreateParamFnMap;
+};

language-extensions/python/include/PythonPathSettings.h

@@ -0,0 +1,53 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonPathSettings.h
+//
+// Purpose:
+//  Global class to keep language runtime settings
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+
+//-------------------------------------------------------------------------------------------------
+// Description:
+//  Global class storing the language runtime paths and parameters
+//
+class PythonPathSettings
+{
+public:
+	// Initialize this global class
+	//
+	static void Init(
+		const SQLCHAR *languageParams,
+		const SQLCHAR *languagePath,
+		const SQLCHAR *publicLibraryPath,
+		const SQLCHAR *privateLibraryPath);
+
+	// Get the private library path sent by SQL Server
+	//
+	static const std::string& PrivateLibraryPath() { return sm_privateLibraryPath; }
+
+	// Get the public library path sent by SQL Server
+	//
+	static const std::string& PublicLibraryPath() { return sm_publicLibraryPath; }
+
+	// Get the extension root folder
+	//
+	static const std::string& RootPath() { return sm_languagePath; }
+
+	// Get the language parameters sent by SQL Server
+	//
+	static const std::string& Params() { return sm_languageParams; }
+
+private:
+	static std::string sm_languagePath;
+	static std::string sm_languageParams;
+	static std::string sm_privateLibraryPath;
+	static std::string sm_publicLibraryPath;
+};

language-extensions/python/include/PythonSession.h

@@ -0,0 +1,125 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonSession.h
+//
+// Purpose:
+//  Class encapsulating operations performed per session
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+#include "PythonColumn.h"
+#include "PythonDataSet.h"
+#include "PythonParam.h"
+#include "PythonParamContainer.h"
+
+// data pertaining to a session
+//
+class PythonSession
+{
+public:
+
+	// Init the session
+	//
+	void Init(
+		const SQLGUID *sessionId,
+		SQLUSMALLINT  taskId,
+		SQLUSMALLINT  numTasks,
+		SQLCHAR       *script,
+		SQLULEN       scriptLength,
+		SQLUSMALLINT  inputSchemaColumnsNumber,
+		SQLUSMALLINT  parametersNumber,
+		SQLCHAR       *inputDataName,
+		SQLUSMALLINT  inputDataNameLength,
+		SQLCHAR       *outputDataName,
+		SQLUSMALLINT  outputDataNameLength);
+
+	// Init the input column
+	//
+	void InitColumn(
+		SQLUSMALLINT  columnNumber,
+		const SQLCHAR *columnName,
+		SQLSMALLINT   columnNameLength,
+		SQLSMALLINT   dataType,
+		SQLULEN       columnSize,
+		SQLSMALLINT   decimalDigits,
+		SQLSMALLINT   nullable,
+		SQLSMALLINT   partitionByNumber,
+		SQLSMALLINT   orderByNumber);
+
+	// Init the input parameter
+	//
+	void InitParam(
+		SQLUSMALLINT  paramNumber,
+		const SQLCHAR *paramName,
+		SQLSMALLINT   paramNameLength,
+		SQLSMALLINT   dataType,
+		SQLULEN       paramSize,
+		SQLSMALLINT   decimalDigits,
+		SQLPOINTER    paramValue,
+		SQLINTEGER    strLen_or_Ind,
+		SQLSMALLINT   InputOutputType);
+
+	// Execute the workflow for the session
+	//
+	void ExecuteWorkflow(
+		SQLULEN      rowsNumber,
+		SQLPOINTER   *data,
+		SQLINTEGER   **strLen_or_Ind,
+		SQLUSMALLINT *outputSchemaColumnsNumber);
+
+	// Get the metadata for the output column
+	//
+	void GetResultColumn(
+		SQLUSMALLINT columnNumber,
+		SQLSMALLINT  *dataType,
+		SQLULEN      *columnSize,
+		SQLSMALLINT  *decimalDigits,
+		SQLSMALLINT  *nullable);
+
+	// Get the results
+	//
+	void GetResults(
+		SQLULEN    *rowsNumber,
+		SQLPOINTER **data,
+		SQLINTEGER ***strLen_or_Ind);
+
+	// Get the the output parameter
+	//
+	void GetOutputParam(
+		SQLUSMALLINT paramNumber,
+		SQLPOINTER   *paramValue,
+		SQLINTEGER   *strLen_or_Ind);
+
+	// Cleanup session
+	//
+	void Cleanup();
+
+private:
+	boost::python::object m_mainModule; // The boost python module which contains the namespace.
+
+	// The underlying boost::python namespace, which contains all the python variables.
+	// We execute any python scripts on this namespace.
+	//
+	boost::python::object m_mainNamespace;
+
+	SQLGUID m_sessionId{ 0, 0, 0, {0} };
+	SQLUSMALLINT m_taskId = 0;
+	SQLUSMALLINT m_numTasks = 0;
+
+	std::string m_script;
+	SQLULEN m_scriptLength;
+
+	PythonInputDataSet m_inputDataSet;
+	PythonOutputDataSet m_outputDataSet;
+
+	// Parameters
+	//
+	PythonParamContainer m_paramContainer;
+
+};

language-extensions/python/src/CMakeLists.txt

@@ -0,0 +1,119 @@
+# CMakeLists.txt
+
+cmake_minimum_required (VERSION 3.5)
+
+set(PYTHONEXTENSION_VERSION_MAJOR "1")
+set(PYTHONEXTENSION_VERSION_MINOR "0")
+set(PYTHONEXTENSION_VERSION ${PYTHONEXTENSION_VERSION_MAJOR}.${PYTHONEXTENSION_VERSION_MINOR})
+
+# this is what the final library is going to be named
+#
+project(PythonExtension VERSION ${PYTHONEXTENSION_VERSION} LANGUAGES CXX)
+
+# All string comparisons are CASE SENSITIVE in CMAKE. Make all strings lower before comparisons!
+#
+string(TOLOWER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE)
+string(TOLOWER ${PLATFORM} PLATFORM)
+
+# To build shared libraries in Windows, we set CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS to TRUE.
+# See https://cmake.org/cmake/help/v3.4/variable/CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS.html
+# See https://blog.kitware.com/create-dlls-on-windows-without-declspec-using-new-cmake-export-all-feature/
+#
+set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
+
+file(TO_CMAKE_PATH ${ENL_ROOT}/language-extensions/python PYTHONEXTENSION_HOME)
+file(TO_CMAKE_PATH ${PYTHONEXTENSION_HOME}/src PYTHONEXTENSION_SRC_DIR)
+file(TO_CMAKE_PATH ${PYTHONEXTENSION_HOME}/include PYTHONEXTENSION_INCLUDE_DIR)
+file(TO_CMAKE_PATH ${ENL_ROOT}/extension-host/include EXTENSION_API_INCLUDE_DIR)
+file(TO_CMAKE_PATH ${ENL_ROOT}/build-output/pythonextension/${PLATFORM} PYTHONEXTENSION_WORKING_DIR)
+
+# C++ code; only these files are compiled-in
+#
+file(GLOB PYTHONEXTENSION_SOURCE_FILES ${PYTHONEXTENSION_SRC_DIR}/*.cpp ${PYTHONEXTENSION_SRC_DIR}/${PLATFORM}/*.cpp)
+
+# Create the target library
+#
+add_library(PythonExtension SHARED
+	${PYTHONEXTENSION_SOURCE_FILES}
+)
+
+set_target_properties(PythonExtension
+	PROPERTIES
+	VERSION ${PYTHONEXTENSION_VERSION}
+	SOVERSION ${PYTHONEXTENSION_VERSION_MAJOR}
+)
+
+if (${PLATFORM} STREQUAL linux)
+	target_compile_options(PythonExtension PRIVATE -Wall -Wextra -g -O2 -fPIC -Werror -std=c++17 -Wno-unused-parameter -fshort-wchar)
+
+	set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--as-needed -Wl,--export-dynamic -fopenmp")
+
+	set(USR_LIB_PATH /usr/local/lib)
+	set(PYTHON_LIB_PATH ${PYTHONHOME}/lib/python3.7/config-3.7m-x86_64-linux-gnu)
+
+	find_library(PYTHON_LIB python3.7 ${PYTHON_LIB_PATH})
+	find_library(BOOST_PYTHON_LIB boost_python37 ${BOOST_PYTHON_ROOT})
+	find_library(BOOST_NUMPY_LIB boost_numpy37 ${BOOST_PYTHON_ROOT})
+
+	file(TO_CMAKE_PATH ${INCLUDE_ROOT}/python3.7 PYTHON_INCLUDE)
+	file(TO_CMAKE_PATH ${INCLUDE_ROOT}/boost BOOST_INCLUDE)
+	file(TO_CMAKE_PATH ${INCLUDE_ROOT}/boost/python BOOST_PYTHON_INCLUDE)
+
+	set(ADDITIONAL_INCLUDES ${PYTHON_INCLUDE} ${BOOST_INCLUDE} ${BOOST_PYTHON_INCLUDE})
+
+	target_link_libraries(PythonExtension 
+		stdc++fs
+	)
+elseif(${PLATFORM} STREQUAL windows)
+	add_definitions(-DWIN_EXPORT -D_WIN64 -D_WINDOWS)
+
+	# Compile option /c to compile only
+	#
+	set(COMPILE_OPTIONS /c /std:c++17)
+
+	find_library(PYTHON_LIB python37 ${PYTHONHOME}/libs)
+
+	# MDd is for debug DLL and MD is for release DLL
+	#
+	if (${CMAKE_BUILD_TYPE} STREQUAL debug)
+		set(COMPILE_OPTIONS ${COMPILE_OPTIONS} /MDd)
+		find_library(BOOST_PYTHON_LIB libboost_python37-vc140-mt-gd-x64-1_69 ${BOOST_PYTHON_ROOT})
+		find_library(BOOST_NUMPY_LIB libboost_numpy37-vc140-mt-gd-x64-1_69 ${BOOST_PYTHON_ROOT})
+	else()
+		set(COMPILE_OPTIONS ${COMPILE_OPTIONS} /MD)
+		find_library(BOOST_PYTHON_LIB libboost_python37-vc140-mt-x64-1_69 ${BOOST_PYTHON_ROOT})
+		find_library(BOOST_NUMPY_LIB libboost_numpy37-vc140-mt-x64-1_69 ${BOOST_PYTHON_ROOT})
+	endif()
+
+	target_compile_options(PythonExtension PRIVATE ${COMPILE_OPTIONS})
+
+	# Set the DLLEXPORT variable to export symbols
+	#
+	target_compile_definitions(PythonExtension PRIVATE WIN_EXPORT)
+
+	file(TO_CMAKE_PATH ${PYTHONHOME}/include PYTHON_INCLUDE)
+	file(TO_CMAKE_PATH ${BOOST_ROOT}/include BOOST_INCLUDE)
+
+	set(ADDITIONAL_INCLUDES ${PYTHON_INCLUDE} ${BOOST_INCLUDE})
+endif()
+
+add_definitions(-DBOOST_USE_STATIC_LIBS -DBOOST_PYTHON_STATIC_LIB -DBOOST_ALL_NO_LIB -DBOOST_NUMPY_STATIC_LIB)
+
+if (${CMAKE_BUILD_TYPE} STREQUAL debug)
+	add_definitions(-D_DEBUG)
+else()
+	add_definitions(-DNDEBUG)
+endif()
+
+
+target_link_libraries(PythonExtension 
+	${PYTHON_LIB} 
+	${BOOST_PYTHON_LIB} 
+	${BOOST_NUMPY_LIB}
+)
+
+target_include_directories(PythonExtension
+	PRIVATE ${EXTENSION_API_INCLUDE_DIR} ${PYTHONEXTENSION_INCLUDE_DIR} ${ADDITIONAL_INCLUDES} 
+)
+
+install(TARGETS PythonExtension DESTINATION "${PYTHONEXTENSION_WORKING_DIR}" LIBRARY NAMELINK_ONLY)

language-extensions/python/src/Logger.cpp

@@ -0,0 +1,91 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: Logger.cpp
+//
+// Purpose:
+//  Logging functions for extension
+//
+//*************************************************************************************************
+
+#include "Logger.h"
+
+#define TIMESTAMP_LENGTH 35
+using namespace std;
+
+char Logger::sm_timestampBuffer[TIMESTAMP_LENGTH] = { 0 };
+
+//-------------------------------------------------------------------------------------------------
+// Name: LogError
+//
+// Description:
+//  Log an error to stderr with format "TIMESTAMP Error: <message>".
+//
+void Logger::LogError(const string &errorMsg)
+{
+	cerr << GetCurrentTimestamp() << "Error: " << errorMsg << endl;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: LogException
+//
+// Description:
+//  Log a c++ exception to stderr with format "TIMESTAMP Exception
+//  occurred: <message>".
+//
+void Logger::LogException(const exception &e)
+{
+	cerr << GetCurrentTimestamp() << "Exception occurred: " << e.what() << endl;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: Log
+//
+// Description:
+//  Log a message to stdout with format "TIMESTAMP <message>".
+//
+void Logger::Log(const string &msg)
+{
+#if defined(_DEBUG)
+	cout << GetCurrentTimestamp() << msg << endl;
+#endif
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: GetCurrentTimestamp
+//
+// Description:
+//  Gets the current system time and format it to the SQL log format
+//  (Year-Month-Day Hour:Minute:Second.Millisecond).
+//
+const std::string Logger::GetCurrentTimestamp()
+{
+#if defined ( _MSC_VER )
+	SYSTEMTIME sysTime;
+
+	GetLocalTime(&sysTime);
+
+	sprintf_s(sm_timestampBuffer, TIMESTAMP_LENGTH,
+		"%04d-%02d-%02d %02d:%02d:%02d.%02d\t",
+		sysTime.wYear,
+		sysTime.wMonth,
+		sysTime.wDay,
+		sysTime.wHour,
+		sysTime.wMinute,
+		sysTime.wSecond,
+		sysTime.wMilliseconds / 10);
+
+	return sm_timestampBuffer;
+#else
+	time_t now = time(0);
+	struct tm tstruct;
+	char buf[80];
+
+	tstruct = *localtime(&now);
+	strftime(buf, sizeof(buf), "%Y-%m-%d %H:%M:%S.000\t", &tstruct); // No millisecond support
+	return buf;
+#endif
+}

language-extensions/python/src/PythonColumn.cpp

@@ -0,0 +1,51 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonColumn.cpp
+//
+// Purpose:
+//  Encapsulate data column attributes
+//
+//*************************************************************************************************
+
+#include "PythonColumn.h"
+
+using namespace std;
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonColumn::PythonColumn
+//
+// Description:
+//  Initializes the class that encapsulates one column in the Python DataFrame/Dictionary
+//
+PythonColumn::PythonColumn(
+	const SQLCHAR *columnName,
+	SQLSMALLINT   columnNameLength,
+	SQLSMALLINT   dataType,
+	SQLULEN       columnSize,
+	SQLSMALLINT   decimalDigits,
+	SQLSMALLINT   nullable) :
+	m_dataType(dataType),
+	m_size(columnSize),
+	m_decimalDigits(decimalDigits),
+	m_nullable(nullable)
+{
+	const char *name = static_cast<const char*>(static_cast<const void*>(columnName));
+
+	// columnNameLength does not include the null terminator.
+	//
+#if defined(_DEBUG)
+	if (static_cast<size_t>(columnNameLength) != strlen(name))
+	{
+		throw invalid_argument("Invalid column name length, it doesn't match string length.");
+	}
+#endif
+
+	// Store the information for this column
+	//
+	m_name = string(name, columnNameLength);
+}
+

language-extensions/python/src/PythonExtension.cpp

@@ -0,0 +1,761 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonExtension.cpp
+//
+// Purpose:
+//  Python extension DLL that can be loaded by ExtHost. This library loads the
+//  Python dll, handles communication with ExtHost, and executes user-specified
+//  Python script
+//
+//*************************************************************************************************
+
+#include <boost/python.hpp>
+#include <experimental/filesystem>
+#include <iostream>
+#include <fstream>
+#include <unordered_map>
+
+#include "Logger.h"
+#include "PythonExtensionUtils.h"
+#include "PythonLibrarySession.h"
+#include "PythonNamespace.h"
+#include "PythonPathSettings.h"
+#include "PythonSession.h"
+#include "sqlexternallanguage.h"
+#include "sqlexternallibrary.h"
+
+using namespace std;
+namespace bp = boost::python;
+namespace fs = std::experimental::filesystem;
+
+#ifndef _WIN64
+#include <dlfcn.h>
+const string x_PythonSoFile = "libpython3.7m.so.1.0";
+#endif
+
+static unordered_map<string, PythonSession *> g_pySessionMap;
+
+//-------------------------------------------------------------------------------------------------
+// Name: GetInterfaceVersion
+//
+// Description:
+// Returns the API interface version for the extension
+//
+// Returns:
+// EXTERNAL_LANGUAGE_EXTENSION_API
+//
+SQLUSMALLINT
+GetInterfaceVersion()
+{
+	return EXTERNAL_LANGUAGE_EXTENSION_API;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: Init
+//
+// Description:
+//  Initialize the python extension. Until registration, nothing is needed here.
+//  We call Py_Initialize to initialize python in C++ and allow boost to work
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN Init(
+	SQLCHAR *extensionParams,
+	SQLULEN extensionParamsLength,
+	SQLCHAR *extensionPath,
+	SQLULEN extensionPathLength,
+	SQLCHAR *publicLibraryPath,
+	SQLULEN publicLibraryPathLength,
+	SQLCHAR *privateLibraryPath,
+	SQLULEN privateLibraryPathLength
+)
+{
+	LOG("Init");
+
+	SQLRETURN result = SQL_SUCCESS;
+
+	try
+	{
+
+#ifndef _WIN64
+		// Preload the python so in Linux so that numpy knows about it.
+		// Without this line, the numpy .so cannot find python and will fail to load.
+		//
+		dlopen(x_PythonSoFile.c_str(), RTLD_LAZY | RTLD_GLOBAL);
+#endif
+
+		// Initialize Python using the Python/C API.
+		// This allows us to start using Python API and boost functions.
+		//
+		Py_Initialize();
+
+		if (!Py_IsInitialized())
+		{
+			throw runtime_error("Python did not initialize properly, "
+				"check paths and dependencies.");
+		}
+
+		bp::numpy::initialize();
+
+		PythonPathSettings::Init(
+			extensionParams,
+			extensionPath,
+			publicLibraryPath,
+			privateLibraryPath);
+
+		PythonNamespace::Init();
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function Init()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: InitSession
+//
+// Description:
+//  Initializes session-specific data. We store the schema and find the main class and
+//  method to execute here.
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN InitSession(
+	SQLGUID      sessionId,
+	SQLUSMALLINT taskId,
+	SQLUSMALLINT numTasks,
+	SQLCHAR      *script,
+	SQLULEN      scriptLength,
+	SQLUSMALLINT inputSchemaColumnsNumber,
+	SQLUSMALLINT parametersNumber,
+	SQLCHAR      *inputDataName,
+	SQLUSMALLINT inputDataNameLength,
+	SQLCHAR      *outputDataName,
+	SQLUSMALLINT outputDataNameLength
+)
+{
+	LOG("InitSession");
+	SQLRETURN result = SQL_SUCCESS;
+
+	string session = PythonExtensionUtils::ConvertGuidToString(&sessionId);
+	try
+	{
+		g_pySessionMap[session] = new PythonSession();
+		g_pySessionMap[session]->Init(
+			&sessionId,
+			taskId,
+			numTasks,
+			script,
+			scriptLength,
+			inputSchemaColumnsNumber,
+			parametersNumber,
+			inputDataName,
+			inputDataNameLength,
+			outputDataName,
+			outputDataNameLength);
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function InitSession()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: InitColumn
+//
+// Description:
+//  Initializes column-specific data. We store the name and the data type of the column
+//  here.
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN InitColumn(
+	SQLGUID      sessionId,
+	SQLUSMALLINT taskId,
+	SQLUSMALLINT columnNumber,
+	SQLCHAR      *columnName,
+	SQLSMALLINT  columnNameLength,
+	SQLSMALLINT  dataType,
+	SQLULEN      columnSize,
+	SQLSMALLINT  decimalDigits,
+	SQLSMALLINT  nullable,
+	SQLSMALLINT  partitionByNumber,
+	SQLSMALLINT  orderByNumber
+)
+{
+	LOG("InitColumn");
+	SQLRETURN result = SQL_SUCCESS;
+	string session = PythonExtensionUtils::ConvertGuidToString(&sessionId);
+	try
+	{
+		g_pySessionMap[session]->InitColumn(
+			columnNumber,
+			columnName,
+			columnNameLength,
+			dataType,
+			columnSize,
+			decimalDigits,
+			nullable,
+			partitionByNumber,
+			orderByNumber);
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function InitColumn()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: InitParam
+//
+// Description:
+//  Initializes parameter-specific data.
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN InitParam(
+	SQLGUID      sessionId,
+	SQLUSMALLINT taskId,
+	SQLUSMALLINT paramNumber,
+	SQLCHAR      *paramName,
+	SQLSMALLINT  paramNameLength,
+	SQLSMALLINT  dataType,
+	SQLULEN      paramSize,
+	SQLSMALLINT  decimalDigits,
+	SQLPOINTER   paramValue,
+	SQLINTEGER   strLen_or_Ind,
+	SQLSMALLINT  inputOutputType)
+{
+	LOG("InitParam");
+	SQLRETURN result = SQL_SUCCESS;
+	string session = PythonExtensionUtils::ConvertGuidToString(&sessionId);
+	try
+	{
+		g_pySessionMap[session]->InitParam(
+			paramNumber,
+			paramName,
+			paramNameLength,
+			dataType,
+			paramSize,
+			decimalDigits,
+			paramValue,
+			strLen_or_Ind,
+			inputOutputType);
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function InitParam()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: Execute
+//
+// Description:
+//  Given the data from ExtHost, convert and populate the arrays in the user python program. Then,
+//  invoke the specified function and retrieve the output schema and convert the data back to
+//  ODBC types.
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN Execute(
+	SQLGUID      sessionId,
+	SQLUSMALLINT taskId,
+	SQLULEN      rowsNumber,
+	SQLPOINTER   *data,
+	SQLINTEGER   **strLen_or_Ind,
+	SQLUSMALLINT *outputSchemaColumnsNumber
+)
+{
+	LOG("Execute");
+	SQLRETURN result = SQL_SUCCESS;
+	string session = PythonExtensionUtils::ConvertGuidToString(&sessionId);
+	try
+	{
+		g_pySessionMap[session]->ExecuteWorkflow(rowsNumber,
+									data,
+									strLen_or_Ind,
+									outputSchemaColumnsNumber);
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function Execute()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: GetResultColumn
+//
+// Description:
+//  Returns information about the output column
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN GetResultColumn(
+	SQLGUID      sessionId,
+	SQLUSMALLINT taskId,
+	SQLUSMALLINT columnNumber,
+	SQLSMALLINT  *dataType,
+	SQLULEN      *columnSize,
+	SQLSMALLINT  *decimalDigits,
+	SQLSMALLINT  *nullable
+)
+{
+	LOG("GetResultColumn");
+	SQLRETURN result = SQL_SUCCESS;
+	string session = PythonExtensionUtils::ConvertGuidToString(&sessionId);
+	try
+	{
+		g_pySessionMap[session]->GetResultColumn(
+			columnNumber,
+			dataType,
+			columnSize,
+			decimalDigits,
+			nullable);
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function Execute()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: GetResults
+//
+// Description:
+//  Returns the output data as well as the null map retrieved from the user program
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN GetResults(
+	SQLGUID      sessionId,
+	SQLUSMALLINT taskId,
+	SQLULEN      *rowsNumber,
+	SQLPOINTER   **data,
+	SQLINTEGER   ***strLen_or_Ind
+)
+{
+	LOG("GetResults");
+	SQLRETURN result = SQL_SUCCESS;
+	string session = PythonExtensionUtils::ConvertGuidToString(&sessionId);
+	try
+	{
+		g_pySessionMap[session]->GetResults(
+			rowsNumber,
+			data,
+			strLen_or_Ind);
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function GetResults()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: GetOutputParam
+//
+// Description:
+//  Returns the output parameter's data.
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN GetOutputParam(
+	SQLGUID      sessionId,
+	SQLUSMALLINT taskId,
+	SQLUSMALLINT paramNumber,
+	SQLPOINTER   *paramValue,
+	SQLINTEGER   *strLen_or_Ind
+)
+{
+	LOG("GetOutputParam");
+	SQLRETURN result = SQL_SUCCESS;
+	string session = PythonExtensionUtils::ConvertGuidToString(&sessionId);
+	try
+	{
+		g_pySessionMap[session]->GetOutputParam(
+			paramNumber,
+			paramValue,
+			strLen_or_Ind);
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function GetOutputParam()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: CleanupSession
+//
+// Description:
+//  Cleans up the output data buffers that we persist for
+//  ExtHost to finish processing the data
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN CleanupSession(
+	SQLGUID      sessionId,
+	SQLUSMALLINT taskId
+)
+{
+	LOG("CleanupSession");
+	SQLRETURN result = SQL_SUCCESS;
+	string session = PythonExtensionUtils::ConvertGuidToString(&sessionId);
+	try
+	{
+		if (g_pySessionMap.count(session) > 0)
+		{
+			g_pySessionMap[session]->Cleanup();
+			delete g_pySessionMap[session];
+			g_pySessionMap.erase(session);
+		}
+	}
+	catch (const exception &ex)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR(ex.what());
+	}
+	catch (const bp::error_already_set&)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Python error: " + PythonExtensionUtils::ParsePythonException());
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		LOG_ERROR("Unexpected exception occurred in function CleanupSession()");
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: Cleanup
+//
+// Description:
+//  Completely clean up the extension
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN Cleanup()
+{
+	LOG("Cleanup");
+	SQLRETURN result = SQL_SUCCESS;
+
+	PythonNamespace::Cleanup();
+
+	return result;
+}
+
+
+//-------------------------------------------------------------------------------------------------
+// External Library APIs
+//-------------------------------------------------------------------------------------------------
+
+//-------------------------------------------------------------------------------------------------
+// Name: InstallExternalLibrary
+//
+// Description:
+//  Installs an external library to the specified directory
+//  The library file is expected to be a zip containing the python package inside.
+//  We unzip the file then install the python package inside using a subprocess call to pip.
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN InstallExternalLibrary(
+	SQLGUID    setupSessionId,
+	SQLCHAR    *libraryName,
+	SQLINTEGER libraryNameLength,
+	SQLCHAR    *libraryFile,
+	SQLINTEGER libraryFileLength,
+	SQLCHAR    *libraryInstallDirectory,
+	SQLINTEGER libraryInstallDirectoryLength,
+	SQLCHAR    **libraryError,
+	SQLINTEGER *libraryErrorLength)
+{
+	LOG("InstallExternalLibrary");
+	SQLRETURN result = SQL_ERROR;
+
+	string errorString;
+
+	string installDir = string(reinterpret_cast<char *>(libraryInstallDirectory), libraryInstallDirectoryLength);
+	installDir = PythonExtensionUtils::NormalizePathString(installDir);
+
+	string tempFolder = PythonExtensionUtils::NormalizePathString(fs::path(installDir).append("tmp").string());
+
+	try
+	{
+		PythonLibrarySession librarySession = PythonLibrarySession();
+
+		librarySession.Init(&setupSessionId);
+
+		result = librarySession.InstallLibrary(
+			tempFolder,
+			libraryName,
+			libraryNameLength,
+			libraryFile,
+			libraryFileLength,
+			libraryInstallDirectory,
+			libraryInstallDirectoryLength);
+	}
+	catch (const exception & ex)
+	{
+		result = SQL_ERROR;
+
+		errorString = string(ex.what());
+		LOG_ERROR(errorString);
+	}
+	catch (const bp::error_already_set &)
+	{
+		result = SQL_ERROR;
+
+		errorString = PythonExtensionUtils::ParsePythonException();
+
+		LOG_ERROR("Python error: " + errorString);
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		errorString = "Unexpected exception occurred in function InstallExternalLibrary()";
+
+		LOG_ERROR(errorString);
+	}
+
+	// Clean up the temp installation folder
+	//
+	if (fs::exists(tempFolder))
+	{
+		fs::remove_all(tempFolder);
+	}
+
+	if (!errorString.empty())
+	{
+		*libraryErrorLength = errorString.length();
+
+		string *pError = new string(errorString);
+		SQLCHAR *error = const_cast<SQLCHAR*>(reinterpret_cast<const SQLCHAR *>(pError->c_str()));
+
+		*libraryError = error;
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: UninstallExternalLibrary
+//
+// Description:
+//  Uninstalls an external library from the specified directory.
+//  We add the directory to the python path, then call pip uninstall in a subprocess.
+//  If pip fails for some reason, we try to manually uninstall the package by deleting the
+//  top level package folder as well as any dist/egg/.py files that were left behind.
+//
+// Returns:
+//  SQL_SUCCESS on success, else SQL_ERROR
+//
+SQLRETURN UninstallExternalLibrary(
+	SQLGUID    setupSessionId,
+	SQLCHAR    *libraryName,
+	SQLINTEGER libraryNameLength,
+	SQLCHAR    *libraryInstallDirectory,
+	SQLINTEGER libraryInstallDirectoryLength,
+	SQLCHAR    **libraryError,
+	SQLINTEGER *libraryErrorLength)
+{
+	LOG("UninstallExternalLibrary");
+	SQLRETURN result = SQL_SUCCESS;
+
+	string errorString;
+
+	try
+	{
+		PythonLibrarySession librarySession = PythonLibrarySession();
+
+		librarySession.Init(&setupSessionId);
+		
+		result = librarySession.UninstallLibrary(
+			libraryName,
+			libraryNameLength,
+			libraryInstallDirectory,
+			libraryInstallDirectoryLength);
+	}
+	catch (const exception & ex)
+	{
+		result = SQL_ERROR;
+
+		errorString = string(ex.what());
+	}
+	catch (const bp::error_already_set &)
+	{
+		result = SQL_ERROR;
+
+		errorString = PythonExtensionUtils::ParsePythonException();
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		errorString = "Unexpected exception occurred in function UninstallExternalLibrary()";
+	}
+
+	if (!errorString.empty())
+	{
+		LOG_ERROR(errorString);
+		result = SQL_ERROR;
+
+		*libraryErrorLength = errorString.length();
+
+		string *pError = new string(errorString);
+		SQLCHAR *error = const_cast<SQLCHAR *>(reinterpret_cast<const SQLCHAR *>(pError->c_str()));
+
+		*libraryError = error;
+	}
+
+	return result;
+}

language-extensions/python/src/PythonExtensionUtils.cpp

@@ -0,0 +1,208 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonExtensionUtils.cpp
+//
+// Purpose:
+//  Non-platform specific utility functions for PythonExtension
+//
+//*************************************************************************************************
+
+#include "Logger.h"
+#include "PythonExtensionUtils.h"
+#include "PythonPathSettings.h"
+
+using namespace std;
+namespace bp = boost::python;
+
+// Null values
+//
+const static float sm_floatNull = NAN;
+const static int sm_intNull = 0;
+const static bool sm_boolNull = false;
+
+// Map to store the ODBC C datatype to null value in Python.
+//
+const unordered_map<SQLSMALLINT, const void*> PythonExtensionUtils::sm_DataTypeToNullMap =
+{
+	{SQL_C_BIT, static_cast<const void*>(&sm_boolNull)},
+	{SQL_C_SLONG, static_cast<const void*>(&sm_intNull)},
+	{SQL_C_FLOAT, static_cast<const void*>(&sm_floatNull)},
+	{SQL_C_DOUBLE, static_cast<const void*>(&sm_floatNull)},
+	{SQL_C_SBIGINT, static_cast<const void*>(&sm_intNull)},
+	{SQL_C_SSHORT, static_cast<const void*>(&sm_intNull)},
+	{SQL_C_UTINYINT, static_cast<const void*>(&sm_intNull)}
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::ParsePythonException
+//
+// Description:
+//  Parses the value of the active python exception.
+//  Type, value, and traceback are in separate pointers.
+//
+// Returns:
+//  String version of the python error
+//
+string PythonExtensionUtils::ParsePythonException()
+{
+	PyObject *pType = NULL;
+	PyObject *pValue = NULL;
+	PyObject *pTraceback = NULL;
+
+	// Fetch the exception info from the Python C API
+	//
+	PyErr_Fetch(&pType, &pValue, &pTraceback);
+
+	// Fallback error
+	//
+	string ret("Unfetchable Python error");
+
+	// If the fetch got a type pointer, parse the type into the exception string
+	//
+	if (pType != NULL)
+	{
+		string type = ExtractString(pType);
+
+		// If a valid string extraction is available, use it
+		// otherwise use fallback string
+		//
+		if (type.empty())
+		{
+			ret = "Unknown exception type";
+		}
+		else
+		{
+			ret = type;
+		}
+	}
+
+	// Do the same for the exception value (the stringification of the exception)
+	//
+	if (pValue != NULL)
+	{
+		string value = ExtractString(pValue);
+
+		if (value.empty())
+		{
+			ret += string(": Unparseable Python error: ");
+		}
+		else
+		{
+			ret += ": " + value;
+		}
+	}
+
+	// Parse lines from the traceback using the Python traceback module
+	//
+	if (pTraceback != NULL)
+	{
+		bp::handle<> handleTrace(pTraceback);
+
+		// Load the traceback module and the format_tb function
+		//
+		bp::object traceback(bp::import("traceback"));
+		bp::object format_tb(traceback.attr("format_tb"));
+
+		// Call format_tb to get a list of traceback strings
+		//
+		bp::object traceList(format_tb(handleTrace));
+
+		// Join the traceback strings into a single string
+		//
+		bp::object tracePyStr(bp::str("\n").join(traceList));
+
+		// Extract the string, check the extraction, and fallback if necessary
+		//
+		string trace = ExtractString(tracePyStr);
+
+		if (trace.empty())
+		{
+			ret += string(": Unparseable Python traceback");
+		}
+		else
+		{
+			ret += ": " + trace;
+		}
+	}
+
+	return ret;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::ExtractString
+//
+// Description:
+//  Extract the string from a PyObject, then DECREFS the PyObject.
+//  NOTE: This function STEALS the reference and destroys it.
+//
+// Returns:
+//  String inside the PyObject
+//
+string PythonExtensionUtils::ExtractString(PyObject *pObj)
+{
+	bp::handle<> handle(pObj);
+
+	return PythonExtensionUtils::ExtractString(bp::object(handle));
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::ExtractString
+//
+// Description:
+//  Extract the string from a boost::python object
+//
+// Returns:
+//  String inside the boost::python object
+//
+string PythonExtensionUtils::ExtractString(bp::object handle)
+{
+	string ret;
+	bp::str pyStr(handle);
+
+	// Extract the string from the boost::python object
+	//
+	bp::extract<string> extracted(pyStr);
+
+	// If a valid string extraction is available, use it
+	// otherwise return empty string
+	//
+	if (extracted.check())
+	{
+		ret = extracted();
+	}
+
+	return ret;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::NormalizePathString
+//
+// Description:
+//  Normalize path strings by replacting \ with /
+//
+// Returns:
+//  The normalized path string
+//
+string PythonExtensionUtils::NormalizePathString(string pathString)
+{
+	replace(pathString.begin(), pathString.end(), '\\', '/');
+	return pathString;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::IsBitTrue
+//
+// Description:
+//  Check if a SQLCHAR bit is True (not 0 or '0').
+//
+// Returns:
+//  What the boolean value of the bit is
+//
+bool PythonExtensionUtils::IsBitTrue(SQLCHAR bitValue)
+{
+	return bitValue != '0' && bitValue != 0;
+}

language-extensions/python/src/PythonLibrarySession.cpp

@@ -0,0 +1,327 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonLibrarySession.cpp
+//
+// Purpose:
+//  Class encapsulating operations performed in a library management session
+//
+//*************************************************************************************************
+
+#include <experimental/filesystem>
+#include <fstream>
+#include <iostream>
+#include <regex>
+#include <unordered_map>
+
+#include "Logger.h"
+#include "PythonExtensionUtils.h"
+#include "PythonLibrarySession.h"
+#include "PythonNamespace.h"
+
+using namespace std;
+namespace bp = boost::python;
+namespace fs = std::experimental::filesystem;
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonLibrarySession::Init
+//
+// Description:
+//  Initializes the Python library session, initialize the main namespace.
+//
+void PythonLibrarySession::Init(
+	const SQLGUID *sessionId)
+{
+	LOG("PythonLibrarySession::Init");
+
+	m_mainNamespace = PythonNamespace::MainNamespace();
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonLibrarySession::InstallLibrary
+//
+// Description:
+//  Install the specified library.
+//
+// Returns:
+//  The result of installation
+//
+SQLRETURN PythonLibrarySession::InstallLibrary(
+	string     tempFolder,
+	SQLCHAR    *libraryName,
+	SQLINTEGER libraryNameLength,
+	SQLCHAR    *libraryFile,
+	SQLINTEGER libraryFileLength,
+	SQLCHAR    *libraryInstallDirectory,
+	SQLINTEGER libraryInstallDirectoryLength)
+{
+	LOG("PythonLibrarySession::InstallExternalLibrary");
+	SQLRETURN result = SQL_ERROR;
+
+	string errorString;
+
+	string libName = string(reinterpret_cast<char *>(libraryName), libraryNameLength);
+
+	string installDir = string(reinterpret_cast<char *>(libraryInstallDirectory), libraryInstallDirectoryLength);
+	installDir = PythonExtensionUtils::NormalizePathString(installDir);
+	string libFilePath = string(reinterpret_cast<char *>(libraryFile), libraryFileLength);
+	libFilePath = PythonExtensionUtils::NormalizePathString(libFilePath);
+	
+	string extractScript = "import zipfile\n"
+		"with zipfile.ZipFile('" + libFilePath + "') as zip:\n"
+		"    zip.extractall('" + tempFolder + "')";
+
+	bp::exec(extractScript.c_str(), m_mainNamespace);
+
+	string installPath = "";
+
+	// Find the python package inside the zip to use for installation.
+	//
+	for (const fs::directory_entry &entry : fs::directory_iterator(tempFolder))
+	{
+		string type = entry.path().extension().generic_string();
+
+		if (type.compare(".whl") == 0 ||
+			type.compare(".zip") == 0 ||
+			type.compare(".gz") == 0)
+		{
+			installPath = entry.path().generic_string();
+			break;
+		}
+	}
+
+	if (installPath.empty())
+	{
+		throw runtime_error("Could not find the package inside the zip - "
+			"external library must be a python package inside a zip.");
+	}
+
+	string pathToPython = PythonExtensionUtils::GetPathToPython();
+
+	string installScript = "import subprocess;pipresult = subprocess.run(['" + pathToPython +
+		"', '-m', 'pip', 'install', '" + installPath +
+		"', '--no-deps', '--ignore-installed', '--no-cache-dir', '-t', '" + installDir + "']).returncode";
+
+	bp::exec(installScript.c_str(), m_mainNamespace);
+
+	int pipResult = bp::extract<int>(m_mainNamespace["pipresult"]);
+
+	if (pipResult != 0)
+	{
+		throw runtime_error("Pip failed to install the package with exit code " + to_string(pipResult));
+	}
+
+	result = SQL_SUCCESS;
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonLibrarySession::UninstallLibrary
+//
+// Description:
+//  Uninstall the specified library from the specified library directory
+//
+// Returns:
+//  The result of uninstallation
+//
+SQLRETURN PythonLibrarySession::UninstallLibrary(
+	SQLCHAR    *libraryName,
+	SQLINTEGER libraryNameLength,
+	SQLCHAR    *libraryInstallDirectory,
+	SQLINTEGER libraryInstallDirectoryLength
+)
+{
+	LOG("PythonLibrarySession::UninstallExternalLibrary");
+	SQLRETURN result = SQL_ERROR;
+
+	string errorString;
+	vector<fs::directory_entry> artifacts;
+
+	string libName = string(reinterpret_cast<char *>(libraryName), libraryNameLength);
+
+	string installDir = string(reinterpret_cast<char *>(libraryInstallDirectory), libraryInstallDirectoryLength);
+	installDir = PythonExtensionUtils::NormalizePathString(installDir);
+
+	try
+	{
+		// Save the top_level items so we can delete them if the pip uninstall fails.
+		// If pip uninstall succeeds, we won't need this.
+		//
+		artifacts = GetTopLevel(libName, installDir);
+
+		string pathToPython = PythonExtensionUtils::GetPathToPython();
+
+		string uninstallScript =
+			"newPath = ['" + installDir + "'] + _originalpath\n"
+			"os.environ['PYTHONPATH'] = os.pathsep.join(newPath)\n"
+			"import subprocess\n"
+			"pipresult = subprocess.run(['" + pathToPython +
+			"', '-m', 'pip', 'uninstall', '" + libName + "', '-y']).returncode\n";
+
+		bp::exec(uninstallScript.c_str(), m_mainNamespace);
+
+		int pipResult = bp::extract<int>(m_mainNamespace["pipresult"]);
+
+		if (pipResult == 0)
+		{
+			result = SQL_SUCCESS;
+		}
+		else
+		{
+			throw runtime_error("Pip failed to fully uninstall the package with exit code " + to_string(pipResult));
+		}
+	}
+	catch (const exception & ex)
+	{
+		result = SQL_ERROR;
+
+		errorString = string(ex.what());
+	}
+	catch (const bp::error_already_set &)
+	{
+		result = SQL_ERROR;
+
+		errorString = PythonExtensionUtils::ParsePythonException();
+	}
+	catch (...)
+	{
+		result = SQL_ERROR;
+
+		errorString = "Unexpected exception occurred in function UninstallExternalLibrary()";
+	}
+
+	// If pip fails for some reason, we try to manually uninstall the package by deleting the
+	// top level package folder as well as any dist/egg/.py files that were left behind.
+	//
+	if (result != SQL_SUCCESS && fs::exists(installDir))
+	{
+		LOG("Failed to fully uninstall " + libName + " with pip, deleting files manually");
+
+		for (fs::directory_entry entry : artifacts)
+		{
+			fs::remove_all(entry);
+		}
+
+		vector<fs::directory_entry> newArtifacts = GetAllArtifacts(libName, installDir);
+
+		for (fs::directory_entry entry : newArtifacts)
+		{
+			fs::remove_all(entry);
+		}
+
+		// If we successfully removed all the files, then we have a SUCCESS result.
+		//
+		result = SQL_SUCCESS;
+	}
+
+	return result;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonLibrarySession::GetTopLevel
+//
+// Description:
+//  Get top level directory/ies for a package
+//
+// Returns:
+//  A vector of directory_entries of the top level artifacts of the package
+//
+vector<fs::directory_entry> PythonLibrarySession::GetTopLevel(string libName, string installDir)
+{
+	vector<fs::directory_entry> artifacts;
+	regex_constants::syntax_option_type caseInsensitive = regex_constants::icase;
+
+	// Normalize library names by replacing all dashes and underscores with regex for either
+	//
+	string regexLibName = regex_replace(libName, regex("(-|_)"), "(-|_)");
+
+	if (fs::exists(installDir))
+	{
+		for (const fs::directory_entry &entry : fs::directory_iterator(installDir))
+		{
+			string pathFilename = entry.path().filename().string();
+
+			// The top_level.txt file is in the egg-info or dist-info folder
+			//
+			regex egg("^" + regexLibName + "-(.*)egg(.*)", caseInsensitive);
+			regex distinfo("^" + regexLibName + "-(.*)dist-info", caseInsensitive);
+
+			if (regex_match(pathFilename, egg) ||
+				regex_match(pathFilename, distinfo))
+			{
+				artifacts.push_back(entry);
+
+				// The top_level.txt file tells us what items this package put into the installation directory
+				// that we will need to delete to uninstall.
+				//
+				fs::path topLevelPath = entry.path();
+				topLevelPath = topLevelPath.append("top_level.txt");
+
+				if (fs::exists(topLevelPath))
+				{
+					// Read in the top_level file to find what the top_level folders and files are
+					//
+					ifstream topLevelFile(topLevelPath);
+					string str;
+					while (getline(topLevelFile, str))
+					{
+						if (str.size() > 0)
+						{
+							fs::path path(installDir);
+							artifacts.push_back(fs::directory_entry(path.append(str)));
+						}
+					}
+
+					topLevelFile.close();
+					break;
+				}
+
+				break;
+			}
+		}
+	}
+
+	return artifacts;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonLibrarySession::GetAllArtifacts
+//
+// Description:
+//  Get all the artifacts we can find of a package that are in the path
+//
+// Returns:
+//  A vector of directory_entries of the artifacts
+//
+vector<fs::directory_entry> PythonLibrarySession::GetAllArtifacts(string libName, string path)
+{
+	vector<fs::directory_entry> artifacts;
+	regex_constants::syntax_option_type caseInsensitive = regex_constants::icase;
+
+	// Normalize library names by replacing all dashes with underscores
+	//
+	string regexLibName = regex_replace(libName, regex("(-|_)"), "(-|_)");
+
+	if (fs::exists(path))
+	{
+		for (const fs::directory_entry &entry : fs::directory_iterator(path))
+		{
+			string pathFilename = entry.path().filename().string();
+
+			regex pth("^" + regexLibName + "-(.*).pth", caseInsensitive);
+			regex pyFile("^" + regexLibName + ".py", caseInsensitive);
+
+			if (regex_match(pathFilename, pyFile) ||
+				regex_match(pathFilename, pth))
+			{
+				artifacts.push_back(entry);
+			}
+		}
+	}
+
+	return artifacts;
+}

language-extensions/python/src/PythonNamespace.cpp

@@ -0,0 +1,86 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonNamespace.cpp
+//
+// Purpose:
+//  Global class to keep the global python namespace
+//
+//*************************************************************************************************
+
+#include "PythonNamespace.h"
+#include "PythonPathSettings.h"
+
+using namespace std;
+namespace bp = boost::python;
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonNamespace::Init
+//
+// Description:
+//  Initialize the class
+//
+void PythonNamespace::Init()
+{
+	sm_mainModule = bp::import("__main__");
+	sm_mainNamespace = sm_mainModule.attr("__dict__");
+
+	// Check that the module and namespace are populated, not None objects
+	//
+	if (sm_mainModule == bp::object() ||
+		sm_mainNamespace == bp::object())
+	{
+		throw runtime_error("Main module or namespace was None");
+	}
+
+	// Setup the devnull device (which we do not have access to in Windows)
+	// to redirect to a file. We create that file here for future use.
+	// Also, import packages that we will need in later functions.
+	//
+	string setupScript =
+		"import os\n"
+		"import sys\n"
+		"import platform\n"
+		"from pandas import DataFrame\n"
+		"import numpy as np\n"
+		"_originalpath = list(sys.path)\n"
+		"if platform.system() == 'Windows':\n"
+		"    oldnulldevice = os.devnull\n"
+		"    os.devnull = 'nulldeviceout.txt'\n"
+		"    nulhandle = open(os.devnull, 'w+')\n"
+		"    nulhandle.close()";
+
+	bp::exec(setupScript.c_str(), sm_mainNamespace);
+
+	string privateLibPath = PythonPathSettings::PrivateLibraryPath();
+	string publicLibPath = PythonPathSettings::PublicLibraryPath();
+
+	// Setup the path to include private and public external library paths
+	// so that we can find the external packages that are installed.
+	// We set the private/public paths in front of the sys.path so they are searched first.
+	//
+	sm_originalPath = bp::extract<bp::list>(bp::eval("sys.path", sm_mainNamespace));
+	bp::list newPath(sm_originalPath);
+	newPath.insert(0, bp::str(publicLibPath));
+	newPath.insert(0, bp::str(privateLibPath));
+
+	PySys_SetObject("path", newPath.ptr());
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonNamespace::Cleanup
+//
+// Description:
+//  Cleanup, reset the Python syspath
+//
+void PythonNamespace::Cleanup()
+{
+	PySys_SetObject("path", sm_originalPath.ptr());
+}
+
+bp::object PythonNamespace::sm_mainNamespace;
+bp::object PythonNamespace::sm_mainModule;
+bp::object PythonNamespace::sm_originalPath;

language-extensions/python/src/PythonParam.cpp

@@ -0,0 +1,661 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonParam.cpp
+//
+// Purpose:
+//  Class storing information about the PythonExtension input/output parameter.
+//
+//*************************************************************************************************
+
+#include "Logger.h"
+#include "PythonExtensionUtils.h"
+#include "PythonParam.h"
+
+// datetime.h includes macros that define the PyDateTime APIs
+//
+#include <datetime.h>
+
+using namespace std;
+namespace bp = boost::python;
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonParam
+//
+// Description:
+//  Constructor.
+//
+PythonParam::PythonParam(
+	SQLUSMALLINT  id,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   type,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType) :
+	m_id(id),
+	m_type(type),
+	m_size(paramSize),
+	m_decimalDigits(decimalDigits),
+	m_strLenOrInd(strLen_or_Ind),
+	m_inputOutputType(inputOutputType)
+{
+
+	// Remove "@" from the front of the name
+	//
+	const char *name = static_cast<const char*>(static_cast<const void*>(paramName + 1));
+
+	// paramNameLength includes @, we remove it
+	//
+#if defined(_DEBUG)
+	if (static_cast<size_t>(paramNameLength - 1) != strlen(name))
+	{
+		throw invalid_argument("Invalid parameter name length, it doesn't match string length.");
+	}
+#endif
+
+	// Store the information for this column
+	//
+	m_name = string(name, paramNameLength - 1);
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: CheckParamSize
+//
+// Description:
+//  Verifies if m_Size is equal to the size of the template type T.
+//  Returns nothing if the check succeeds, throws an exception otherwise.
+//
+template<class T>
+void PythonParam::CheckParamSize()
+{
+	size_t dataTypeSize = sizeof(T);
+	if (dataTypeSize != m_size)
+	{
+		string error("The parameter size(" + to_string(m_size) +
+			") does not match the size of the supported datatype(" +
+			to_string(dataTypeSize) + ").");
+		LOG_ERROR(error);
+		throw invalid_argument(error);
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonParamTemplate
+//
+// Description:
+//  Constructor.
+//  Calls the base constructor then populates m_pyObject with a boost::python object that contains
+//  the parameter value, in a way that python can use, or bp::object which is None.
+//
+template<class SQLType>
+PythonParamTemplate<SQLType>::PythonParamTemplate(
+	SQLUSMALLINT  id,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   type,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLPOINTER    paramValue,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType)
+	: PythonParam(id,
+		paramName,
+		paramNameLength,
+		type,
+		paramSize,
+		decimalDigits,
+		strLen_or_Ind,
+		inputOutputType)
+{
+	CheckParamSize<SQLType>();
+
+	if (strLen_or_Ind != SQL_NULL_DATA)
+	{
+		m_pyObject = bp::object(*static_cast<SQLType*>(paramValue));
+	}
+	else
+	{
+		// Use None object for NULLs
+		//
+		m_pyObject = bp::object();
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonParamTemplate::RetrieveValueAndStrLenInd
+//
+// Description:
+//  Retrieves the value from the namespace and populates m_value and m_strLenOrInd.
+//  Template for int/float types.
+//
+template<class SQLType>
+void PythonParamTemplate<SQLType>::RetrieveValueAndStrLenInd(bp::object mainNamespace)
+{
+	bp::dict dictNamespace = bp::extract<bp::dict>(mainNamespace);
+
+	m_strLenOrInd = SQL_NULL_DATA;
+
+	if (dictNamespace.has_key(m_name))
+	{
+		bp::object tempObj = mainNamespace[m_name];
+
+		if(!tempObj.is_none())
+		{
+			bp::extract<SQLType> extractedObj(tempObj);
+			if(extractedObj.check())
+			{
+				m_value.push_back(SQLType(extractedObj));
+				m_strLenOrInd = sizeof(SQLType);
+			}
+		}
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonBooleanParam
+//
+// Description:
+//  Constructor.
+//  Calls the base constructor then populates m_pyObject with a boost::python object that contains
+//  the parameter value, in a way that python can use, or bp::object which is None.
+//
+PythonBooleanParam::PythonBooleanParam(
+	SQLUSMALLINT  id,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   type,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLPOINTER    paramValue,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType)
+	: PythonParam(id,
+				paramName,
+				paramNameLength,
+				type,
+				paramSize,
+				decimalDigits,
+				strLen_or_Ind,
+				inputOutputType)
+{
+	if (strLen_or_Ind != SQL_NULL_DATA)
+	{
+		SQLCHAR val = *static_cast<SQLCHAR *>(paramValue);
+		bool value = PythonExtensionUtils::IsBitTrue(val);
+		m_pyObject = bp::object(value);
+	}
+	else
+	{
+		// Use None object for NULLs
+		//
+		m_pyObject = bp::object();
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonBooleanParam::RetrieveValueAndStrLenInd
+//
+// Description:
+//  Retrieves the value from the namespace and populates m_value and m_strLenOrInd
+//
+void PythonBooleanParam::RetrieveValueAndStrLenInd(bp::object mainNamespace)
+{
+	bp::dict dictNamespace = bp::extract<bp::dict>(mainNamespace);
+	if (dictNamespace.has_key(m_name))
+	{
+		bp::object tempObj = mainNamespace[m_name];
+
+		m_strLenOrInd = SQL_NULL_DATA;
+
+		if (!tempObj.is_none())
+		{
+			bp::extract<bool> extractedObj(tempObj);
+			if (extractedObj.check())
+			{
+				m_value.push_back(static_cast<SQLCHAR>(bool(extractedObj)));
+				m_strLenOrInd = sizeof(bool);
+			}
+		}
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonStringParam
+//
+// Description:
+//  Constructor.
+//  Calls the base constructor then populates m_pyObject with a boost::python object that contains
+//  the parameter value, in a way that python can use, or bp::object which is None.
+//  We use StrLen_or_Ind to calculate how long the string is before creating the python object.
+//
+template<class CharType>
+PythonStringParam<CharType>::PythonStringParam(
+	SQLUSMALLINT  id,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   type,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLPOINTER    paramValue,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType)
+	: PythonParam(id,
+		paramName,
+		paramNameLength,
+		type,
+		paramSize,
+		decimalDigits,
+		strLen_or_Ind,
+		inputOutputType)
+{
+	if (strLen_or_Ind != SQL_NULL_DATA)
+	{
+		SQLINTEGER strlen = strLen_or_Ind / sizeof(CharType);
+
+		// Create a string PyObject from the str and strLen.
+		// This DOES copy the underlying string into a new buffer and null terminates it.
+		// Then, convert to a boost object so that boost handles ref counting.
+		//
+		m_pyObject = bp::object(bp::handle<>(
+			PyUnicode_FromKindAndData(sizeof(CharType), paramValue, strlen)
+		));
+	}
+	else
+	{
+		// Use None object for NULLs
+		//
+		m_pyObject = bp::object();
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonStringParam::RetrieveValueAndStrLenInd
+//
+// Description:
+//  Retrieves the value from the namespace and populates m_value and m_strLenOrInd
+//
+template<class CharType>
+void PythonStringParam<CharType>::RetrieveValueAndStrLenInd(bp::object mainNamespace)
+{
+	bp::dict dictNamespace = bp::extract<bp::dict>(mainNamespace);
+	if (dictNamespace.has_key(m_name))
+	{
+		bp::object tempObj = mainNamespace[m_name];
+
+		m_strLenOrInd = SQL_NULL_DATA;
+
+		if (!tempObj.is_none())
+		{
+			if constexpr (is_same_v<CharType,char>)
+			{
+				// Check to make sure the extracted data exists and is of the correct type
+				//
+				bp::extract<string> extractedObj(tempObj);
+
+				if (extractedObj.check())
+				{
+					// Extract and copy the string characters into the vector.
+					//
+					string value = extractedObj;
+					m_value = vector<CharType>(value.begin(), value.end());
+				}
+			}
+			else
+			{
+				// Get length of the unicode object in pyObj
+				//
+				int size = PyUnicode_GET_LENGTH(tempObj.ptr());
+
+				// Get a byte representation of the string as UTF16.
+				// PyUnicode_AsUTF16String adds a BOM to the front of every string.
+				//
+				char *utf16str = PyBytes_AsString(PyUnicode_AsUTF16String(tempObj.ptr()));
+
+				// Reinterpret the bytes as wchar_t *, which we will return.
+				//
+				CharType *wData = reinterpret_cast<CharType*>(utf16str);
+
+				// Ignore 2 byte BOM at front of wData that was added by PyUnicode_AsUTF16String
+				//
+				m_value = vector<CharType>(wData + 1, wData + 1 + size);
+			}
+
+			// Truncate the return data to only be the size specified when creating
+			//
+			if (m_value.size() > m_size)
+			{
+				m_value.resize(m_size);
+				m_value.shrink_to_fit();
+			}
+
+			m_strLenOrInd = m_value.size() * sizeof(CharType);
+		}
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonRawParam
+//
+// Description:
+//  Constructor.
+//  Calls the base constructor then populates m_pyObject with a boost::python object that contains
+//  the bytes object of the data, in a way that python can use, or bp::object which is None.
+//
+PythonRawParam::PythonRawParam(
+	SQLUSMALLINT  id,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   type,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLPOINTER    paramValue,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType)
+	: PythonParam(id,
+		paramName,
+		paramNameLength,
+		type,
+		paramSize,
+		decimalDigits,
+		strLen_or_Ind,
+		inputOutputType)
+{
+	if (strLen_or_Ind != SQL_NULL_DATA)
+	{
+		SQLINTEGER strlen = strLen_or_Ind / sizeof(SQLCHAR);
+
+		// Create a Python bytes object from binary
+		//
+		m_pyObject = bp::object(bp::handle<>(
+			PyBytes_FromObject(PyMemoryView_FromMemory(
+				static_cast<char *>(paramValue), strlen, PyBUF_READ
+			))
+			));
+	}
+	else
+	{
+		// Use None object for NULLs
+		//
+		m_pyObject = bp::object();
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonRawParam::RetrieveValueAndStrLenInd
+//
+// Description:
+//  Retrieves the value from the namespace and populates m_value and m_strLenOrInd
+//
+void PythonRawParam::RetrieveValueAndStrLenInd(bp::object mainNamespace)
+{
+	bp::dict dictNamespace = bp::extract<bp::dict>(mainNamespace);
+	if (dictNamespace.has_key(m_name))
+	{
+		bp::object tempObj = mainNamespace[m_name];
+
+		m_strLenOrInd = SQL_NULL_DATA;
+
+		if (!tempObj.is_none())
+		{
+			// The uninitialized iterator is equivalent to the end of the iterable
+			//
+			bp::stl_input_iterator<SQLCHAR> begin(tempObj), end;
+
+			// Copy the py_buffer into a local buffer.
+			//
+			m_value = vector<SQLCHAR>(begin, end);
+
+			// Truncate the return data to only be the size specified when creating
+			//
+			if (m_value.size() > m_size)
+			{
+				m_value.resize(m_size);
+				m_value.shrink_to_fit();
+			}
+
+			m_strLenOrInd = m_value.size();
+		}
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonDateTimeParam
+//
+// Description:
+//  Constructor.
+//  Calls the base constructor then populates m_pyObject with a boost::python object that contains
+//  the date/datetime object of the data, in a way that python can use, or bp::object which is None.
+//
+template<SQLSMALLINT DataType>
+PythonDateTimeParam<DataType>::PythonDateTimeParam(
+	SQLUSMALLINT  id,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   type,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLPOINTER    paramValue,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType)
+	: PythonParam(id,
+		paramName,
+		paramNameLength,
+		type,
+		paramSize,
+		decimalDigits,
+		strLen_or_Ind,
+		inputOutputType)
+{
+	if (strLen_or_Ind != SQL_NULL_DATA)
+	{
+		// Use the PyDateTime_IMPORT macro to get the Python Date/Time APIs
+		//
+		PyDateTime_IMPORT;
+		PyObject *dtObject = Py_None;
+
+		// SQL_C_TYPE_DATE for Date objects in SQL, SQL_C_TYPE_TIMESTAMP for Datetime
+		//
+		if (DataType == SQL_C_TYPE_DATE)
+		{
+			SQL_DATE_STRUCT dateParam = *(static_cast<SQL_DATE_STRUCT *>(paramValue));
+
+			// Create a Python Date object
+			//
+			dtObject = PyDate_FromDate(dateParam.year, dateParam.month, dateParam.day);
+		}
+		else if (DataType == SQL_C_TYPE_TIMESTAMP)
+		{
+			SQL_TIMESTAMP_STRUCT timeStampParam = *(static_cast<SQL_TIMESTAMP_STRUCT *>(paramValue));
+			
+			// "fraction" is stored in nanoseconds, we need microseconds.
+			//
+			SQLUINTEGER usec = timeStampParam.fraction / 1000;
+
+			// Create a Python DateTime object
+			//
+			dtObject = PyDateTime_FromDateAndTime(
+				timeStampParam.year, timeStampParam.month, timeStampParam.day,
+				timeStampParam.hour, timeStampParam.minute, timeStampParam.second, usec);
+		}
+
+		m_pyObject = bp::object(bp::handle<>(dtObject));
+	}
+	else
+	{
+		// Use None object for NULLs
+		//
+		m_pyObject = bp::object();
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonDateTimeParam::RetrieveValueAndStrLenInd
+//
+// Description:
+//  Retrieves the value from the namespace and populates m_value and m_strLenOrInd
+//
+template<SQLSMALLINT DataType>
+void PythonDateTimeParam<DataType>::RetrieveValueAndStrLenInd(bp::object mainNamespace)
+{
+	bp::dict dictNamespace = bp::extract<bp::dict>(mainNamespace);
+	if (dictNamespace.has_key(m_name))
+	{
+		bp::object tempObj = mainNamespace[m_name];
+
+		bp::exec("from numpy import isnat, datetime64", mainNamespace);
+		bool isNaT = false;
+
+		if (DataType == SQL_C_TYPE_TIMESTAMP)
+		{
+			string checkNaTScript = "isnat(datetime64(" + m_name + "))";
+			isNaT = bp::extract<bool>(bp::eval(checkNaTScript.c_str(), mainNamespace));
+		}
+
+		m_strLenOrInd = SQL_NULL_DATA;
+
+		// Make sure the boost object is not pointing at Python None.
+		// Also check the object type for NaT (Not a Time), a special timestamp type,
+		// because that should be NULL in SQL as well.
+		//
+		if (!tempObj.is_none() && !isNaT)
+		{
+			PyDateTime_IMPORT;
+			PyObject *dateObject = tempObj.ptr();
+
+			SQLSMALLINT year = PyDateTime_GET_YEAR(dateObject);
+			SQLUSMALLINT month = PyDateTime_GET_MONTH(dateObject);
+			SQLUSMALLINT day = PyDateTime_GET_DAY(dateObject);
+			SQLUSMALLINT hour = PyDateTime_DATE_GET_HOUR(dateObject);
+			SQLUSMALLINT minute = PyDateTime_DATE_GET_MINUTE(dateObject);
+			SQLUSMALLINT second = PyDateTime_DATE_GET_SECOND(dateObject);
+			SQLUINTEGER usec = PyDateTime_DATE_GET_MICROSECOND(dateObject);
+
+			// "fraction" in TIMESTAMP_STRUCT is stored in nanoseconds, we convert from microseconds.
+			//
+			SQL_TIMESTAMP_STRUCT datetime = { year, month, day, hour, minute, second, usec * 1000 };
+
+			m_value.push_back(datetime);
+			m_strLenOrInd = sizeof(SQL_TIMESTAMP_STRUCT);
+		}
+	}
+}
+
+//------------------------------------------------------------------------------------------------
+// Do explicit template instantiations, so that object code is generated for these
+// and the linker is able to find their definitions even after instantiations are in different
+// translation units (i.e. PythonParamTemplate instantiation is in PythonParamContainer.cpp)
+//
+template PythonParamTemplate<SQLINTEGER>::PythonParamTemplate(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonParamTemplate<SQLREAL>::PythonParamTemplate(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonParamTemplate<SQLDOUBLE>::PythonParamTemplate(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonParamTemplate<SQLBIGINT>::PythonParamTemplate(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonParamTemplate<SQLSMALLINT>::PythonParamTemplate(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonParamTemplate<SQLCHAR>::PythonParamTemplate(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonStringParam<char>::PythonStringParam(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonStringParam<wchar_t>::PythonStringParam(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonDateTimeParam<SQL_C_TYPE_DATE>::PythonDateTimeParam(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);
+
+template PythonDateTimeParam<SQL_C_TYPE_TIMESTAMP>::PythonDateTimeParam(
+	SQLUSMALLINT,
+	const SQLCHAR *,
+	SQLSMALLINT,
+	SQLSMALLINT,
+	SQLULEN,
+	SQLSMALLINT,
+	SQLPOINTER,
+	SQLINTEGER,
+	SQLSMALLINT);

language-extensions/python/src/PythonParamContainer.cpp

@@ -0,0 +1,183 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonParamContainer.cpp
+//
+// Purpose:
+//  PythonExtension input/output parameters wrappers,
+//  along with the container consolidating them.
+//
+//*************************************************************************************************
+
+
+#include "Logger.h"
+#include "PythonParam.h"
+#include "PythonParamContainer.h"
+
+using namespace std;
+namespace bp = boost::python;
+
+// Function map - maps a SQL data type to the appropriate param creator
+//
+unordered_map<SQLSMALLINT, PythonParamContainer::fnCreateParam>
+PythonParamContainer::sm_FnCreateParamMap =
+{
+	{static_cast<SQLSMALLINT>(SQL_C_BIT),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonBooleanParam>)},
+	{static_cast<SQLSMALLINT>(SQL_C_SLONG),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonParamTemplate<SQLINTEGER>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_DOUBLE),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonParamTemplate<SQLDOUBLE>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_FLOAT),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonParamTemplate<SQLREAL>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_SSHORT),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonParamTemplate<SQLSMALLINT>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_UTINYINT),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonParamTemplate<SQLCHAR>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_SBIGINT),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonParamTemplate<SQLBIGINT>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_CHAR),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonStringParam<char>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_WCHAR),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonStringParam<wchar_t>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_BINARY),
+	 static_cast<fnCreateParam>(&PythonParamContainer::CreateParam<PythonRawParam>)},
+	{static_cast<SQLSMALLINT>(SQL_C_TYPE_DATE), static_cast<fnCreateParam>(
+		 &PythonParamContainer::CreateParam<PythonDateTimeParam<SQL_C_TYPE_DATE>>)},
+	{static_cast<SQLSMALLINT>(SQL_C_TYPE_TIMESTAMP), static_cast<fnCreateParam>(
+		 &PythonParamContainer::CreateParam<PythonDateTimeParam<SQL_C_TYPE_TIMESTAMP>>)},
+};
+
+//-------------------------------------------------------------------------------------------------
+// Name: Init
+//
+// Description:
+//  Initialize this container with the number of parameters.
+//
+void PythonParamContainer::Init(SQLSMALLINT paramsNumber)
+{
+	LOG("PythonParamContainer::Init");
+
+	m_params.resize(paramsNumber);
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: AddParamToNamespace
+//
+// Description:
+//  Creates a PythonParam object and adds it to the boost:python namespace.
+//  Eventually, adds the PythonParam to m_params for future use.
+//  Creation is done by finding the appropriate function from the function map.
+//
+void PythonParamContainer::AddParamToNamespace(
+	bp::object    nameSpace,
+	SQLUSMALLINT  paramNumber,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   dataType,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLPOINTER    paramValue,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType)
+{
+	LOG("PythonParamContainer::AddParamToNamespace");
+
+	CreateParamFnMap::const_iterator it = sm_FnCreateParamMap.find(dataType);
+
+	if (it == sm_FnCreateParamMap.end())
+	{
+		throw runtime_error("Unsupported parameter type " + to_string(dataType) +
+			" encountered when creating param #" + to_string(paramNumber));
+	}
+
+	(this->*it->second)(
+		nameSpace,
+		paramNumber,
+		paramName,
+		paramNameLength,
+		dataType,
+		paramSize,
+		decimalDigits,
+		paramValue,
+		strLen_or_Ind,
+		inputOutputType
+		);
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: CreateParam
+//
+// Description:
+//  Template to create a parameter and add it to the python namespace
+//
+template<class ParamType>
+void PythonParamContainer::CreateParam(
+	bp::object    nameSpace,
+	SQLUSMALLINT  paramNumber,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   dataType,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLPOINTER    paramValue,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType)
+{
+	unique_ptr<PythonParam> paramToBeAdded = make_unique<ParamType>(
+		paramNumber,
+		paramName,
+		paramNameLength,
+		dataType,
+		paramSize,
+		decimalDigits,
+		paramValue,
+		strLen_or_Ind,
+		inputOutputType
+		);
+
+	string name = paramToBeAdded.get()->Name();
+	nameSpace[name] = paramToBeAdded.get()->PythonObject();
+	m_params[paramNumber] = std::move(paramToBeAdded);
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonParamContainer::GetParamValueAndStrLenInd
+//
+// Description:
+//  For the given paramNumber, call RetriveValueAndStrLenOrInd() to retrieve the value from python
+//  and return it via paramValue. Return the strLenOrInd as well.
+//  Note the value returned is allocated on the heap and
+//  will be cleaned up when param is destructed.
+//
+void PythonParamContainer::GetParamValueAndStrLenInd(
+	bp::object   mainNamespace,
+	SQLUSMALLINT paramNumber,
+	SQLPOINTER   *paramValue,
+	SQLINTEGER   *strLen_or_Ind)
+{
+	LOG("PythonParamContainer::GetParamValueAndStrLenInd");
+
+	if (m_params[paramNumber] == nullptr)
+	{
+		throw runtime_error("InitParam not called for paramNumber " + to_string(paramNumber));
+	}
+
+	PythonParam *param = m_params[paramNumber].get();
+
+	if (param->InputOutputType() <= SQL_PARAM_INPUT)
+	{
+		throw runtime_error("Requested param #" + to_string(paramNumber) +
+			" is not initialized as an output parameter");
+	}
+
+	// Retrieve the value from Python namespace
+	//
+	param->RetrieveValueAndStrLenInd(mainNamespace);
+
+	*paramValue = param->Value();
+	*strLen_or_Ind = param->StrLenOrInd();
+}

language-extensions/python/src/PythonPathSettings.cpp

@@ -0,0 +1,55 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonPathSettings.cpp
+//
+// Purpose:
+//  Global class to keep language runtime settings
+//
+//*************************************************************************************************
+
+#include "PythonPathSettings.h"
+#include "PythonExtensionUtils.h"
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonPathSettings::Init
+//
+// Description:
+//  Initialize the class
+//
+void
+PythonPathSettings::Init(
+	const SQLCHAR *languageParams,
+	const SQLCHAR *languagePath,
+	const SQLCHAR *publicLibraryPath,
+	const SQLCHAR *privateLibraryPath)
+{
+	// nullptrs are mapped to empty strings - has the same effect when
+	// the paths are used and avoids an additional flag.
+	//
+	sm_languageParams =
+		(languageParams == nullptr) ? "" : reinterpret_cast<const char *>(languageParams);
+
+	sm_languagePath =
+		(languagePath == nullptr) ? "" : reinterpret_cast<const char *>(languagePath);
+
+	sm_publicLibraryPath =
+		(publicLibraryPath == nullptr) ? "" : reinterpret_cast<const char *>(publicLibraryPath);
+
+	sm_privateLibraryPath =
+		(privateLibraryPath == nullptr) ? "" : reinterpret_cast<const char *>(privateLibraryPath);
+
+	// Remove single slashes and replace with forward slashes in the paths
+	//
+	sm_languagePath = PythonExtensionUtils::NormalizePathString(sm_languagePath);
+	sm_privateLibraryPath = PythonExtensionUtils::NormalizePathString(sm_privateLibraryPath);
+	sm_publicLibraryPath = PythonExtensionUtils::NormalizePathString(sm_publicLibraryPath);
+}
+
+std::string PythonPathSettings::sm_languageParams;
+std::string PythonPathSettings::sm_languagePath;
+std::string PythonPathSettings::sm_privateLibraryPath;
+std::string PythonPathSettings::sm_publicLibraryPath;

language-extensions/python/src/PythonSession.cpp

@@ -0,0 +1,329 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonSession.cpp
+//
+// Purpose:
+//  Class encapsulating operations performed per session
+//
+//*************************************************************************************************
+
+#include "Logger.h"
+#include "PythonExtensionUtils.h"
+#include "PythonNamespace.h"
+#include "PythonPathSettings.h"
+#include "PythonSession.h"
+
+using namespace std;
+namespace bp = boost::python;
+namespace np = boost::python::numpy;
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonSession::Init
+//
+// Description:
+//  Initializes the Python session, storing all the information passed in.
+//
+void PythonSession::Init(
+	const SQLGUID *sessionId,
+	SQLUSMALLINT  taskId,
+	SQLUSMALLINT  numTasks,
+	SQLCHAR       *script,
+	SQLULEN       scriptLength,
+	SQLUSMALLINT  inputSchemaColumnsNumber,
+	SQLUSMALLINT  parametersNumber,
+	SQLCHAR       *inputDataName,
+	SQLUSMALLINT  inputDataNameLength,
+	SQLCHAR       *outputDataName,
+	SQLUSMALLINT  outputDataNameLength)
+{
+	LOG("PythonSession::Init");
+
+	m_mainNamespace = PythonNamespace::MainNamespace();
+
+	// Initialize the script
+	//
+	if (script == nullptr)
+	{
+		throw invalid_argument("Invalid script, the script value cannot be NULL");
+	}
+
+	// Initialize and store the user script
+	//
+	m_script = string(reinterpret_cast<const char*>(script), scriptLength);
+	m_scriptLength = scriptLength;
+
+	// Initialize the parameters container.
+	//
+	m_paramContainer.Init(parametersNumber);
+
+	// Initialize the InputDataSet
+	//
+	m_inputDataSet.Init(inputDataName, inputDataNameLength, inputSchemaColumnsNumber, m_mainNamespace);
+
+	// Initialize the OutputDataSet
+	//
+	m_outputDataSet.Init(outputDataName, outputDataNameLength, 0, m_mainNamespace);
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonSession::Init
+//
+// Description:
+//  Initializes the input column for this session
+//
+void PythonSession::InitColumn(
+	SQLUSMALLINT  columnNumber,
+	const SQLCHAR *columnName,
+	SQLSMALLINT   columnNameLength,
+	SQLSMALLINT   dataType,
+	SQLULEN       columnSize,
+	SQLSMALLINT   decimalDigits,
+	SQLSMALLINT   nullable,
+	SQLSMALLINT   partitionByNumber,
+	SQLSMALLINT   orderByNumber)
+{
+	LOG("PythonSession::InitColumn #" + to_string(columnNumber));
+
+	m_inputDataSet.InitColumn(
+		columnNumber,
+		columnName,
+		columnNameLength,
+		dataType,
+		columnSize,
+		decimalDigits,
+		nullable);
+ }
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonSession::InitParam
+//
+// Description:
+//  Initializes an input parameter for this session
+//
+void PythonSession::InitParam(
+	SQLUSMALLINT  paramNumber,
+	const SQLCHAR *paramName,
+	SQLSMALLINT   paramNameLength,
+	SQLSMALLINT   dataType,
+	SQLULEN       paramSize,
+	SQLSMALLINT   decimalDigits,
+	SQLPOINTER    paramValue,
+	SQLINTEGER    strLen_or_Ind,
+	SQLSMALLINT   inputOutputType)
+{
+	LOG("PythonSession::InitParam #" + to_string(paramNumber));
+
+	if (paramName == nullptr)
+	{
+		throw invalid_argument("Invalid input parameter name supplied");
+	}
+	else if (paramNumber >= m_paramContainer.GetSize())
+	{
+		throw invalid_argument("Invalid input param id supplied: " + to_string(paramNumber));
+	}
+
+	// Add parameter to the container and boost::python nameSpace.
+	//
+	m_paramContainer.AddParamToNamespace(
+		m_mainNamespace,
+		paramNumber,
+		paramName,
+		paramNameLength,
+		dataType,
+		paramSize,
+		decimalDigits,
+		paramValue,
+		strLen_or_Ind,
+		inputOutputType);
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonSession::ExecuteWorkflow
+//
+// Description:
+//  Execute the workflow for the session
+//
+void PythonSession::ExecuteWorkflow(
+	SQLULEN      rowsNumber,
+	SQLPOINTER   *data,
+	SQLINTEGER   **strLen_or_Ind,
+	SQLUSMALLINT *outputSchemaColumnsNumber)
+{
+	LOG("PythonSession::ExecuteWorkflow");
+
+	*outputSchemaColumnsNumber = 0;
+
+	// Add columns to the input DataFrame.
+	//
+	m_inputDataSet.AddColumnsToDictionary(rowsNumber, data, strLen_or_Ind);
+
+	// Add the dictionary for InputDataSet to the python namespace and convert to a DataFrame.
+	//
+	m_inputDataSet.AddDictionaryToNamespace();
+
+	// Initialize a dictionary for OutputDataSet to the python namespace .
+	//
+	m_outputDataSet.InitializeDataFrameInNamespace();
+
+	// Scripts to redirect stdout and stderr to variables to extract afterwards
+	//
+	string redirectPyOut = "import sys; from io import StringIO\n"
+		"_temp_out_ = StringIO(); _temp_err_ = StringIO()\n"
+		"sys.stdout = _temp_out_; sys.stderr = _temp_err_\n"
+		"_original_stdout_ = sys.stdout; _original_stderr_ = sys.stderr";
+
+	string resetPyOut = "sys.stdout = _original_stdout_\n"
+		"sys.stderr = _original_stderr_\n"
+		"_temp_out_ = _temp_out_.getvalue()\n"
+		"_temp_err_ = _temp_err_.getvalue()";
+
+	// Execute script and capture output
+	//
+	bp::exec(redirectPyOut.c_str(), m_mainNamespace);
+	bp::exec(m_script.c_str(), m_mainNamespace);
+	bp::exec(resetPyOut.c_str(), m_mainNamespace);
+
+	string pyStdOut = bp::extract<string>(m_mainNamespace["_temp_out_"]);
+	string pyStdErr = bp::extract<string>(m_mainNamespace["_temp_err_"]);
+
+	cout << pyStdOut << endl;
+	cerr << pyStdErr << endl;
+
+	// In case of streaming clean up the previous stream batch's output buffers
+	//
+	m_outputDataSet.CleanupColumns();
+
+	// Get the column number from the underlying DataFrame
+	// and set it to be the outputSchemaColumnsNumber.
+	//
+	*outputSchemaColumnsNumber = m_outputDataSet.GetDataFrameColumnsNumber();
+
+	if (*outputSchemaColumnsNumber > 0)
+	{
+		m_outputDataSet.PopulateColumnsDataType();
+		m_outputDataSet.PopulateNumberOfRows();
+		m_outputDataSet.RetrieveColumnsFromDataFrame();
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonSession::GetResultColumn
+//
+// Description:
+//  Returns metadata information about the output column
+//
+void PythonSession::GetResultColumn(
+	SQLUSMALLINT columnNumber,
+	SQLSMALLINT  *dataType,
+	SQLULEN      *columnSize,
+	SQLSMALLINT  *decimalDigits,
+	SQLSMALLINT  *nullable)
+{
+	LOG("PythonSession::GetResultColumn for column #" + to_string(columnNumber));
+
+	*dataType = SQL_UNKNOWN_TYPE;
+	*columnSize = 0;
+	*decimalDigits = 0;
+	*nullable = 0;
+
+	if (columnNumber >= m_outputDataSet.GetVectorColumnsNumber())
+	{
+		throw invalid_argument("Invalid column #" + to_string(columnNumber)
+			+ " provided to GetResultColumn().");
+	}
+
+	const vector<unique_ptr<PythonColumn>>& resultColumns = m_outputDataSet.Columns();
+	PythonColumn *resultColumn = resultColumns[columnNumber].get();
+
+	if(resultColumn != nullptr)
+	{
+		*dataType = resultColumn->DataType();
+		*columnSize = resultColumn->Size();
+		*decimalDigits = resultColumn->DecimalDigits();
+		*nullable = resultColumn->Nullable();
+	}
+	else
+	{
+		throw runtime_error("ResultColumn #" + to_string(columnNumber) +
+			" is not initialized for the output dataset");
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonSession::GetResults
+//
+// Description:
+//  Returns the output data and the null map retrieved from the user program
+//
+void PythonSession::GetResults(
+	SQLULEN    *rowsNumber,
+	SQLPOINTER **data,
+	SQLINTEGER ***strLen_or_Ind)
+{
+	LOG("PythonSession::GetResults");
+
+	if (rowsNumber != nullptr && data != nullptr && strLen_or_Ind != nullptr)
+	{
+		*rowsNumber = m_outputDataSet.RowsNumber();
+		*data = m_outputDataSet.GetData();
+		*strLen_or_Ind = m_outputDataSet.GetColumnNullMap();
+	}
+	else
+	{
+		throw runtime_error("Invalid parameters provided to GetResults()");
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonSession::GetOutputParam
+//
+// Description:
+//  Returns the data and size of the output parameter
+//
+void PythonSession::GetOutputParam(
+	SQLUSMALLINT paramNumber,
+	SQLPOINTER   *paramValue,
+	SQLINTEGER   *strLen_or_Ind)
+{
+	LOG("PythonSession::GetOutputParam - initializing output parameter #"
+		+ to_string(paramNumber));
+
+	if (paramValue == nullptr || strLen_or_Ind == nullptr)
+	{
+		throw invalid_argument("Null arguments supplied to GetOutputParam().");
+	}
+
+	if (paramNumber < m_paramContainer.GetSize())
+	{
+		m_paramContainer.GetParamValueAndStrLenInd(
+			m_mainNamespace,
+			paramNumber,
+			paramValue,
+			strLen_or_Ind);
+	}
+	else
+	{
+		throw invalid_argument("Invalid output parameter id supplied to GetOutputParam(): " +
+			to_string(paramNumber));
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonSession::Cleanup()
+//
+// Description:
+//  Cleans up the python session
+//
+void PythonSession::Cleanup()
+{
+	LOG("PythonSession::Cleanup");
+
+	m_inputDataSet.Cleanup();
+
+	m_outputDataSet.CleanupColumns();
+	m_outputDataSet.Cleanup();
+}

language-extensions/python/src/linux/PythonExtensionUtils_linux.cpp

@@ -0,0 +1,99 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonExtensionUtils_linux.cpp
+//
+// Purpose:
+//  Linux specific utility functions for Python Extension
+//
+//*************************************************************************************************
+
+#include <dlfcn.h>
+#include <experimental/filesystem>
+
+#include "Logger.h"
+#include "PythonExtensionUtils.h"
+
+namespace fs = std::experimental::filesystem;
+
+const CHAR *GuidFormat = "%08lX-%04X-%04X-%02X%02X-%02X%02X%02X%02X%02X%02X";
+
+//--------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::GetEnvVariable
+//
+// Description:
+//  Get the value of an environment variable
+//
+// Returns:
+//  String value of the environment variable requested
+//
+std::string PythonExtensionUtils::GetEnvVariable(const std::string &envVarName)
+{
+	char* envVarValue;
+	envVarValue = getenv(envVarName.c_str());
+
+	if (envVarValue == NULL)
+	{
+		throw std::runtime_error("Error while loading " + envVarName);
+	}
+
+	return std::string(envVarValue);
+}
+
+//--------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::ConvertGuidToString
+//
+// Description:
+//  Converts a SQLGUID to a string
+//
+// Returns:
+//  string of the guid
+//
+std::string PythonExtensionUtils::ConvertGuidToString(const SQLGUID *guid)
+{
+	// 32 hex chars + 4 hyphens + null terminator, so 37 characters.
+	//
+	char guidString[37];
+
+	snprintf(guidString, sizeof(guidString) / sizeof(guidString[0]),
+		GuidFormat,
+		static_cast<unsigned long>(guid->Data1), guid->Data2, guid->Data3,
+		guid->Data4[0], guid->Data4[1], guid->Data4[2], guid->Data4[3],
+		guid->Data4[4], guid->Data4[5], guid->Data4[6], guid->Data4[7]);
+
+	std::string s(guidString);
+
+	return s;
+}
+
+//--------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::FreeDLL
+//
+// Description:
+//  Close an open dll handle
+//
+void PythonExtensionUtils::FreeDLL(void *pDll)
+{
+	if (pDll != nullptr)
+	{
+		dlclose(pDll);
+	}
+}
+
+//--------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::GetEnvVariable
+//
+// Description:
+//  Get the path to the python executable
+//
+// Returns:
+//  String value of the python executable path
+//
+std::string PythonExtensionUtils::GetPathToPython()
+{
+	fs::path pathToPython = fs::path("python3.7");
+	return pathToPython.string();
+}

language-extensions/python/src/windows/PythonExtensionUtils_win.cpp

@@ -0,0 +1,138 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonExtensionUtils_win.cpp
+//
+// Purpose:
+//  Windows specific utility functions for Python Extension
+//
+//*************************************************************************************************
+
+#include <experimental/filesystem>
+
+#include "Logger.h"
+#include "PythonExtensionUtils.h"
+
+namespace fs = std::experimental::filesystem;
+
+const CHAR *GuidFormat = "%08lX-%04X-%04X-%02X%02X-%02X%02X%02X%02X%02X%02X";
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::GetEnvVariable
+//
+// Description:
+//  Get the value of an environment variable
+//
+// Returns:
+//  String value of the environment variable requested
+//
+std::string PythonExtensionUtils::GetEnvVariable(const std::string &envVarName)
+{
+	std::string envVarValue;
+
+	DWORD result;
+
+	// First call to get the length of the environment variable
+	//
+	result = GetEnvironmentVariableA(envVarName.c_str(), nullptr, 0);
+
+	// If result is 0, there was an error.
+	// Check GetLastError for the exact code.
+	//
+	if (result == 0)
+	{
+		DWORD dwError = GetLastError();
+		if (dwError == ERROR_ENVVAR_NOT_FOUND)
+		{
+			throw std::runtime_error("Error: could not find environment variable " + envVarName);
+		}
+		else
+		{
+			std::string errorHex;
+			errorHex.resize(8); // hex is max size 8 bytes
+			sprintf_s(&errorHex[0], errorHex.size() + 1, "%08X", dwError);
+			throw std::runtime_error("Error while loading " + envVarName + ": 0x" + errorHex);
+		}
+	}
+
+	// Resize the return string to the length returned by GetEnvironmentVariableA,
+	// minus null terminator because strings implicitly have null terminator
+	//
+	envVarValue.resize(result - 1);
+
+	// Second call gets the actual environment variable
+	//
+	result = GetEnvironmentVariableA(envVarName.c_str(), &envVarValue[0], result);
+
+	if (result == 0)
+	{
+		DWORD dwError = GetLastError();
+		std::string errorHex;
+		errorHex.resize(8); // hex is max size 8 bytes
+		sprintf_s(&errorHex[0], errorHex.size() + 1, "%08X", dwError);
+		throw std::runtime_error("Error while loading " + envVarName + ": 0x" + errorHex);
+	}
+
+	return envVarValue;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::ConvertGuidToString
+//
+// Description:
+//  Converts a SQLGUID to a string
+//
+// Returns:
+//  string of the guid
+//
+std::string PythonExtensionUtils::ConvertGuidToString(const SQLGUID *guid)
+{
+	// 32 hex chars + 4 hyphens + null terminator, so 37 characters.
+	//
+	char guidString[37];
+
+	sprintf_s(guidString, sizeof(guidString) / sizeof(guidString[0]),
+		GuidFormat,
+		guid->Data1, guid->Data2, guid->Data3,
+		guid->Data4[0], guid->Data4[1], guid->Data4[2], guid->Data4[3],
+		guid->Data4[4], guid->Data4[5], guid->Data4[6], guid->Data4[7]);
+
+	std::string s(guidString);
+	return s;
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::FreeDLL
+//
+// Description:
+//  Close an open dll handle
+//
+void PythonExtensionUtils::FreeDLL(void *pDll)
+{
+	if (static_cast<HMODULE>(pDll) != nullptr)
+	{
+		FreeLibrary(static_cast<HMODULE>(pDll));
+	}
+}
+
+//-------------------------------------------------------------------------------------------------
+// Name: PythonExtensionUtils::GetEnvVariable
+//
+// Description:
+//  Get the path to the python executable
+//
+// Returns:
+//  String value of the python executable path
+//
+std::string PythonExtensionUtils::GetPathToPython()
+{
+	std::string pythonhome = GetEnvVariable("PYTHONHOME");
+	fs::path pathToPython = fs::absolute(pythonhome) / "python.exe";
+
+	std::string pathString = NormalizePathString(pathToPython.string());
+
+	return pathString;
+}

language-extensions/python/test/build/linux/build-pythonextension-test.sh

@@ -0,0 +1,111 @@
+#!/bin/bash
+
+function check_exit_code {
+	EXIT_CODE=$?
+	if [ ${EXIT_CODE} -eq 0 ]; then
+		echo $1
+	else
+		echo $2
+		exit ${EXIT_CODE}
+	fi
+}
+
+function build {
+	# Set cmake config to first arg
+	#
+	CMAKE_CONFIGURATION=$1
+
+	rm -rf ${PYTHONEXTENSIONTEST_WORKING_DIR}/${CMAKE_CONFIGURATION}
+	mkdir -p ${PYTHONEXTENSIONTEST_WORKING_DIR}/${CMAKE_CONFIGURATION}
+
+	pushd ${PYTHONEXTENSIONTEST_WORKING_DIR}
+
+	# Compile
+	#
+	cmake -DCMAKE_INSTALL_PREFIX:PATH=${PYTHONEXTENSIONTEST_WORKING_DIR}/${CMAKE_CONFIGURATION} \
+		-DPLATFORM=linux \
+		-DCMAKE_CONFIGURATION=${CMAKE_CONFIGURATION} \
+		-DENL_ROOT=${ENL_ROOT} \
+		-DINCLUDE_ROOT=${INCLUDE_ROOT} \
+		-DPYTHONHOME=${PYTHONHOME} \
+		-DBOOST_PYTHON_ROOT=${BOOST_PYTHON_ROOT} \
+		${PYTHONEXTENSIONTEST_SRC_DIR}
+	cmake --build ${PYTHONEXTENSIONTEST_WORKING_DIR} --config ${CMAKE_CONFIGURATION} --target install
+
+	# Check the exit code of the compiler and exit appropriately so that build will fail.
+	#
+	check_exit_code "Success: Built pythonextension-test" "Error: Failed to build pythonextension-test"
+
+	# Move the generated libs to configuration folder
+	#
+	mv pythonextension-test ${CMAKE_CONFIGURATION}/
+
+	popd
+}
+
+# Enlistment root and location of pythonextension-test
+#
+SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+ENL_ROOT=${SCRIPTDIR}/../../../../..
+PYTHONEXTENSIONTEST_HOME=${ENL_ROOT}/language-extensions/python/test/
+
+# Set environment variables required in Cmake
+#
+PYTHONEXTENSIONTEST_SRC_DIR=${PYTHONEXTENSIONTEST_HOME}/src
+PYTHONEXTENSIONTEST_WORKING_DIR=${ENL_ROOT}/build-output/pythonextension-test/linux
+
+INCLUDE_ROOT=/usr/include
+
+DEFAULT_PYTHONHOME=/usr
+DEFAULT_BOOST_ROOT=/usr/lib/boost_1_69_0
+
+# Find PYTHONHOME from user, or set to default for tests.
+# Error code 1 is generic bash error.
+# 
+if [ -z "${PYTHONHOME}" ]; then
+	if [ -x "${DEFAULT_PYTHONHOME}" ]; then 
+		PYTHONHOME=${DEFAULT_PYTHONHOME}
+	else
+		echo "PYTHONHOME is empty but needs to be set to build the python extension test"
+		exit 1
+	fi
+fi
+
+# Find BOOST_ROOT from user, or set to default for tests.
+# Error code 1 is generic bash error.
+# 
+if [ -z "${BOOST_ROOT}" ]; then
+	if [ -x "${DEFAULT_BOOST_ROOT}" ]; then 
+		BOOST_ROOT=${DEFAULT_BOOST_ROOT}
+	else
+		echo "BOOST_ROOT is empty but needs to be set to build the python extension test"
+		exit 1
+	fi
+fi
+
+DEFAULT_BOOST_PYTHON_ROOT=${BOOST_ROOT}/stage/lib
+
+# Find BOOST_PYTHON_ROOT from user, or set to default for tests.
+# Error code 1 is generic bash error.
+# 
+if [ -z "${BOOST_PYTHON_ROOT}" ]; then
+	if [ -x "${DEFAULT_BOOST_PYTHON_ROOT}" ]; then 
+		BOOST_PYTHON_ROOT=${DEFAULT_BOOST_PYTHON_ROOT}
+	else
+		echo "BOOST_PYTHON_ROOT is empty but needs to be set to build the python extension test"
+		exit 1
+	fi
+fi
+
+# Build in release mode if nothing is specified
+#
+if [ "$1" == "" ]; then
+	set -- release
+fi
+
+while [ "$1" != "" ]; do
+	# Advance arg passed to build.cmd
+	#
+	build $1
+	shift
+done;

language-extensions/python/test/build/linux/run-pythonextension-test.sh

@@ -0,0 +1,69 @@
+#!/bin/bash
+
+function check_exit_code {
+	EXIT_CODE=$?
+	if [ ${EXIT_CODE} -eq 0 ]; then
+		echo $1
+	else
+		echo $2
+		exit ${EXIT_CODE}
+	fi
+}
+
+function build {
+	# Set cmake config to first arg
+	#
+	CMAKE_CONFIGURATION=$1
+	if [ -z "${CMAKE_CONFIGURATION}" ]; then
+		CMAKE_CONFIGURATION=release
+	fi
+
+	pushd ${PYTHONEXTENSIONTEST_WORKING_DIR}/${CMAKE_CONFIGURATION}
+	
+	# Move the generated libs to configuration folder
+	#
+	cp ${PYTHONEXTENSION_WORKING_DIR}/${CMAKE_CONFIGURATION}/libPythonExtension.so.1.0 .
+	cp /usr/src/gtest/*.so .
+	
+	ENL_ROOT=${ENL_ROOT} ./pythonextension-test --gtest_output=xml:${ENL_ROOT}/out/TestReport_PythonExtension-test.xml
+
+	# Check the exit code of the tests.
+	#
+	check_exit_code "Success: Ran pythonextension-test" "Error: pythonextension-test failed"
+
+	popd
+}
+
+# Enlistment root and location of pythonextension-test
+#
+SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+ENL_ROOT=${SCRIPTDIR}/../../../../..
+PYTHONEXTENSION_WORKING_DIR=${ENL_ROOT}/build-output/pythonextension/linux
+PYTHONEXTENSIONTEST_WORKING_DIR=${ENL_ROOT}/build-output/pythonextension-test/linux
+PACKAGES_ROOT=${ENL_ROOT}/packages
+DEFAULT_PYTHONHOME=/usr
+
+# Find PYTHONHOME from user, or set to default for tests.
+# Error code 1 is generic bash error.
+# 
+if [ -z "${PYTHONHOME}" ]; then
+	if [ -x "${DEFAULT_PYTHONHOME}" ]; then 
+		PYTHONHOME=${DEFAULT_PYTHONHOME}
+	else
+		echo "PYTHONHOME is empty"
+		exit 1
+	fi
+fi
+
+# Build in release mode if nothing is specified
+#
+if [ "$1" == "" ]; then
+	set -- release
+fi
+
+while [ "$1" != "" ]; do
+	# Advance arg passed to build.cmd
+	#
+	build $1
+	shift
+done;

language-extensions/python/test/build/windows/build-pythonextension-test.cmd

@@ -0,0 +1,135 @@
+@ECHO off
+SETLOCAL
+
+REM Set environment variables
+REM
+SET ENL_ROOT=%~dp0..\..\..\..\..
+SET PACKAGES_ROOT=%ENL_ROOT%\packages
+
+SET PYTHONEXTENSION_TEST_HOME=%ENL_ROOT%\language-extensions\python\test
+SET PYTHONEXTENSION_TEST_WORKING_DIR=%ENL_ROOT%\build-output\pythonextension-test\windows
+
+RMDIR /s /q %PYTHONEXTENSION_TEST_WORKING_DIR%
+MKDIR %PYTHONEXTENSION_TEST_WORKING_DIR%
+
+SET DEFAULT_BOOST_ROOT=%PACKAGES_ROOT%\External-Boost.master.Boost.1.69.0.1020
+SET DEFAULT_BOOST_PYTHON_ROOT=%DEFAULT_BOOST_ROOT%\windows\lib
+SET DEFAULT_PYTHONHOME=%PACKAGES_ROOT%\python
+SET DEFAULT_CMAKE_ROOT=%PACKAGES_ROOT%\CMake-win64.3.15.5
+
+REM Find boost, python, and cmake paths from user, or set to default for tests.
+REM 
+SET ENVVAR_NOT_FOUND=203
+
+IF "%BOOST_ROOT%" == "" (
+	IF EXIST %DEFAULT_BOOST_ROOT% (
+		SET BOOST_ROOT=%DEFAULT_BOOST_ROOT%
+	) ELSE (
+		CALL :CHECKERROR %ENVVAR_NOT_FOUND% "Error: BOOST_ROOT variable must be set to build the python extension test" || EXIT /b %ENVVAR_NOT_FOUND%
+	)
+)
+
+IF "%BOOST_PYTHON_ROOT%" == "" (
+	IF EXIST %DEFAULT_BOOST_PYTHON_ROOT% (
+		SET BOOST_PYTHON_ROOT=%DEFAULT_BOOST_PYTHON_ROOT%
+	) ELSE (
+		CALL :CHECKERROR %ENVVAR_NOT_FOUND% "Error: BOOST_PYTHON_ROOT variable must be set to build the python extension test" || EXIT /b %ENVVAR_NOT_FOUND%
+	)
+)
+
+IF "%PYTHONHOME%" == "" (
+	IF EXIST %DEFAULT_PYTHONHOME% (
+		SET PYTHONHOME=%DEFAULT_PYTHONHOME%
+	) ELSE (
+		CALL :CHECKERROR %ENVVAR_NOT_FOUND% "Error: PYTHONHOME variable must be set to build the python extension test" || EXIT /b %ENVVAR_NOT_FOUND%
+	)
+)
+
+IF "%CMAKE_ROOT%" == "" (
+	IF EXIST %DEFAULT_CMAKE_ROOT% (
+		SET CMAKE_ROOT=%DEFAULT_CMAKE_ROOT%
+	) ELSE (
+		CALL :CHECKERROR %ENVVAR_NOT_FOUND% "Error: CMAKE_ROOT variable must be set to build the python extension test" || EXIT /b %ENVVAR_NOT_FOUND%
+	)
+)
+
+:LOOP
+
+REM Set cmake config to first arg
+REM
+SET CMAKE_CONFIGURATION=%1
+
+REM *Setting CMAKE_CONFIGURATION to anything but "debug" will set CMAKE_CONFIGURATION to "release".
+REM The string comparison for CMAKE_CONFIGURATION is case-insensitive.
+REM
+IF NOT DEFINED CMAKE_CONFIGURATION (SET CMAKE_CONFIGURATION=release)
+IF /I %CMAKE_CONFIGURATION%==debug (SET CMAKE_CONFIGURATION=debug) ELSE (SET CMAKE_CONFIGURATION=release)
+
+REM VSCMD_START_DIR set the working directory to this variable after calling VsDevCmd.bat
+REM otherwise, it will default to %USERPROFILE%\Source
+REM
+SET VSCMD_START_DIR=%PYTHONEXTENSION_TEST_WORKING_DIR%
+
+REM Do not call VsDevCmd if the environment is already set. Otherwise, it will keep appending
+REM to the PATH environment variable and it will be too long for windows to handle.
+REM
+if not defined DevEnvDir (
+	call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\Tools\VsDevCmd.bat" -arch=amd64 -host_arch=amd64
+)
+
+ECHO "[INFO] Generating pythonextension test project build files using CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%"
+
+SET BUILD_OUTPUT=%PYTHONEXTENSION_TEST_WORKING_DIR%\%CMAKE_CONFIGURATION%
+MKDIR %BUILD_OUTPUT%
+PUSHD %BUILD_OUTPUT%
+
+REM Call cmake
+REM
+call "%CMAKE_ROOT%\bin\cmake.exe" ^
+	-G "Visual Studio 15 2017 Win64" ^
+	-DPLATFORM=windows ^
+	-DENL_ROOT="%ENL_ROOT%" ^
+	-DCMAKE_CONFIGURATION=%CMAKE_CONFIGURATION% ^
+	-DPYTHONHOME="%PYTHONHOME%" ^
+	-DBOOST_ROOT="%BOOST_ROOT%" ^
+	-DBOOST_PYTHON_ROOT="%BOOST_PYTHON_ROOT%" ^
+	 %PYTHONEXTENSION_TEST_HOME%\src
+
+CALL :CHECKERROR %ERRORLEVEL% "Error: Failed to generate make files for CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%" || EXIT /b %ERRORLEVEL%
+
+ECHO "[INFO] Building pythonextension test project using CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%"
+
+
+REM Call cmake build
+REM
+CALL "%CMAKE_ROOT%\bin\cmake.exe" --build . --config %CMAKE_CONFIGURATION% --target INSTALL
+
+CALL :CHECKERROR %ERRORLEVEL% "Error: Failed to build Python extension test for CMAKE_CONFIGURATION=%CMAKE_CONFIGURATION%" || EXIT /b %ERRORLEVEL%
+
+REM Advance arg passed to build-pythonextension-test.cmd
+REM
+SHIFT
+
+REM Continue building using more configs until argv has been exhausted
+REM
+IF NOT "%~1"=="" GOTO LOOP
+
+REM Save exit code of compiler
+REM
+SET EX=%ERRORLEVEL%
+
+:CLEANUP
+
+if "%EX%" neq "0" (
+	echo "Build failed"
+)
+
+EXIT /b %ERRORLEVEL%
+
+:CHECKERROR
+	IF %1 NEQ 0 (
+		ECHO %2
+		EXIT /b %1
+	)
+
+	EXIT /b 0

language-extensions/python/test/build/windows/run-pythonextension-test.cmd

@@ -0,0 +1,51 @@
+@ECHO off
+SETLOCAL
+
+REM Set environment variables
+REM
+SET ENL_ROOT=%~dp0..\..\..\..\..
+SET PACKAGES_ROOT=%ENL_ROOT%\packages
+SET PYTHONEXTENSION_TEST_WORKING_DIR=%ENL_ROOT%\build-output\pythonextension-test\windows
+SET PYTHONEXTENSION_WORKING_DIR=%ENL_ROOT%\build-output\pythonextension\windows
+SET GTEST_HOME=%PACKAGES_ROOT%\Microsoft.googletest.v140.windesktop.msvcstl.dyn.rt-dyn.1.8.1.3
+SET GTEST_LIB_PATH=%GTEST_HOME%\lib\native\v140\windesktop\msvcstl\dyn\rt-dyn\x64
+
+:LOOP
+
+REM Set cmake config to first arg
+REM
+SET CMAKE_CONFIGURATION=%1
+
+REM *Setting CMAKE_CONFIGURATION to anything but "debug" will set CMAKE_CONFIGURATION to "release".
+REM The string comparison for CMAKE_CONFIGURATION is case-insensitive.
+REM
+IF NOT DEFINED CMAKE_CONFIGURATION (SET CMAKE_CONFIGURATION=release)
+IF /I %CMAKE_CONFIGURATION%==debug (SET CMAKE_CONFIGURATION=debug) ELSE (SET CMAKE_CONFIGURATION=release)
+
+pushd %PYTHONEXTENSION_TEST_WORKING_DIR%\%CMAKE_CONFIGURATION%
+copy %PYTHONEXTENSION_WORKING_DIR%\%CMAKE_CONFIGURATION%\pythonextension.* .
+copy %GTEST_LIB_PATH%\%CMAKE_CONFIGURATION%\gtest* .
+
+IF "%PYTHONHOME%"=="" (SET PYTHONHOME=%PACKAGES_ROOT%\python)
+
+SET PATH=%PATH%;%PYTHONHOME%;
+
+pythonextension-test.exe --gtest_output=xml:%ENL_ROOT%\out\TestReport_PythonExtension-test.xml
+IF %ERRORLEVEL% NEQ 0 GOTO error
+popd
+
+REM Advance arg passed to build-pythonextension-test.cmd
+REM
+SHIFT
+
+REM Continue running using more configs until argv has been exhausted
+REM
+IF NOT "%~1"=="" GOTO LOOP
+
+:error
+EXIT /b %ERRORLEVEL%
+
+:USAGE
+echo.
+echo Usage:
+echo %0 { debug ^| release }

language-extensions/python/test/include/Common.h

@@ -0,0 +1,29 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: Common.h
+//
+// Purpose:
+//  Common headers for the python test project
+//
+//*************************************************************************************************
+#pragma once
+
+#ifdef _WIN64
+#include <windows.h>
+#endif
+
+#include <boost/python.hpp>
+#include <boost/python/numpy.hpp>
+
+#include <iostream>
+#include <string>
+#include <sql.h>
+#include <sqlext.h>
+#include <sqltypes.h>
+
+#include "gtest/gtest.h"
+#include "sqlexternallibrary.h"

language-extensions/python/test/include/PythonExtensionApiTests.h

@@ -0,0 +1,460 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonExtensionApiTests.h
+//
+// Purpose:
+//  Define common methods and variables needed to test the Extension API
+//
+//*************************************************************************************************
+#pragma once
+#include "Common.h"
+#include <unordered_map>
+
+namespace ExtensionApiTest
+{
+	// Forward declaration
+	//
+	template<class SQLType>
+	class ColumnInfo;
+
+	// All the tests in the PythonextensionApiTest suite run one after the other
+	//
+	class PythonExtensionApiTests : public ::testing::Test
+	{
+	protected:
+
+		// Code here will be called immediately after the constructor (right
+		// before each test).
+		//
+		void SetUp() override;
+
+		// Code here will be called immediately after each test (right
+		// before the destructor).
+		//
+		void TearDown() override;
+
+		// Initialize a valid session.
+		//
+		void InitializeSession(
+			SQLUSMALLINT parametersNumber = 0,
+			SQLUSMALLINT inputSchemaColumnsNumber = 0,
+			std::string  scriptString = "");
+
+		// Initialize a column
+		//
+		void InitializeColumn(
+			SQLSMALLINT columnNumber,
+			std::string columnNameString,
+			SQLSMALLINT dataType,
+			SQLULEN     columnSize);
+
+		// Get max length of all strings from strLenOrInd.
+		//
+		SQLINTEGER GetMaxLength(SQLINTEGER *strLenOrInd, SQLULEN rowsNumber);
+
+		// Get length of a wstring
+		//
+		SQLULEN GetWStringLength(const wchar_t *str);
+
+		template<class SQLType, SQLSMALLINT dataType>
+		void InitializeColumns(ColumnInfo<SQLType> *ColumnInfo);
+
+		// Set up default, valid variables for use in tests
+		//
+		void SetupVariables();
+
+		// Call Cleanup on the PythonExtension.
+		// Testing if Cleanup is implemented correctly.
+		//
+		void DoCleanup();
+
+		// Template to test all input parameter data types
+		//
+		template<class SQLType, SQLSMALLINT dataType>
+		void TestParameter(
+			int         paramNumber,
+			SQLType     paramValue,
+			bool        isNull = false,
+			SQLSMALLINT inputOutputType = SQL_PARAM_INPUT_OUTPUT,
+			bool        validate = true);
+
+		// Template to test date/datetime data types
+		//
+		template<class DateTimeStruct, SQLSMALLINT dataType>
+		void TestDateTimeParameter(
+			int            paramNumber,
+			DateTimeStruct paramValue,
+			bool           isNull = false,
+			SQLSMALLINT    inputOutputType = SQL_PARAM_INPUT_OUTPUT,
+			bool           validate = true);
+
+		// Test a string parameter
+		//
+		void TestStringParameter(
+			int         paramNumber,
+			const char  *paramValue,
+			SQLULEN     paramSize,
+			bool        isFixedType,
+			SQLSMALLINT inputOutputType = SQL_PARAM_INPUT_OUTPUT,
+			bool        validate = true);
+
+		// Test a wstring parameter
+		//
+		void TestWStringParameter(
+			int           paramNumber,
+			const wchar_t *paramValue,
+			SQLULEN       paramSize,
+			bool          isFixedType,
+			SQLSMALLINT   inputOutputType = SQL_PARAM_INPUT_OUTPUT,
+			bool          validate = true);
+
+		// Test a binary parameter
+		//
+		void TestRawParameter(
+			int           paramNumber,
+			const SQLCHAR *paramValue,
+			SQLULEN       paramSize,
+			bool          isFixedType,
+			SQLSMALLINT   inputOutputType = SQL_PARAM_INPUT_OUTPUT,
+			bool          validate = true);
+
+		// Fill a contiguous array columnData with members from the given columnVector
+		// having lengths defined in strLenOrInd, unless it is SQL_NULL_DATA.
+		//
+		template<class SQLType>
+		std::vector<SQLType> GenerateContiguousData(
+			std::vector<const SQLType*> columnVector,
+			SQLINTEGER                  *strLenOrInd);
+
+		// Template function to Test Execute with default script
+		// The "validate" parameter can be false to run the execution
+		// without validating DataSets to set up outputs.
+		//
+		template<class SQLType, SQLSMALLINT dataType>
+		void TestExecute(
+			SQLULEN                  rowsNumber,
+			void                     **dataSet,
+			SQLINTEGER               **strLen_or_Ind,
+			std::vector<std::string> columnNames,
+			bool                     validate = true);
+
+		// Template function to compare the given integer/float column for equality.
+		//
+		template<class SQLType>
+		void CheckColumnEquality(
+			SQLULEN             expectedRowsNumber,
+			boost::python::dict columnToTest,
+			void                *expectedColumn,
+			SQLINTEGER          *strLen_or_Ind);
+
+		// Compare a given boolean column with another for equality
+		//
+		void CheckBooleanColumnEquality(
+			SQLULEN             expectedRowsNumber,
+			boost::python::dict columnToTest,
+			void                *expectedColumn,
+			SQLINTEGER          *strLen_or_Ind);
+
+		// Compare a given string column with another for equality
+		//
+		void CheckStringColumnEquality(
+			SQLULEN             expectedRowsNumber,
+			boost::python::dict columnToTest,
+			void                *expectedColumn,
+			SQLINTEGER          *strLen_or_Ind);
+
+		// Compare a given wstring column with another for equality
+		//
+		void CheckWStringColumnEquality(
+			SQLULEN             expectedRowsNumber,
+			boost::python::dict columnToTest,
+			void                *expectedColumn,
+			SQLINTEGER          *strLen_or_Ind);
+
+		// Compare a given binary column with another for equality
+		//
+		void CheckRawColumnEquality(
+			SQLULEN             expectedRowsNumber,
+			boost::python::dict columnToTest,
+			void                *expectedColumn,
+			SQLINTEGER          *strLen_or_Ind);
+
+		// Compare a given datetime column with another for equality
+		//
+		template<class DateTimeStruct>
+		void CheckDateTimeColumnEquality(
+			SQLULEN             expectedRowsNumber,
+			boost::python::dict columnToTest,
+			void                *expectedColumn,
+			SQLINTEGER          *strLen_or_Ind);
+
+		// Test GetResultColumn to verify the expected result column information.
+		//
+		void TestGetResultColumn(
+			SQLUSMALLINT columnNumber,
+			SQLSMALLINT  expectedDataType,
+			SQLULEN      expectedColumnSize,
+			SQLSMALLINT  expectedDecimalDigits,
+			SQLSMALLINT  expectedNullable);
+
+		// Test GetResults to verify the expected results are obtained.
+		// For numeric, boolean and integer types.
+		//
+		template<class SQLType, class InputCType, SQLSMALLINT dataType>
+		void TestGetResults(
+			SQLULEN                  expectedRowsNumber,
+			SQLPOINTER               *expectedData,
+			SQLINTEGER               **expectedStrLen_or_Ind,
+			std::vector<std::string> columnNames);
+
+		// Test GetResults to verify the expected string results are obtained.
+		//
+		void TestGetStringResults(
+			SQLULEN                  expectedRowsNumber,
+			SQLPOINTER               *expectedData,
+			SQLINTEGER               **expectedStrLen_or_Ind,
+			std::vector<std::string> columnNames);
+
+		// Test GetResults to verify the expected raw results are obtained.
+		//
+		void TestGetRawResults(
+			SQLULEN                  expectedRowsNumber,
+			SQLPOINTER               *expectedData,
+			SQLINTEGER               **expectedStrLen_or_Ind,
+			std::vector<std::string> columnNames);
+
+		// Test GetResults to verify the expected datetime results are obtained.
+		//
+		template<class DateTimeStruct>
+		void TestGetDateTimeResults(
+			SQLULEN                  expectedRowsNumber,
+			SQLPOINTER               *expectedData,
+			SQLINTEGER               **expectedStrLen_or_Ind,
+			std::vector<std::string> columnNames);
+
+		// Template function to compare the given column data and nullMap for equality.
+		// For numeric, boolean and integer types.
+		//
+		template<class SQLType>
+		void CheckColumnDataEquality(
+			SQLULEN    rowsNumber,
+			SQLType    *expectedColumnData,
+			SQLType    *columnData,
+			SQLINTEGER *expectedColumnStrLenOrInd,
+			SQLINTEGER *columnStrLenOrInd);
+
+		// Template function to compare the given column data and nullMap for equality.
+		// For numeric, boolean and integer types, WITH NULLS.
+		// Because NULLS for float types are NAN, there are complications that need to be checked for.
+		//
+		template<class SQLType, class DefaultType>
+		void CheckColumnDataEqualityForNullable(
+			SQLULEN    rowsNumber,
+			SQLType    *expectedColumnData,
+			void       *pColumnData,
+			SQLINTEGER *expectedColumnStrLenOrInd,
+			SQLINTEGER *columnStrLenOrInd);
+
+		// Compare the given string column data and nullMap for equality.
+		//
+		void CheckStringDataEquality(
+			SQLULEN    rowsNumber,
+			char       *expectedColumnData,
+			char       *columnData,
+			SQLINTEGER *expectedColumnStrLenOrInd,
+			SQLINTEGER *columnStrLenOrInd);
+
+		// Compare the given raw column data and nullMap for equality.
+		//
+		void CheckRawDataEquality(
+			SQLULEN    rowsNumber,
+			SQLCHAR    *expectedColumnData,
+			SQLCHAR    *columnData,
+			SQLINTEGER *expectedColumnStrLenOrInd,
+			SQLINTEGER *columnStrLenOrInd);
+
+		// Compare the given datetime column data and nullMap for equality.
+		//
+		template<class DateTimeStruct>
+		void CheckDateTimeDataEquality(
+			SQLULEN        rowsNumber,
+			DateTimeStruct *expectedColumnData,
+			DateTimeStruct *columnData,
+			SQLINTEGER     *expectedColumnStrLenOrInd,
+			SQLINTEGER     *columnStrLenOrInd);
+
+		// Template function to test output param value and strLenOrInd is as expected.
+		//
+		template<class SQLType>
+		void TestGetOutputParam(
+			std::vector<SQLType*>   expectedParamValueVector,
+			std::vector<SQLINTEGER> expectedStrLenOrIndVector);
+
+		// Template function to test date/datetime output param value and strLenOrInd is as expected.
+		//
+		template<class DateTimeStruct>
+		void TestGetDateTimeOutputParam(
+			std::vector<DateTimeStruct *> expectedParamValueVector,
+			std::vector<SQLINTEGER>       expectedStrLenOrIndVector);
+
+		// Test output string param value and strLenOrInd is as expected.
+		//
+		void TestGetStringOutputParam(
+			std::vector<const char*> expectedParamValueVector,
+			std::vector<SQLINTEGER>  expectedStrLenOrIndVector);
+
+		// Test output string param value and strLenOrInd is as expected.
+		//
+		void TestGetWStringOutputParam(
+			std::vector<const wchar_t*> expectedParamValueVector,
+			std::vector<SQLINTEGER>     expectedStrLenOrIndVector);
+
+		// Test output raw param value and strLenOrInd is as expected.
+		//
+		void TestGetRawOutputParam(
+			std::vector<SQLCHAR*>   expectedParamValueVector,
+			std::vector<SQLINTEGER> expectedStrLenOrIndVector);
+
+		// Objects declared here can be used by all tests in the test suite.
+		//
+		SQLGUID *m_sessionId;
+		SQLUSMALLINT m_taskId;
+		SQLUSMALLINT m_numTasks;
+
+		SQLCHAR *m_script;
+		std::string m_scriptString;
+		SQLSMALLINT m_scriptLength;
+
+		SQLCHAR *m_inputDataName;
+		std::string m_inputDataNameString;
+		SQLSMALLINT m_inputDataNameLength;
+
+		SQLCHAR *m_outputDataName;
+		std::string m_outputDataNameString;
+		SQLSMALLINT m_outputDataNameLength;
+
+		const std::string m_printMessage = "Hello PythonExtension!";
+
+		// A value of 2'147'483'648
+		//
+		const SQLINTEGER m_MaxInt = (std::numeric_limits<SQLINTEGER>::max)();
+
+		// A value of -2'147'483'648
+		//
+		const SQLINTEGER m_MinInt = (std::numeric_limits<SQLINTEGER>::min)();
+
+		// A value of 9'223'372'036'854'775'807LL
+		//
+		const SQLBIGINT m_MaxBigInt = (std::numeric_limits<SQLBIGINT>::max)();
+
+		// A value of -9'223'372'036'854'775'808LL
+		//
+		const SQLBIGINT m_MinBigInt = (std::numeric_limits<SQLBIGINT>::min)();
+
+		// A value of 32'767
+		//
+		const SQLSMALLINT m_MaxSmallInt = (std::numeric_limits<SQLSMALLINT>::max)();
+
+		// A value of -32'768
+		//
+		const SQLSMALLINT m_MinSmallInt = (std::numeric_limits<SQLSMALLINT>::min)();
+
+		// A value of 255
+		//
+		const SQLCHAR m_MaxTinyInt = (std::numeric_limits<SQLCHAR>::max)();
+
+		// A value of 0
+		//
+		const SQLCHAR m_MinTinyInt = (std::numeric_limits<SQLCHAR>::min)();
+
+		// For floating types, not using numeric_limits because they can't be
+		// used for equality comparisons.
+		//
+		const SQLREAL m_MaxReal = 3.4e38F;
+		const SQLREAL m_MinReal = -3.4e38F;
+		const SQLDOUBLE m_MaxDouble = 1.79e308;
+		const SQLDOUBLE m_MinDouble = -1.79e308;
+
+		const SQLINTEGER m_IntSize = sizeof(SQLINTEGER);
+		const SQLINTEGER m_BooleanSize = sizeof(SQLCHAR);
+		const SQLINTEGER m_RealSize = sizeof(SQLREAL);
+		const SQLINTEGER m_DoubleSize = sizeof(SQLDOUBLE);
+		const SQLINTEGER m_BigIntSize = sizeof(SQLBIGINT);
+		const SQLINTEGER m_SmallIntSize = sizeof(SQLSMALLINT);
+		const SQLINTEGER m_TinyIntSize = sizeof(SQLCHAR);
+		const SQLINTEGER m_CharSize = sizeof(SQLCHAR);
+		const SQLINTEGER m_WCharSize = sizeof(wchar_t);
+		const SQLINTEGER m_BinarySize = sizeof(SQLCHAR);
+		const SQLINTEGER m_DateTimeSize = sizeof(SQL_TIMESTAMP_STRUCT);
+		const SQLINTEGER m_DateSize = sizeof(SQL_DATE_STRUCT);
+
+		std::unique_ptr<ColumnInfo<SQLINTEGER>> m_integerInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQLCHAR>> m_booleanInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQLREAL>> m_realInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQLDOUBLE>> m_doubleInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQLBIGINT>> m_bigIntInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQLSMALLINT>> m_smallIntInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQLCHAR>> m_tinyIntInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQLCHAR>> m_charInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQL_TIMESTAMP_STRUCT>> m_dateTimeInfo = nullptr;
+		std::unique_ptr<ColumnInfo<SQL_DATE_STRUCT>> m_dateInfo = nullptr;
+
+		const float m_floatNull = NAN;
+		const int m_intNull = 0;
+		const bool m_boolNull = false;
+
+		// The boost python module; python will run in this object
+		//
+		boost::python::object m_mainModule;
+
+		// The boost python namespace; dictionary containing all python variables
+		//
+		boost::python::object m_mainNamespace;
+
+		// Check column equality function pointer definition
+		//
+		using fnCheckColumnEquality = void (PythonExtensionApiTests::*)(
+			SQLULEN             expectedRowsNumber,
+			boost::python::dict columnToTest,
+			void                *expectedColumn,
+			SQLINTEGER          *strLen_or_Ind);
+
+		// Function map to add columns to the data frame and its typedef
+		//
+		static const std::unordered_map<SQLSMALLINT, fnCheckColumnEquality> sm_FnCheckColumnEqualityMap;
+		typedef std::unordered_map<SQLSMALLINT, fnCheckColumnEquality> CheckColumnEqualityFnMap;
+	};
+
+	// ColumnInfo template class to store information
+	// about integer, basic numeric and boolean columns.
+	// This assumes two columns and five rows.
+	//
+	template<class SQLType>
+	class ColumnInfo
+	{
+	public:
+		ColumnInfo(
+			std::string column1Name, std::vector<SQLType> column1,
+			std::vector<SQLINTEGER> col1StrLenOrInd,
+			std::string column2Name, std::vector<SQLType> column2,
+			std::vector<SQLINTEGER> col2StrLenOrInd);
+
+		SQLUSMALLINT GetColumnsNumber() const
+		{
+			return m_columnNames.size();
+		}
+
+		static const SQLULEN sm_rowsNumber = 5;
+		std::vector<std::string> m_columnNames;
+		std::vector<SQLType> m_column1;
+		std::vector<SQLType> m_column2;
+		std::vector<void*> m_dataSet;
+		std::vector<SQLINTEGER> m_col1StrLenOrInd;
+		std::vector<SQLINTEGER> m_col2StrLenOrInd;
+		std::vector<SQLINTEGER*> m_strLen_or_Ind;
+	};
+}

language-extensions/python/test/include/PythonLibraryTests.h

@@ -0,0 +1,96 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonLibraryTests.h
+//
+// Purpose:
+//  Define the gtest overrides and the External Library tests
+//
+//*************************************************************************************************
+#pragma once
+#include "Common.h"
+#include <experimental/filesystem>
+namespace fs = std::experimental::filesystem;
+
+namespace LibraryApiTests
+{
+	// All the tests in the LibraryApiTests suite run one after the other
+	//
+	class ExternalLibraryApiTests : public ::testing::Test
+	{
+	protected:
+		// Code here will be called immediately after the constructor (right
+		// before each test).
+		//
+		void SetUp() override;
+
+		// Code here will be called immediately after each test (right
+		// before the destructor).
+		//
+		void TearDown() override;
+
+		// Set up default, valid variables for use in tests
+		//
+		void SetupVariables();
+
+		// Call Cleanup on the PythonExtension.
+		// Testing if Cleanup is implemented correctly.
+		//
+		void DoCleanup();
+
+		// Cleanup sys.modules in python to remove import traces
+		//
+		void CleanModules(std::string extLibName, std::string moduleName);
+
+		// Install and test a package
+		//
+		void InstallAndTest(
+			std::string extLibName,
+			std::string moduleName,
+			std::string pathToPackage,
+			std::string installDir,
+			std::string expectedVersion,
+			std::string expectedLocation = "",
+			bool successfulInstall = true,
+			bool successfulImport = true);
+
+		// Uninstall and test a package
+		//
+		void UninstallAndTest(
+			std::string extLibName,
+			std::string moduleName,
+			std::string installDir,
+			bool otherInstallationExists = false);
+
+		// Initialize pythonextension for library management
+		//
+		void Initialize();
+
+		// Filesystem path to the packages we will install
+		//
+		fs::path m_packagesPath;
+
+		// Some temp paths for public and private libraries
+		//
+		std::string m_libraryPath = "testInstallPkgs";
+		std::string m_publicLibraryPath;
+		std::string m_privateLibraryPath;	
+		std::string m_pathToPython;
+
+		// The boost python module; python will run in this object
+		//
+		boost::python::object m_mainModule;
+
+		// The boost python namespace; backup dictionary containing all builtins
+		//
+		boost::python::dict m_backupNamespace;
+
+		// The main namespace. We use this for all execution
+		//
+		boost::python::dict m_mainNamespace;
+
+	};
+}

language-extensions/python/test/include/PythonTestUtilities.h

@@ -0,0 +1,30 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonTestUtilities.h
+//
+// Purpose:
+//  Utility functions for the tests
+//
+//*************************************************************************************************
+
+#pragma once
+#include "Common.h"
+
+class PythonTestUtilities
+{
+public:
+
+	// Parses the value of the active python exception
+	// Type, value, and traceback are in separate pointers
+	//
+	static std::string ParsePythonException();
+
+	// Extract the string from a boost::python object
+	//
+	static std::string ExtractString(PyObject * pObj);
+	static std::string ExtractString(boost::python::object handle);
+};

language-extensions/python/test/src/CMakeLists.txt

@@ -0,0 +1,129 @@
+cmake_minimum_required (VERSION 3.5)
+
+# this is what the final executable is going to be named
+project(pythonextension-test VERSION 1.0 LANGUAGES CXX)
+
+# All string comparisons are CASE SENSITIVE in CMAKE. Make all strings lower before comparisons!
+#
+string(TOLOWER ${CMAKE_CONFIGURATION} CMAKE_CONFIGURATION)
+
+file(TO_CMAKE_PATH ${ENL_ROOT}/extension-host EXTENSION_API_HOME)
+file(TO_CMAKE_PATH ${ENL_ROOT}/language-extensions/python PYTHONEXTENSION_HOME)
+file(TO_CMAKE_PATH ${ENL_ROOT}/language-extensions/python/test PYTHONEXTENSION_TEST_HOME)
+file(TO_CMAKE_PATH ${PYTHONEXTENSION_TEST_HOME}/src PYTHONEXTENSION_TEST_SRC_DIR)
+
+file(TO_CMAKE_PATH ${ENL_ROOT}/build-output/pythonextension/${PLATFORM} PYTHONEXTENSION_WORKING_DIR)
+file(TO_CMAKE_PATH ${ENL_ROOT}/build-output/pythonextension-test/${PLATFORM} PYTHONEXTENSION_TEST_WORKING_DIR)
+file(TO_CMAKE_PATH ${PYTHONEXTENSION_TEST_WORKING_DIR}/${CMAKE_CONFIGURATION} PYTHONEXTENSION_TEST_INSTALL_DIR)
+
+find_library(PYTHONEXTENSION_LIB PythonExtension ${PYTHONEXTENSION_WORKING_DIR}/${CMAKE_CONFIGURATION})
+
+# C++ unit tests code; only these files are compiled-in
+#
+file(GLOB PYTHONEXTENSION_TEST_SOURCE_FILES ${PYTHONEXTENSION_TEST_SRC_DIR}/*.cpp ${PYTHONEXTENSION_TEST_SRC_DIR}/${PLATFORM}/*.cpp)
+
+add_executable(pythonextension-test
+  ${PYTHONEXTENSION_TEST_SOURCE_FILES}
+)
+
+if (${PLATFORM} STREQUAL linux)	
+	target_compile_options(pythonextension-test PRIVATE -Wall -Wextra -g -O2 -fPIC -Werror -std=c++17 -Wno-unused-parameter -fshort-wchar)
+	set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-rpath,'$ORIGIN:${CMAKE_INSTALL_PREFIX}' -Wl,--no-as-needed -Wl,--export-dynamic")
+	
+	set(USR_LIB_PATH /usr/local/lib)
+	set(PYTHON_LIB_PATH ${PYTHONHOME}/lib/python3.7/config-3.7m-x86_64-linux-gnu)
+	
+	find_library(PYTHON_LIB python3.7 ${PYTHON_LIB_PATH})
+	find_library(BOOST_PYTHON_LIB boost_python37 ${BOOST_PYTHON_ROOT})
+	find_library(BOOST_NUMPY_LIB boost_numpy37 ${BOOST_PYTHON_ROOT})
+
+	file(TO_CMAKE_PATH ${INCLUDE_ROOT}/python3.7 PYTHON_INCLUDE)
+	file(TO_CMAKE_PATH ${INCLUDE_ROOT}/boost BOOST_INCLUDE)
+	
+	set(ADDITIONAL_INCLUDES ${PYTHON_INCLUDE} ${BOOST_INCLUDE})
+
+	find_library(DL dl ${USR_LIB_PATH})
+	
+	set(THREADS_PREFER_PTHREAD_FLAG ON)
+	find_package(Threads REQUIRED)
+	
+	target_link_libraries(pythonextension-test
+		${DL}
+		Threads::Threads
+		stdc++fs
+	)
+
+	file(TO_CMAKE_PATH ${ENL_ROOT}/build-output/googletest/${PLATFORM} GTEST_HOME)
+	file(TO_CMAKE_PATH ${GTEST_HOME}/googletest-src/googletest/include GTEST_INCLUDE_DIR)
+	file(TO_CMAKE_PATH ${GTEST_HOME}/lib/libgtest.a GTEST_LIB)
+	
+elseif(${PLATFORM} STREQUAL windows)
+	add_definitions(-DWIN_EXPORT -D_WIN64 -D_WINDOWS -D_SILENCE_TR1_NAMESPACE_DEPRECATION_WARNING)
+	set(COMPILE_OPTIONS /std:c++17)
+	
+	find_library(PYTHON_LIB python37 ${PYTHONHOME}/libs)
+	
+
+	file(TO_CMAKE_PATH ${ENL_ROOT}/packages/Microsoft.googletest.v140.windesktop.msvcstl.dyn.rt-dyn.1.8.1.3 GTEST_HOME)
+	file(TO_CMAKE_PATH ${GTEST_HOME}/build/native/include GTEST_INCLUDE_DIR)
+	file(TO_CMAKE_PATH ${GTEST_HOME}/lib/native/v140/windesktop/msvcstl/dyn/rt-dyn/x64/${CMAKE_CONFIGURATION} GTEST_LIB_PATH)
+
+	# MDd is for debug DLL and MD is for release DLL
+	#
+	if (${CMAKE_CONFIGURATION} STREQUAL debug)
+		set(COMPILE_OPTIONS ${COMPILE_OPTIONS} /MDd)
+		find_library(BOOST_PYTHON_LIB libboost_python37-vc140-mt-gd-x64-1_69 ${BOOST_PYTHON_ROOT})
+		find_library(BOOST_NUMPY_LIB libboost_numpy37-vc140-mt-gd-x64-1_69 ${BOOST_PYTHON_ROOT})
+		find_library(GTEST_LIB gtestd ${GTEST_LIB_PATH})
+	else()
+		set(COMPILE_OPTIONS ${COMPILE_OPTIONS} /MD)
+		find_library(BOOST_PYTHON_LIB libboost_python37-vc140-mt-x64-1_69 ${BOOST_PYTHON_ROOT})
+		find_library(BOOST_NUMPY_LIB libboost_numpy37-vc140-mt-x64-1_69 ${BOOST_PYTHON_ROOT})
+		find_library(GTEST_LIB gtest ${GTEST_LIB_PATH})
+	endif()
+	
+	target_compile_options(pythonextension-test PRIVATE ${COMPILE_OPTIONS})
+
+	# Set the DLLEXPORT variable to export symbols
+	#
+	target_compile_definitions(pythonextension-test PRIVATE ${COMPILE_OPTIONS})
+
+	file(TO_CMAKE_PATH ${PYTHONHOME}/include PYTHON_INCLUDE)
+	file(TO_CMAKE_PATH ${BOOST_ROOT}/include BOOST_INCLUDE)
+
+	set(ADDITIONAL_INCLUDES ${PYTHON_INCLUDE} ${BOOST_INCLUDE})
+endif()
+
+add_definitions(-DBOOST_USE_STATIC_LIBS -DBOOST_PYTHON_STATIC_LIB -DBOOST_ALL_NO_LIB -DBOOST_NUMPY_STATIC_LIB)
+
+# this is not a standard include path so test projects need
+# to add it explicitly
+#
+include_directories(
+	"${GTEST_INCLUDE_DIR}"
+	"${EXTENSION_API_HOME}/include"
+	"${PYTHONEXTENSION_HOME}/include"
+	"${PYTHONEXTENSION_TEST_HOME}/include"
+)
+
+# The string comparison is case sensitive
+#
+if (${CMAKE_CONFIGURATION} STREQUAL debug)
+	add_definitions(-D_DEBUG)
+else()
+	add_definitions(-DNDEBUG)
+endif()
+
+target_link_libraries(pythonextension-test
+	${GTEST_LIB}
+	${PYTHONEXTENSION_LIB}
+	${PYTHON_LIB} 
+	${BOOST_PYTHON_LIB}
+	${BOOST_NUMPY_LIB}
+)
+
+target_include_directories(pythonextension-test
+	PRIVATE ${ADDITIONAL_INCLUDES} 
+)
+
+install(TARGETS pythonextension-test DESTINATION ${PYTHONEXTENSION_TEST_INSTALL_DIR})

language-extensions/python/test/src/PythonExecuteTests.cpp

@@ -0,0 +1,609 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonExecuteTests.cpp
+//
+// Purpose:
+//  Tests the PythonExtension's implementation of the external language Execute API.
+//
+//*************************************************************************************************
+
+#include <sstream>
+
+#include "PythonExtensionApiTests.h"
+#include "PythonTestUtilities.h"
+
+using namespace std;
+namespace bp = boost::python;
+
+namespace ExtensionApiTest
+{
+	// Name: ExecuteIntegerColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of Integer columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteIntegerColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_integerInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLINTEGER, SQL_C_SLONG>(m_integerInfo.get());
+
+		TestExecute<SQLINTEGER, SQL_C_SLONG>(
+			ColumnInfo<SQLINTEGER>::sm_rowsNumber,
+			(*m_integerInfo).m_dataSet.data(),
+			(*m_integerInfo).m_strLen_or_Ind.data(),
+			(*m_integerInfo).m_columnNames);
+	}
+
+	// Name: ExecuteBooleanColumnsTest
+	//
+	// Description:
+	//  Test Execute using an InputDataSet of Boolean columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteBooleanColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_booleanInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLCHAR, SQL_C_BIT>(m_booleanInfo.get());
+
+		TestExecute<SQLCHAR, SQL_C_BIT>(
+			ColumnInfo<SQLCHAR>::sm_rowsNumber,
+			(*m_booleanInfo).m_dataSet.data(),
+			(*m_booleanInfo).m_strLen_or_Ind.data(),
+			(*m_booleanInfo).m_columnNames);
+	}
+
+	// Name: ExecuteRealColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of Real columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteRealColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_realInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLREAL, SQL_C_FLOAT>(m_realInfo.get());
+
+		TestExecute<SQLREAL, SQL_C_FLOAT>(
+			ColumnInfo<SQLREAL>::sm_rowsNumber,
+			(*m_realInfo).m_dataSet.data(),
+			(*m_realInfo).m_strLen_or_Ind.data(),
+			(*m_realInfo).m_columnNames);
+	}
+
+	// Name: ExecuteDoubleColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of Double columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteDoubleColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_doubleInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLDOUBLE, SQL_C_DOUBLE>(m_doubleInfo.get());
+
+		TestExecute<SQLDOUBLE, SQL_C_DOUBLE>(
+			ColumnInfo<SQLDOUBLE>::sm_rowsNumber,
+			(*m_doubleInfo).m_dataSet.data(),
+			(*m_doubleInfo).m_strLen_or_Ind.data(),
+			(*m_doubleInfo).m_columnNames);
+	}
+
+	// Name: ExecuteBigIntColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of BigInteger columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteBigIntColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_bigIntInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLBIGINT, SQL_C_SBIGINT>(m_bigIntInfo.get());
+
+		TestExecute<SQLBIGINT, SQL_C_SBIGINT>(
+			ColumnInfo<SQLBIGINT>::sm_rowsNumber,
+			(*m_bigIntInfo).m_dataSet.data(),
+			(*m_bigIntInfo).m_strLen_or_Ind.data(),
+			(*m_bigIntInfo).m_columnNames);
+	}
+
+	// Name: ExecuteSmallIntColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of SmallInt columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteSmallIntColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_smallIntInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLSMALLINT, SQL_C_SSHORT>(m_smallIntInfo.get());
+
+		TestExecute<SQLSMALLINT, SQL_C_SSHORT>(
+			ColumnInfo<SQLSMALLINT>::sm_rowsNumber,
+			(*m_smallIntInfo).m_dataSet.data(),
+			(*m_smallIntInfo).m_strLen_or_Ind.data(),
+			(*m_smallIntInfo).m_columnNames);
+	}
+
+	// Name: ExecuteTinyIntColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of TinyInt columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteTinyIntColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_tinyIntInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLCHAR, SQL_C_UTINYINT>(m_tinyIntInfo.get());
+
+		TestExecute<SQLCHAR, SQL_C_UTINYINT>(
+			ColumnInfo<SQLCHAR>::sm_rowsNumber,
+			(*m_tinyIntInfo).m_dataSet.data(),
+			(*m_tinyIntInfo).m_strLen_or_Ind.data(),
+			(*m_tinyIntInfo).m_columnNames);
+	}
+
+	// Name: ExecuteStringColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of string columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteStringColumnsTest)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = 3;
+
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			inputSchemaColumnsNumber,
+			m_scriptString);
+
+		string stringColumn1Name = "StringColumn1";
+		InitializeColumn(0, stringColumn1Name, SQL_C_CHAR, m_CharSize);
+
+		string stringColumn2Name = "StringColumn2";
+		InitializeColumn(1, stringColumn2Name, SQL_C_CHAR, m_CharSize);
+
+		string stringColumn3Name = "StringColumn3";
+		InitializeColumn(2, stringColumn3Name, SQL_C_CHAR, m_CharSize);
+
+		vector<const char*> stringCol1{ "Hello", "test", "data", "World", "-123" };
+		vector<const char*> stringCol2{ "", 0, nullptr, "verify", "-1" };
+
+		vector<SQLINTEGER> strLenOrIndCol1 =
+		{ static_cast<SQLINTEGER>(strlen(stringCol1[0])),
+		  static_cast<SQLINTEGER>(strlen(stringCol1[1])),
+		  static_cast<SQLINTEGER>(strlen(stringCol1[2])),
+		  static_cast<SQLINTEGER>(strlen(stringCol1[3])),
+		  static_cast<SQLINTEGER>(strlen(stringCol1[4])) };
+		vector<SQLINTEGER> strLenOrIndCol2 =
+		{ 0, SQL_NULL_DATA, SQL_NULL_DATA,
+		  static_cast<SQLINTEGER>(strlen(stringCol2[3])),
+		  static_cast<SQLINTEGER>(strlen(stringCol2[4])) };
+
+		vector<SQLINTEGER*> strLen_or_Ind{ strLenOrIndCol1.data(),
+			strLenOrIndCol2.data(), nullptr };
+
+		// Coalesce the arrays of each row of each column
+		// into a contiguous array for each column.
+		//
+		vector<char> stringCol1Data = GenerateContiguousData<char>(stringCol1, strLenOrIndCol1.data());
+		vector<char> stringCol2Data = GenerateContiguousData<char>(stringCol2, strLenOrIndCol2.data());
+
+		void* dataSet[] = { stringCol1Data.data(),
+							stringCol2Data.data(),
+							nullptr };
+
+		int rowsNumber = stringCol1.size();
+
+		vector<string> columnNames{ stringColumn1Name, stringColumn2Name, stringColumn3Name };
+
+		TestExecute<SQLCHAR, SQL_C_CHAR>(
+			rowsNumber,
+			dataSet,
+			strLen_or_Ind.data(),
+			columnNames);
+	}
+
+	// Name: ExecuteWStringColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of wstring columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteWStringColumnsTest)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = 3;
+
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			inputSchemaColumnsNumber,
+			m_scriptString);
+
+		string wstringColumn1Name = "WStringColumn1";
+		InitializeColumn(0, wstringColumn1Name, SQL_C_WCHAR, m_WCharSize);
+
+		string wstringColumn2Name = "WStringColumn2";
+		InitializeColumn(1, wstringColumn2Name, SQL_C_WCHAR, m_WCharSize);
+
+		string wstringColumn3Name = "WStringColumn3";
+		InitializeColumn(2, wstringColumn3Name, SQL_C_WCHAR, m_WCharSize);
+
+		vector<const wchar_t*> wstringCol1{ L"Hello", L"test", L"data", L"World", L"你好" };
+		vector<const wchar_t*> wstringCol2{ L"", 0, nullptr, L"verify", L"-1" };
+
+		vector<SQLINTEGER> strLenOrIndCol1 =
+		{ static_cast<SQLINTEGER>(5 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(4 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(4 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(5 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(2 * sizeof(wchar_t)) };
+		vector<SQLINTEGER> strLenOrIndCol2 =
+		{ 0, SQL_NULL_DATA, SQL_NULL_DATA,
+		  static_cast<SQLINTEGER>(6 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(2 * sizeof(wchar_t)) };
+
+		vector<SQLINTEGER*> strLen_or_Ind{ strLenOrIndCol1.data(),
+			strLenOrIndCol2.data(), nullptr };
+
+		// Coalesce the arrays of each row of each column
+		// into a contiguous array for each column.
+		//
+		vector<wchar_t> wstringCol1Data = GenerateContiguousData<wchar_t>(wstringCol1, strLenOrIndCol1.data());
+		vector<wchar_t> wstringCol2Data = GenerateContiguousData<wchar_t>(wstringCol2, strLenOrIndCol2.data());
+
+		void* dataSet[] = { wstringCol1Data.data(),
+							wstringCol2Data.data(),
+							nullptr };
+
+		int rowsNumber = wstringCol1.size();
+
+		vector<string> columnNames{ wstringColumn1Name, wstringColumn2Name, wstringColumn3Name };
+
+		TestExecute<wchar_t, SQL_C_WCHAR>(
+			rowsNumber,
+			dataSet,
+			strLen_or_Ind.data(),
+			columnNames);
+	}
+
+	// Name: ExecuteRawColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of binary columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteRawColumnsTest)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = 3;
+
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			inputSchemaColumnsNumber,
+			m_scriptString);
+
+		const SQLCHAR BinaryValue1[] = { 0x01, 0x01, 0xe2, 0x40 };
+		const SQLCHAR BinaryValue2[] = { 0x04, 0x05, 0xe1 };
+		const SQLCHAR BinaryValue3[] = { 0x00, 0x00, 0x00, 0x01 };
+		const SQLCHAR BinaryValue4[] = { 0xff };
+
+		const SQLCHAR BinaryValue5[] = { 0x00 };
+		const SQLCHAR BinaryValue6[] = { 0xff, 0xff, 0xff, 0xff };
+		const SQLCHAR BinaryValue7[] = { 0x00, 0x12, 0xd2, 0xff, 0x00, 0x12, 0xd2, 0xff, 0x00, 0x12, 0xd2, 0xff };
+
+		string binaryColumn1Name = "BinaryColumn1";
+		InitializeColumn(0, binaryColumn1Name, SQL_C_BINARY, m_BinarySize);
+
+		string binaryColumn2Name = "BinaryColumn2";
+		InitializeColumn(1, binaryColumn2Name, SQL_C_BINARY, m_BinarySize);
+
+		string binaryColumn3Name = "BinaryColumn3";
+		InitializeColumn(2, binaryColumn3Name, SQL_C_BINARY, m_BinarySize);
+
+		vector<const SQLCHAR*> binaryCol1{ BinaryValue1, BinaryValue2, BinaryValue3, BinaryValue4 };
+		vector<const SQLCHAR*> binaryCol2{ BinaryValue5, BinaryValue6, nullptr, BinaryValue7};
+
+		SQLINTEGER strLenOrIndCol1[] =
+		{
+			static_cast<SQLINTEGER>(sizeof(BinaryValue1) / m_BinarySize),
+			static_cast<SQLINTEGER>(sizeof(BinaryValue2) / m_BinarySize),
+			static_cast<SQLINTEGER>(sizeof(BinaryValue3) / m_BinarySize),
+			static_cast<SQLINTEGER>(sizeof(BinaryValue4) / m_BinarySize)
+		};
+
+		SQLINTEGER strLenOrIndCol2[] =
+		{
+			SQL_NULL_DATA,
+			static_cast<SQLINTEGER>(sizeof(BinaryValue6) / m_BinarySize),
+			SQL_NULL_DATA,
+			static_cast<SQLINTEGER>(sizeof(BinaryValue7) / m_BinarySize)
+		};
+
+		vector<SQLINTEGER*> strLen_or_Ind{ strLenOrIndCol1, strLenOrIndCol2, nullptr };
+
+		// Coalesce the arrays of each row of each column
+		// into a contiguous array for each column.
+		//
+		int rowsNumber = binaryCol1.size();
+
+		vector<SQLCHAR> binaryCol1Data = GenerateContiguousData<SQLCHAR>(binaryCol1, strLenOrIndCol1);
+		vector<SQLCHAR> binaryCol2Data = GenerateContiguousData<SQLCHAR>(binaryCol2, strLenOrIndCol2);
+
+		void* dataSet[] = { binaryCol1Data.data(),
+							binaryCol2Data.data(),
+							nullptr };
+
+		vector<string> columnNames{ binaryColumn1Name, binaryColumn2Name, binaryColumn3Name };
+
+		TestExecute<SQLCHAR, SQL_C_BINARY>(
+			rowsNumber,
+			dataSet,
+			strLen_or_Ind.data(),
+			columnNames);
+	}
+
+	// Name: ExecuteDifferentColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of different column types.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteDifferentColumnsTest)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = 3;
+
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			inputSchemaColumnsNumber,
+			m_scriptString);
+
+		string integerColumnName = "IntegerColumn";
+		InitializeColumn(0, integerColumnName, SQL_C_SLONG, m_IntSize);
+
+		string doubleColumnName = "DoubleColumn";
+		InitializeColumn(1, doubleColumnName, SQL_C_DOUBLE, m_DoubleSize);
+
+		string stringColumnName = "StringColumn";
+		InitializeColumn(2, stringColumnName, SQL_C_CHAR, m_CharSize);
+
+		vector<SQLINTEGER> intColData{ m_MaxInt, m_MinInt, 0, 1320, -1 };
+		vector<SQLDOUBLE> doubleColData{ m_MinDouble, 1.33, 83.98, 72.45, m_MaxDouble };
+		vector<const char*> stringCol{ "Hello", "test", "data", "World", "-123" };
+
+		SQLINTEGER strLenOrIndCol1[] = { 0, 0, SQL_NULL_DATA, SQL_NULL_DATA, 0 };
+		SQLINTEGER strLenOrIndCol3[] =
+		{ static_cast<SQLINTEGER>(strlen(stringCol[0])),
+		static_cast<SQLINTEGER>(strlen(stringCol[1])),
+		static_cast<SQLINTEGER>(strlen(stringCol[2])),
+		static_cast<SQLINTEGER>(strlen(stringCol[3])),
+		static_cast<SQLINTEGER>(strlen(stringCol[4])) };
+		vector<SQLINTEGER*> strLen_or_Ind{ strLenOrIndCol1, nullptr, strLenOrIndCol3 };
+
+		int rowsNumber = intColData.size();
+
+		vector<char> stringColData = GenerateContiguousData<char>(stringCol, strLenOrIndCol3);
+
+		vector<void *> dataSet{ intColData.data(), doubleColData.data(), stringColData.data() };
+
+		testing::internal::CaptureStdout();
+
+		SQLUSMALLINT outputschemaColumnsNumber = 0;
+		SQLRETURN result = Execute(
+			*m_sessionId,
+			m_taskId,
+			rowsNumber,
+			dataSet.data(),
+			strLen_or_Ind.data(),
+			&outputschemaColumnsNumber);
+		ASSERT_EQ(result, SQL_SUCCESS);
+
+		// Test print message was printed correctly
+		//
+		string output = testing::internal::GetCapturedStdout();
+		cout << output;
+		ASSERT_TRUE(output.find(m_printMessage) != string::npos);
+
+		try {
+			string createDictScript = m_inputDataNameString + ".to_dict()";
+			bp::dict inputDataSet = bp::extract<bp::dict>(bp::eval(createDictScript.c_str(), m_mainNamespace));
+
+			createDictScript = m_outputDataNameString + ".to_dict()";
+			bp::dict outputDataSet = bp::extract<bp::dict>(bp::eval(createDictScript.c_str(), m_mainNamespace));
+
+			for(bp::dict ds : {inputDataSet, outputDataSet})
+			{
+				bp::dict intColumn = bp::extract<bp::dict>(ds.get(integerColumnName));
+				CheckColumnEquality<SQLINTEGER>(
+					rowsNumber,
+					intColumn,
+					dataSet[0],
+					strLen_or_Ind[0]);
+
+				bp::dict numericColumn = bp::extract<bp::dict>(ds.get(doubleColumnName));
+				CheckColumnEquality<SQLDOUBLE>(
+					rowsNumber,
+					numericColumn,
+					dataSet[1],
+					strLen_or_Ind[1]);
+
+				bp::dict stringColumn = bp::extract<bp::dict>(ds.get(stringColumnName));
+				CheckStringColumnEquality(
+					rowsNumber,
+					stringColumn,
+					dataSet[2],
+					strLen_or_Ind[2]);
+			}
+		}
+		catch (bp::error_already_set &)
+		{
+			string pyError = PythonTestUtilities::ParsePythonException();
+			throw runtime_error("Error running python:\n" + pyError);
+		}
+	}
+
+	// Name: ExecuteDateTimeColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of DateTime columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteDateTimeColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_dateTimeInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQL_TIMESTAMP_STRUCT, SQL_C_TYPE_TIMESTAMP>(m_dateTimeInfo.get());
+
+		TestExecute<SQL_TIMESTAMP_STRUCT, SQL_C_TYPE_TIMESTAMP>(
+			ColumnInfo<SQL_TIMESTAMP_STRUCT>::sm_rowsNumber,
+			(*m_dateTimeInfo).m_dataSet.data(),
+			(*m_dateTimeInfo).m_strLen_or_Ind.data(),
+			(*m_dateTimeInfo).m_columnNames);
+	}
+
+	// Name: ExecuteDateColumnsTest
+	//
+	// Description:
+	//  Test Execute with default script using an InputDataSet of Date columns.
+	//
+	TEST_F(PythonExtensionApiTests, ExecuteDateColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_dateInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQL_DATE_STRUCT, SQL_C_TYPE_DATE>(m_dateInfo.get());
+
+		TestExecute<SQL_DATE_STRUCT, SQL_C_TYPE_DATE>(
+			ColumnInfo<SQL_DATE_STRUCT>::sm_rowsNumber,
+			(*m_dateInfo).m_dataSet.data(),
+			(*m_dateInfo).m_strLen_or_Ind.data(),
+			(*m_dateInfo).m_columnNames);
+	}
+
+	// Name: TestExecute
+	//
+	// Description:
+	//  Template function to Test Execute with default script that assigns Input to Output.
+	//  It tests the correctness of the:
+	//   1. Executed script,
+	//   2. InputDataSet and
+	//   3. OutputDataSet
+	//  This can also be run without the validation steps by setting "validate" to false.
+	//
+	template<class SQLType, SQLSMALLINT dataType>
+	void PythonExtensionApiTests::TestExecute(
+		SQLULEN        rowsNumber,
+		void           **dataSet,
+		SQLINTEGER     **strLen_or_Ind,
+		vector<string> columnNames,
+		bool           validate)
+	{
+		testing::internal::CaptureStdout();
+
+		SQLUSMALLINT outputschemaColumnsNumber = 0;
+		SQLRETURN result = Execute(
+			*m_sessionId,
+			m_taskId,
+			rowsNumber,
+			dataSet,
+			strLen_or_Ind,
+			&outputschemaColumnsNumber);
+
+		ASSERT_EQ(result, SQL_SUCCESS);
+
+		string output = testing::internal::GetCapturedStdout();
+		cout << output;
+
+		if (validate)
+		{
+			// Verify print message was printed correctly
+			//
+			ASSERT_TRUE(output.find(m_printMessage) != string::npos);
+
+			try {
+				string createDictScript = m_inputDataNameString + ".to_dict()";
+				bp::dict inputDataSet = bp::extract<bp::dict>(bp::eval(createDictScript.c_str(), m_mainNamespace));
+
+				createDictScript = m_outputDataNameString + ".to_dict()";
+				bp::dict outputDataSet = bp::extract<bp::dict>(bp::eval(createDictScript.c_str(), m_mainNamespace));
+
+				for (SQLUSMALLINT columnIndex = 0; columnIndex < columnNames.size(); columnIndex++)
+				{
+					bp::dict inputColumnToTest = bp::extract<bp::dict>(inputDataSet.get(columnNames[columnIndex]));
+					bp::dict outputColumnToTest = bp::extract<bp::dict>(outputDataSet.get(columnNames[columnIndex]));
+
+					for (bp::dict column : { inputColumnToTest, outputColumnToTest })
+					{
+						CheckColumnEqualityFnMap::const_iterator it = sm_FnCheckColumnEqualityMap.find(dataType);
+
+						if (it == sm_FnCheckColumnEqualityMap.end())
+						{
+							throw runtime_error("Unsupported column type encountered when testing column equality");
+						}
+
+						(this->*it->second)(
+							rowsNumber,
+							column,
+							dataSet[columnIndex],
+							strLen_or_Ind[columnIndex]);
+					}
+				}
+			}
+			catch (bp::error_already_set &)
+			{
+				string pyError = PythonTestUtilities::ParsePythonException();
+				throw runtime_error("Error running python:\n" + pyError);
+			}
+		}
+	}
+}

language-extensions/python/test/src/PythonExtensionApiTests.cpp

@@ -0,0 +1,744 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonExtensionApiTests.cpp
+//
+// Purpose:
+//  Define the common member functions of the PythonExtensionApiTests class
+//
+//*************************************************************************************************
+#include "PythonExtensionApiTests.h"
+
+#include <datetime.h>
+
+using namespace std;
+namespace bp = boost::python;
+
+namespace ExtensionApiTest
+{
+	// Function map - maps a SQL data type to the appropriate function that
+	// adds a column to the dictionary
+	//
+	const PythonExtensionApiTests::CheckColumnEqualityFnMap PythonExtensionApiTests::sm_FnCheckColumnEqualityMap =
+	{
+		{static_cast<SQLSMALLINT>(SQL_C_BIT),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckBooleanColumnEquality)},
+		{static_cast<SQLSMALLINT>(SQL_C_SLONG),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckColumnEquality<SQLINTEGER>)},
+		{static_cast<SQLSMALLINT>(SQL_C_DOUBLE),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckColumnEquality<SQLDOUBLE>)},
+		{static_cast<SQLSMALLINT>(SQL_C_FLOAT),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckColumnEquality<SQLREAL>)},
+		{static_cast<SQLSMALLINT>(SQL_C_SSHORT),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckColumnEquality<SQLSMALLINT>)},
+		{static_cast<SQLSMALLINT>(SQL_C_UTINYINT),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckColumnEquality<SQLCHAR>)},
+		{static_cast<SQLSMALLINT>(SQL_C_SBIGINT),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckColumnEquality<SQLBIGINT>)},
+		{static_cast<SQLSMALLINT>(SQL_C_CHAR),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckStringColumnEquality)},
+		{static_cast<SQLSMALLINT>(SQL_C_WCHAR),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckWStringColumnEquality)},
+		{static_cast<SQLSMALLINT>(SQL_C_BINARY),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckRawColumnEquality)},
+		{static_cast<SQLSMALLINT>(SQL_C_TYPE_TIMESTAMP),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckDateTimeColumnEquality<SQL_TIMESTAMP_STRUCT>)},
+		{static_cast<SQLSMALLINT>(SQL_C_TYPE_DATE),
+		 static_cast<fnCheckColumnEquality>(&PythonExtensionApiTests::CheckDateTimeColumnEquality<SQL_DATE_STRUCT>)},
+	};
+
+	// Name: SetUp
+	//
+	// Description:
+	//  Code here will be called immediately after the constructor (right
+	//  before each test).
+	//
+	void PythonExtensionApiTests::SetUp()
+	{
+		Py_Initialize();
+
+		boost::python::numpy::initialize();
+
+		SetupVariables();
+	}
+
+	// Name: TearDown
+	//
+	// Description:
+	//  Code here will be called immediately after each test 
+	//  (right before the destructor).
+	//
+	void PythonExtensionApiTests::TearDown()
+	{
+		DoCleanup();
+	}
+
+	// Name: SetupVariables
+	//
+	// Description:
+	//  Set up default, valid variables for use in tests
+	//
+	void PythonExtensionApiTests::SetupVariables()
+	{
+		m_sessionId = new SQLGUID();
+		m_taskId = 0;
+		m_numTasks = 1;
+
+		m_scriptString = "print('" + m_printMessage + "');"
+			"OutputDataSet = InputDataSet;"
+			"print('InputDataSet:'); print(InputDataSet);"
+			"print('OutputDataSet:'); print(OutputDataSet);";
+		m_script = static_cast<SQLCHAR *>(static_cast<void *>(const_cast<char *>(m_scriptString.c_str())));
+		m_scriptLength = m_scriptString.length();
+
+		m_inputDataNameString = "InputDataSet";
+		m_inputDataName = static_cast<SQLCHAR *>(
+			static_cast<void *>(const_cast<char *>(m_inputDataNameString.c_str())));
+		m_inputDataNameLength = m_inputDataNameString.length();
+
+		m_outputDataNameString = "OutputDataSet";
+		m_outputDataName = static_cast<SQLCHAR *>(
+			static_cast<void *>(const_cast<char *>(m_outputDataNameString.c_str())));
+		m_outputDataNameLength = m_outputDataNameString.length();
+
+		m_integerInfo = make_unique<ColumnInfo<SQLINTEGER>>(
+			"IntegerColumn1",
+			vector<SQLINTEGER>{ 0, 1, 2, 3, 4},
+			vector<SQLINTEGER>(ColumnInfo<SQLINTEGER>::sm_rowsNumber, m_IntSize),
+			"IntegerColumn2",
+			vector<SQLINTEGER>{ m_MaxInt, m_MinInt, 0, 0, -1 },
+			vector<SQLINTEGER>{ m_IntSize, m_IntSize, SQL_NULL_DATA, SQL_NULL_DATA, m_IntSize });
+
+		m_booleanInfo = make_unique<ColumnInfo<SQLCHAR>>(
+			"BooleanColumn1",
+			vector<SQLCHAR>{ '1', '0', '1', 0, 1 },
+			vector<SQLINTEGER>(ColumnInfo<SQLCHAR>::sm_rowsNumber, m_BooleanSize),
+			"BooleanColumn2",
+			vector<SQLCHAR>{ '\0', '2', '1', '0', '\0' },
+			vector<SQLINTEGER>{ SQL_NULL_DATA, m_BooleanSize, m_BooleanSize, m_BooleanSize, SQL_NULL_DATA });
+
+		m_realInfo = make_unique<ColumnInfo<SQLREAL>>(
+			"RealColumn1",
+			vector<SQLREAL>{ 0.34F, 1.33F, m_MaxReal, m_MinReal, 68e10F },
+			vector<SQLINTEGER>(ColumnInfo<SQLREAL>::sm_rowsNumber, m_RealSize),
+			"RealColumn2",
+			vector<SQLREAL>{  0, -1, NAN, NAN, NAN },
+			vector<SQLINTEGER>{ m_RealSize, m_RealSize, SQL_NULL_DATA, SQL_NULL_DATA, SQL_NULL_DATA });
+
+		m_doubleInfo = make_unique<ColumnInfo<SQLDOUBLE>>(
+			"DoubleColumn1",
+			vector<SQLDOUBLE>{ -1.79e301, 1.33, m_MaxDouble, m_MinDouble, 1.79e30 },
+			vector<SQLINTEGER>(ColumnInfo<SQLDOUBLE>::sm_rowsNumber, m_DoubleSize),
+			"DoubleColumn2",
+			vector<SQLDOUBLE>{  0, -1, NAN, NAN, NAN },
+			vector<SQLINTEGER>{ m_DoubleSize, m_DoubleSize, SQL_NULL_DATA, SQL_NULL_DATA, SQL_NULL_DATA });
+
+		m_bigIntInfo = make_unique<ColumnInfo<SQLBIGINT>>(
+			"BigIntColumn1",
+			vector<SQLBIGINT>{ m_MaxBigInt, 1, 88883939, m_MinBigInt, -622280108 },
+			vector<SQLINTEGER>(ColumnInfo<SQLBIGINT>::sm_rowsNumber, m_BigIntSize),
+			"BigIntColumn2",
+			vector<SQLBIGINT>{0, 0, 0, 12341512213, -12341512213 },
+			vector<SQLINTEGER>{ SQL_NULL_DATA, SQL_NULL_DATA,
+			SQL_NULL_DATA, m_BigIntSize, m_BigIntSize });
+
+		m_smallIntInfo = make_unique<ColumnInfo<SQLSMALLINT>>(
+			"SmallIntColumn1",
+			vector<SQLSMALLINT>{ 223, 33, 9811, -725, 6810 },
+			vector<SQLINTEGER>(ColumnInfo<SQLSMALLINT>::sm_rowsNumber, m_SmallIntSize),
+			"SmallIntColumn2",
+			vector<SQLSMALLINT>{ m_MaxSmallInt, m_MinSmallInt, 0, 0, 3'276 },
+			vector<SQLINTEGER>{ m_SmallIntSize, m_SmallIntSize,
+			SQL_NULL_DATA, SQL_NULL_DATA, m_SmallIntSize });
+
+		m_tinyIntInfo = make_unique<ColumnInfo<SQLCHAR>>(
+			"TinyIntColumn1",
+			vector<SQLCHAR>{ 34, 133, 98, 72, 10 },
+			vector<SQLINTEGER>(ColumnInfo<SQLCHAR>::sm_rowsNumber, m_TinyIntSize),
+			"TinyIntColumn2",
+			vector<SQLCHAR>{ m_MaxTinyInt, m_MinTinyInt, 0, 0, 128 },
+			vector<SQLINTEGER>{ m_TinyIntSize, m_TinyIntSize,
+			SQL_NULL_DATA, SQL_NULL_DATA, m_TinyIntSize });
+
+		m_dateTimeInfo = make_unique<ColumnInfo<SQL_TIMESTAMP_STRUCT>>(
+			"DateTimeColumn1",
+			vector<SQL_TIMESTAMP_STRUCT>{
+				{ 9518, 8, 25, 19, 11, 40, 528934000 },
+				{ 5712, 3, 9, 2, 24, 32, 770483000 },
+				{ 1470, 7, 27, 17, 47, 52, 123456000 },
+				{ 2020, 4, 16, 15, 5, 12, 169012000 },
+				{ 231, 2, 14, 22, 36, 18, 489102000 },
+			},
+			vector<SQLINTEGER>(ColumnInfo<SQL_TIMESTAMP_STRUCT>::sm_rowsNumber, m_DateTimeSize),
+			"DateTimeColumn2",
+			vector<SQL_TIMESTAMP_STRUCT>{
+				{ 9999, 12, 31, 23, 59, 59, 999999000 },
+				{ 1,1,1,0,0,0,0 },
+				{},
+				{},
+				{}
+			},
+			vector<SQLINTEGER>{ m_DateTimeSize, m_DateTimeSize,
+			SQL_NULL_DATA, SQL_NULL_DATA, SQL_NULL_DATA });
+
+		m_dateInfo = make_unique<ColumnInfo<SQL_DATE_STRUCT>>(
+			"DateColumn1",
+			vector<SQL_DATE_STRUCT>{
+				{ 9518, 8, 25 },
+				{ 5712, 3, 9 },
+				{ 1470, 7, 27 },
+				{ 2020, 4, 16 },
+				{ 231, 2, 14, },
+			},
+			vector<SQLINTEGER>(ColumnInfo<SQL_DATE_STRUCT>::sm_rowsNumber, m_DateSize),
+			"DateColumn2",
+			vector<SQL_DATE_STRUCT>{
+				{ 9999, 12, 31 },
+				{ 1,1,1 },
+				{},
+				{},
+				{}
+			},
+			vector<SQLINTEGER>{ m_DateSize, m_DateSize,
+			SQL_NULL_DATA, SQL_NULL_DATA, SQL_NULL_DATA });
+
+		try
+		{
+			m_mainModule = bp::import("__main__");
+			m_mainNamespace = m_mainModule.attr("__dict__");
+		}
+		catch (bp::error_already_set&)
+		{
+			throw runtime_error("Error loading main module and namespace");
+		}
+
+		// Check that the module and namespace are populated, not None objects
+		//
+		if (m_mainModule == boost::python::object() ||
+			m_mainNamespace == boost::python::object())
+		{
+			throw runtime_error("Main module or namespace was None");
+		}
+	}
+
+	// Name: InitializeSession
+	//
+	// Description:
+	//  Initialize a valid, default session for non-Init tests
+	//  Tests InitSession API
+	//
+	void PythonExtensionApiTests::InitializeSession(
+		SQLUSMALLINT parametersNumber,
+		SQLUSMALLINT inputSchemaColumnsNumber,
+		string       scriptString)
+	{
+		SQLRETURN result = SQL_ERROR;
+
+		SQLCHAR *script = static_cast<SQLCHAR*>(
+			static_cast<void*>(const_cast<char*>(scriptString.c_str())));
+
+		result = Init(
+			nullptr, // Extension Params
+			0,       // Extension Params Length
+			nullptr, // Extension Path
+			0,       // Extension Path Length
+			nullptr, // Public Library Path
+			0,       // Public Library Path Length
+			nullptr, // Private Library Path
+			0        // Private Library Path Length
+		);
+		EXPECT_EQ(result, SQL_SUCCESS);
+
+		result = InitSession(
+			*m_sessionId,
+			m_taskId,
+			m_numTasks,
+			script,
+			scriptString.length(),
+			inputSchemaColumnsNumber,
+			parametersNumber,
+			m_inputDataName,
+			m_inputDataNameLength,
+			m_outputDataName,
+			m_outputDataNameLength
+		);
+		EXPECT_EQ(result, SQL_SUCCESS);
+	}
+
+
+	// Name: DoCleanup
+	//
+	// Description:
+	//  Call Cleanup on the PythonExtension.
+	//  Testing if Cleanup is implemented correctly.
+	//
+	void PythonExtensionApiTests::DoCleanup()
+	{
+		SQLRETURN result = SQL_ERROR;
+
+		result = CleanupSession(*m_sessionId, m_taskId);
+		EXPECT_EQ(result, SQL_SUCCESS);
+
+		result = Cleanup();
+		EXPECT_EQ(result, SQL_SUCCESS);
+	}
+
+	// Name: InitializeColumns
+	//
+	// Description:
+	//  Template function to call InitializeColumn for all columns.
+	//
+	template<class SQLType, SQLSMALLINT dataType>
+	void PythonExtensionApiTests::InitializeColumns(ColumnInfo<SQLType> *ColumnInfo)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = ColumnInfo->GetColumnsNumber();
+		for (SQLUSMALLINT columnNumber = 0; columnNumber < inputSchemaColumnsNumber; ++columnNumber)
+		{
+			InitializeColumn(columnNumber,
+				ColumnInfo->m_columnNames[columnNumber],
+				dataType,
+				sizeof(SQLType));
+		}
+	}
+
+	// Template instantiations
+	//
+	template void PythonExtensionApiTests::InitializeColumns<SQLINTEGER, SQL_C_SLONG>(
+		ColumnInfo<SQLINTEGER> *ColumnInfo);
+	template void PythonExtensionApiTests::InitializeColumns<SQLCHAR, SQL_C_BIT>(
+		ColumnInfo<SQLCHAR> *ColumnInfo);
+	template void PythonExtensionApiTests::InitializeColumns<SQLREAL, SQL_C_FLOAT>(
+		ColumnInfo<SQLREAL> *ColumnInfo);
+	template void PythonExtensionApiTests::InitializeColumns<SQLDOUBLE, SQL_C_DOUBLE>(
+		ColumnInfo<SQLDOUBLE> *ColumnInfo);
+	template void PythonExtensionApiTests::InitializeColumns<SQLBIGINT, SQL_C_SBIGINT>(
+		ColumnInfo<SQLBIGINT> *ColumnInfo);
+	template void PythonExtensionApiTests::InitializeColumns<SQLSMALLINT, SQL_C_SSHORT>(
+		ColumnInfo<SQLSMALLINT> *ColumnInfo);
+	template void PythonExtensionApiTests::InitializeColumns<SQLCHAR, SQL_C_UTINYINT>(
+		ColumnInfo<SQLCHAR> *ColumnInfo);
+	template void PythonExtensionApiTests::InitializeColumns<SQL_TIMESTAMP_STRUCT, SQL_C_TYPE_TIMESTAMP>(
+		ColumnInfo<SQL_TIMESTAMP_STRUCT> *ColumnInfo);
+	template void PythonExtensionApiTests::InitializeColumns<SQL_DATE_STRUCT, SQL_C_TYPE_DATE>(
+		ColumnInfo<SQL_DATE_STRUCT> *ColumnInfo);
+
+	// Name: InitializeColumn
+	//
+	// Description:
+	//  Call InitColumn for the given columnNumber, columnName, dataType and columnSize.
+	//
+	void PythonExtensionApiTests::InitializeColumn(
+		SQLSMALLINT columnNumber,
+		string      columnNameString,
+		SQLSMALLINT dataType,
+		SQLULEN     columnSize)
+	{
+		SQLCHAR *columnName = static_cast<SQLCHAR *>(
+			static_cast<void *>(const_cast<char *>(columnNameString.c_str()))
+			);
+
+		SQLRETURN result = SQL_ERROR;
+
+		result = InitColumn(
+			*m_sessionId,
+			m_taskId,
+			columnNumber,
+			columnName,
+			columnNameString.length(),
+			dataType,
+			columnSize,
+			0,         // decimalDigits
+			1,         // nullable
+			-1,        // partitionByNumber
+			-1);       // orderByNumber
+
+		EXPECT_EQ(result, SQL_SUCCESS);
+	}
+
+	// Name: GenerateContiguousData
+	//
+	// Description:
+	//  Fill a contiguous array columnData with members from the given columnVector
+	//  having lengths defined in strLenOrInd, unless it is SQL_NULL_DATA.
+	//
+	template<class SQLType>
+	vector<SQLType> PythonExtensionApiTests::GenerateContiguousData(
+		vector<const SQLType*> columnVector,
+		SQLINTEGER             *strLenOrInd)
+	{
+		vector<SQLType> retVal;
+
+		for (SQLULEN index = 0; index < columnVector.size(); ++index)
+		{
+			if (strLenOrInd[index] != SQL_NULL_DATA)
+			{
+				SQLINTEGER strLen = strLenOrInd[index] / sizeof(SQLType);
+				vector<SQLType> data(columnVector[index], columnVector[index] + strLen);
+				retVal.insert(retVal.end(), data.begin(), data.end());
+			}
+		}
+
+		return retVal;
+	}
+
+	template vector<char> PythonExtensionApiTests::GenerateContiguousData(
+		vector<const char*> columnVector,
+		SQLINTEGER          *strLenOrInd);
+	template vector<SQLCHAR> PythonExtensionApiTests::GenerateContiguousData(
+		vector<const SQLCHAR*> columnVector,
+		SQLINTEGER             *strLenOrInd);
+	template vector<wchar_t> PythonExtensionApiTests::GenerateContiguousData(
+		vector<const wchar_t*> columnVector,
+		SQLINTEGER             *strLenOrInd);
+
+	// Name: GetMaxLength
+	//
+	// Description:
+	//  Get max length of all strings from strLenOrInd.
+	//
+	SQLINTEGER PythonExtensionApiTests::GetMaxLength(
+		SQLINTEGER *strLenOrInd,
+		SQLULEN    rowsNumber)
+	{
+		SQLINTEGER maxLen = 0;
+		for (SQLULEN index = 0; index < rowsNumber; ++index)
+		{
+			if (strLenOrInd[index] != SQL_NULL_DATA && maxLen < strLenOrInd[index])
+			{
+				maxLen = strLenOrInd[index];
+			}
+		}
+
+		return maxLen;
+	}
+
+	// Name: GetWStringLength
+	//
+	// Description:
+	//  Utility function to get the length of a wchar_t *.
+	//  wcslen does not work in Linux with -fshort-wchar, so we use this function instead.
+	//
+	SQLULEN PythonExtensionApiTests::GetWStringLength(const wchar_t *wstr)
+	{
+		SQLULEN distance = -1;
+
+		// If nullptr, return
+		//
+		if (wstr)
+		{
+			// Get distance from end of string to beginning
+			//
+			const wchar_t *newstr = wstr;
+			while (*newstr)
+			{
+				++newstr;
+			}
+
+			distance = newstr - wstr;
+		}
+
+		return distance;
+	}
+
+	// Name: ColumnInfo
+	//
+	// Description:
+	//  Template constructor for the type information.
+	//  Useful for ColumnInfo of integer, basic numeric and boolean types.
+	//
+	template<class SQLType>
+	ColumnInfo<SQLType>::ColumnInfo(
+		string column1Name, vector<SQLType> column1, vector<SQLINTEGER> col1StrLenOrInd,
+		string column2Name, vector<SQLType> column2, vector<SQLINTEGER> col2StrLenOrInd)
+	{
+		m_columnNames = { column1Name, column2Name };
+		m_column1 = column1;
+		m_column2 = column2;
+		m_dataSet = { m_column1.data(), m_column2.data() };
+		m_col1StrLenOrInd = col1StrLenOrInd;
+		if (m_col1StrLenOrInd.empty())
+		{
+			m_strLen_or_Ind.push_back(nullptr);
+		}
+		else
+		{
+			m_strLen_or_Ind.push_back(m_col1StrLenOrInd.data());
+		}
+
+		m_col2StrLenOrInd = col2StrLenOrInd;
+		if (m_col2StrLenOrInd.empty())
+		{
+			m_strLen_or_Ind.push_back(nullptr);
+		}
+		else
+		{
+			m_strLen_or_Ind.push_back(m_col2StrLenOrInd.data());
+		}
+
+	}
+
+	// Name: CheckColumnEquality
+	//
+	// Description:
+	//  Template function to compare the given int/float columns for equality
+	//
+	template<class SQLType>
+	void PythonExtensionApiTests::CheckColumnEquality(
+		SQLULEN    expectedRowsNumber,
+		bp::dict   columnToTest,
+		void       *expectedColumn,
+		SQLINTEGER *strLen_or_Ind)
+	{
+		ASSERT_EQ(static_cast<SQLULEN>(bp::len(columnToTest)), expectedRowsNumber);
+
+		for (SQLULEN index = 0; index < expectedRowsNumber; ++index)
+		{
+			bp::object val = columnToTest[index];
+
+			if (strLen_or_Ind != nullptr && strLen_or_Ind[index] == SQL_NULL_DATA)
+			{
+				EXPECT_TRUE(val.is_none());
+			}
+			else
+			{
+				SQLType typeVal = bp::extract<SQLType>(val);
+				SQLType expectedValue = static_cast<SQLType*>(expectedColumn)[index];
+				EXPECT_EQ(typeVal, expectedValue);
+			}
+		}
+	}
+
+	// Name: CheckBooleanColumnEquality
+	//
+	// Description:
+	//  Check boolean columns for equality
+	//
+	void PythonExtensionApiTests::CheckBooleanColumnEquality(
+		SQLULEN    expectedRowsNumber,
+		bp::dict   columnToTest,
+		void       *expectedColumn,
+		SQLINTEGER *strLen_or_Ind)
+	{
+		ASSERT_EQ(static_cast<SQLULEN>(bp::len(columnToTest)), expectedRowsNumber);
+
+		for (SQLULEN index = 0; index < expectedRowsNumber; ++index)
+		{
+			bp::object val = columnToTest[index];
+
+			if (strLen_or_Ind != nullptr && strLen_or_Ind[index] == SQL_NULL_DATA)
+			{
+				EXPECT_TRUE(val.is_none());
+			}
+			else
+			{
+				bool expectedValue = static_cast<bool*>(expectedColumn)[index];
+				bool typeVal = bp::extract<bool>(val);
+				EXPECT_EQ(typeVal, expectedValue);
+			}
+		}
+	}
+
+	// Name: CheckStringColumnEquality
+	//
+	// Description:
+	//  Compare string column with the given data and corresponding strLen_or_Ind.
+	//  The expectedData is input as a void*, hence we input the expectedRowsNumber as well.
+	//  Where strLen_or_Ind == SQL_NULL_DATA, check for None.
+	//
+	void PythonExtensionApiTests::CheckStringColumnEquality(
+		SQLULEN    expectedRowsNumber,
+		bp::dict   columnToTest,
+		void       *expectedColumn,
+		SQLINTEGER *strLen_or_Ind)
+	{
+		ASSERT_EQ(static_cast<SQLULEN>(bp::len(columnToTest)), expectedRowsNumber);
+
+		SQLINTEGER cumulativeLength = 0;
+
+		for (SQLULEN index = 0; index < expectedRowsNumber; ++index)
+		{
+			bp::object val = columnToTest[index];
+			if (strLen_or_Ind == nullptr ||
+				(strLen_or_Ind != nullptr && strLen_or_Ind[index] == SQL_NULL_DATA))
+			{
+				EXPECT_TRUE(val.is_none());
+			}
+			else
+			{
+				string typeVal = bp::extract<string>(val);
+				string expectedString = string(
+					static_cast<char*>(expectedColumn) + cumulativeLength,
+					strLen_or_Ind[index]);
+
+				EXPECT_EQ(typeVal, expectedString);
+				cumulativeLength += strLen_or_Ind[index];
+			}
+		}
+	}
+
+	// Name: CheckWStringColumnEquality
+	//
+	// Description:
+	//  Compare wstring column with the given data and corresponding strLen_or_Ind.
+	//  The expectedData is input as a void*, hence we input the expectedRowsNumber as well.
+	//  Where strLen_or_Ind == SQL_NULL_DATA, check for None.
+	//  We have to compare byte by byte because EXPECT_EQ/STREQ do not
+	//  work properly for wstrings in Linux.
+	//
+	void PythonExtensionApiTests::CheckWStringColumnEquality(
+		SQLULEN    expectedRowsNumber,
+		bp::dict   columnToTest,
+		void       *expectedColumn,
+		SQLINTEGER *strLen_or_Ind)
+	{
+		ASSERT_EQ(static_cast<SQLULEN>(bp::len(columnToTest)), expectedRowsNumber);
+
+		SQLINTEGER cumulativeLength = 0;
+
+		for (SQLULEN index = 0; index < expectedRowsNumber; ++index)
+		{
+			bp::object val = columnToTest[index];
+			if (strLen_or_Ind == nullptr ||
+				(strLen_or_Ind != nullptr && strLen_or_Ind[index] == SQL_NULL_DATA))
+			{
+				EXPECT_TRUE(val.is_none());
+			}
+			else
+			{
+				// Get length of the unicode object in val and make sure it is the size we expect
+				//
+				SQLINTEGER size = PyUnicode_GET_LENGTH(val.ptr());
+				SQLINTEGER strLen = strLen_or_Ind[index] / sizeof(wchar_t);
+
+				EXPECT_EQ(strLen, size);
+
+				// Get a byte representation of the string as UTF16.
+				// PyUnicode_AsUTF16String adds a 2-byte BOM to the front of every string, so we ignore it.
+				//
+				char *paramBytes = PyBytes_AsString(PyUnicode_AsUTF16String(val.ptr())) + 2;
+
+				char *expectedParamBytes = static_cast<char*>(expectedColumn) + cumulativeLength;
+
+				// Compare the two wstrings byte by byte
+				//
+				for (SQLINTEGER i = 0; i < strLen_or_Ind[index]; ++i)
+				{
+					EXPECT_EQ(paramBytes[i], expectedParamBytes[i]);
+				}
+
+				cumulativeLength += strLen_or_Ind[index];
+			}
+		}
+	}
+
+	// Name: CheckRawColumnEquality
+	//
+	// Description:
+	//  Compare raw column with the given data and corresponding strLen_or_Ind.
+	//  The expectedData is input as a void*, hence we input the expectedRowsNumber as well.
+	//  Where strLen_or_Ind == SQL_NULL_DATA, check for None.
+	//
+	void PythonExtensionApiTests::CheckRawColumnEquality(
+		SQLULEN    expectedRowsNumber,
+		bp::dict   columnToTest,
+		void       *expectedColumn,
+		SQLINTEGER *strLen_or_Ind)
+	{
+		ASSERT_EQ(static_cast<SQLULEN>(bp::len(columnToTest)), expectedRowsNumber);
+
+		SQLINTEGER cumulativeLength = 0;
+
+		for (SQLULEN index = 0; index < expectedRowsNumber; ++index)
+		{
+			bp::object val = columnToTest[index];
+
+			if (strLen_or_Ind == nullptr ||
+				(strLen_or_Ind != nullptr && strLen_or_Ind[index] == SQL_NULL_DATA))
+			{
+				EXPECT_TRUE(val.is_none());
+			}
+			else
+			{
+				SQLCHAR *expectedValue = static_cast<SQLCHAR *>(expectedColumn) + cumulativeLength;
+				SQLCHAR *bytes = static_cast<SQLCHAR *>(static_cast<void *>(PyBytes_AsString(val.ptr())));
+
+				for (SQLINTEGER i = 0; i < strLen_or_Ind[index]; ++i)
+				{
+					EXPECT_EQ(bytes[i], expectedValue[i]);
+				}
+
+				cumulativeLength += strLen_or_Ind[index];
+			}
+		}
+	}
+
+	// Name: CheckDateTimeColumnEquality
+	//
+	// Description:
+	//  Compare datetime column with the given data and corresponding strLen_or_Ind.
+	//
+	template<class DateTimeStruct>
+	void PythonExtensionApiTests::CheckDateTimeColumnEquality(
+		SQLULEN    expectedRowsNumber,
+		bp::dict   columnToTest,
+		void       *expectedColumn,
+		SQLINTEGER *strLen_or_Ind)
+	{
+		ASSERT_EQ(static_cast<SQLULEN>(bp::len(columnToTest)), expectedRowsNumber);
+
+		DateTimeStruct *expectedDateTimeColumn = static_cast<DateTimeStruct *>(expectedColumn);
+
+		for (SQLULEN index = 0; index < expectedRowsNumber; ++index)
+		{
+			bp::object val = columnToTest[index];
+
+			if (strLen_or_Ind != nullptr && strLen_or_Ind[index] == SQL_NULL_DATA)
+			{
+				EXPECT_TRUE(val.is_none() || 
+					strcmp(val.ptr()->ob_type->tp_name, "NaTType") == 0);
+			}
+			else
+			{
+				// Import the PyDateTime API
+				//
+				PyDateTime_IMPORT;
+
+				PyObject *dateObject = val.ptr();
+
+				DateTimeStruct expectedValue = expectedDateTimeColumn[index];
+
+				SQLSMALLINT year = PyDateTime_GET_YEAR(dateObject);
+				SQLUSMALLINT month = PyDateTime_GET_MONTH(dateObject);
+				SQLUSMALLINT day = PyDateTime_GET_DAY(dateObject);
+
+				if constexpr (is_same_v<DateTimeStruct, SQL_TIMESTAMP_STRUCT>)
+				{
+					EXPECT_TRUE(PyDateTime_CheckExact(dateObject));
+					SQLUSMALLINT hour = PyDateTime_DATE_GET_HOUR(dateObject);
+					SQLUSMALLINT minute = PyDateTime_DATE_GET_MINUTE(dateObject);
+					SQLUSMALLINT second = PyDateTime_DATE_GET_SECOND(dateObject);
+					SQLUINTEGER usec = PyDateTime_DATE_GET_MICROSECOND(dateObject);
+
+					EXPECT_EQ(expectedValue.year, year);
+					EXPECT_EQ(expectedValue.month, month);
+					EXPECT_EQ(expectedValue.day, day);
+					EXPECT_EQ(expectedValue.hour, hour);
+					EXPECT_EQ(expectedValue.minute, minute);
+					EXPECT_EQ(expectedValue.second, second);
+					EXPECT_EQ(expectedValue.fraction, usec * 1000);
+				}
+				else
+				{
+					EXPECT_TRUE(PyDate_CheckExact(dateObject));
+
+					EXPECT_EQ(expectedValue.year, year);
+					EXPECT_EQ(expectedValue.month, month);
+					EXPECT_EQ(expectedValue.day, day);
+				}
+			}
+		}
+	}
+}

language-extensions/python/test/src/PythonGetResultColumnTests.cpp

@@ -0,0 +1,817 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonGetResultColumnTests.cpp
+//
+// Purpose:
+//  Tests the Python Extension's implementation of the external language GetResultColumn API.
+//
+//*************************************************************************************************
+#include "PythonExtensionApiTests.h"
+
+using namespace std;
+
+namespace ExtensionApiTest
+{
+	// Name: GetIntegerResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script expecting an OutputDataSet of Integer columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetIntegerResultColumnsTest)
+	{
+		InitializeSession(0, // parametersNumber
+			(*m_integerInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLINTEGER, SQL_C_SLONG>(m_integerInfo.get());
+
+		TestExecute<SQLINTEGER, SQL_C_SLONG>(
+			ColumnInfo<SQLINTEGER>::sm_rowsNumber,
+			(*m_integerInfo).m_dataSet.data(),
+			(*m_integerInfo).m_strLen_or_Ind.data(),
+			(*m_integerInfo).m_columnNames,
+			false);  // validate
+
+		TestGetResultColumn(0,  // columnNumber
+			SQL_C_SLONG,        // dataType
+			m_IntSize,          // columnSize
+			0,                  // decimalDigits
+			SQL_NO_NULLS);      // nullable
+
+		// Returns most generic int type because NULLs change the column to object
+		//
+		TestGetResultColumn(1,  // columnNumber
+			SQL_C_SBIGINT,      // dataType
+			m_BigIntSize,       // columnSize
+			0,                  // decimalDigits
+			SQL_NULLABLE);      // nullable
+	}
+
+	// Name: GetBooleanResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of Boolean columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetBooleanResultColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_booleanInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLCHAR, SQL_C_BIT>(m_booleanInfo.get());
+
+		TestExecute<SQLCHAR, SQL_C_BIT>(
+			ColumnInfo<SQLCHAR>::sm_rowsNumber,
+			(*m_booleanInfo).m_dataSet.data(),
+			(*m_booleanInfo).m_strLen_or_Ind.data(),
+			(*m_booleanInfo).m_columnNames,
+			false);  // validate
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_BIT,         // dataType
+			m_BooleanSize,     // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_BIT,         // dataType
+			m_BooleanSize,     // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetRealResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of Real columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetRealResultColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_realInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLREAL, SQL_C_FLOAT>(m_realInfo.get());
+
+		TestExecute<SQLREAL, SQL_C_FLOAT>(
+			ColumnInfo<SQLREAL>::sm_rowsNumber,
+			(*m_realInfo).m_dataSet.data(),
+			(*m_realInfo).m_strLen_or_Ind.data(),
+			(*m_realInfo).m_columnNames,
+			false);  // validate
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_FLOAT,       // dataType
+			m_RealSize,        // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		// Returns most generic float type because NULLs change the column to object
+		//
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_DOUBLE,      // dataType
+			m_DoubleSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetDoubleResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of Double columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetDoubleResultColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_doubleInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLDOUBLE, SQL_C_DOUBLE>(m_doubleInfo.get());
+
+		TestExecute<SQLDOUBLE, SQL_C_DOUBLE>(
+			ColumnInfo<SQLDOUBLE>::sm_rowsNumber,
+			(*m_doubleInfo).m_dataSet.data(),
+			(*m_doubleInfo).m_strLen_or_Ind.data(),
+			(*m_doubleInfo).m_columnNames,
+			false);  // validate
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_DOUBLE,      // dataType
+			m_DoubleSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_DOUBLE,      // dataType
+			m_DoubleSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetBigIntResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of BigInteger columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetBigIntResultColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_bigIntInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLBIGINT, SQL_C_SBIGINT>(m_bigIntInfo.get());
+
+		TestExecute<SQLBIGINT, SQL_C_SBIGINT>(
+			ColumnInfo<SQLBIGINT>::sm_rowsNumber,
+			(*m_bigIntInfo).m_dataSet.data(),
+			(*m_bigIntInfo).m_strLen_or_Ind.data(),
+			(*m_bigIntInfo).m_columnNames,
+			false); // validate
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_SBIGINT,     // dataType
+			m_BigIntSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_SBIGINT,     // dataType
+			m_BigIntSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetSmallIntResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of SmallInt columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetSmallIntResultColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_smallIntInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLSMALLINT, SQL_C_SSHORT>(m_smallIntInfo.get());
+
+		TestExecute<SQLSMALLINT, SQL_C_SSHORT>(
+			ColumnInfo<SQLSMALLINT>::sm_rowsNumber,
+			(*m_smallIntInfo).m_dataSet.data(),
+			(*m_smallIntInfo).m_strLen_or_Ind.data(),
+			(*m_smallIntInfo).m_columnNames,
+			false);  // validate
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_SSHORT,      // dataType
+			m_SmallIntSize,    // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		// Returns most generic int type because NULLs change the column to object
+		//
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_SBIGINT,     // dataType
+			m_BigIntSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetTinyIntResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of TinyInt columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetTinyIntResultColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_tinyIntInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQLCHAR, SQL_C_UTINYINT>(m_tinyIntInfo.get());
+
+		TestExecute<SQLCHAR, SQL_C_UTINYINT>(
+			ColumnInfo<SQLCHAR>::sm_rowsNumber,
+			(*m_tinyIntInfo).m_dataSet.data(),
+			(*m_tinyIntInfo).m_strLen_or_Ind.data(),
+			(*m_tinyIntInfo).m_columnNames,
+			false);  // validate
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_UTINYINT,    // dataType
+			m_TinyIntSize,     // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		// Returns most generic int type because NULLs change the column to object
+		//
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_SBIGINT,     // dataType
+			m_BigIntSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetStringResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of String columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetStringResultColumnsTest)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = 3;
+
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			inputSchemaColumnsNumber,
+			m_scriptString);
+
+		string stringColumn1Name = "StringColumn1";
+		InitializeColumn(0, stringColumn1Name, SQL_C_CHAR, m_CharSize);
+
+		string stringColumn2Name = "StringColumn2";
+		InitializeColumn(1, stringColumn2Name, SQL_C_CHAR, m_CharSize);
+
+		string stringColumn3Name = "StringColumn3";
+		InitializeColumn(2, stringColumn3Name, SQL_C_CHAR, m_CharSize);
+
+		vector<const char *> stringCol1{ "Hello", "test", "data", "World", "-123" };
+		vector<const char *> stringCol2{ "", 0, nullptr, "verify", "-1" };
+
+		vector<SQLINTEGER> strLenOrIndCol1 =
+		{ static_cast<SQLINTEGER>(strlen(stringCol1[0])),
+		  static_cast<SQLINTEGER>(strlen(stringCol1[1])),
+		  static_cast<SQLINTEGER>(strlen(stringCol1[2])),
+		  static_cast<SQLINTEGER>(strlen(stringCol1[3])),
+		  static_cast<SQLINTEGER>(strlen(stringCol1[4])) };
+		vector<SQLINTEGER> strLenOrIndCol2 =
+		{ 0, SQL_NULL_DATA, SQL_NULL_DATA,
+		  static_cast<SQLINTEGER>(strlen(stringCol2[3])),
+		  static_cast<SQLINTEGER>(strlen(stringCol2[4])) };
+
+		vector<SQLINTEGER *> strLen_or_Ind{ strLenOrIndCol1.data(),
+			strLenOrIndCol2.data(), nullptr };
+
+		// Coalesce the arrays of each row of each column
+		// into a contiguous array for each column.
+		//
+		vector<char> stringCol1Data = GenerateContiguousData<char>(stringCol1, strLenOrIndCol1.data());
+		vector<char> stringCol2Data = GenerateContiguousData<char>(stringCol2, strLenOrIndCol2.data());
+
+		void *dataSet[] = { stringCol1Data.data(),
+							stringCol2Data.data(),
+							nullptr };
+
+		int rowsNumber = stringCol1.size();
+
+		vector<string> columnNames{ stringColumn1Name, stringColumn2Name, stringColumn3Name };
+
+		TestExecute<SQLCHAR, SQL_C_CHAR>(
+			rowsNumber,
+			dataSet,
+			strLen_or_Ind.data(),
+			columnNames,
+			false); // validate
+
+		SQLULEN maxCol1Len = GetMaxLength(strLenOrIndCol1.data(), rowsNumber);
+		SQLULEN maxCol2Len = GetMaxLength(strLenOrIndCol2.data(), rowsNumber);
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_CHAR,        // dataType
+			maxCol1Len,        // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_CHAR,        // dataType
+			maxCol2Len,        // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+
+		TestGetResultColumn(2, // columnNumber
+			SQL_C_CHAR,        // dataType
+			0,                 // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetWStringResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of wstring columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetWStringResultColumnsTest)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = 3;
+
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			inputSchemaColumnsNumber,
+			m_scriptString);
+
+		string wstringColumn1Name = "WstringColumn1";
+		InitializeColumn(0, wstringColumn1Name, SQL_C_WCHAR, m_CharSize);
+
+		string wstringColumn2Name = "WstringColumn2";
+		InitializeColumn(1, wstringColumn2Name, SQL_C_WCHAR, m_CharSize);
+
+		string wstringColumn3Name = "WstringColumn3";
+		InitializeColumn(2, wstringColumn3Name, SQL_C_WCHAR, m_CharSize);
+
+		vector<const wchar_t *> wstringCol1{ L"Hello", L"test", L"data", L"World", L"你" };
+		vector<const wchar_t *> wstringCol2{ L"", 0, nullptr, L"verify", L"-1" };
+
+		int rowsNumber = wstringCol1.size();
+
+		vector<SQLINTEGER> strLenOrIndCol1 =
+		{ static_cast<SQLINTEGER>(5 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(4 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(4 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(5 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(1 * sizeof(wchar_t)) };
+		vector<SQLINTEGER> strLenOrIndCol2 =
+		{ 0, SQL_NULL_DATA, SQL_NULL_DATA,
+		  static_cast<SQLINTEGER>(6 * sizeof(wchar_t)),
+		  static_cast<SQLINTEGER>(2 * sizeof(wchar_t)) };
+		vector<SQLINTEGER> strLenOrIndCol3(rowsNumber, SQL_NULL_DATA);
+
+		vector<SQLINTEGER *> strLen_or_Ind{ strLenOrIndCol1.data(),
+			strLenOrIndCol2.data(), strLenOrIndCol3.data() };
+
+		// Coalesce the arrays of each row of each column
+		// into a contiguous array for each column.
+		//
+		vector<wchar_t> wstringCol1Data =
+			GenerateContiguousData<wchar_t>(wstringCol1, strLenOrIndCol1.data());
+
+		vector<wchar_t> wstringCol2Data =
+			GenerateContiguousData<wchar_t>(wstringCol2, strLenOrIndCol2.data());
+
+		void *dataSet[] = { wstringCol1Data.data(),
+							wstringCol2Data.data(),
+							nullptr };
+
+		vector<string> columnNames{ wstringColumn1Name, wstringColumn2Name, wstringColumn3Name };
+
+		TestExecute<wchar_t, SQL_C_WCHAR>(
+			rowsNumber,
+			dataSet,
+			strLen_or_Ind.data(),
+			columnNames,
+			false); // validate
+
+		// Because Python is UTF-8, we always return UTF-8 strings. 
+		// Thus, we expect output result columns to be strings, not wstrings, so we need to resize
+		// our expected column size by dividing by the sizeof(wchar_t).
+		//
+		SQLULEN maxCol1Len = GetMaxLength(strLenOrIndCol1.data(), rowsNumber) / sizeof(wchar_t);
+		SQLULEN maxCol2Len = GetMaxLength(strLenOrIndCol2.data(), rowsNumber) / sizeof(wchar_t);
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_CHAR,        // expectedDataType
+			maxCol1Len,        // expectedColumnSize
+			0,                 // expectedDecimalDigits
+			SQL_NO_NULLS);     // expectedNullable
+
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_CHAR,        // expectedDataType
+			maxCol2Len,        // expectedColumnSize
+			0,                 // expectedDecimalDigits
+			SQL_NULLABLE);     // expectedNullable
+
+		TestGetResultColumn(2, // columnNumber
+			SQL_C_CHAR,        // expectedDataType
+			0,                 // expectedColumnSize
+			0,                 // expectedDecimalDigits
+			SQL_NULLABLE);     // expectedNullable
+	}
+
+	// Name: GetRawResultColumnTest
+	//
+	// Description:
+	//  Test GetResultColumn with a script that returns OutputDataSet with raw columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetRawResultColumnsTest)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = 3;
+
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			inputSchemaColumnsNumber,
+			m_scriptString);
+
+		const SQLCHAR BinaryValue1[] = { 0x01, 0x01, 0xe2, 0x40 };
+		const SQLCHAR BinaryValue2[] = { 0x04, 0x05, 0xe1 };
+		const SQLCHAR BinaryValue3[] = { 0x00, 0x00, 0x00, 0x01 };
+		const SQLCHAR BinaryValue4[] = { 0xff };
+
+		const SQLCHAR BinaryValue5[] = { 0x00 };
+		const SQLCHAR BinaryValue6[] = { 0xff, 0xff, 0xff, 0xff };
+		const SQLCHAR BinaryValue7[] = { 0x00, 0x12, 0xd2, 0xff, 0x00, 0x12, 0xd2, 0xff, 0x00, 0x12, 0xd2, 0xff };
+
+		string binaryColumn1Name = "BinaryColumn1";
+		InitializeColumn(0, binaryColumn1Name, SQL_C_BINARY, m_BinarySize);
+
+		string binaryColumn2Name = "BinaryColumn2";
+		InitializeColumn(1, binaryColumn2Name, SQL_C_BINARY, m_BinarySize);
+
+		string binaryColumn3Name = "BinaryColumn3";
+		InitializeColumn(2, binaryColumn3Name, SQL_C_BINARY, m_BinarySize);
+
+		vector<const SQLCHAR*> binaryCol1{ BinaryValue1, BinaryValue2, BinaryValue3, BinaryValue4 };
+		vector<const SQLCHAR*> binaryCol2{ BinaryValue5, BinaryValue6, nullptr, BinaryValue7 };
+
+		SQLINTEGER strLenOrIndCol1[] =
+		{
+			static_cast<SQLINTEGER>(sizeof(BinaryValue1) / m_BinarySize),
+			static_cast<SQLINTEGER>(sizeof(BinaryValue2) / m_BinarySize),
+			static_cast<SQLINTEGER>(sizeof(BinaryValue3) / m_BinarySize),
+			static_cast<SQLINTEGER>(sizeof(BinaryValue4) / m_BinarySize)
+		};
+
+		SQLINTEGER strLenOrIndCol2[] =
+		{
+			SQL_NULL_DATA,
+			static_cast<SQLINTEGER>(sizeof(BinaryValue6) / m_BinarySize),
+			SQL_NULL_DATA,
+			static_cast<SQLINTEGER>(sizeof(BinaryValue7) / m_BinarySize)
+		};
+
+		vector<SQLINTEGER*> strLen_or_Ind{ strLenOrIndCol1, strLenOrIndCol2, nullptr };
+
+		// Coalesce the arrays of each row of each column
+		// into a contiguous array for each column.
+		//
+		int rowsNumber = binaryCol1.size();
+
+		vector<SQLCHAR> binaryCol1Data = GenerateContiguousData<SQLCHAR>(binaryCol1, strLenOrIndCol1);
+		vector<SQLCHAR> binaryCol2Data = GenerateContiguousData<SQLCHAR>(binaryCol2, strLenOrIndCol2);
+
+		void* dataSet[] = { binaryCol1Data.data(),
+							binaryCol2Data.data(),
+							nullptr };
+
+		vector<string> columnNames{ binaryColumn1Name, binaryColumn2Name, binaryColumn3Name };
+
+		SQLULEN maxCol1Len = GetMaxLength(strLenOrIndCol1, rowsNumber);
+		SQLULEN maxCol2Len = GetMaxLength(strLenOrIndCol2, rowsNumber);
+
+		TestExecute<SQLCHAR, SQL_C_BINARY>(
+			rowsNumber,
+			dataSet,
+			strLen_or_Ind.data(),
+			columnNames,
+			false);
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_BINARY,      // dataType
+			maxCol1Len,        // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_BINARY,      // dataType
+			maxCol2Len,        // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+
+		TestGetResultColumn(2, // columnNumber
+			SQL_C_CHAR,        // dataType; since the whole column is None, we use CHAR NoneType
+			0,                 // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetDateTimeResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script expecting an OutputDataSet of DateTime columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetDateTimeResultColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_dateTimeInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQL_TIMESTAMP_STRUCT, SQL_C_TYPE_TIMESTAMP>(m_dateTimeInfo.get());
+
+		TestExecute<SQL_TIMESTAMP_STRUCT, SQL_C_TYPE_TIMESTAMP>(
+			ColumnInfo<SQL_TIMESTAMP_STRUCT>::sm_rowsNumber,
+			(*m_dateTimeInfo).m_dataSet.data(),
+			(*m_dateTimeInfo).m_strLen_or_Ind.data(),
+			(*m_dateTimeInfo).m_columnNames,
+			false);
+
+		TestGetResultColumn(0,    // columnNumber
+			SQL_C_TYPE_TIMESTAMP, // dataType
+			m_DateTimeSize,       // columnSize
+			0,                    // decimalDigits
+			SQL_NO_NULLS);        // nullable
+
+		TestGetResultColumn(1,    // columnNumber
+			SQL_C_TYPE_TIMESTAMP, // dataType
+			m_DateTimeSize,       // columnSize
+			0,                    // decimalDigits
+			SQL_NULLABLE);        // nullable
+	}
+
+	// Name: GetDateResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script expecting an OutputDataSet of Date columns.
+	//
+	TEST_F(PythonExtensionApiTests, GetDateResultColumnsTest)
+	{
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			(*m_dateInfo).GetColumnsNumber(),
+			m_scriptString);
+
+		InitializeColumns<SQL_DATE_STRUCT, SQL_C_TYPE_DATE>(m_dateInfo.get());
+
+		TestExecute<SQL_DATE_STRUCT, SQL_C_TYPE_DATE>(
+			ColumnInfo<SQL_DATE_STRUCT>::sm_rowsNumber,
+			(*m_dateInfo).m_dataSet.data(),
+			(*m_dateInfo).m_strLen_or_Ind.data(),
+			(*m_dateInfo).m_columnNames,
+			false);
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_TYPE_DATE,   // dataType
+			m_DateSize,        // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_TYPE_DATE,   // dataType
+			m_DateSize,        // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+	}
+
+	// Name: GetDifferentResultColumnsTest
+	//
+	// Description:
+	//  Test GetResultColumn with default script using an OutputDataSet of different column types.
+	//
+	TEST_F(PythonExtensionApiTests, GetDifferentResultColumnsTest)
+	{
+		SQLUSMALLINT inputSchemaColumnsNumber = 3;
+
+		// Initialize with a default Session that prints Hello PythonExtension
+		// and assigns InputDataSet to OutputDataSet
+		//
+		InitializeSession(0, // parametersNumber
+			inputSchemaColumnsNumber,
+			m_scriptString);
+
+		string integerColumnName = "IntegerColumn";
+		InitializeColumn(0, integerColumnName, SQL_C_SLONG, m_IntSize);
+
+		string doubleColumnName = "DoubleColumn";
+		InitializeColumn(1, doubleColumnName, SQL_C_DOUBLE, m_DoubleSize);
+
+		string stringColumnName = "StringColumn";
+		InitializeColumn(2, stringColumnName, SQL_C_CHAR, m_CharSize);
+
+		vector<SQLINTEGER> intColData{ m_MaxInt, m_MinInt, 0, 1320, -1 };
+		vector<SQLDOUBLE> doubleColData{ m_MinDouble, 1.33, 83.98, 72.45, m_MaxDouble };
+		vector<const char*> stringCol{ "Hello", "test", "data", "World", "-123" };
+
+		const SQLINTEGER intSize = m_IntSize;
+		SQLINTEGER strLenOrIndCol1[] = { intSize, intSize, SQL_NULL_DATA, SQL_NULL_DATA, intSize };
+		SQLINTEGER strLenOrIndCol3[] =
+		{ static_cast<SQLINTEGER>(strlen(stringCol[0])),
+		static_cast<SQLINTEGER>(strlen(stringCol[1])),
+		static_cast<SQLINTEGER>(strlen(stringCol[2])),
+		static_cast<SQLINTEGER>(strlen(stringCol[3])),
+		static_cast<SQLINTEGER>(strlen(stringCol[4])) };
+		vector<SQLINTEGER*> strLen_or_Ind{ strLenOrIndCol1, nullptr, strLenOrIndCol3 };
+
+		int rowsNumber = intColData.size();
+
+		vector<char> stringColData = GenerateContiguousData<char>(stringCol, strLenOrIndCol3);
+
+		vector<void*> dataSet{ intColData.data(), doubleColData.data(), stringColData.data() };
+
+		SQLUSMALLINT outputschemaColumnsNumber = 0;
+		SQLRETURN result = Execute(
+			*m_sessionId,
+			m_taskId,
+			rowsNumber,
+			dataSet.data(),
+			strLen_or_Ind.data(),
+			&outputschemaColumnsNumber);
+		ASSERT_EQ(result, SQL_SUCCESS);
+
+		TestGetResultColumn(0, // columnNumber
+			SQL_C_SBIGINT,     // dataType
+			m_BigIntSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NULLABLE);     // nullable
+
+		TestGetResultColumn(1, // columnNumber
+			SQL_C_DOUBLE,      // dataType
+			m_DoubleSize,      // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+
+		SQLULEN maxLen = GetMaxLength(strLenOrIndCol3, rowsNumber);
+
+		TestGetResultColumn(2, // columnNumber
+			SQL_C_CHAR,        // dataType
+			maxLen,            // columnSize
+			0,                 // decimalDigits
+			SQL_NO_NULLS);     // nullable
+	}
+
+	// Name: GetEmptyResultColumnTest
+	//
+	// Description:
+	//  Test GetResultColumn with a script that returns OutputDataSet with empty rows.
+	//
+	TEST_F(PythonExtensionApiTests, GetEmptyResultColumnTest)
+	{
+		string scriptString = "from pandas import DataFrame; import numpy as np;"
+		"OutputDataSet = DataFrame({'intCol' : np.array([], dtype=np.int32)});"
+		"print(OutputDataSet)";
+
+		// Initialize with a Session that executes the above script
+		// that creates an empty row 1 column OutputDataSet.
+		//
+		InitializeSession(0, // parametersNumber
+			0,               // inputSchemaColumnsNumber
+			scriptString);
+
+		SQLUSMALLINT outputschemaColumnsNumber = 0;
+		SQLRETURN result = Execute(
+			*m_sessionId,
+			m_taskId,
+			0,
+			nullptr,
+			nullptr,
+			&outputschemaColumnsNumber);
+		ASSERT_EQ(result, SQL_SUCCESS);
+
+		EXPECT_EQ(outputschemaColumnsNumber, 1);
+
+		TestGetResultColumn(0,  // columnNumber
+			SQL_C_SLONG,        // dataType
+			m_IntSize,          // columnSize
+			0,                  // decimalDigits
+			SQL_NO_NULLS);      // nullable
+	}
+
+	// Name: MixedColumnNamesTest
+	//
+	// Description:
+	//  Test GetResultColumn with a script that returns a dataset with mixed type column names
+	//  i.e. integer and string.
+	//  This makes sure that we can access columns with integer column names.
+	//
+	TEST_F(PythonExtensionApiTests, MixedColumnNamesTest)
+	{
+		// With this script, we create a DataFrame with unusual column names.
+		// This tests non-string and mixed type column names.
+		//
+		string scriptString = "from pandas import DataFrame;"
+			"OutputDataSet = DataFrame([['ABC', 123],['DEF', 456]], columns=[3, 'col2']);"
+			"print(OutputDataSet)";
+
+		// Initialize with a Session that executes the above script
+		// that creates an OutputDataSet with mixed column names.
+		//
+		InitializeSession(0, // parametersNumber
+			0,               // inputSchemaColumnsNumber
+			scriptString);
+
+		SQLUSMALLINT outputschemaColumnsNumber = 0;
+		SQLRETURN result = Execute(
+			*m_sessionId,
+			m_taskId,
+			0,
+			nullptr,
+			nullptr,
+			&outputschemaColumnsNumber);
+		ASSERT_EQ(result, SQL_SUCCESS);
+
+		EXPECT_EQ(outputschemaColumnsNumber, 2);
+		
+		TestGetResultColumn(0,  // columnNumber
+			SQL_C_CHAR,         // dataType
+			3,                  // columnSize
+			0,                  // decimalDigits
+			SQL_NO_NULLS);      // nullable
+
+		TestGetResultColumn(1,  // columnNumber
+			SQL_C_SBIGINT,      // dataType
+			m_BigIntSize,       // columnSize
+			0,                  // decimalDigits
+			SQL_NO_NULLS);      // nullable
+	}
+
+	// Name: TestGetResultColumn
+	//
+	// Description:
+	//  Test GetResultColumn to verify the expected result column information is obtained.
+	//
+	void PythonExtensionApiTests::TestGetResultColumn(
+		SQLUSMALLINT columnNumber,
+		SQLSMALLINT  expectedDataType,
+		SQLULEN      expectedColumnSize,
+		SQLSMALLINT  expectedDecimalDigits,
+		SQLSMALLINT  expectedNullable)
+	{
+		SQLSMALLINT dataType = 0;
+		SQLULEN columnSize = 0;
+		SQLSMALLINT decimalDigits = 0;
+		SQLSMALLINT nullable = 0;
+
+		SQLRETURN result = GetResultColumn(
+			*m_sessionId,
+			m_taskId,
+			columnNumber,
+			&dataType,
+			&columnSize,
+			&decimalDigits,
+			&nullable);
+		ASSERT_EQ(result, SQL_SUCCESS);
+
+		EXPECT_EQ(dataType, expectedDataType);
+		EXPECT_EQ(columnSize, expectedColumnSize);
+		EXPECT_EQ(decimalDigits, expectedDecimalDigits);
+		EXPECT_EQ(nullable, expectedNullable);
+	}
+}

language-extensions/python/test/src/PythonInitColumnTests.cpp

@@ -0,0 +1,104 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonInitColumnTests.cpp
+//
+// Purpose:
+//  Test the Python extension columns using the Extension API
+//
+//*************************************************************************************************
+#include "PythonExtensionApiTests.h"
+
+namespace ExtensionApiTest
+{
+	// Name: InitColumnTest
+	//
+	// Description:
+	//  Test InitColumn() API with valid values
+	//
+	TEST_F(PythonExtensionApiTests, InitColumnTest)
+	{
+		InitializeSession(0, // paramsNumber
+			1); // inputSchemaColumns
+
+		SQLCHAR * columnName = static_cast<SQLCHAR *>(
+			static_cast<void *>(const_cast<char *>("Column1")));
+		SQLSMALLINT columnNameLength = 7;
+		SQLRETURN result = InitColumn(
+			*m_sessionId,
+			m_taskId,
+			0,                // Column Number
+			columnName,
+			columnNameLength,
+			SQL_C_SLONG,      // Data Type
+			m_IntSize,        // Column Size
+			0,                // Decimal Digits
+			1,                // Nullable
+			-1,               // PartitionByNumber
+			-1                // OrderByNumber
+		);
+		EXPECT_EQ(result, SQL_SUCCESS);
+	}
+	
+	//
+	// Negative Tests
+	//
+
+	// Name: InitInvalidColumnTest
+	//
+	// Description:
+	//  Test InitColumn() API with null column name
+	//
+	TEST_F(PythonExtensionApiTests, InitInvalidColumnTest)
+	{
+		InitializeSession(0, // paramsNumber
+			1); // inputSchemaColumns
+
+		SQLRETURN result = InitColumn(
+			*m_sessionId,
+			m_taskId,
+			0,           // Column Number
+			nullptr,     // Column Name
+			0,           // Column Name Length
+			SQL_C_SLONG, // Data Type
+			m_IntSize,   // Column Size
+			0,           // Decimal Digits
+			1,           // Nullable
+			-1,          // PartitionByNumber
+			-1           // OrderByNumber
+		);
+		EXPECT_EQ(result, SQL_ERROR);
+	}
+
+	// Name: InitInvalidColumnNumberTest
+	//
+	// Description:
+	//  Test InitColumn() API with bad column numbers (too big)
+	//
+	TEST_F(PythonExtensionApiTests, InitInvalidColumnNumberTest)
+	{
+		InitializeSession(1);
+
+		SQLCHAR * columnName = static_cast<SQLCHAR *>(
+			static_cast<void *>(const_cast<char *>("Column1")));
+		SQLSMALLINT columnNameLength = 7;
+
+		SQLRETURN result = InitColumn(
+			*m_sessionId,
+			m_taskId,
+			2,                // column number greater than initialized columns
+			columnName,
+			columnNameLength,
+			SQL_C_SLONG,      // Data Type
+			m_IntSize,        // Column Size
+			0,                // Decimal Digits
+			1,                // Nullable
+			-1,               // PartitionByNumber
+			-1                // OrderByNumber
+		);
+		EXPECT_EQ(result, SQL_ERROR);
+	}
+}

language-extensions/python/test/src/PythonInitTests.cpp

@@ -0,0 +1,76 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonInitTests.cpp
+//
+// Purpose:
+//  Test the Python extension initialization and
+//  session initialization using the Extension API
+//
+//*************************************************************************************************
+#include "PythonExtensionApiTests.h"
+
+using namespace std;
+
+namespace ExtensionApiTest
+{
+	// Name: TestInitWithNulls
+	//
+	// Description:
+	//  Test Init() API with all nullptrs.
+	//  This tests the extension initialization, using nullptrs because
+	//  param and library paths are optional.
+	//
+	TEST_F(PythonExtensionApiTests, TestInitWithNulls)
+	{
+		SQLRETURN result = Init(
+			nullptr, // Extension Params
+			0,       // Extension Params Length
+			nullptr, // Extension Path
+			0,       // Extension Path Length
+			nullptr, // Public Library Path
+			0,       // Public Library Path Length
+			nullptr, // Private Library Path
+			0        // Private Library Path Length
+		);
+
+		EXPECT_EQ(result, SQL_SUCCESS);
+	}
+
+	// Name: TestInitWithValues
+	//
+	// Description:
+	//  Test Init() API with actual values.
+	//
+	TEST_F(PythonExtensionApiTests, TestInitWithValues)
+	{
+		SQLCHAR *extensionParams = nullptr;
+		SQLULEN extensionParamsLength = 0;
+		string extensionPath = "C:/Path/To/ExternalLanguages/1/65554";
+		SQLULEN extensionPathLength = extensionPath.length();
+		string publicLibraryPath = "C:/Path/To/ExternalLanguages/1/65554/1";
+		SQLULEN publicLibraryPathLength = publicLibraryPath.length();
+		SQLCHAR *privateLibraryPath = nullptr;
+		SQLULEN privateLibraryPathLength = 0;
+
+		SQLCHAR *unsignedExtensionPath = static_cast<SQLCHAR *>(
+			static_cast<void *>(const_cast<char *>(extensionPath.c_str())));
+		SQLCHAR *unsignedPublicLibraryPath = static_cast<SQLCHAR *>(
+			static_cast<void *>(const_cast<char *>(publicLibraryPath.c_str())));
+
+		SQLRETURN result = Init(extensionParams,
+			extensionParamsLength,
+			unsignedExtensionPath,
+			extensionPathLength,
+			unsignedPublicLibraryPath,
+			publicLibraryPathLength,
+			privateLibraryPath,
+			privateLibraryPathLength
+		);
+
+		EXPECT_EQ(result, SQL_SUCCESS);
+	}
+}

language-extensions/python/test/src/PythonLibraryTests.cpp

@@ -0,0 +1,682 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonLibraryTests.cpp
+//
+// Purpose:
+//  Define the gtest overrides and the External Library tests
+//
+//*************************************************************************************************
+#include "PythonLibraryTests.h"
+#include "PythonTestUtilities.h"
+
+using namespace std;
+namespace bp = boost::python;
+
+namespace LibraryApiTests
+{
+	// Name: SetUp
+	//
+	// Description:
+	//  Code here will be called immediately after the constructor (right
+	//  before each test).
+	//
+	void ExternalLibraryApiTests::SetUp()
+	{
+		fs::path libPath = fs::absolute(m_libraryPath);
+
+		if (fs::exists(libPath))
+		{
+			fs::remove_all(libPath);
+		}
+
+		EXPECT_FALSE(fs::exists(libPath));
+
+		SetupVariables();
+		Initialize();
+	}
+
+	// Name: TearDown
+	//
+	// Description:
+	//  Code here will be called immediately after each test (right
+	//  before the destructor).
+	//
+	void ExternalLibraryApiTests::TearDown()
+	{
+		fs::path libPath = fs::absolute(m_libraryPath);
+
+		if (fs::exists(libPath))
+		{
+			fs::remove_all(libPath);
+		}
+
+		EXPECT_FALSE(fs::exists(libPath));
+
+		DoCleanup();
+	}
+
+	// Name: SetupVariables
+	//
+	// Description:
+	//  Set up default, valid variables for use in tests
+	//
+	void ExternalLibraryApiTests::SetupVariables()
+	{
+		m_libraryPath = fs::absolute(m_libraryPath).string();
+
+		string enlRoot = getenv("ENL_ROOT");
+
+		m_packagesPath = fs::absolute(enlRoot);
+
+		m_packagesPath = m_packagesPath / "language-extensions" / "python" / "test" / "test_packages";
+
+		ASSERT_TRUE(fs::exists(m_packagesPath));
+
+		fs::path libPath = m_libraryPath;
+		m_publicLibraryPath = (libPath / "1").string();
+		m_privateLibraryPath = (libPath / "2").string();
+	}
+
+	// Name: Initialize
+	//
+	// Description:
+	//  Initialize pythonextension for library management
+	//
+	void ExternalLibraryApiTests::Initialize()
+	{
+		SQLRETURN result = SQL_ERROR;
+
+		result = Init(
+			nullptr, // Extension Params
+			0,       // Extension Params Length
+			nullptr, // Extension Path
+			0,       // Extension Path Length
+			reinterpret_cast<SQLCHAR *>(const_cast<char *>(m_publicLibraryPath.c_str())),
+			m_publicLibraryPath.length(),
+			reinterpret_cast<SQLCHAR *>(const_cast<char *>(m_privateLibraryPath.c_str())),
+			m_privateLibraryPath.length()
+		);
+
+		EXPECT_EQ(result, SQL_SUCCESS);
+
+		m_mainModule = bp::import("__main__");
+		m_mainNamespace = bp::extract<bp::dict>(m_mainModule.attr("__dict__"));
+
+		if (!m_mainNamespace.has_key("originalKeys"))
+		{
+			string getKeysScript = "import sys\n"
+				"originalKeys = list(sys.modules.keys())";
+			bp::exec(getKeysScript.c_str(), m_mainNamespace);
+		}
+
+		ASSERT_NE(m_mainModule, boost::python::object());
+		ASSERT_NE(m_mainNamespace, boost::python::object());
+	}
+
+	// Name: DoCleanup
+	//
+	// Description:
+	//  Call Cleanup on the PythonExtension.
+	//  Testing if Cleanup is implemented correctly.
+	//
+	void ExternalLibraryApiTests::DoCleanup()
+	{
+		SQLRETURN result = Cleanup();
+		EXPECT_EQ(result, SQL_SUCCESS);
+	}
+
+	// Name: CleanModules
+	//
+	// Description:
+	//  Clean sys.modules in python to remove import traces.
+	//  This keeps the system clean for the next installation and import test.
+	//
+	void ExternalLibraryApiTests::CleanModules(
+		string extLibName,
+		string moduleName)
+	{
+		try
+		{
+			string delScript = "import sys\n"
+				"keys = [key for key in sys.modules if '" + extLibName +
+				"' in key or '" + moduleName + "' in key]\n"
+				"for k in keys:"
+				"    del sys.modules[k]";
+
+			bp::exec(delScript.c_str(), m_mainNamespace);
+		}
+		catch (const bp::error_already_set &)
+		{
+			string s = PythonTestUtilities::ParsePythonException();
+			cout << s << endl;
+		}
+	}
+
+	// Name: InstallAndTest
+	//
+	// Description:
+	//  Install a package and test it.
+	//  If we expect a successful install, then we check the version and location.
+	//  We then try importing the package and check whether it imports as we expect.
+	//
+	void ExternalLibraryApiTests::InstallAndTest(
+		string extLibName,
+		string moduleName,
+		string pathToPackage,
+		string installDir,
+		string expectedVersion,
+		string expectedLocation,
+		bool   successfulInstall,
+		bool   successfulImport)
+	{
+		SQLCHAR *libError = nullptr;
+		SQLINTEGER libErrorLength = 0;
+
+		SQLRETURN result = InstallExternalLibrary(
+			SQLGUID(),
+			reinterpret_cast<SQLCHAR *>(const_cast<char *>(extLibName.c_str())),
+			extLibName.length(),
+			reinterpret_cast<SQLCHAR *>(const_cast<char *>(pathToPackage.c_str())),
+			pathToPackage.length(),
+			reinterpret_cast<SQLCHAR *>(const_cast<char *>(installDir.c_str())),
+			installDir.length(),
+			&libError,
+			&libErrorLength);
+
+		if (successfulInstall)
+		{
+			EXPECT_EQ(result, SQL_SUCCESS);
+
+			EXPECT_EQ(libError, nullptr);
+			EXPECT_EQ(libErrorLength, 0);
+
+			try
+			{
+				CleanModules(extLibName, moduleName);
+
+				string infoScript = "import pkg_resources; import importlib;import sys\n"
+					"importlib.invalidate_caches();\n"
+					"pkg_resources._initialize_master_working_set()\n"
+					"dist = pkg_resources.get_distribution('" + extLibName + "');\n"
+					"location = dist.location; version = dist.version;";
+
+				bp::exec(infoScript.c_str(), m_mainNamespace);
+				string version = bp::extract<string>(m_mainNamespace["version"]);
+				string location = bp::extract<string>(m_mainNamespace["location"]);
+
+				if (expectedLocation.empty())
+				{
+					expectedLocation = installDir;
+				}
+
+				EXPECT_TRUE(fs::equivalent(location, expectedLocation));
+				EXPECT_EQ(version, expectedVersion);
+
+				// Import the module then delete it.
+				//
+				string importScript = "import " + moduleName + "; del " + moduleName;
+				bp::exec(importScript.c_str(), m_mainNamespace);
+
+				EXPECT_TRUE(successfulImport);
+			}
+			catch (const bp::error_already_set &)
+			{
+				EXPECT_FALSE(successfulImport);
+				string s = PythonTestUtilities::ParsePythonException();
+				cout << s << endl;
+			}
+
+			CleanModules(extLibName, moduleName);
+		}
+		else
+		{
+			EXPECT_EQ(result, SQL_ERROR);
+		}
+	}
+
+	// Name: UninstallAndTest
+	//
+	// Description:
+	//  Uninstall and test a package.
+	//  If there is a different installation of the same package, then uninstalling won't
+	//  fail on import, but the version in our specified dir should be gone.
+	//
+	void ExternalLibraryApiTests::UninstallAndTest(
+		string extLibName,
+		string moduleName,
+		string installDir,
+		bool   otherInstallationExists)
+	{
+		SQLCHAR *libError = nullptr;
+		SQLINTEGER libErrorLength = 0;
+
+		SQLRETURN result = UninstallExternalLibrary(
+			SQLGUID(),
+			reinterpret_cast<SQLCHAR *>(const_cast<char *>(extLibName.c_str())),
+			extLibName.length(),
+			reinterpret_cast<SQLCHAR *>(const_cast<char *>(installDir.c_str())),
+			installDir.length(),
+			&libError,
+			&libErrorLength);
+
+		// Try to find the package - if it successfully imports, 
+		// the uninstall either failed or there is another
+		// installation of the same package in a different location.
+		//
+		try
+		{
+			CleanModules(extLibName, moduleName);
+
+			string infoScript = "import pkg_resources; import importlib;import sys\n"
+				"importlib.invalidate_caches();\n"
+				"pkg_resources._initialize_master_working_set()\n"
+				"dist = pkg_resources.get_distribution('" + extLibName + "');\n"
+				"location = dist.location; version = dist.version;";
+
+			bp::exec(infoScript.c_str(), m_mainNamespace);
+
+			string version = bp::extract<string>(m_mainNamespace["version"]);
+			string location = bp::extract<string>(m_mainNamespace["location"]);
+
+			// If another installation exists, 
+			// then the uninstall should succeed and import should still work
+			//
+			if (otherInstallationExists)
+			{
+				EXPECT_EQ(result, SQL_SUCCESS);
+				EXPECT_EQ(libError, nullptr);
+				EXPECT_EQ(libErrorLength, 0);
+
+				EXPECT_TRUE(fs::exists(location));
+				if (fs::exists(installDir))
+				{
+					EXPECT_FALSE(fs::equivalent(location, installDir));
+				}
+			}
+			else
+			{
+				EXPECT_EQ(result, SQL_ERROR);
+			}
+		}
+		catch (const bp::error_already_set &)
+		{
+			EXPECT_EQ(result, SQL_SUCCESS);
+			EXPECT_EQ(libError, nullptr);
+			EXPECT_EQ(libErrorLength, 0);
+		}
+	}
+
+	// Name: BasicZipInstallTest
+	//
+	// Description:
+	//  Install a ZIP python package.
+	//  This zip package also has some random extra_file.txt, 
+	//  which we should ignore without failure.
+	//
+	TEST_F(ExternalLibraryApiTests, BasicZipInstallTest)
+	{
+		string libName = "testpackageA";
+		fs::path pkgPath = m_packagesPath / "testpackageA-ZIP.zip";
+		string version = "0.0.1";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,
+			version);
+
+		UninstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			m_publicLibraryPath);
+	}
+
+	// Name: BasicWhlInstallTest
+	//
+	// Description:
+	//  Install a WHL python package
+	//
+	TEST_F(ExternalLibraryApiTests, BasicWhlInstallTest)
+	{
+		string libName = "astor";
+		fs::path pkgPath = m_packagesPath / "astor-0.7.1-WHL.zip";
+		string version = "0.7.1";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,
+			version);
+
+		UninstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			m_publicLibraryPath);
+	}
+
+	// Name: BasicTarInstallTest
+	//
+	// Description:
+	//  Install a TAR GZ python package
+	//
+	TEST_F(ExternalLibraryApiTests, BasicTarInstallTest)
+	{
+		// The library name and module name are not the same
+		// We install with "pip install absl-py", but import with "import absl"
+		//
+		string libName = "absl-py";
+		string moduleName = "absl";
+		fs::path pkgPath = m_packagesPath / "absl-py-0.1.13-TAR.zip";
+		string version = "0.1.13";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(libName, moduleName, pkgPath.string(), m_publicLibraryPath, version);
+
+		UninstallAndTest(libName, moduleName, m_publicLibraryPath);
+	}
+
+	// Name: InstallMultipleTest
+	//
+	// Description:
+	//  Install multiple packages of different types without cleaning
+	//
+	TEST_F(ExternalLibraryApiTests, InstallMultipleTest)
+	{
+		vector<string> libNames{ "absl-py", "astor", "testpackageA" };
+		vector<string> moduleNames{ "absl", "astor", "testpackageA" };
+
+		vector<fs::path> pkgPaths{ m_packagesPath / "absl-py-0.1.13-TAR.zip",
+			 m_packagesPath / "astor-0.7.1-WHL.zip",
+			 m_packagesPath / "testpackageA-ZIP.zip" };
+
+		vector<string> versions{ "0.1.13", "0.7.1", "0.0.1" };
+
+		for (size_t i = 0; i < libNames.size(); ++i)
+		{
+			EXPECT_TRUE(fs::exists(pkgPaths[i]));
+			InstallAndTest(libNames[i], moduleNames[i], pkgPaths[i].string(), m_publicLibraryPath, versions[i]);
+		}
+
+		for (size_t i = 0; i < libNames.size(); ++i)
+		{
+			UninstallAndTest(libNames[i], moduleNames[i], m_publicLibraryPath);
+		}
+	}
+
+	// Name: InstallPublicPrivate
+	//
+	// Description:
+	//  Install package to public then private and make sure public is loaded, then private second time
+	//
+	TEST_F(ExternalLibraryApiTests, InstallPublicPrivate)
+	{
+		string libName = "testpackageA";
+
+		fs::path pkgPath;
+		string version;
+
+		// First install in public path
+		//
+		pkgPath = m_packagesPath / "testpackageA-v2-ZIP.zip";
+		version = "0.0.2";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,
+			version);
+
+		// Install same package in different context to see if it is loaded instead of public
+		//
+		pkgPath = m_packagesPath / "testpackageA-ZIP.zip";
+		version = "0.0.1";
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,              // extLibName
+			libName,              // moduleName
+			pkgPath.string(),
+			m_privateLibraryPath, // USE PRIVATE LIB PATH
+			version);
+
+		// Uninstall private then public, make sure we get still get the public after
+		// uninstalling private.
+		//
+		UninstallAndTest(libName, libName, m_privateLibraryPath, true); // otherInstallationExists
+		UninstallAndTest(libName, libName, m_publicLibraryPath, false);
+	}
+
+	// Name: InstallPrivatePublic
+	//
+	// Description:
+	//  Install package to private then public and make sure private is loaded BOTH times
+	//
+	TEST_F(ExternalLibraryApiTests, InstallPrivatePublic)
+	{
+		string libName = "testpackageA";
+
+		fs::path pkgPath;
+		string version;
+
+		// First install in public path
+		//
+		pkgPath = m_packagesPath / "testpackageA-v2-ZIP.zip";
+		version = "0.0.2";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,               // extLibName
+			libName,               // moduleName
+			pkgPath.string(),
+			m_privateLibraryPath,  // USE PRIVATE LIB PATH
+			version,
+			m_privateLibraryPath); // Expected location
+
+		// Install same package in different context
+		//
+		pkgPath = m_packagesPath / "testpackageA-ZIP.zip";
+		version = "0.0.1";
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		// We install this package into the PUBLIC scope, but since we have already installed a package into PRIVATE,
+		// we should still be loading PRIVATE over PUBLIC.
+		//
+		InstallAndTest(
+			libName,               // extLibName
+			libName,               // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,   // Install to the public path
+			"0.0.2",               // Expect that we are still importing the private version and location
+			m_privateLibraryPath); // Expected location
+
+		// Uninstall private then public, make sure we get still get the public after
+		// uninstalling private.
+		//
+		UninstallAndTest(libName, libName, m_privateLibraryPath, true); // otherInstallationExists
+		UninstallAndTest(libName, libName, m_publicLibraryPath, false);
+	}
+
+	// Name: DependencyInstallTest
+	//
+	// Description:
+	//  Install a package that requires a dependency
+	//
+	TEST_F(ExternalLibraryApiTests, DependencyInstallTest)
+	{
+		// telemetry depends on statsd - install dep first
+		//
+		string libName = "statsd";
+		fs::path pkgPath = m_packagesPath / "statsd-3.3.0-WHL.zip";
+		string version = "3.3.0";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,
+			version,
+			m_publicLibraryPath, // expectedLocation
+			true,                // successfulInstall
+			true                 // successfulImport
+		);
+
+		// Install telemetry now that the dependency is installed.
+		// Here we try installing the top package into private, which will get the dependency
+		// from public.
+		//
+		libName = "telemetry";
+		pkgPath = m_packagesPath / "telemetry-0.3.2-TAR.zip";
+		version = "0.3.2";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,              // extLibName
+			libName,              // moduleName
+			pkgPath.string(),
+			m_privateLibraryPath, // expectedLocation
+			version);
+
+		// Clean statsd (which was imported by telemetry) 
+		// from the imported modules to not interfere with later tests.
+		//
+		CleanModules(
+			"statsd",  // extLibName
+			"statsd"); // moduleName
+	}
+
+	// Name: UninstallNonexistentTest
+	//
+	// Description:
+	//  Uninstall nonexistent package. pip should automatically passthrough successfully.
+	//
+	TEST_F(ExternalLibraryApiTests, UninstallNonexistentTest)
+	{
+		string libName = "noPackage";
+
+		UninstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			m_publicLibraryPath
+		);
+	}
+
+	//
+	// Negative Tests
+	//
+
+	// Name: NoSetupPackageInstallTest
+	//
+	// Description:
+	//  Try to install a package without setup.py
+	//
+	TEST_F(ExternalLibraryApiTests, NoSetupPackageInstallTest)
+	{
+		string libName = "testpackagebad";
+		fs::path pkgPath = m_packagesPath / "testpackagebad-ZIP.zip";
+		string version = "0.7.1";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,
+			version,
+			m_publicLibraryPath, // expectedLocation
+			false);              // Successful install
+	}
+
+	// Name: NotAZipInstallTest
+	//
+	// Description:
+	//  Try to install a non-zip package
+	//
+	TEST_F(ExternalLibraryApiTests, NotAZipInstallTest)
+	{
+		string libName = "badpackage";
+		fs::path pkgPath = m_packagesPath / "bad-package-ZIP.zip";
+		string version = "0.7.1";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,
+			version,
+			m_publicLibraryPath, // expectedLocation
+			false);              // Successful install
+	}
+
+	// Name: NoPrezipInstallTest
+	//
+	// Description:
+	//  Try to install a whl directly, with no zip over it
+	//  This fails because we expect all our packages to be a zip, with the whl file inside the zip.
+	//  When we get packages from SQL Server, they will not have any special filename, and whl files
+	//  hold information in the name. By requiring a zip on top of the whl, we preserve the name.
+	//
+	TEST_F(ExternalLibraryApiTests, NoPrezipInstallTest)
+	{
+		string libName = "astor";
+		fs::path pkgPath = m_packagesPath / "astor.whl";
+		string version = "0.7.1";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		InstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,
+			version,
+			m_publicLibraryPath, // expectedLocation
+			false);              // Successful install
+	}
+
+	// Name: NoDependencyInstallTest
+	//
+	// Description:
+	//  Install a package that requires a dependency, but without the dependency
+	//
+	TEST_F(ExternalLibraryApiTests, NoDependencyInstallTest)
+	{
+		string libName = "telemetry";
+		fs::path pkgPath = m_packagesPath / "telemetry-0.3.2-TAR.zip";
+		string version = "0.3.2";
+
+		EXPECT_TRUE(fs::exists(pkgPath));
+
+		// Import fails because it depends on statsd
+		//
+		InstallAndTest(
+			libName,             // extLibName
+			libName,             // moduleName
+			pkgPath.string(),
+			m_publicLibraryPath,
+			version,
+			m_publicLibraryPath, // expectedLocation
+			true,                // Successful install
+			false);              // Successful import
+	}
+}

language-extensions/python/test/src/PythonTestUtilities.cpp

@@ -0,0 +1,155 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: PythonTestUtilities.cpp
+//
+// Purpose:
+//  Utility functions for Python Tests
+//
+//*************************************************************************************************
+
+#include "PythonTestUtilities.h"
+
+using namespace std;
+namespace bp = boost::python;
+
+// Name: PythonTestUtilities::ParsePythonException
+//
+// Description:
+//  Parses the value of the active python exception.
+//  Type, value, and traceback are in separate pointers.
+//
+// Returns:
+//  String version of the python error
+//
+string PythonTestUtilities::ParsePythonException()
+{
+	PyObject *pType = NULL;
+	PyObject *pValue = NULL;
+	PyObject *pTraceback = NULL;
+
+	// Fetch the exception info from the Python C API
+	//
+	PyErr_Fetch(&pType, &pValue, &pTraceback);
+
+	// Fallback error
+	//
+	string ret("Unfetchable Python error");
+
+	// If the fetch got a type pointer, parse the type into the exception string
+	//
+	if (pType != NULL)
+	{
+		string type = ExtractString(pType);
+
+		// If a valid string extraction is available, use it
+		// otherwise use fallback string
+		//
+		if (type.empty())
+		{
+			ret = "Unknown exception type";
+		}
+		else
+		{
+			ret = type;
+		}
+	}
+
+	// Do the same for the exception value (the stringification of the exception)
+	//
+	if (pValue != NULL)
+	{
+		string value = ExtractString(pValue);
+
+		if (value.empty())
+		{
+			ret += string(": Unparseable Python error: ");
+		}
+		else
+		{
+			ret += ": " + value;
+		}
+	}
+
+	// Parse lines from the traceback using the Python traceback module
+	//
+	if (pTraceback != NULL)
+	{
+		bp::handle<> handleTrace(pTraceback);
+
+		// Load the traceback module and the format_tb function
+		//
+		bp::object traceback(bp::import("traceback"));
+		bp::object format_tb(traceback.attr("format_tb"));
+
+		// Call format_tb to get a list of traceback strings
+		//
+		bp::object traceList(format_tb(handleTrace));
+
+		// Join the traceback strings into a single string
+		//
+		bp::object tracePyStr(bp::str("\n").join(traceList));
+
+		// Extract the string, check the extraction, and fallback if necessary
+		//
+		string trace = ExtractString(tracePyStr);
+
+		if (trace.empty())
+		{
+			ret += string(": Unparseable Python traceback");
+		}
+		else
+		{
+			ret += ": " + trace;
+		}
+	}
+
+	return ret;
+}
+
+// Name: PythonExtensionUtils::ExtractString
+//
+// Description:
+//  Extract the string from a PyObject, then DECREFS the PyObject.
+//  NOTE: This function STEALS the reference and destroys it.
+//
+// Returns:
+//  String inside the PyObject
+//
+string PythonTestUtilities::ExtractString(PyObject *pObj)
+{
+	bp::handle<> handle(pObj);
+
+	return ExtractString(bp::object(handle));
+}
+
+// Name: PythonExtensionUtils::ExtractString
+//
+// Description:
+//  Extract the string from a boost::python object
+//
+// Returns:
+//  String inside the boost::python object
+//
+string PythonTestUtilities::ExtractString(bp::object handle)
+{
+	string ret;
+	bp::str pyStr(handle);
+
+	// Extract the string from the boost::python object
+	//
+	bp::extract<string> extracted(pyStr);
+
+	// If a valid string extraction is available, use it
+	// otherwise return empty string
+	//
+	if (extracted.check())
+	{
+		ret = extracted();
+	}
+
+	return ret;
+}

language-extensions/python/test/src/main.cpp

@@ -0,0 +1,28 @@
+//*************************************************************************************************
+// Copyright (C) Microsoft Corporation.
+// Distributed under the Boost Software License, Version 1.0.
+// (See accompanying file LICENSE_1_0.txt or copy at
+// https://www.boost.org/LICENSE_1_0.txt)
+//
+// @File: main.cpp
+//
+// Purpose:
+//  Initialize gtest and run all Python extension tests
+//
+//*************************************************************************************************
+#include <gtest/gtest.h>
+
+int main(int argc, const char **argv)
+{
+	// banner
+	//
+	std::cout << "Running PythonExtension C++ unit tests.\n";
+
+	// First, initiate Google Test framework - this will remove
+	// framework-specific parameters from argc and argv
+	//
+	::testing::InitGoogleTest(&argc, const_cast<char**>(argv));
+	int rc = RUN_ALL_TESTS();
+
+	return rc;
+}