A library of cross-platform, cross-framework, cross-programming language featurizers (operations in a data processing pipeline) that can be used during Machine Learning training and prediction activities.

Перейти к файлу

Michael Sharp 2dd988cfce Merged PR 5201: Updated Featurizers SimpleSchema Updated SimpleSchema to allow for better mapping of input->output types. Added support for multiple template types. Note: Tuple types aren't supported yet, so the code is generated for RobustScalar but we will need to add that code in manually in the custom section.		2019-10-04 15:28:51 +00:00
CI	Merged PR 5176: Fix build error(artifact download) when triggered by another build(completion)	2019-09-30 18:22:06 +00:00
Libraries/Python/DataPipelines/v1.0/DataPipelines	Merged PR 5081: Added new build configurations, removing unused code	2019-09-10 19:54:09 +00:00
Scripts	Merged PR 4717: Moved CppToJson python code to its own directory	2019-07-12 20:32:44 +00:00
Tools/Nuget/v5.2.0/Windows	Merged PR 5115: Building nuget packages	2019-09-17 18:04:29 +00:00
src	Merged PR 5201: Updated Featurizers SimpleSchema	2019-10-04 15:28:51 +00:00
.gitignore	Merged PR 4976: DateTimeTransformer implemented in ML.NET with assocated C++ wrapper	2019-08-19 16:45:08 +00:00
Activate_custom.py	Initial checkin	2019-05-22 10:10:31 -07:00
Readme.md	Merged PR 4542: Updated Readme.md and ignoring build dirs	2019-06-12 21:13:22 +00:00
ScmHook_custom.py	Initial checkin	2019-05-22 10:10:31 -07:00
Setup.cmd	Initial checkin	2019-05-22 10:10:31 -07:00
Setup.sh	Merged PR 4455: Added CI files	2019-05-24 16:18:38 +00:00
Setup_custom.py	Merged PR 5081: Added new build configurations, removing unused code	2019-09-10 19:54:09 +00:00
__RepositoryId__	Initial checkin	2019-05-22 10:10:31 -07:00
bootstrap.cmd	Merged PR 4902: Pass command line arguments to setup in the bootstrap process	2019-08-05 17:38:36 +00:00
bootstrap.sh	Merged PR 4902: Pass command line arguments to setup in the bootstrap process	2019-08-05 17:38:36 +00:00
bootstrap_impl.py	Merged PR 5081: Added new build configurations, removing unused code	2019-09-10 19:54:09 +00:00

Readme.md

Data Pipelines

Data Pipelines are used to convert from arbitrary data into a matrix/tensor that can be consumed by machine learning algorithms.

Developer Quick Start

Note that these commands should be invoked within the root of the repository.

Linux

Run sudo bootstrap.sh ..

This will need to happen once per machine or after a repository has moved to a different directory. Note that sudo is not necessary when running within a Docker container.
Run source ./Activate.sh x64

This will need to happen each time a new terminal window is opened.

Windows

Run bootstrap.cmd ..

This will need to happen once per machine or after a repository has moved to a different directory.
Run Activate.cmd x64

This will need to happen each time a new terminal window is opened.

Useful Development Commands

The following commands are available within an activated terminal window.

Name	Command	Description
DevEnvScripts	`DevEnvScripts.sh	.cmd`
Builder	`Builder.sh	.cmd`
Tester	`Tester.sh	.cmd`
Formatter	`Formatter.sh	.cmd`

Invoking CMake

Native code is built via CMake. The following commands can be used to build any folder that contains a CMakeLists.txt file. In each example, create a build directory that will contain the generated CMake content and eventual binaries.

Assumed directory structure:

| - <Workspace dir>
    | - CMakeLists.txt
    | - ...
    | - build
        | - <Initially empty>

Within <Workspace Dir>/build run...

[Debug] cmake -G Ninja ..
[Debug with Code Coverage] cmake -G Ninja -D CppCommon_CODE_COVERAGE=ON ..
[Release] cmake -G Ninja -D CMAKE_BUILD_TYPE=Release ..

Once CMake has generated the build files, within <WorkSpace Dir>/build run...

[Build] cmake --build . or ninja -v (if the build generator was Ninja)
[Test] ctest --parallel