Added examples as e2e tests
This commit is contained in:
Родитель
756e27a645
Коммит
bc2c4f3a82
Двоичные данные
Documentation/Documents/PerformanceChart.png
Двоичные данные
Documentation/Documents/PerformanceChart.png
Двоичный файл не отображается.
До Ширина: | Высота: | Размер: 35 KiB После Ширина: | Высота: | Размер: 34 KiB |
|
@ -17,8 +17,8 @@ precision = "float"
|
|||
modelPath = "$ModelDir$/01_OneHidden"
|
||||
ndlMacros = "$ConfigDir$/Macros.ndl"
|
||||
|
||||
# comment the following line to write logs to the console
|
||||
stderr = "$OutputDir$/01_OneHidden_out"
|
||||
# uncomment the following line to write logs to a file
|
||||
# stderr = "$OutputDir$/01_OneHidden_out"
|
||||
traceLevel=1
|
||||
numMBsToShowResult=500
|
||||
|
||||
|
|
|
@ -17,8 +17,8 @@ precision = "float"
|
|||
modelPath = "$ModelDir$/02_Convolution"
|
||||
ndlMacros = "$ConfigDir$/Macros.ndl"
|
||||
|
||||
# comment the following line to write logs to the console
|
||||
stderr = "$OutputDir$/02_Convolution_out"
|
||||
# uncomment the following line to write logs to a file
|
||||
# stderr = "$OutputDir$/02_Convolution_out"
|
||||
traceLevel=1
|
||||
numMBsToShowResult=500
|
||||
|
||||
|
|
|
@ -17,8 +17,8 @@ precision = "float"
|
|||
modelPath = "$ModelDir$/03_ConvBatchNorm"
|
||||
ndlMacros = "$ConfigDir$/Macros.ndl"
|
||||
|
||||
# comment the following line to write logs to the console
|
||||
stderr = "$OutputDir$/03_ConvBatchNorm_out"
|
||||
# uncomment the following line to write logs to a file
|
||||
# stderr = "$OutputDir$/03_ConvBatchNorm_out"
|
||||
|
||||
#######################################
|
||||
# TRAINING CONFIG #
|
||||
|
|
|
@ -4,10 +4,10 @@
|
|||
|
||||
|Data: |The MNIST database (http://yann.lecun.com/exdb/mnist/) of handwritten digits.
|
||||
|:---------|:---
|
||||
|Purpose: |This example demonstrates usage of the NDL (Network Description Language) to define networks.
|
||||
|Network: |NDLNetworkBuilder, simple feed forward and convolutional networks, cross entropy with softmax.
|
||||
|Training: |Stochastic gradient descent both with and without momentum.
|
||||
|Comments: |There are two config files, details are provided below.
|
||||
|Purpose |This example demonstrates usage of the NDL (Network Description Language) to define networks.
|
||||
|Network |NDLNetworkBuilder, simple feed forward and convolutional networks, cross entropy with softmax.
|
||||
|Training |Stochastic gradient descent both with and without momentum.
|
||||
|Comments |There are two config files, details are provided below.
|
||||
|
||||
## Running the example
|
||||
|
||||
|
|
|
@ -21,7 +21,9 @@ precision = "float"
|
|||
traceLevel = 1
|
||||
modelPath = "$ModelDir$/multigpu.dnn"
|
||||
outputNodeNames = ScaledLogLikelihood
|
||||
stderr = "$OutputDir$/DemoOut"
|
||||
|
||||
# uncomment the following line to write logs to a file
|
||||
# stderr = "$OutputDir$/DemoOut"
|
||||
parallelTrain = true
|
||||
|
||||
#######################################
|
||||
|
|
|
@ -4,7 +4,7 @@ This folder contains demos and examples to get started with CNTK.
|
|||
The examples are structured by topic into Image, Speech, Text and Other.
|
||||
The individual folders contain on the first level at least one self-contained example,
|
||||
which cover different types of networks including FF, CNN, RNN and LSTM.
|
||||
Further examples for for each category are provided in the corresponding Miscellaneous subfolder.
|
||||
Further examples for each category are provided in the corresponding Miscellaneous subfolder.
|
||||
Each folder contains a Readme file that explains how to run the example on Windows and Linux.
|
||||
How to run the examples on Philly (https://philly) is explained in the Philly portal wiki
|
||||
(Philly is an internal GPU cluster for Microsoft production runs).
|
||||
|
|
|
@ -7,6 +7,8 @@ The data uses the format required by the HTKMLFReader. For details please refer
|
|||
The [AN4 dataset](http://www.speech.cs.cmu.edu/databases/an4) is a part of CMU audio databases.
|
||||
This modified version of dataset is distributed under the terms of a AN4 license which can be found in 'AdditionalFiles/AN4.LICENSE.html'
|
||||
|
||||
See License.md in the root level folder of the CNTK repository for full license information.
|
||||
|
||||
## Overview
|
||||
|
||||
|Data: |Speech data from the CMU Audio Database aka AN4 (http://www.speech.cs.cmu.edu/databases/an4)
|
||||
|
|
|
@ -27,9 +27,9 @@ numCPUThreads = 1
|
|||
confVocabSize = 10000
|
||||
confClassSize = 50
|
||||
|
||||
trainFile = "ptb.train.cntk.txt"
|
||||
validFile = "ptb.valid.cntk.txt"
|
||||
testFile = "ptb.test.cntk.txt"
|
||||
trainFile = "ptb.train.txt"
|
||||
validFile = "ptb.valid.txt"
|
||||
testFile = "ptb.test.txt"
|
||||
|
||||
writeWordAndClassInfo = [
|
||||
action = "writeWordAndClass"
|
||||
|
|
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
|
@ -2,23 +2,26 @@
|
|||
|
||||
## License
|
||||
|
||||
Note: The data is not checked into the repository currently since a license is required for the penn treebank data.
|
||||
CNTK distribution contains a subset of the data of The Penn Treebank Project (https://www.cis.upenn.edu/~treebank/):
|
||||
|
||||
Marcus, Mitchell, Beatrice Santorini, and Mary Ann Marcinkiewicz. Treebank-2 LDC95T7. Web Download. Philadelphia: Linguistic Data Consortium, 1995.
|
||||
|
||||
See License.md in the root level folder of the CNTK repository for full license information.
|
||||
|
||||
## Overview
|
||||
|
||||
|Data: |The Penn Treebank Project (https://www.cis.upenn.edu/~treebank/) annotates naturally-occuring text for linguistic structure .
|
||||
|Data |The Penn Treebank Project (https://www.cis.upenn.edu/~treebank/) annotates naturally-occuring text for linguistic structure .
|
||||
|:---------|:---|
|
||||
|Purpose: |Showcase how to train a recurrent network for text data.
|
||||
|Network: |SimpleNetworkBuilder for recurrent network with two hidden layers.
|
||||
|Training: |Stochastic gradient descent with adjusted learning rate.
|
||||
|Comments: |The provided configuration file performs class based RNN training.
|
||||
|Purpose |Showcase how to train a recurrent network for text data.
|
||||
|Network |SimpleNetworkBuilder for recurrent network with two hidden layers.
|
||||
|Training |Stochastic gradient descent with adjusted learning rate.
|
||||
|Comments |The provided configuration file performs class based RNN training.
|
||||
|
||||
## Running the example
|
||||
|
||||
### Getting the data
|
||||
|
||||
The data is not checked into the repository currently since a license is required for the penn treebank data.
|
||||
Please visit https://www.cis.upenn.edu/~treebank/
|
||||
The data for this example is already contained in the folder PennTreebank/Data/.
|
||||
|
||||
### Setup
|
||||
|
||||
|
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1,10 @@
|
|||
#!/bin/bash
|
||||
|
||||
. $TEST_ROOT_DIR/run-test-common
|
||||
|
||||
ConfigDir=$TEST_DIR/../../../../../../Examples/Image/MNIST/Config
|
||||
CleanDataDir=$(cygpath -aw $DataDir)
|
||||
|
||||
# cntkrun <CNTK config file name> <additional CNTK args>
|
||||
cntkrun 01_OneHidden.config "train=[reader=[file=$CleanDataDir/Train.txt]] test=[reader=[file=$CleanDataDir/Test.txt]] train=[SGD=[maxEpochs=1]] train=[SGD=[epochSize=100]] train=[reader=[randomize=none]]" || exit $?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
dataDir: ../../../../Image/Data
|
||||
|
||||
tags:
|
||||
# TODO: create new leg for Example-related e2e tests?
|
||||
- examples (flavor=='debug')
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
patterns:
|
||||
- ^COMPLETED
|
||||
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1,10 @@
|
|||
#!/bin/bash
|
||||
|
||||
. $TEST_ROOT_DIR/run-test-common
|
||||
|
||||
ConfigDir=$TEST_DIR/../../../../../../Examples/Image/MNIST/Config
|
||||
CleanDataDir=$(cygpath -aw $DataDir)
|
||||
|
||||
# cntkrun <CNTK config file name> <additional CNTK args>
|
||||
cntkrun 02_Convolution.config "train=[reader=[file=$CleanDataDir/Train.txt]] test=[reader=[file=$CleanDataDir/Test.txt]] train=[SGD=[maxEpochs=1]] train=[reader=[randomize=none]]" || exit $?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
dataDir: ../../../../Image/Data
|
||||
|
||||
tags:
|
||||
# TODO: create new leg for Example-related e2e tests?
|
||||
- examples (flavor=='release')
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
patterns:
|
||||
- ^COMPLETED
|
||||
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1,10 @@
|
|||
#!/bin/bash
|
||||
|
||||
. $TEST_ROOT_DIR/run-test-common
|
||||
|
||||
ConfigDir=$TEST_DIR/../../../../../../Examples/Image/MNIST/Config
|
||||
CleanDataDir=$(cygpath -aw $DataDir)
|
||||
|
||||
# cntkrun <CNTK config file name> <additional CNTK args>
|
||||
cntkrun 03_ConvBatchNorm.config "train=[reader=[file=$CleanDataDir/Train.txt]] test=[reader=[file=$CleanDataDir/Test.txt]] train=[SGD=[maxEpochs=1]] train=[SGD=[epochSize=100]] train=[reader=[randomize=none]]" || exit $?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
dataDir: ../../../../Image/Data
|
||||
|
||||
tags:
|
||||
# TODO: create new leg for Example-related e2e tests?
|
||||
- examples (flavor=='debug')
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
patterns:
|
||||
- ^COMPLETED
|
||||
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/bash
|
||||
|
||||
. $TEST_ROOT_DIR/run-test-common
|
||||
|
||||
ConfigDir=$TEST_DIR/../../../../../../Examples/Other/Simple2d/Config
|
||||
|
||||
# cntkrun <CNTK config file name> <additional CNTK args>
|
||||
cntkrun Multigpu.config "Simple_Demo_Train=[SGD=[maxEpochs=1]] Simple_Demo_Train=[SGD=[epochSize=100]] Simple_Demo_Train=[reader=[randomize=none]]" || exit $?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
dataDir: ../../../../../../Examples/Other/Simple2d/Data
|
||||
|
||||
tags:
|
||||
# TODO: create new leg for Example-related e2e tests?
|
||||
- examples (flavor=='debug') and (device=='gpu') and (build_sku == '1bitsgd')
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
patterns:
|
||||
- ^COMPLETED
|
||||
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/bash
|
||||
|
||||
. $TEST_ROOT_DIR/run-test-common
|
||||
|
||||
ConfigDir=$TEST_DIR/../../../../../../Examples/Other/Simple2d/Config
|
||||
|
||||
# cntkrun <CNTK config file name> <additional CNTK args>
|
||||
cntkrun Simple.config "Simple_Demo_Train=[SGD=[maxEpochs=1]] Simple_Demo_Train=[SGD=[epochSize=100]] Simple_Demo_Train=[reader=[randomize=none]]" || exit $?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
dataDir: ../../../../../../Examples/Other/Simple2d/Data
|
||||
|
||||
tags:
|
||||
# TODO: create new leg for Example-related e2e tests?
|
||||
- examples (flavor=='debug') & (device=='cpu')
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
patterns:
|
||||
- ^COMPLETED
|
||||
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/bash
|
||||
|
||||
. $TEST_ROOT_DIR/run-test-common
|
||||
|
||||
ConfigDir=$TEST_DIR/../../../../../../Examples/Speech/AN4/Config
|
||||
|
||||
# cntkrun <CNTK config file name> <additional CNTK args>
|
||||
cntkrun FeedForward.config "speechTrain=[SGD=[maxEpochs=1]] speechTrain=[SGD=[epochSize=2048]]" || exit $?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
dataDir: ../../../../../../Examples/Speech/AN4/Data
|
||||
|
||||
tags:
|
||||
# TODO: create new leg for Example-related e2e tests?
|
||||
- examples (flavor=='release') and (build_sku == '1bitsgd')
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
patterns:
|
||||
- ^COMPLETED
|
||||
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/bash
|
||||
|
||||
. $TEST_ROOT_DIR/run-test-common
|
||||
|
||||
ConfigDir=$TEST_DIR/../../../../../../Examples/Speech/AN4/Config
|
||||
|
||||
# cntkrun <CNTK config file name> <additional CNTK args>
|
||||
cntkrun LSTM-NDL.config "speechTrain=[SGD=[maxEpochs=1]] speechTrain=[SGD=[epochSize=64]] parallelTrain=false" || exit $?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
dataDir: ../../../../../../Examples/Speech/AN4/Data
|
||||
|
||||
tags:
|
||||
# TODO: create new leg for Example-related e2e tests?
|
||||
- examples (flavor=='release') and (build_sku == '1bitsgd')
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
patterns:
|
||||
- ^COMPLETED
|
||||
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1 @@
|
|||
COMPLETED
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/bash
|
||||
|
||||
. $TEST_ROOT_DIR/run-test-common
|
||||
|
||||
ConfigDir=$TEST_DIR/../../../../../../Examples/Text/PennTreebank/Config
|
||||
|
||||
# cntkrun <CNTK config file name> <additional CNTK args>
|
||||
cntkrun rnn.config "train=[SGD=[maxEpochs=1]] train=[epochSize=4096]]" || exit $?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
dataDir: ../../../../../../Examples/Text/PennTreebank/Data
|
||||
|
||||
tags:
|
||||
# TODO: create new leg for Example-related e2e tests?
|
||||
- examples (flavor=='release') and (device=='gpu')
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
patterns:
|
||||
- ^COMPLETED
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
# End-to-end tests
|
||||
|
||||
How to run and debug end-to-end tests is described in the CNTK wiki on github at
|
||||
|
||||
https://github.com/Microsoft/CNTK/wiki/How-to-Test#end-to-end-tests
|
|
@ -10,11 +10,11 @@
|
|||
#
|
||||
# ---- Adding the tests: -------
|
||||
# File system organization:
|
||||
# Each test suite (e.g. Speech) has its own directory inside Tests
|
||||
# Each test suite (e.g. Speech) has its own directory inside EndToEndTests
|
||||
# Each test (e.g. QuickE2E) has its own directory within test suite
|
||||
#
|
||||
# Each test directory has a following components:
|
||||
# - testcases.yml - main test confuguration file, whcih defines all test cases
|
||||
# - testcases.yml - main test configuration file, which defines all test cases
|
||||
# - run-test - (run-test) script
|
||||
# - baseline*.txt - baseline files whith a captured expected output of run-test script
|
||||
#
|
||||
|
@ -40,7 +40,7 @@
|
|||
# .....
|
||||
#
|
||||
# ----- pattern language --------
|
||||
# Multpile patterns of the same testcase are matching a *single* line of text
|
||||
# Multiple patterns of the same testcase are matching a *single* line of text
|
||||
# Pattern is essentiually a substring which has to be found in a line
|
||||
# if pattern starts with ^ then matching is constrained to look only at the beginning of the line
|
||||
#
|
||||
|
@ -544,6 +544,8 @@ def runCommand(args):
|
|||
if len(args.test) > 0:
|
||||
testsToRun = []
|
||||
for name in args.test:
|
||||
if name[len(name)-1] == '/':
|
||||
name = name[:-1]
|
||||
if name.lower() in Test.allTestsIndexedByFullName:
|
||||
testsToRun.append(Test.allTestsIndexedByFullName[name.lower()])
|
||||
else:
|
||||
|
|
|
@ -18,6 +18,7 @@ LogFileName=
|
|||
ConfigDir=$TEST_DIR
|
||||
RunDir=$TEST_RUN_DIR
|
||||
DataDir=$TEST_DATA_DIR
|
||||
OutputDir=$TEST_RUN_DIR
|
||||
|
||||
MPIMode=0
|
||||
MPIArgs=
|
||||
|
@ -66,9 +67,10 @@ cntkrun()
|
|||
ConfigDir=$(cygpath -aw $ConfigDir)
|
||||
RunDir=$(cygpath -aw $RunDir)
|
||||
DataDir=$(cygpath -aw $DataDir)
|
||||
OutputDir=$(cygpath -aw $OutputDir)
|
||||
fi
|
||||
|
||||
CNTKArgs="configFile=$ConfigDir/$configFileName currentDirectory=$DataDir RunDir=$RunDir DataDir=$DataDir ConfigDir=$ConfigDir DeviceId=$CNTKDeviceId $additionalCNTKArgs"
|
||||
CNTKArgs="configFile=$ConfigDir/$configFileName currentDirectory=$DataDir RunDir=$RunDir DataDir=$DataDir ConfigDir=$ConfigDir OutputDir=$OutputDir DeviceId=$CNTKDeviceId $additionalCNTKArgs"
|
||||
if [ "$LogFileName" != "" ]; then
|
||||
CNTKArgs="$CNTKArgs stderr=$RunDir/$LogFileName"
|
||||
fi
|
||||
|
|
|
@ -0,0 +1,9 @@
|
|||
# Unit tests and end-to-end tests
|
||||
|
||||
The unit tests and end-to-end tests provided in this folder are run as part
|
||||
of the checkin workflow on the build server. You can run all tests locally
|
||||
to verify that your code changes didn't break any test.
|
||||
|
||||
How to run unit tests and end-to-end test is described in the CNTK wiki on github at
|
||||
|
||||
https://github.com/Microsoft/CNTK/wiki/How-to-Test
|
Загрузка…
Ссылка в новой задаче