merging from master

Examples/Image/Classification/ConvNet/ConvNet_CIFAR10.cntk

@@ -17,7 +17,7 @@ TrainConvNet = {
         imageShape = 32:32:3
         labelDim = 10
 
-		featMean = 128
+        featMean = 128
         featScale = 1/256
         Normalize{m,f} = x => f .* (x - m)
 

Examples/Image/Classification/ConvNet/README.md

@@ -15,7 +15,7 @@
 
 we use the MNIST and CIFAR-10 datasets to demonstrate how to train a `convolutional neural network (CNN)`. CNN has been one of the most popular neural networks for image-related tasks. A very well-known early work on CNN is the [LeNet](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf). In 2012 Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton won the ILSVRC-2012 competition using a [CNN architecture](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf). And most state-of-the-art neural networks on image classification tasks today adopts a modified CNN architecture, such as [VGG](../VGG), [GoogLeNet](../GoogLeNet), [ResNet](../ResNet), etc.
 
-MNIST and CIFAR-10 dataset is not included in the CNTK distribution but can be easily downloaded and converted by following the instructions in [DataSets/MNIST](../../DataSets/MNIST) and [DataSets/CIFAR10](../../DataSets/CIFAR10). We recommend you to keep the downloaded data in the respective folder while downloading, as the configuration files in this folder assumes that by default.
+MNIST and CIFAR-10 dataset is not included in the CNTK distribution but can be easily downloaded and converted by following the instructions in [DataSets/MNIST](../../DataSets/MNIST) and [DataSets/CIFAR-10](../../DataSets/CIFAR-10). We recommend you to keep the downloaded data in the respective folder while downloading, as the configuration files in this folder assumes that by default.
 
 ## Details
 
@@ -48,4 +48,4 @@ Run the example from the current folder using:
 
 `cntk configFile=ConvNet_CIFAR10_DataAug.cntk`
 
-As seen in the cntk configuration file [ConvNet_CIFAR10_DataAug.cntk](./ConvNet_CIFAR10_DataAug.cntk), we use a fix crop ratio of `0.8` (effectively we only do translation transform without scaling), and scale the image to `32x32` pixels for training. The accuracy of the network on test data is `14.21%`, which is a lot better than the previous model.
+As seen in the cntk configuration file [ConvNet_CIFAR10_DataAug.cntk](./ConvNet_CIFAR10_DataAug.cntk), we use a fix crop ratio of `0.8` and scale the image to `32x32` pixels for training. Since all training images are pre-padded to `40x40` pixels, effectively we only perfrom translation transform without scaling. The accuracy of the network on test data is `14.21%`, which is a lot better than the previous model.

Examples/Image/Regression/README.md

@@ -13,13 +13,13 @@
 
 ### Getting the data
 
-we use the CIFAR-10 dataset to demonstrate how to perform regression on images. CIFAR-10 dataset is not included in the CNTK distribution but can be easily downloaded and converted by following the instructions in [DataSets/CIFAR10](../DataSets/CIFAR10). We recommend you to keep the downloaded data in the respective folder while downloading, as the configuration files in this folder assumes that by default.
+we use the CIFAR-10 dataset to demonstrate how to perform regression on images. CIFAR-10 dataset is not included in the CNTK distribution but can be easily downloaded and converted by following the instructions in [DataSets/CIFAR-10](../DataSets/CIFAR-10). We recommend you to keep the downloaded data in the respective folder while downloading, as the configuration files in this folder assumes that by default.
 
 ## Details
 
 ### RegrSimple_CIFAR10.cntk
 
-In this example, we set up a very simple task to have a neural network predict the average RGB values of images normalized to [0,1). To generate the ground truth labels for this regression task, the CIFAR-10 installation script in [DataSets/CIFAR10](../DataSets/CIFAR10) will generate two additional files, `train_regrLabels.txt` and `test_regrLabels.txt`, for train and test respectively.
+In this example, we set up a very simple task to have a neural network predict the average RGB values of images normalized to [0,1). To generate the ground truth labels for this regression task, the CIFAR-10 installation script in [DataSets/CIFAR-10](../DataSets/CIFAR-10) will generate two additional files, `train_regrLabels.txt` and `test_regrLabels.txt`, for train and test respectively.
 
 Run the example from the current folder using:
 
@@ -27,4 +27,4 @@ Run the example from the current folder using:
 
 The network produces root-mean-square error (rmse) of around 0.00098, which indicates that the regression accuracy is very high for this simple task.
 
-A few notes on the cntk configuration file. The network is a linear one without nonlinearity. This is intended as we know that computing the average RGB values of images is a linear operation. The reader is a composite reader that uses the `ImageReader` to read images and the `CNTKTextFormatReader` to read the regression ground truth labels. The configuration file also demonstrates how to write the network prediction for the test data into a output file.
+You may examine the cntk configuration file [RegrSimple_CIFAR10.cntk](./RegrSimple_CIFAR10.cntk) for more details. Note the network is a linear one without nonlinearity. This is intended as we know that computing the average RGB values of images is a linear operation. The reader is a composite reader that uses the `ImageReader` to read images and the `CNTKTextFormatReader` to read the regression ground truth labels. The configuration file also demonstrates how to write the network prediction for the test data into an output file.

Examples/README.md

@@ -16,5 +16,5 @@ Please refer to the Readme file in the corresponding folder for further details.
 |:------------------------|:-------------------------------------------------|:----------------|
 |Other/Simple2d           | Synthetic 2d data                                | FF (CPU and GPU)
 |Speech/AN4               | Speech data (CMU AN4)                            | FF and LSTM
-|Image/MNIST              | Image data (MNIST handwritten digit recognition) | CNN 
+|Image/GettingStarted     | Image data (MNIST handwritten digit recognition) | CNN 
 |Text/PennTreebank        | Text data (penn treebank)                        | RNN

Makefile

@@ -734,8 +734,11 @@ IMAGE_READER_LIBS += -lopencv_core -lopencv_imgproc -lopencv_imgcodecs
 
 ifdef LIBZIP_PATH
   CPPFLAGS += -DUSE_ZIP
+  #both directories are needed for building libzip
+  INCLUDEPATH += $(LIBZIP_PATH)/include
   INCLUDEPATH += $(LIBZIP_PATH)/lib/libzip/include
   IMAGE_READER_LIBS += -lzip
+  LIBPATH += $(LIBZIP_PATH)/lib
 endif
 
 IMAGEREADER_SRC =\

Source/EvalDll/CNTKEval.cpp

@@ -414,7 +414,7 @@ void CNTKEvalExtended<ElemType>::ForwardPassT(const std::vector<ValueBuffer<Elem
 template<typename ElemType>
 void CNTKEvalExtended<ElemType>::ForwardPass(const Values<ElemType>& inputs, Values<ElemType>& outputs)
 {
-    ForwardPassT(inputs, outputs, false);
+    ForwardPassT(inputs, outputs, true);
 }
 
 template<typename ElemType>
@@ -426,7 +426,7 @@ void CNTKEvalExtended<ElemType>::ForwardPass(const Values<ElemType>& inputs, Val
 template<typename ElemType>
 void CNTKEvalExtended<ElemType>::ForwardPass(const ValueRefs<ElemType>& inputs, ValueRefs<ElemType>& outputs)
 {
-    ForwardPassT(inputs, outputs, false);
+    ForwardPassT(inputs, outputs, true);
 }
 
 template<typename ElemType>

Tests/EndToEndTests/Examples/Image/GettingStarted/01_OneHidden/testcases.yml

@@ -1,10 +1,8 @@
 dataDir: ../../../../../../Examples/Image/DataSets
 
 tags:
-    # In BVT, run Release GPU
     - bvt-e (build_sku=='gpu') and (device=='gpu') and (flavor=='release') 
-    # In Nightly on Linux, additionally run Debug GPU and Release CPU
+    - nightly-e (build_sku=='gpu') and (device=='gpu') and ((flavor=='release') or (os=='linux'))
-    - nightly-e (build_sku=='gpu') and (((device=='gpu') and (flavor=='release')) or (os=='linux' and ((flavor=='debug') ^ (device=='cpu'))))
 
 testCases:
   CNTK Run must be completed:

Tests/EndToEndTests/Examples/Image/GettingStarted/02_OneConv/testcases.yml

@@ -1,10 +1,8 @@
 dataDir: ../../../../../../Examples/Image/DataSets/MNIST
 
 tags:
-    # In BVT, run Release GPU
     - bvt-e (build_sku=='gpu') and (device=='gpu') and (flavor=='release') 
-    # In Nightly on Linux, additionally run Debug GPU and Release CPU
+    - nightly-e (build_sku=='gpu') and (device=='gpu') and ((flavor=='release') or (os=='linux'))
-    - nightly-e (build_sku=='gpu') and (((device=='gpu') and (flavor=='release')) or (os=='linux' and ((flavor=='debug') ^ (device=='cpu'))))
 
 testCases:
   CNTK Run must be completed:

Tests/EndToEndTests/Examples/Image/GettingStarted/03_OneConvDropout/testcases.yml

@@ -1,10 +1,8 @@
 dataDir: ../../../../../../Examples/Image/DataSets
 
 tags:
-    # In BVT, run Release GPU
     - bvt-e (build_sku=='gpu') and (device=='gpu') and (flavor=='release') 
-    # In Nightly on Linux, additionally run Debug GPU and Release CPU
+    - nightly-e (build_sku=='gpu') and (device=='gpu') and ((flavor=='release') or (os=='linux'))
-    - nightly-e (build_sku=='gpu') and (((device=='gpu') and (flavor=='release')) or (os=='linux' and ((flavor=='debug') ^ (device=='cpu'))))
 
 testCases:
   CNTK Run must be completed:

Tests/EndToEndTests/Examples/Image/GettingStarted/05_OneConvRegr/testcases.yml

@@ -1,10 +1,8 @@
 dataDir: ../../../../../../Examples/Image/DataSets
 
 tags:
-    # In BVT, run Release GPU
     - bvt-e (build_sku=='gpu') and (device=='gpu') and (flavor=='release') 
-    # In Nightly on Linux, additionally run Debug GPU and Release CPU
+    - nightly-e (build_sku=='gpu') and (device=='gpu') and ((flavor=='release') or (os=='linux'))
-    - nightly-e (build_sku=='gpu') and (((device=='gpu') and (flavor=='release')) or (os=='linux' and ((flavor=='debug') ^ (device=='cpu'))))
 
 testCases:
   CNTK Run must be completed:

bindings/python/examples/CifarResNet/CifarResNet.py

@@ -171,9 +171,12 @@ def cifar_resnet(base_path, debug_output=False):
 
     # Get minibatches of images to train with and perform model training
     mb_size = 32
-    training_progress_output_freq = 20
+    training_progress_output_freq = 60
     num_mbs = 1000
 
+    if debug_output:
+        training_progress_output_freq = training_progress_output_freq/3
+
     for i in range(0, num_mbs):
         mb = minibatch_source.get_next_minibatch(mb_size)
 
@@ -183,8 +186,7 @@ def cifar_resnet(base_path, debug_output=False):
             features_si].m_data, label_var: mb[labels_si].m_data}
         trainer.train_minibatch(arguments)
 
-        if debug_output:
+        print_training_progress(trainer, i, training_progress_output_freq)
-            print_training_progress(trainer, i, training_progress_output_freq)
 
     test_minibatch_source = create_test_mb_source(feats_stream_name, labels_stream_name,
                     image_height, image_width, num_channels, num_classes, base_path)
@@ -219,4 +221,5 @@ if __name__ == '__main__':
 
     os.chdir(os.path.join(base_path, '..'))
 
-    cifar_resnet(base_path)
+    error = cifar_resnet(base_path)
+    print("Error: %f" % error)

bindings/python/examples/MNIST/SimpleMNIST.py

@@ -68,7 +68,11 @@ def simple_mnist(debug_output=False):
     num_samples_per_sweep = 60000
     num_sweeps_to_train_with = 1
     num_minibatches_to_train = (num_samples_per_sweep * num_sweeps_to_train_with) / minibatch_size
-    training_progress_output_freq = 20
+    training_progress_output_freq = 80
+
+    if debug_output:
+        training_progress_output_freq = training_progress_output_freq/4
+
     for i in range(0, int(num_minibatches_to_train)):
         mb = mb_source.get_next_minibatch(minibatch_size)
 
@@ -78,8 +82,7 @@ def simple_mnist(debug_output=False):
                      label: mb[labels_si].m_data}
         trainer.train_minibatch(arguments)
 
-        if debug_output:
+        print_training_progress(trainer, i, training_progress_output_freq)
-            print_training_progress(trainer, i, training_progress_output_freq)
 
     # Load test data
     try:
@@ -123,4 +126,4 @@ if __name__=='__main__':
     DeviceDescriptor.set_default_device(target_device)
 
     error = simple_mnist()
-    print("test: %f" % error)
+    print("Error: %f" % error)

bindings/python/examples/NumpyInterop/FeedForwardNet.py

@@ -59,15 +59,18 @@ def ffnet(debug_output=False):
     num_sweeps_to_train_with = 2
     num_minibatches_to_train = (
         num_samples_per_sweep * num_sweeps_to_train_with) / minibatch_size
-    training_progress_output_freq = 20
+    training_progress_output_freq = 60
+
+    if debug_output:
+        training_progress_output_freq = training_progress_output_freq/3
+
     for i in range(0, int(num_minibatches_to_train)):
         features, labels = generate_random_data(
             minibatch_size, input_dim, num_output_classes)
         # Specify the mapping of input variables in the model to actual
         # minibatch data to be trained with
         trainer.train_minibatch({input: features, label: labels})
-        if debug_output:
+        print_training_progress(trainer, i, training_progress_output_freq)
-            print_training_progress(trainer, i, training_progress_output_freq)
 
     test_features, test_labels = generate_random_data(
         minibatch_size, input_dim, num_output_classes)
@@ -83,4 +86,4 @@ if __name__ == '__main__':
     DeviceDescriptor.set_default_device(target_device)
 
     error = ffnet()
-    print("test: %f" % error)
+    print("Error: %f" % error)

bindings/python/examples/Sequence2Sequence/Sequence2Sequence.py

@@ -116,7 +116,10 @@ def sequence_to_sequence_translator(debug_output=False):
 
     # Get minibatches of sequences to train with and perform model training
     minibatch_size = 72
-    training_progress_output_freq = 10
+    training_progress_output_freq = 30
+    if debug_output:
+        training_progress_output_freq = training_progress_output_freq/3
+
     while True:
         mb = mb_source.get_next_minibatch(minibatch_size)
         if len(mb) == 0:
@@ -128,10 +131,8 @@ def sequence_to_sequence_translator(debug_output=False):
                      raw_labels: mb[labels_si].m_data}
         trainer.train_minibatch(arguments)
 
-        if debug_output:
+        print_training_progress(trainer, i, training_progress_output_freq)
-            print_training_progress(trainer, i, training_progress_output_freq)
+        i += 1
-
-            i += 1
 
     rel_path = r"../../../../Examples/SequenceToSequence/CMUDict/Data/cmudict-0.7b.test.ctf"
     path = os.path.join(os.path.dirname(os.path.abspath(__file__)), rel_path)
@@ -177,4 +178,4 @@ if __name__ == '__main__':
     DeviceDescriptor.set_default_device(target_device)
 
     error = sequence_to_sequence_translator()
-    print("test: %f" % error)
+    print("Error: %f" % error)

bindings/python/examples/SequenceClassification/SequenceClassification.py

@@ -27,7 +27,7 @@ def LSTM_sequence_classifer_net(input, num_output_classes, embedding_dim, LSTM_d
 
 # Creates and trains a LSTM sequence classification model
 
-def train_sequence_classifier():
+def train_sequence_classifier(debug_output=False):
     input_dim = 2000
     cell_dim = 25
     hidden_dim = 25
@@ -66,6 +66,10 @@ def train_sequence_classifier():
     minibatch_size = 200
     training_progress_output_freq = 10
     i = 0
+
+    if debug_output:
+        training_progress_output_freq = training_progress_output_freq/3
+
     while True:
         mb = mb_source.get_next_minibatch(minibatch_size)
 
@@ -79,7 +83,6 @@ def train_sequence_classifier():
         trainer.train_minibatch(arguments)
 
         print_training_progress(trainer, i, training_progress_output_freq)
-
         i += 1
 
     import copy
@@ -98,4 +101,4 @@ if __name__ == '__main__':
     DeviceDescriptor.set_default_device(target_device)
 
     error, _ = train_sequence_classifier()
-    print("test: %f" % error)
+    print("Error: %f" % error)

bindings/python/examples/test/cifar_resnet_test.py

@@ -11,7 +11,7 @@ from cntk.io import ReaderConfig, ImageDeserializer
 
 from examples.CifarResNet.CifarResNet import cifar_resnet
 
-TOLERANCE_ABSOLUTE = 1E-1
+TOLERANCE_ABSOLUTE = 2E-1
 
 def test_cifar_resnet_error(device_id):
     target_device = DeviceDescriptor.gpu_device(0)

bindings/python/examples/test/sequence_classification_test.py

@@ -9,7 +9,7 @@ from cntk import DeviceDescriptor
 
 from examples.SequenceClassification.SequenceClassification import train_sequence_classifier
 
-TOLERANCE_ABSOLUTE = 1E-2
+TOLERANCE_ABSOLUTE = 1E-1
 
 def test_seq_classification_error(device_id):
     from cntk.utils import cntk_device
@@ -17,6 +17,5 @@ def test_seq_classification_error(device_id):
 
     evaluation_avg, loss_avg = train_sequence_classifier()
 
-    # Temporarily disable the comparison against baseline as it needs to be updated
+    expected_avg = [0.68181, 1.5661]
-    # expected_avg = [0.1595744, 0.35799171]
+    assert np.allclose([evaluation_avg, loss_avg], expected_avg, atol=TOLERANCE_ABSOLUTE)
-    # assert np.allclose([evaluation_avg, loss_avg], expected_avg, atol=TOLERANCE_ABSOLUTE)

bindings/python/examples/test/sequence_to_sequence_test.py

@@ -17,6 +17,5 @@ def test_sequence_to_sequence(device_id):
 
     error = sequence_to_sequence_translator()
 
-    # Temporarily disable the comparison against baseline as it needs to be updated
+    expected_error = 0.8596881547969316
-    # expected_error = 0.758458
+    assert np.allclose(error, expected_error, atol=TOLERANCE_ABSOLUTE)
-    # assert np.allclose(error, expected_error, atol=TOLERANCE_ABSOLUTE)

bindings/python/test.bat

@@ -17,26 +17,11 @@ pytest
 echo(
 popd
 
-pushd examples\CifarResNet
+pushd examples\test
-echo RUNNING Cifar ResNet example...
+echo RUNNING cntk\examples\test tests
-python CifarResNet.py
+pytest
 echo(
 popd
 
-echo RUNNING MNIST feed-forward classifier example...
-python examples\MNIST\SimpleMNIST.py
-echo(
-
-echo RUNNING feed-forward numpy interop example...
-python examples\NumpyInterop\FeedForwardNet.py
-echo(
-
-echo RUNNING sequence-to-sequence example...
-python examples\Sequence2Sequence\Sequence2Sequence.py
-echo(
-
-echo RUNNING sequence classification example...
-python examples\SequenceClassification\SequenceClassification.py
-echo(
 
 endlocal