merging from master
This commit is contained in:
Thilo Will
Коммит
e90b140f5b
|
|
@ -17,7 +17,7 @@ TrainConvNet = {
|
|||
imageShape = 32:32:3
|
||||
labelDim = 10
|
||||
|
||||
featMean = 128
|
||||
featMean = 128
|
||||
featScale = 1/256
|
||||
Normalize{m,f} = x => f .* (x - m)
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
|
||||
we use the MNIST and CIFAR-10 datasets to demonstrate how to train a `convolutional neural network (CNN)`. CNN has been one of the most popular neural networks for image-related tasks. A very well-known early work on CNN is the [LeNet](http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf). In 2012 Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton won the ILSVRC-2012 competition using a [CNN architecture](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf). And most state-of-the-art neural networks on image classification tasks today adopts a modified CNN architecture, such as [VGG](../VGG), [GoogLeNet](../GoogLeNet), [ResNet](../ResNet), etc.
|
||||
|
||||
MNIST and CIFAR-10 dataset is not included in the CNTK distribution but can be easily downloaded and converted by following the instructions in [DataSets/MNIST](../../DataSets/MNIST) and [DataSets/CIFAR10](../../DataSets/CIFAR10). We recommend you to keep the downloaded data in the respective folder while downloading, as the configuration files in this folder assumes that by default.
|
||||
MNIST and CIFAR-10 dataset is not included in the CNTK distribution but can be easily downloaded and converted by following the instructions in [DataSets/MNIST](../../DataSets/MNIST) and [DataSets/CIFAR-10](../../DataSets/CIFAR-10). We recommend you to keep the downloaded data in the respective folder while downloading, as the configuration files in this folder assumes that by default.
|
||||
|
||||
## Details
|
||||
|
||||
|
|
@ -41,11 +41,11 @@ The network achieves an error rate of `18.51%` after 30 epochs. This is comparab
|
|||
|
||||
### ConvNet_CIFAR10_DataAug.cntk
|
||||
|
||||
The third example uses the same CNN as the previous example, but it improves by adding data augmentation to training. For this purpose, we use the `ImageReader` instead of the `CNTKTextFormatReader` to load the data. The ImageReader currently supports crop, flip, scale, color jittering, and mean subtraction.
|
||||
The third example uses the same CNN as the previous example, but it improves by adding data augmentation to training. For this purpose, we use the `ImageReader` instead of the `CNTKTextFormatReader` to load the data. The ImageReader currently supports crop, flip, scale, color jittering, and mean subtraction.
|
||||
For a reference on image reader and transforms, please check [here](https://github.com/Microsoft/CNTK/wiki/Image-reader).
|
||||
|
||||
Run the example from the current folder using:
|
||||
|
||||
`cntk configFile=ConvNet_CIFAR10_DataAug.cntk`
|
||||
|
||||
As seen in the cntk configuration file [ConvNet_CIFAR10_DataAug.cntk](./ConvNet_CIFAR10_DataAug.cntk), we use a fix crop ratio of `0.8` (effectively we only do translation transform without scaling), and scale the image to `32x32` pixels for training. The accuracy of the network on test data is `14.21%`, which is a lot better than the previous model.
|
||||
As seen in the cntk configuration file [ConvNet_CIFAR10_DataAug.cntk](./ConvNet_CIFAR10_DataAug.cntk), we use a fix crop ratio of `0.8` and scale the image to `32x32` pixels for training. Since all training images are pre-padded to `40x40` pixels, effectively we only perfrom translation transform without scaling. The accuracy of the network on test data is `14.21%`, which is a lot better than the previous model.
|
||||
|
|
|
|||
|
|
@ -13,13 +13,13 @@
|
|||
|
||||
### Getting the data
|
||||
|
||||
we use the CIFAR-10 dataset to demonstrate how to perform regression on images. CIFAR-10 dataset is not included in the CNTK distribution but can be easily downloaded and converted by following the instructions in [DataSets/CIFAR10](../DataSets/CIFAR10). We recommend you to keep the downloaded data in the respective folder while downloading, as the configuration files in this folder assumes that by default.
|
||||
we use the CIFAR-10 dataset to demonstrate how to perform regression on images. CIFAR-10 dataset is not included in the CNTK distribution but can be easily downloaded and converted by following the instructions in [DataSets/CIFAR-10](../DataSets/CIFAR-10). We recommend you to keep the downloaded data in the respective folder while downloading, as the configuration files in this folder assumes that by default.
|
||||
|
||||
## Details
|
||||
|
||||
### RegrSimple_CIFAR10.cntk
|
||||
|
||||
In this example, we set up a very simple task to have a neural network predict the average RGB values of images normalized to [0,1). To generate the ground truth labels for this regression task, the CIFAR-10 installation script in [DataSets/CIFAR10](../DataSets/CIFAR10) will generate two additional files, `train_regrLabels.txt` and `test_regrLabels.txt`, for train and test respectively.
|
||||
In this example, we set up a very simple task to have a neural network predict the average RGB values of images normalized to [0,1). To generate the ground truth labels for this regression task, the CIFAR-10 installation script in [DataSets/CIFAR-10](../DataSets/CIFAR-10) will generate two additional files, `train_regrLabels.txt` and `test_regrLabels.txt`, for train and test respectively.
|
||||
|
||||
Run the example from the current folder using:
|
||||
|
||||
|
|
@ -27,4 +27,4 @@ Run the example from the current folder using:
|
|||
|
||||
The network produces root-mean-square error (rmse) of around 0.00098, which indicates that the regression accuracy is very high for this simple task.
|
||||
|
||||
A few notes on the cntk configuration file. The network is a linear one without nonlinearity. This is intended as we know that computing the average RGB values of images is a linear operation. The reader is a composite reader that uses the `ImageReader` to read images and the `CNTKTextFormatReader` to read the regression ground truth labels. The configuration file also demonstrates how to write the network prediction for the test data into a output file.
|
||||
You may examine the cntk configuration file [RegrSimple_CIFAR10.cntk](./RegrSimple_CIFAR10.cntk) for more details. Note the network is a linear one without nonlinearity. This is intended as we know that computing the average RGB values of images is a linear operation. The reader is a composite reader that uses the `ImageReader` to read images and the `CNTKTextFormatReader` to read the regression ground truth labels. The configuration file also demonstrates how to write the network prediction for the test data into an output file.
|
||||
|
|
|
|||
|
|
@ -16,5 +16,5 @@ Please refer to the Readme file in the corresponding folder for further details.
|
|||
|:------------------------|:-------------------------------------------------|:----------------|
|
||||
|Other/Simple2d | Synthetic 2d data | FF (CPU and GPU)
|
||||
|Speech/AN4 | Speech data (CMU AN4) | FF and LSTM
|
||||
|Image/MNIST | Image data (MNIST handwritten digit recognition) | CNN
|
||||
|Image/GettingStarted | Image data (MNIST handwritten digit recognition) | CNN
|
||||
|Text/PennTreebank | Text data (penn treebank) | RNN
|
||||
|
|
|
|||
3
Makefile
3
Makefile
|
|
@ -734,8 +734,11 @@ IMAGE_READER_LIBS += -lopencv_core -lopencv_imgproc -lopencv_imgcodecs
|
|||
|
||||
ifdef LIBZIP_PATH
|
||||
CPPFLAGS += -DUSE_ZIP
|
||||
#both directories are needed for building libzip
|
||||
INCLUDEPATH += $(LIBZIP_PATH)/include
|
||||
INCLUDEPATH += $(LIBZIP_PATH)/lib/libzip/include
|
||||
IMAGE_READER_LIBS += -lzip
|
||||
LIBPATH += $(LIBZIP_PATH)/lib
|
||||
endif
|
||||
|
||||
IMAGEREADER_SRC =\
|
||||
|
|
|
|||
|
|
@ -414,7 +414,7 @@ void CNTKEvalExtended<ElemType>::ForwardPassT(const std::vector<ValueBuffer<Elem
|
|||
template<typename ElemType>
|
||||
void CNTKEvalExtended<ElemType>::ForwardPass(const Values<ElemType>& inputs, Values<ElemType>& outputs)
|
||||
{
|
||||
ForwardPassT(inputs, outputs, false);
|
||||
ForwardPassT(inputs, outputs, true);
|
||||
}
|
||||
|
||||
template<typename ElemType>
|
||||
|
|
@ -426,7 +426,7 @@ void CNTKEvalExtended<ElemType>::ForwardPass(const Values<ElemType>& inputs, Val
|
|||
template<typename ElemType>
|
||||
void CNTKEvalExtended<ElemType>::ForwardPass(const ValueRefs<ElemType>& inputs, ValueRefs<ElemType>& outputs)
|
||||
{
|
||||
ForwardPassT(inputs, outputs, false);
|
||||
ForwardPassT(inputs, outputs, true);
|
||||
}
|
||||
|
||||
template<typename ElemType>
|
||||
|
|
|
|||
|
|
@ -1,10 +1,8 @@
|
|||
dataDir: ../../../../../../Examples/Image/DataSets
|
||||
|
||||
tags:
|
||||
# In BVT, run Release GPU
|
||||
- bvt-e (build_sku=='gpu') and (device=='gpu') and (flavor=='release')
|
||||
# In Nightly on Linux, additionally run Debug GPU and Release CPU
|
||||
- nightly-e (build_sku=='gpu') and (((device=='gpu') and (flavor=='release')) or (os=='linux' and ((flavor=='debug') ^ (device=='cpu'))))
|
||||
- nightly-e (build_sku=='gpu') and (device=='gpu') and ((flavor=='release') or (os=='linux'))
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
|
|
@ -35,4 +33,4 @@ testCases:
|
|||
patterns:
|
||||
- "Final Results: Minibatch[{{integer}}-{{integer}}]"
|
||||
# - errs = {{float,tolerance=1%}}% * {{integer}}
|
||||
# - ce = {{float,tolerance=1%}} * {{integer}}
|
||||
# - ce = {{float,tolerance=1%}} * {{integer}}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,8 @@
|
|||
dataDir: ../../../../../../Examples/Image/DataSets/MNIST
|
||||
|
||||
tags:
|
||||
# In BVT, run Release GPU
|
||||
- bvt-e (build_sku=='gpu') and (device=='gpu') and (flavor=='release')
|
||||
# In Nightly on Linux, additionally run Debug GPU and Release CPU
|
||||
- nightly-e (build_sku=='gpu') and (((device=='gpu') and (flavor=='release')) or (os=='linux' and ((flavor=='debug') ^ (device=='cpu'))))
|
||||
- nightly-e (build_sku=='gpu') and (device=='gpu') and ((flavor=='release') or (os=='linux'))
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
|
|
@ -35,4 +33,4 @@ testCases:
|
|||
patterns:
|
||||
- "Final Results: Minibatch[{{integer}}-{{integer}}]"
|
||||
# - errs = {{float,tolerance=1%}}% * {{integer}}
|
||||
# - ce = {{float,tolerance=1%}} * {{integer}}
|
||||
# - ce = {{float,tolerance=1%}} * {{integer}}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,8 @@
|
|||
dataDir: ../../../../../../Examples/Image/DataSets
|
||||
|
||||
tags:
|
||||
# In BVT, run Release GPU
|
||||
- bvt-e (build_sku=='gpu') and (device=='gpu') and (flavor=='release')
|
||||
# In Nightly on Linux, additionally run Debug GPU and Release CPU
|
||||
- nightly-e (build_sku=='gpu') and (((device=='gpu') and (flavor=='release')) or (os=='linux' and ((flavor=='debug') ^ (device=='cpu'))))
|
||||
- nightly-e (build_sku=='gpu') and (device=='gpu') and ((flavor=='release') or (os=='linux'))
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
|
|
@ -35,4 +33,4 @@ testCases:
|
|||
patterns:
|
||||
- "Final Results: Minibatch[{{integer}}-{{integer}}]"
|
||||
# - errs = {{float,tolerance=1%}}% * {{integer}}
|
||||
# - ce = {{float,tolerance=1%}} * {{integer}}
|
||||
# - ce = {{float,tolerance=1%}} * {{integer}}
|
||||
|
|
|
|||
|
|
@ -1,10 +1,8 @@
|
|||
dataDir: ../../../../../../Examples/Image/DataSets
|
||||
|
||||
tags:
|
||||
# In BVT, run Release GPU
|
||||
- bvt-e (build_sku=='gpu') and (device=='gpu') and (flavor=='release')
|
||||
# In Nightly on Linux, additionally run Debug GPU and Release CPU
|
||||
- nightly-e (build_sku=='gpu') and (((device=='gpu') and (flavor=='release')) or (os=='linux' and ((flavor=='debug') ^ (device=='cpu'))))
|
||||
- nightly-e (build_sku=='gpu') and (device=='gpu') and ((flavor=='release') or (os=='linux'))
|
||||
|
||||
testCases:
|
||||
CNTK Run must be completed:
|
||||
|
|
|
|||
|
|
@ -171,9 +171,12 @@ def cifar_resnet(base_path, debug_output=False):
|
|||
|
||||
# Get minibatches of images to train with and perform model training
|
||||
mb_size = 32
|
||||
training_progress_output_freq = 20
|
||||
training_progress_output_freq = 60
|
||||
num_mbs = 1000
|
||||
|
||||
if debug_output:
|
||||
training_progress_output_freq = training_progress_output_freq/3
|
||||
|
||||
for i in range(0, num_mbs):
|
||||
mb = minibatch_source.get_next_minibatch(mb_size)
|
||||
|
||||
|
|
@ -183,8 +186,7 @@ def cifar_resnet(base_path, debug_output=False):
|
|||
features_si].m_data, label_var: mb[labels_si].m_data}
|
||||
trainer.train_minibatch(arguments)
|
||||
|
||||
if debug_output:
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
|
||||
test_minibatch_source = create_test_mb_source(feats_stream_name, labels_stream_name,
|
||||
image_height, image_width, num_channels, num_classes, base_path)
|
||||
|
|
@ -219,4 +221,5 @@ if __name__ == '__main__':
|
|||
|
||||
os.chdir(os.path.join(base_path, '..'))
|
||||
|
||||
cifar_resnet(base_path)
|
||||
error = cifar_resnet(base_path)
|
||||
print("Error: %f" % error)
|
||||
|
|
|
|||
|
|
@ -68,7 +68,11 @@ def simple_mnist(debug_output=False):
|
|||
num_samples_per_sweep = 60000
|
||||
num_sweeps_to_train_with = 1
|
||||
num_minibatches_to_train = (num_samples_per_sweep * num_sweeps_to_train_with) / minibatch_size
|
||||
training_progress_output_freq = 20
|
||||
training_progress_output_freq = 80
|
||||
|
||||
if debug_output:
|
||||
training_progress_output_freq = training_progress_output_freq/4
|
||||
|
||||
for i in range(0, int(num_minibatches_to_train)):
|
||||
mb = mb_source.get_next_minibatch(minibatch_size)
|
||||
|
||||
|
|
@ -78,8 +82,7 @@ def simple_mnist(debug_output=False):
|
|||
label: mb[labels_si].m_data}
|
||||
trainer.train_minibatch(arguments)
|
||||
|
||||
if debug_output:
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
|
||||
# Load test data
|
||||
try:
|
||||
|
|
@ -123,4 +126,4 @@ if __name__=='__main__':
|
|||
DeviceDescriptor.set_default_device(target_device)
|
||||
|
||||
error = simple_mnist()
|
||||
print("test: %f" % error)
|
||||
print("Error: %f" % error)
|
||||
|
|
|
|||
|
|
@ -59,15 +59,18 @@ def ffnet(debug_output=False):
|
|||
num_sweeps_to_train_with = 2
|
||||
num_minibatches_to_train = (
|
||||
num_samples_per_sweep * num_sweeps_to_train_with) / minibatch_size
|
||||
training_progress_output_freq = 20
|
||||
training_progress_output_freq = 60
|
||||
|
||||
if debug_output:
|
||||
training_progress_output_freq = training_progress_output_freq/3
|
||||
|
||||
for i in range(0, int(num_minibatches_to_train)):
|
||||
features, labels = generate_random_data(
|
||||
minibatch_size, input_dim, num_output_classes)
|
||||
# Specify the mapping of input variables in the model to actual
|
||||
# minibatch data to be trained with
|
||||
trainer.train_minibatch({input: features, label: labels})
|
||||
if debug_output:
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
|
||||
test_features, test_labels = generate_random_data(
|
||||
minibatch_size, input_dim, num_output_classes)
|
||||
|
|
@ -83,4 +86,4 @@ if __name__ == '__main__':
|
|||
DeviceDescriptor.set_default_device(target_device)
|
||||
|
||||
error = ffnet()
|
||||
print("test: %f" % error)
|
||||
print("Error: %f" % error)
|
||||
|
|
|
|||
|
|
@ -116,7 +116,10 @@ def sequence_to_sequence_translator(debug_output=False):
|
|||
|
||||
# Get minibatches of sequences to train with and perform model training
|
||||
minibatch_size = 72
|
||||
training_progress_output_freq = 10
|
||||
training_progress_output_freq = 30
|
||||
if debug_output:
|
||||
training_progress_output_freq = training_progress_output_freq/3
|
||||
|
||||
while True:
|
||||
mb = mb_source.get_next_minibatch(minibatch_size)
|
||||
if len(mb) == 0:
|
||||
|
|
@ -128,10 +131,8 @@ def sequence_to_sequence_translator(debug_output=False):
|
|||
raw_labels: mb[labels_si].m_data}
|
||||
trainer.train_minibatch(arguments)
|
||||
|
||||
if debug_output:
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
|
||||
i += 1
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
i += 1
|
||||
|
||||
rel_path = r"../../../../Examples/SequenceToSequence/CMUDict/Data/cmudict-0.7b.test.ctf"
|
||||
path = os.path.join(os.path.dirname(os.path.abspath(__file__)), rel_path)
|
||||
|
|
@ -177,4 +178,4 @@ if __name__ == '__main__':
|
|||
DeviceDescriptor.set_default_device(target_device)
|
||||
|
||||
error = sequence_to_sequence_translator()
|
||||
print("test: %f" % error)
|
||||
print("Error: %f" % error)
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ def LSTM_sequence_classifer_net(input, num_output_classes, embedding_dim, LSTM_d
|
|||
|
||||
# Creates and trains a LSTM sequence classification model
|
||||
|
||||
def train_sequence_classifier():
|
||||
def train_sequence_classifier(debug_output=False):
|
||||
input_dim = 2000
|
||||
cell_dim = 25
|
||||
hidden_dim = 25
|
||||
|
|
@ -66,6 +66,10 @@ def train_sequence_classifier():
|
|||
minibatch_size = 200
|
||||
training_progress_output_freq = 10
|
||||
i = 0
|
||||
|
||||
if debug_output:
|
||||
training_progress_output_freq = training_progress_output_freq/3
|
||||
|
||||
while True:
|
||||
mb = mb_source.get_next_minibatch(minibatch_size)
|
||||
|
||||
|
|
@ -79,7 +83,6 @@ def train_sequence_classifier():
|
|||
trainer.train_minibatch(arguments)
|
||||
|
||||
print_training_progress(trainer, i, training_progress_output_freq)
|
||||
|
||||
i += 1
|
||||
|
||||
import copy
|
||||
|
|
@ -98,4 +101,4 @@ if __name__ == '__main__':
|
|||
DeviceDescriptor.set_default_device(target_device)
|
||||
|
||||
error, _ = train_sequence_classifier()
|
||||
print("test: %f" % error)
|
||||
print("Error: %f" % error)
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ from cntk.io import ReaderConfig, ImageDeserializer
|
|||
|
||||
from examples.CifarResNet.CifarResNet import cifar_resnet
|
||||
|
||||
TOLERANCE_ABSOLUTE = 1E-1
|
||||
TOLERANCE_ABSOLUTE = 2E-1
|
||||
|
||||
def test_cifar_resnet_error(device_id):
|
||||
target_device = DeviceDescriptor.gpu_device(0)
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ from cntk import DeviceDescriptor
|
|||
|
||||
from examples.SequenceClassification.SequenceClassification import train_sequence_classifier
|
||||
|
||||
TOLERANCE_ABSOLUTE = 1E-2
|
||||
TOLERANCE_ABSOLUTE = 1E-1
|
||||
|
||||
def test_seq_classification_error(device_id):
|
||||
from cntk.utils import cntk_device
|
||||
|
|
@ -17,6 +17,5 @@ def test_seq_classification_error(device_id):
|
|||
|
||||
evaluation_avg, loss_avg = train_sequence_classifier()
|
||||
|
||||
# Temporarily disable the comparison against baseline as it needs to be updated
|
||||
# expected_avg = [0.1595744, 0.35799171]
|
||||
# assert np.allclose([evaluation_avg, loss_avg], expected_avg, atol=TOLERANCE_ABSOLUTE)
|
||||
expected_avg = [0.68181, 1.5661]
|
||||
assert np.allclose([evaluation_avg, loss_avg], expected_avg, atol=TOLERANCE_ABSOLUTE)
|
||||
|
|
|
|||
|
|
@ -17,6 +17,5 @@ def test_sequence_to_sequence(device_id):
|
|||
|
||||
error = sequence_to_sequence_translator()
|
||||
|
||||
# Temporarily disable the comparison against baseline as it needs to be updated
|
||||
# expected_error = 0.758458
|
||||
# assert np.allclose(error, expected_error, atol=TOLERANCE_ABSOLUTE)
|
||||
expected_error = 0.8596881547969316
|
||||
assert np.allclose(error, expected_error, atol=TOLERANCE_ABSOLUTE)
|
||||
|
|
|
|||
|
|
@ -17,26 +17,11 @@ pytest
|
|||
echo(
|
||||
popd
|
||||
|
||||
pushd examples\CifarResNet
|
||||
echo RUNNING Cifar ResNet example...
|
||||
python CifarResNet.py
|
||||
pushd examples\test
|
||||
echo RUNNING cntk\examples\test tests
|
||||
pytest
|
||||
echo(
|
||||
popd
|
||||
|
||||
echo RUNNING MNIST feed-forward classifier example...
|
||||
python examples\MNIST\SimpleMNIST.py
|
||||
echo(
|
||||
|
||||
echo RUNNING feed-forward numpy interop example...
|
||||
python examples\NumpyInterop\FeedForwardNet.py
|
||||
echo(
|
||||
|
||||
echo RUNNING sequence-to-sequence example...
|
||||
python examples\Sequence2Sequence\Sequence2Sequence.py
|
||||
echo(
|
||||
|
||||
echo RUNNING sequence classification example...
|
||||
python examples\SequenceClassification\SequenceClassification.py
|
||||
echo(
|
||||
|
||||
endlocal
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче