Note: baselines need to be fixed for
Tests/EndToEndTests/BatchNormalization and
Tests/EndToEndTests/Examples/Image/Miscellaneous/CIFAR-10/02_BatchNormConv.
For batch normalization, running inverse standard deviation becomes
running variance. We mirror this CuDNN v5 change in the CNTK batch
normalization engine. Model version is bumped. When old models are
loaded, this parameter is (approximately) converted.
In the same model version change, let batch normalization count
samples seen rather minibatches (this deals with incorrect averaging
when minibatch size is varied across epochs).
For batch normalization averaging and blending handle initialization
cases, don't rely on mean and variance initial values (set in
NDL/BrainScript).
Update Windows / Linux / Docker build.
With this commit, CuDNN v4 is not supported anymore.
FSAdaGrad, RMSProp: pre-allocate smoothed gradients with expected
number of columns.
Improve configuring built-in learners: add learning rates and
momentums schedules.
Add NDArrayView as a DictionaryValue type.
Add tests for serialization and basic learner functionality.
Use GDK_NVML_LIB_PATH
Simplify Boost libraries
Use ORIGINDIR in rpath of libeval
remove GDK_NVML_LIB_PATH from cppevalclient and Add -lcntkmath to libeval.so
Use GDK_NVML_LIB_PATH to avoid using system-wide one, Simplify
use in libeval.so
Build unit tests only when Boost is available
Configure Boost path in configure
Address CR comments
fix#678
This is a combination of 10 commits.
Adapt makefile to work with Docker container on Linux
This is a combination of 7 commits.
enable unit tests build
remove -liomp5 from math unit test
enable openmp
add -ldl
fix space error
add -dl for reader test
change order of -l
remove gdk installation
use /usr/local/lib as boost library path
Adapt path in unit tests on Linux
adapt path for Linux
adapt path on other test projects for Linux
remove extra blank line
add BOOST_PATH,and build unit tests based on BOOST_PATH
configure boost path
install Boost 1.60.0; use version variable instead of hardcoding.
add comments for different paths on Linux than on Windows.
restore installation of gdk, because the removal of gdk needs more changes and will be done in a separate check-in
use ifdef, fix typos
This is a combination of 7 commits.
minor format changes
adapt makefile and math tests
enable sse4.1 support
adapt to linux
fix shadow param, and adjust order of functions
netowrk tests need .cu
move constant definition into a .cpp file, instead of .h
This is a combination of 25 commits.
Add unit tests
make them work on Linux
add to build eval unit tests.
adapt for Linux
fix incompatablity issues on linux
using runtime_error() instead of exception to make linux happy
add library and path
change order of included files
use dynmaic library on linux
link with boost dynamic libraries
minor changes
use BOOSTLIBS to specify all required boost libraries
Add reader unit tests
add .phony unittests to only build unit tests. (will be removed later)
fix errors
add ifdef for platform specifc stuff
fix for linux
adapt makefile for unit tests
adapt reader test to work on Linux
add missing files for the networks unit test
adapt the networks unit test
add the math unit tests
fix errors
minor changes
remove unexpected tab
remvoed pre-compiler headers and addressed CR comments
simplify naming comments from CR
delete unused files, minor changes
Changes in comments and Todo
Adapt cppevalcient to build both on Windwos and Linux
Add comment for using "this->" explicitly
Add to build LibEval.so in Makefile
minor changes
EvalSample on Linux
Add EvalSampleClient to Makefile
fix errors in Makefile
fix compilation error on Linux
fix more errors
add main function for Linux
Fix captial case for Eval.h on Linux
build cppevalclient
Add dependency processing for .cu
replace \ with / in the file path on Linux
Adapt path on Linux
fix path on Linux
Use main() on Windows, remvoe unused files and path in Makefile
remove unused files
Use main() on Windows, and remove the wrapper for Linux
further remvoe unused files
fine tuning on lib and path
minor changes
* Move caching into a stand-alone ChunkCache class.
Add a config flag 'keepDataInMemory' to cache the whole dataset.
* Allow both tabs and spaces as separators.
* Couple loading/indexing retries with file re-opening (if there was a
file error and the file handle is stale).
* Break 'randomize' config parameter into boolean 'randomize' and
integer 'randomizationWindow'.
* Add 'frameMode' config flag.
* Add an extra check and verify that the expected sequence length
(calculated during indexing) matches the actual length (maximum
number of samples in an input).
Extend sequence and frame packers to support sparse input.
Create a tightly filled layout and base packing off of it.
Add a number of unit test for sequence packing (both sparse and
dense, no randomization, using CNTK text format for input).
new operation Trace() (TraceNode) as a debugging aid;
WriteMinibatchWithFormatting() can now log by FrameRange;
WriteFormattingOptions now in ComputationNode.h
Renames SynchronousNodeEvaluator to NDLNodeEvaluatorImpl
Merges SynchronousExecutionEngine.h into NDLNetworkBuilder.h
Renames SynchronousExecutionEngine.cpp into NDLNetworkBuilder.cpp
Moves DebugUtil functions (PrintCallStack, GetCallStack) to ExceptionWithCallStack
Refactors the PrintCallStack and GetCallStack functions in ExceptionWithCallBack to re-use the common functionality.
Make gcc happy
* Add 'openblas' as mathlib option in configure. Not added to auto-search so
must be specified using --with-openblas
* configure script searches empty tail so that libraries located at default_path_list
roots (ie /usr/local/ + include/openblas_config.h) are found
* Treat ACML as the odd library out in ifdefs since it doesn't conform to typical
BLAS standard. Other libraries like ATLAS should be able to share
OpenBLAS/MKL variants. Add default USE_ACML define in VS projects to match
* Fix 'max' macro define colliding with C++ std::max once openblas headers are included
Usage Notes:
* For best performance, build OpenBLAS with USE_OPENMP=1. When running CNTK, set
OPENBLAS_NUM_THREADS environment var or set numCPUThreads CNTK config variable to the
physical core count or performance will suffer
* OpenBLAS 2.16 (git HEAD) tested in Linux with GCC 4.8.4 and in Windows with
OpenBLAS 2.15 (pre-built binary release + MingGW 64-bit support dlls)
* For Windows, in Math.vcxproj, replace libacml_mp_dll.lib with libopenblas.dll.a and change
USE_ACML define to USE_OPENBLAS. Change ACML_PATH environment variable to your OpenBLAS path.
Modify openblas_config.h as per https://github.com/xianyi/OpenBLAS/issues/708
* On current generation Intel processors, OpenBLAS measures a little faster than
AMD ACML and slower than Intel MKL on MNIST and other examples
Activate LibSparsePCReader on Linux build
Make file name consistent with Windows
Activate LibSparsePCReader on Linux build
Make file name consistent with Windows
Activate LibSparsePCReader on Linux build
Make file name consistent with Windows
Fix reader on Linux
Activate LibSparsePCReader on Linux build
Make file name consistent with Windows
Activate LibSparsePCReader on Linux build
Make file name consistent with Windows
Activate LibSparsePCReader on Linux build
Make file name consistent with Windows
Activate LibSparsePCReader on Linux build
Make file name consistent with Windows
added a dummy DataReader constructor from BrainScript (currently not implemented);
had to disable DelayLoadNofify() as that suddenly caused link errors (seems because DeviceFromConfig() is now called from a different place?);
further unified DoTrain() w.r.t. old CNTK config and BrainScript. Added a new BrainScriptBuilder (but the builders will go away next and get replaced by a lambda). DoTrain now accessible from BS (but not functional yet, too many pieces missing);
all builders now can be constructed from both old ConfigParameters and BrainScript IConfigRecord;
suddenly gcc got very picky, had to fix several places
fixed Linux Makefile after change of GPUMatrixCUDAKernels.cu to GPUMatrixCUDAKernels.cuh. It used to explicitly compile this in the GCC build, although it was included
new source file ComputationNetworkScripting.cpp;
changed ComputationNetwork::SetDeviceId(). Now just takes the input and sets that as the device. Before it did somethinf funky with m_deviceId, which looks like a bug;
in LSTM test config, removed remnants of old NDL, now completely based on BS
moved DisableLegacyTruncationSettings() away from being the very first piece of code in CNTK.cpp, hiding it close to where it is used. Also fixed its spelling and made it 'static'
Project Philly
Zhou Wang
Qiwei Ye
Mark Hillebrand
Frank Seide
Jasha Droppo
Amit Agarwal
Alexey Reznichenko
Wolfgang Manousek
Ivan Rodriguez
U-FAREAST\fseide
anthonyaue
Marko Radmilac
Eldar Akchurin
Yongqiang Wang
Clemens Marschner
Alexey Kamenev
Alexey Kamenev
Gaizka Navarro
Ross Wightman
Chris Basoglu
Alexey Orlov
Philipp Kranen
thhoens
yzhang87
Vladimir Ivanov