Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
Перейти к файлу
Marko Radmilac c518eb2203 Make default output directory under cntk for simplicity, and address code review feedback 2015-06-10 17:45:42 -07:00
CheckInSuites added a global test runner BAT file as a Build action, so far it is running, but CNTK output does not appear in the Build window; 2015-05-22 20:00:40 -07:00
Common add NVML (Linux) related information to BestGPU.cpp 2015-05-24 19:16:51 -07:00
DataReader Fix the comile on linux for kaldi reader 2015-06-07 18:19:26 -04:00
Demos Some minor changes to the Simple Demo config file to allow specifying the path to input files using the RootDir variable 2015-06-03 11:36:35 -07:00
Documentation change CNTK tech report's listing font to type writer fonts 2015-05-25 11:23:16 +08:00
ExampleSetups cpu nce lstmlm expected ppl=122.54 2015-05-20 19:19:14 -07:00
MachineLearning In sequencereader move labelOutput to DeviceId and remove useless code in NCE-LSTM, which will move data from GPU to CPU 2015-06-01 23:29:25 -07:00
Math Fixed a bug that was incorrectly deleting the CPUMatrix external buffer 2015-06-03 14:16:28 -07:00
Scripts Make default output directory under cntk for simplicity, and address code review feedback 2015-06-10 17:45:42 -07:00
license First Release of CNTK 2014-08-29 16:21:42 -07:00
.gitignore Make default output directory under cntk for simplicity, and address code review feedback 2015-06-10 17:45:42 -07:00
CNTK.sln added a global test runner BAT file as a Build action, so far it is running, but CNTK output does not appear in the Build window; 2015-05-22 20:00:40 -07:00
Makefile.cpu Merge branch 'master' of https://git.codeplex.com/cntk into linux-gcc 2015-05-19 16:04:47 -07:00
Makefile.gpu Adding script for build and test 2015-06-02 16:39:53 -07:00
Makefile_kaldi.cpu Visual Studio Solution cleanup: 2015-05-18 20:32:39 -07:00
Makefile_kaldi.gpu Visual Studio Solution cleanup: 2015-05-18 20:32:39 -07:00
Makefile_kaldi2.cpu Fix the comile on linux for kaldi reader 2015-06-07 18:19:26 -04:00
Makefile_kaldi2.gpu For all the wprint related stuff, we should avoid use %s because wstring is the internal representation (no conversion needed). Replaced it with %ls. 2015-04-15 21:02:55 -04:00
README Merge branch 'master' of https://git.codeplex.com/cntk into linux-gcc 2015-05-19 16:04:47 -07:00
c reworked pathnames in CheckInSuites\ASR 2015-05-22 18:55:44 -07:00
kaldi_vars.mk Clean the warnings. 2015-04-09 01:09:58 -04:00

README

== Author of the README ==
    Wengong Jin,
    Shanghai Jiao Tong University
    email: acmgokun@gmail.com

    Hakan Erdogan
    MERL, Sabanci University
    email: haerdogan@sabanciuniv.edu

    Yu Zhang, Leo Liu
    CSAIL, Massachusetts Institute of Technology
    email: yzhang87@csail.mit.edu
    email: leoliu_cu@sbcglobal.net

== Preeliminaries ==
To build the cpu version, you have to install intel MKL blas library or ACML library first. Note that ACML is free, where MKL may not be.

for MKL:
1. Download from https://software.intel.com/en-us/intel-mkl
2. You can modify variable MKL_PATH in makefile.cpu to change your mkl path.
Then add ${MKL_PATH}/mkl/lib/intel64, ${MKL_PATH}/mkl/lib/mic, ${MKL_PATH}/compiler/lib/intel64. ${MKL_PATH}/compiler/lib/mic to your ${LD_LIBRARY_PATH} to make sure the program links the library correctly.

for ACML:
1. Download from http://developer.amd.com/tools-and-sdks/cpu-development/amd-core-math-library-acml/
2. Modify ACML_PATH in the makefile.cpu and makefile.gpu to provide your ACML library path.
You need to add ${ACML_PATH}/lib to your ${LD_LIBRARY_PATH}.

To build the gpu version, you have to install NIVIDIA CUDA first
You can modify the path CUDA_PATH in makefile.cpu to change your cuda path
We use cuda-7.0 as default.
Then add ${CUDA_PATH}/lib, ${CUDA_PATH}/lib64 to your ${LD_LIBRARY_PATH} to make sure the program links to the library correctly.

... TODO: add documentation on nvml lib

== Build ==
To build the cpu version, run
	make -f Makefile.cpu 
To build the gpu version, run
	make -f Makefile.gpu
To clean the compile, just run
	make -f Makefile.cpu clean
or
	make -f Makefile.gpu clean

== Run ==
All executables are in bin/ directory:
	cn.exe: The main executable for CNTK
	*.so: shared library for corresponding reader, these readers will be linked and loaded dynamically at runtime.

To run the executable, make sure bin/ is in your ${LD_LIBRARY_PATH}, if not, running cn.exe will fail when cn.exe tries to link the corresponding reader. Once it's done, run in command line:
	./cn.exe configFile=${your config file}

== Kaldi Reader ==
This is a HTKMLF reader and kaldi writer (for decode)

To build the cpu/gpu version, run
    make -f Makefile_kaldi.cpu/gpu

The feature section is like:

writer=[
    writerType=KaldiReader
    readMethod=blockRandomize
    frameMode=false
    miniBatchMode=Partial
    randomize=Auto
    verbosity=1
    ScaledLogLikelihood=[
        dim=$labelDim$
        Kaldicmd="ark:-" # will pipe to the Kaldi decoder latgen-faster-mapped
        scpFile=$outputSCP$ # the file key of the features
    ]
]

== Kaldi2 Reader ==
This is a kaldi reader and kaldi writer (for decode)

To build the cpu/gpu version, run
    make -f Makefile_kaldi2.cpu/gpu


The features section is different:

features=[
    dim=
    rx=
    scpFile=
    featureTransform=
]

rx is a text file which contains:

    one Kaldi feature rxspecifier readable by RandomAccessBaseFloatMatrixReader.
    'ark:' specifiers don't work; only 'scp:' specifiers work.

scpFile is a text file generated by running:

    feat-to-len FEATURE_RXSPECIFIER_FROM_ABOVE ark,t:- > TEXT_FILE_NAME

    scpFile should contain one line per utterance.

    If you want to run with fewer utterances, just shorten this file.
    (It will load the feature rxspecifier but ignore utterances not present in scpFile).

featureTransform is the name of a Kaldi feature transform file:
    
    Kaldi feature transform files are used for stacking / applying transforms to features.

    An empty string (if permitted by the config file reader?) or the special string: NO_FEATURE_TRANSFORM
    says to ignore this option.

********** Labels **********

The labels section is also different.

labels=[
    mlfFile=
    labelDim=
    labelMappingFile=
]

Only difference is mlfFile. mlfFile is a different format now. It is a text file which contains:

    one Kaldi label rxspecifier readable by Kaldi's copy-post binary.