154 строки
5.0 KiB
Plaintext
154 строки
5.0 KiB
Plaintext
== Dev branch ==
|
|
This branch contains some features that are not yet checked into main branch. To enlist this branch, run
|
|
|
|
git checkout origin/Dev
|
|
|
|
== To-do ==
|
|
Add descriptions to LSTMnode
|
|
Add descriptions to 0/1 mask segmentation in feature reader, delay node, and crossentropywithsoftmax node
|
|
Change criterion node to use the 0/1 mask, following example in crossentropywithsoftmax node
|
|
Add description of encoder-decoder simple network builder
|
|
Add description of time-reverse node, simple network builder and NDL builder for bi-directional models
|
|
|
|
== Author of the README ==
|
|
Kaisheng Yao
|
|
Microsoft Research
|
|
email: kaisheny@microsoft.com
|
|
|
|
Wengong Jin,
|
|
Shanghai Jiao Tong University
|
|
email: acmgokun@gmail.com
|
|
|
|
Yu Zhang, Leo Liu
|
|
CSAIL, Massachusetts Institute of Technology
|
|
email: yzhang87@csail.mit.edu
|
|
email: leoliu_cu@sbcglobal.net
|
|
|
|
Guoguo Chen
|
|
CLSP, Johns Hopkins University
|
|
email: guoguo@jhu.edu
|
|
|
|
== Preeliminaries ==
|
|
To build the cpu version, you have to install intel MKL blas library or ACML library first. Note that ACML is free, where MKL may not be.
|
|
|
|
for MKL:
|
|
1. Download from https://software.intel.com/en-us/intel-mkl
|
|
2. You can modify variable MKL_PATH in makefile.cpu to change your mkl path.
|
|
Then add ${MKL_PATH}/mkl/lib/intel64, ${MKL_PATH}/mkl/lib/mic, ${MKL_PATH}/compiler/lib/intel64. ${MKL_PATH}/compiler/lib/mic to your ${LD_LIBRARY_PATH} to make sure the program links the library correctly.
|
|
|
|
for ACML:
|
|
1. Download from http://developer.amd.com/tools-and-sdks/cpu-development/amd-core-math-library-acml/
|
|
2. Modify ACML_PATH in the makefile.cpu and makefile.gpu to provide your ACML library path.
|
|
You need to add ${ACML_PATH}/lib to your ${LD_LIBRARY_PATH}.
|
|
|
|
for Kaldi:
|
|
1. In kaldi-trunk/tools/Makefile, uncomment # OPENFST_VERSION = 1.4.1, and
|
|
re-install OpenFst using the makefile.
|
|
2. In kaldi-trunk/src/, do ./configure --shared; make depend -j 8; make -j 8;
|
|
and re-compile Kaldi (the -j option is for parallelization).
|
|
3. Set KALDI_PATH in kaldi_vars.mk
|
|
4. When running the binaries, make sure you add $KALDI_PATH/tools/openfst/lib
|
|
and $KALDI_PATH/src/lib/ to your $LD_LIBRARY_PATH
|
|
|
|
To build the gpu version, you have to install NIVIDIA CUDA first
|
|
You can modify the path CUDA_PATH in makefile.cpu to change your cuda path
|
|
We use cuda-7.0 as default.
|
|
Then add ${CUDA_PATH}/lib, ${CUDA_PATH}/lib64 to your ${LD_LIBRARY_PATH} to make sure the program links to the library correctly.
|
|
|
|
... TODO: add documentation on nvml lib
|
|
|
|
== Build ==
|
|
To build the cpu version, run
|
|
make DEVICE=cpu
|
|
To build the gpu version, run
|
|
make
|
|
To clean the compile, just run
|
|
make DEVICE=cpu clean
|
|
or
|
|
make clean
|
|
|
|
For release version, just add BUILDTYPE=release to the make command line.
|
|
|
|
== Run ==
|
|
All executables are in bin/ directory:
|
|
cn.exe: The main executable for CNTK
|
|
*.so: shared library for corresponding reader, these readers will be linked and loaded dynamically at runtime.
|
|
|
|
To run the executable, make sure bin/ is in your ${LD_LIBRARY_PATH}, if not, running cn.exe will fail when cn.exe tries to link the corresponding reader. Once it's done, run in command line:
|
|
./cn.exe configFile=${your config file}
|
|
|
|
== Kaldi Reader ==
|
|
This is a HTKMLF reader and kaldi writer (for decode)
|
|
|
|
To build the cpu/gpu version, run
|
|
make -f Makefile_kaldi.cpu/gpu
|
|
|
|
The feature section is like:
|
|
|
|
writer=[
|
|
writerType=KaldiReader
|
|
readMethod=blockRandomize
|
|
frameMode=false
|
|
miniBatchMode=Partial
|
|
randomize=Auto
|
|
verbosity=1
|
|
ScaledLogLikelihood=[
|
|
dim=$labelDim$
|
|
Kaldicmd="ark:-" # will pipe to the Kaldi decoder latgen-faster-mapped
|
|
scpFile=$outputSCP$ # the file key of the features
|
|
]
|
|
]
|
|
|
|
== Kaldi2 Reader ==
|
|
This is a kaldi reader and kaldi writer (for decode)
|
|
|
|
To build the cpu/gpu version, run
|
|
make -f Makefile_kaldi2.cpu/gpu
|
|
|
|
|
|
The features section is different:
|
|
|
|
features=[
|
|
dim=
|
|
rx=
|
|
scpFile=
|
|
featureTransform=
|
|
]
|
|
|
|
rx is a text file which contains:
|
|
|
|
one Kaldi feature rxspecifier readable by RandomAccessBaseFloatMatrixReader.
|
|
'ark:' specifiers don't work; only 'scp:' specifiers work.
|
|
|
|
scpFile is a text file generated by running:
|
|
|
|
feat-to-len FEATURE_RXSPECIFIER_FROM_ABOVE ark,t:- > TEXT_FILE_NAME
|
|
|
|
scpFile should contain one line per utterance.
|
|
|
|
If you want to run with fewer utterances, just shorten this file.
|
|
(It will load the feature rxspecifier but ignore utterances not present in scpFile).
|
|
|
|
featureTransform is the name of a Kaldi feature transform file:
|
|
|
|
Kaldi feature transform files are used for stacking / applying transforms to features.
|
|
|
|
An empty string (if permitted by the config file reader?) or the special string: NO_FEATURE_TRANSFORM
|
|
says to ignore this option.
|
|
|
|
********** Labels **********
|
|
|
|
The labels section is also different.
|
|
|
|
labels=[
|
|
mlfFile=
|
|
labelDim=
|
|
labelMappingFile=
|
|
]
|
|
|
|
Only difference is mlfFile. mlfFile is a different format now. It is a text file which contains:
|
|
|
|
one Kaldi label rxspecifier readable by Kaldi's copy-post binary.
|
|
|
|
|