microsoft
/
CNTK
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Merge branch 'master' into qiwye/multiverso 

Conflicts:
	Source/CNTK/CNTK.vcxproj
	Source/ComputationNetworkLib/ComputationNetworkLib.vcxproj
	Source/EvalDll/EvalDll.vcxproj
	Source/SGDLib/SGDLib.vcxproj
	Source/SGDLib/SGDLib.vcxproj.filters
This commit is contained in:
Qiwei Ye 2015-12-17 10:35:08 +08:00
Родитель 4c6c57e1de 583f10e4f1
Коммит 254b45e680
963 изменённых файлов: 3011 добавлений и 7479 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

4
.gitignore поставляемый
Просмотреть файл

@ -179,8 +179,8 @@ core
# =========================
# prebuild file
# =========================
MachineLearning/CNTK/buildinfo.h
MachineLearning/CNTK/buildinfo.h$$
Source/CNTK/buildinfo.h
Source/CNTK/buildinfo.h$$
*.lyx#

1061
CNTK.sln
Просмотреть файл

Разница между файлами не показана из-за своего большого размера Загрузить разницу

1
Common/CheckInSuites/ASR/CPU/EvalSimpleNetwork.output
Просмотреть файл

@ -1 +0,0 @@
Final Results: Minibatch[1-3]: Samples Seen = 122487 ErrorPrediction/Sample = 0.84662861 CrossEntropyWithSoftmax/Sample = 3.3569317

5
Common/CheckInSuites/ASR/CPU/TrainAutoEncoder.output
Просмотреть файл

@ -1,5 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 399.03207 EvalErr Per Sample = 399.03207 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=8.8590002
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 339.94937 EvalErr Per Sample = 339.94937 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=7.5630002
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 188.5687 EvalErr Per Sample = 188.5687 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=8.0939999
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 125.43492 EvalErr Per Sample = 125.43492 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=7.5469999
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 100.78821 EvalErr Per Sample = 100.78821 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=8.0629997

5
Common/CheckInSuites/ASR/CPU/TrainMultiInput.output
Просмотреть файл

@ -1,5 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 4.9031577 EvalErr Per Sample = 0.95952296 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=8.7030001
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 4.6207099 EvalErr Per Sample = 0.92985517 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=6.9219999
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 4.18716 EvalErr Per Sample = 0.90271711 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=11.094
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 3.7490704 EvalErr Per Sample = 0.85948026 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=6.9380002
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 3.340929 EvalErr Per Sample = 0.81315506 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=6.4219999

30
Common/CheckInSuites/ASR/CPU/TrainMultiTask.output
Просмотреть файл

@ -1,30 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 3.942008 EvalErr Per Sample [0]=4.9230547 [1]=0.96228278 [2]=0.017820816 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=6.5009999
Finished Epoch[1]: Criterion Node [ObjFcn.O] Per Sample = 3.942008
Finished Epoch[1]: Evaluation Node [CE1] Per Sample = 4.9230547
Finished Epoch[1]: Evaluation Node [FER1] Per Sample = 0.96228278
Finished Epoch[1]: Evaluation Node [CE2] Per Sample = 0.017820816
Finished Epoch[1]: Evaluation Node [FER2] Per Sample = 0
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 3.8613315 EvalErr Per Sample [0]=4.8266635 [1]=0.95328057 [2]=1.5309175e-007 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=5.2350001
Finished Epoch[2]: Criterion Node [ObjFcn.O] Per Sample = 3.8613315
Finished Epoch[2]: Evaluation Node [CE1] Per Sample = 4.8266635
Finished Epoch[2]: Evaluation Node [FER1] Per Sample = 0.95328057
Finished Epoch[2]: Evaluation Node [CE2] Per Sample = 1.5309175e-007
Finished Epoch[2]: Evaluation Node [FER2] Per Sample = 0
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 3.592231 EvalErr Per Sample [0]=4.4902878 [1]=0.91986728 [2]=1.3654688e-006 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=4.5159998
Finished Epoch[3]: Criterion Node [ObjFcn.O] Per Sample = 3.592231
Finished Epoch[3]: Evaluation Node [CE1] Per Sample = 4.4902878
Finished Epoch[3]: Evaluation Node [FER1] Per Sample = 0.91986728
Finished Epoch[3]: Evaluation Node [CE2] Per Sample = 1.3654688e-006
Finished Epoch[3]: Evaluation Node [FER2] Per Sample = 0
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 3.317225 EvalErr Per Sample [0]=4.1465311 [1]=0.90123868 [2]=7.4484177e-007 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=4.6100001
Finished Epoch[4]: Criterion Node [ObjFcn.O] Per Sample = 3.317225
Finished Epoch[4]: Evaluation Node [CE1] Per Sample = 4.1465311
Finished Epoch[4]: Evaluation Node [FER1] Per Sample = 0.90123868
Finished Epoch[4]: Evaluation Node [CE2] Per Sample = 7.4484177e-007
Finished Epoch[4]: Evaluation Node [FER2] Per Sample = 0
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 3.0248218 EvalErr Per Sample [0]=3.7810273 [1]=0.85977596 [2]=2.3329179e-007 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=6.3130002
Finished Epoch[5]: Criterion Node [ObjFcn.O] Per Sample = 3.0248218
Finished Epoch[5]: Evaluation Node [CE1] Per Sample = 3.7810273
Finished Epoch[5]: Evaluation Node [FER1] Per Sample = 0.85977596
Finished Epoch[5]: Evaluation Node [CE2] Per Sample = 2.3329179e-007
Finished Epoch[5]: Evaluation Node [FER2] Per Sample = 0

5
Common/CheckInSuites/ASR/CPU/TrainNDLNetwork.output
Просмотреть файл

@ -1,5 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 4.9007668 EvalErr Per Sample = 0.95962155 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=8.1569996
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 4.5884385 EvalErr Per Sample = 0.92660254 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=5.8439999
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 4.1541729 EvalErr Per Sample = 0.90087724 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=5.7030001
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 3.6861088 EvalErr Per Sample = 0.85389495 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=5.7350001
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 3.3145225 EvalErr Per Sample = 0.8123337 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=5.75

5
Common/CheckInSuites/ASR/CPU/TrainSimpleNetwork.output
Просмотреть файл

@ -1,5 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 4.9007668 EvalErr Per Sample = 0.95962155 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=7.5630002
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 4.5884385 EvalErr Per Sample = 0.92660254 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=10.985
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 4.1541729 EvalErr Per Sample = 0.90087724 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=6.0630002
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 3.6861088 EvalErr Per Sample = 0.85389495 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=6.0310001
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 3.3145225 EvalErr Per Sample = 0.8123337 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=6.2969999

1
Common/CheckInSuites/ASR/GPU/EvalSimpleNetwork.output
Просмотреть файл

@ -1 +0,0 @@
Final Results: Minibatch[1-3]: Samples Seen = 122487 ErrorPrediction/Sample = 0.84766549 CrossEntropyWithSoftmax/Sample = 3.4053154

5
Common/CheckInSuites/ASR/GPU/TrainAutoEncoder.output
Просмотреть файл

@ -1,5 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 399.07224 EvalErr Per Sample = 399.07224 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=0.82800001
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 358.64328 EvalErr Per Sample = 358.64328 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=0.81199998
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 214.13602 EvalErr Per Sample = 214.13602 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=0.81199998
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 134.49174 EvalErr Per Sample = 134.49174 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=0.81199998
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 101.23165 EvalErr Per Sample = 101.23165 Ave Learn Rate Per Sample = 0.0003906250058 Epoch Time=0.81300002

5
Common/CheckInSuites/ASR/GPU/TrainMultiInput.output
Просмотреть файл

@ -1,5 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 4.9008274 EvalErr Per Sample = 0.95968723 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.609
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 4.6037016 EvalErr Per Sample = 0.92785102 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.375
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 4.1639848 EvalErr Per Sample = 0.89992446 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.375
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 3.7287264 EvalErr Per Sample = 0.86164868 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.375
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 3.3417282 EvalErr Per Sample = 0.81598055 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.375

30
Common/CheckInSuites/ASR/GPU/TrainMultiTask.output
Просмотреть файл

@ -1,30 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 3.9423771 EvalErr Per Sample [0]=4.9227123 [1]=0.96231562 [2]=0.021024836 [3]=0.0084108161 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.6720001
Finished Epoch[1]: Criterion Node [ObjFcn.O] Per Sample = 3.9423771
Finished Epoch[1]: Evaluation Node [CE1] Per Sample = 4.9227123
Finished Epoch[1]: Evaluation Node [FER1] Per Sample = 0.96231562
Finished Epoch[1]: Evaluation Node [CE2] Per Sample = 0.021024836
Finished Epoch[1]: Evaluation Node [FER2] Per Sample = 0.0084108161
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 3.8521581 EvalErr Per Sample [0]=4.8151979 [1]=0.95413482 [2]=4.126586e-008 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.391
Finished Epoch[2]: Criterion Node [ObjFcn.O] Per Sample = 3.8521581
Finished Epoch[2]: Evaluation Node [CE1] Per Sample = 4.8151979
Finished Epoch[2]: Evaluation Node [FER1] Per Sample = 0.95413482
Finished Epoch[2]: Evaluation Node [CE2] Per Sample = 4.126586e-008
Finished Epoch[2]: Evaluation Node [FER2] Per Sample = 0
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 3.5730984 EvalErr Per Sample [0]=4.466373 [1]=0.91714036 [2]=3.638261e-007 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.375
Finished Epoch[3]: Criterion Node [ObjFcn.O] Per Sample = 3.5730984
Finished Epoch[3]: Evaluation Node [CE1] Per Sample = 4.466373
Finished Epoch[3]: Evaluation Node [FER1] Per Sample = 0.91714036
Finished Epoch[3]: Evaluation Node [CE2] Per Sample = 3.638261e-007
Finished Epoch[3]: Evaluation Node [FER2] Per Sample = 0
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 3.3172503 EvalErr Per Sample [0]=4.1465631 [1]=0.90458983 [2]=1.3984375e-007 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.375
Finished Epoch[4]: Criterion Node [ObjFcn.O] Per Sample = 3.3172503
Finished Epoch[4]: Evaluation Node [CE1] Per Sample = 4.1465631
Finished Epoch[4]: Evaluation Node [FER1] Per Sample = 0.90458983
Finished Epoch[4]: Evaluation Node [CE2] Per Sample = 1.3984375e-007
Finished Epoch[4]: Evaluation Node [FER2] Per Sample = 0
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 3.0483234 EvalErr Per Sample [0]=3.8104041 [1]=0.8601045 [2]=2.6917903e-008 [3]=0 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.375
Finished Epoch[5]: Criterion Node [ObjFcn.O] Per Sample = 3.0483234
Finished Epoch[5]: Evaluation Node [CE1] Per Sample = 3.8104041
Finished Epoch[5]: Evaluation Node [FER1] Per Sample = 0.8601045
Finished Epoch[5]: Evaluation Node [CE2] Per Sample = 2.6917903e-008
Finished Epoch[5]: Evaluation Node [FER2] Per Sample = 0

5
Common/CheckInSuites/ASR/GPU/TrainNDLNetwork.output
Просмотреть файл

@ -1,5 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 4.9025679 EvalErr Per Sample = 0.96004868 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.516
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 4.6258178 EvalErr Per Sample = 0.92995369 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.313
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 4.193347 EvalErr Per Sample = 0.90271711 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.2970001
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 3.7910109 EvalErr Per Sample = 0.86660975 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.2970001
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 3.3828781 EvalErr Per Sample = 0.81897038 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.2970001

5
Common/CheckInSuites/ASR/GPU/TrainSimpleNetwork.output
Просмотреть файл

@ -1,5 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 4.9025679 EvalErr Per Sample = 0.96004868 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.516
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 4.6258178 EvalErr Per Sample = 0.92995369 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.2970001
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 4.193347 EvalErr Per Sample = 0.90271711 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.312
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 3.7910109 EvalErr Per Sample = 0.86660975 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.359
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 3.3828781 EvalErr Per Sample = 0.81897038 Ave Learn Rate Per Sample = 0.003125000047 Epoch Time=1.2970001

8
Common/CheckInSuites/ASR/config/README.txt
Просмотреть файл

@ -1,8 +0,0 @@
These scripts are similar to those in the TIMIT folder of the ExampleSetups except they use much fewer files (100 utterances) and fewer minibatches. See the README.txt file there for more details about these configurations.
To test on CPU:
CNTK.exe WorkDir=... ExpDir=... LibDir=... ScpDir=... configFile=globals.config+select_cpu.config+<DesiredConfigFile>
To test on GPU:
CNTK.exe WorkDir=... ExpDir=... LibDir=... ScpDir=... configFile=globals.config+select_gpu.config+<DesiredConfigFile>

38
Common/CheckInSuites/ASR/config/TIMIT_EvalSimpleNetwork.config
Просмотреть файл

@ -1,38 +0,0 @@
stderr=$ExpDir$\EvalSimpleNetwork\log\log
command=TIMIT_EvalSimpleNetwork
precision=float
TIMIT_EvalSimpleNetwork=[
action=eval
modelPath=$ExpDir$\TrainSimpleNetwork\model\cntkSpeech.dnn
# deviceId=-1 for CPU, >=0 for GPU devices
deviceId=$DeviceNumber$
traceLevel=1
# Parameter values for the reader
reader=[
# reader to use
readerType=HTKMLFReader
# readMethod=blockRandomize
readMethod=rollingWindow
miniBatchMode=Partial
randomize=None
verbosity=1
features=[
dim=792
scpFile=$ScpDir$\TIMIT.dev.scp.fbank.fullpath
]
labels=[
mlfFile=$MlfDir$\TIMIT.dev.align_cistate.mlf.cntk
labelDim=183
labelMappingFile=$MlfDir$\TIMIT.statelist
]
]
]

59
Common/CheckInSuites/ASR/config/TIMIT_TrainAutoEncoder.config
Просмотреть файл

@ -1,59 +0,0 @@
stderr=$ExpDir$\TrainAutoEncoder\log\log
command=TIMIT_TrainAutoEncoder
precision=float
#######################################
# TRAINING CONFIG (Simple, Fixed LR) #
#######################################
TIMIT_TrainAutoEncoder=[
action=train
modelPath=$ExpDir$\TrainAutoEncoder\model\cntkSpeech.dnn
# deviceId=-1 for CPU, >=0 for GPU devices
deviceId=$DeviceNumber$
traceLevel=1
# notation xxx:yyy*n:zzz is equivalent to xxx, then yyy repeated n times, then zzz
# example: 10:20*3:5 is equivalent to 10:20:20:20:5
NDLNetworkBuilder=[
ndlMacros=$NdlDir$\default_macros.ndl
NetworkDescription=$NdlDir$\ae.ndl
]
SGD=[
# epochSize=0 means epochSize is the size of the training set
epochSize=0
minibatchSize=256
learningRatesPerMB=0.1
momentumPerMB=0.9
dropoutRate=0.0
maxEpochs=$MaxNumEpochs$
]
# Parameter values for the reader
reader=[
# reader to use
readerType=HTKMLFReader
# readMethod=blockRandomize
readMethod=rollingWindow
miniBatchMode=Partial
randomize=Auto
verbosity=1
featIn=[
dim=792
scpFile=$FBankScpShort$
]
featOut=[
scpFile=$FBankScpShort$
dim=792
]
]
]

78
Common/CheckInSuites/ASR/config/TIMIT_TrainMultiInput.config
Просмотреть файл

@ -1,78 +0,0 @@
stderr=$ExpDir$\TrainMultiInput\log\log
command=TIMIT_TrainMultiInput
precision=float
################################################
# TRAINING CONFIG AUTOENCODER (NDL, Fixed LR) #
################################################
TIMIT_TrainMultiInput=[
action=train
modelPath=$ExpDir$\TrainMultiInput\model\cntkSpeech.dnn
#=-1 for CPU, >=0 for GPU devices
deviceId=$DeviceNumber$
traceLevel=1
NDLNetworkBuilder=[
ndlMacros=$NdlDir$\default_macros.ndl
networkDescription=$NdlDir$\mtl_fbank_mfcc.ndl
]
SGD=[
epochSize=0
minibatchSize=256:1024
learningRatesPerMB=0.8:3.2*14:0.08
momentumPerMB=0.9
dropoutRate=0.0
maxEpochs=$MaxNumEpochs$
gradientClippingWithTruncation=true
clippingThresholdPerSample=1#INF
#settings for Auto Adjust Learning Rate
#AutoAdjust=[
# # auto learning rate adjustment
# #autoAdjustLR=SearchBeforeEpoch
# reduceLearnRateIfImproveLessThan=0
# loadBestModel=true
# increaseLearnRateIfImproveMoreThan=1000000000
# learnRateDecreaseFactor=0.618
# learnRateIncreaseFactor=1.382
# numMiniBatch4LRSearch=500
# numPrevLearnRates=5
# numBestSearchEpoch=1
#]
gradientClippingWithTruncation=true
clippingThresholdPerSample=1#INF
]
# Parameter values for the reader
reader=[
readerType=HTKMLFReader
# readMethod=blockRandomize
# frameMode=true
readMethod=rollingWindow
pageFilePath=d:\temp\cntk\TIMIT\CheckInSuites\tempfiles
miniBatchMode=Partial
randomize=Auto
verbosity=1
features1=[
dim=792
scpFile=$FBankScpShort$
type=Real
]
features2=[
dim=39
scpFile=$MfccScpShort$
type=Real
]
labels=[
mlfFile=$MlfDir$\TIMIT.train.align_cistate.mlf.cntk
labelMappingFile=$MlfDir$\TIMIT.statelist
labelDim=183
labelType=Category
]
]
]

72
Common/CheckInSuites/ASR/config/TIMIT_TrainMultiTask.config
Просмотреть файл

@ -1,72 +0,0 @@
stderr=$ExpDir$\TrainMultiTask\log\log
command=TIMIT_TrainMultiTask
precision=float
TIMIT_TrainMultiTask=[
action=train
modelPath=$ExpDir$\TrainMultiTask\model\cntkSpeech.dnn
#=-1 for CPU, >=0 for GPU devices
deviceId=$DeviceNumber$
traceLevel=1
NDLNetworkBuilder=[
ndlMacros=$NdlDir$\default_macros.ndl
networkDescription=$NdlDir$\mtl_senones_dr.ndl
]
SGD=[
epochSize=0
minibatchSize=256:1024
learningRatesPerMB=0.8:3.2*14:0.08
momentumPerMB=0.9
dropoutRate=0.0
maxEpochs=$MaxNumEpochs$
gradientClippingWithTruncation=true
clippingThresholdPerSample=1#INF
#settings for Auto Adjust Learning Rate
#AutoAdjust=[
# # auto learning rate adjustment
# #autoAdjustLR=SearchBeforeEpoch
# reduceLearnRateIfImproveLessThan=0
# loadBestModel=true
# increaseLearnRateIfImproveMoreThan=1000000000
# learnRateDecreaseFactor=0.618
# learnRateIncreaseFactor=1.382
# numMiniBatch4LRSearch=500
# numPrevLearnRates=5
# numBestSearchEpoch=1
#]
gradientClippingWithTruncation=true
clippingThresholdPerSample=1#INF
]
# Parameter values for the reader
reader=[
readerType=HTKMLFReader
#readMethod=blockRandomize
readMethod=rollingWindow
miniBatchMode=Partial
randomize=Auto
verbosity=1
features=[
dim=792
scpFile=$FBankScpShort$
]
labels=[
mlfFile=$MlfDir$\TIMIT.train.align_cistate.mlf.cntk
labelMappingFile=$MlfDir$\TIMIT.statelist
labelDim=183
labelType=Category
]
regions=[
mlfFile=$MlfDir$\TIMIT.train.align_dr.mlf.cntk
labelDim=8
labelType=Category
]
]
]

72
Common/CheckInSuites/ASR/config/TIMIT_TrainNDLNetwork.config
Просмотреть файл

@ -1,72 +0,0 @@
stderr=$ExpDir$\TrainNDLNetwork\log\log
command=TIMIT_TrainNDLNetwork
precision=float
#######################################
# TRAINING CONFIG (NDL, Fixed LR) #
#######################################
TIMIT_TrainNDLNetwork=[
action=train
modelPath=$ExpDir$\TrainNDLNetwork\model\cntkSpeech.dnn
# deviceId=-1 for CPU, >=0 for GPU devices
deviceId=$DeviceNumber$
traceLevel=1
NDLNetworkBuilder=[
ndlMacros=$NdlDir$\default_macros.ndl
networkDescription=$NdlDir$\classify.ndl
]
SGD=[
epochSize=0
minibatchSize=256:1024
learningRatesPerMB=0.8:3.2*14:0.08
momentumPerMB=0.9
dropoutRate=0.0
maxEpochs=$MaxNumEpochs$
#settings for Auto Adjust Learning Rate
#AutoAdjust=[
# # auto learning rate adjustment
# #autoAdjustLR=SearchBeforeEpoch
# reduceLearnRateIfImproveLessThan=0
# loadBestModel=true
# increaseLearnRateIfImproveMoreThan=1000000000
# learnRateDecreaseFactor=0.618
# learnRateIncreaseFactor=1.382
# numMiniBatch4LRSearch=500
# numPrevLearnRates=5
# numBestSearchEpoch=1
#]
gradientClippingWithTruncation=true
clippingThresholdPerSample=1#INF
]
# Parameter values for the reader
reader=[
# reader to use
readerType=HTKMLFReader
# readMethod=blockRandomize
readMethod=rollingWindow
miniBatchMode=Partial
randomize=Auto
verbosity=1
myFeatures=[
dim=792
scpFile=$FBankScpShort$
]
myLabels=[
mlfFile=$MlfDir$\TIMIT.train.align_cistate.mlf.cntk
labelDim=183
labelMappingFile=$MlfDir$\TIMIT.statelist
]
]
]

66
Common/CheckInSuites/ASR/config/TIMIT_TrainSimpleNetwork.config
Просмотреть файл

@ -1,66 +0,0 @@
stderr=$ExpDir$\TrainSimpleNetwork\log\log
command=TIMIT_TrainSimpleNetwork
precision=float
#######################################
# TRAINING CONFIG (Simple, Fixed LR) #
#######################################
TIMIT_TrainSimpleNetwork=[
action=train
modelPath=$ExpDir$\TrainSimpleNetwork\model\cntkSpeech.dnn
# deviceId=-1 for CPU, >=0 for GPU devices
deviceId=$DeviceNumber$
traceLevel=1
# notation xxx:yyy*n:zzz is equivalent to xxx, then yyy repeated n times, then zzz
# example: 10:20*3:5 is equivalent to 10:20:20:20:5
SimpleNetworkBuilder=[
layerSizes=792:512*3:183
trainingCriterion=CrossEntropyWithSoftmax
evalCriterion=ErrorPrediction
layerTypes=Sigmoid
initValueScale=1.0
applyMeanVarNorm=true
uniformInit=true
needPrior=true
]
SGD=[
# epochSize=0 means epochSize is the size of the training set
epochSize=0
minibatchSize=256:1024
learningRatesPerMB=0.8:3.2*14:0.08
momentumPerMB=0.9
dropoutRate=0.0
maxEpochs=$MaxNumEpochs$
]
# Parameter values for the reader
reader=[
# reader to use
readerType=HTKMLFReader
# readMethod=blockRandomize
readMethod=rollingWindow
miniBatchMode=Partial
randomize=Auto
verbosity=1
features=[
dim=792
scpFile=$FBankScpShort$
]
labels=[
mlfFile=$MlfDir$\TIMIT.train.align_cistate.mlf.cntk
labelDim=183
labelMappingFile=$MlfDir$\TIMIT.statelist
]
]
]

40
Common/CheckInSuites/ASR/config/TIMIT_WriteBottleneck.config
Просмотреть файл

@ -1,40 +0,0 @@
stderr=$ExpDir$\WriteBottleneck\log\log
command=TIMIT_WriteBottleneck
precision=float
##################
# WRITER CONFIG #
##################
TIMIT_WriteBottleneck=[
action=write
modelPath=$ExpDir$\TrainAutoEncoder\model\cntkSpeech.dnn
# deviceId=-1 for CPU, >=0 for GPU devices
deviceId=$DeviceNumber$
traceLevel=1
# notation xxx:yyy*3:zzz is equivalent to xxx:yyy:yyy:yyy:zzz
outputNodeNames=L2.BFF.FF.P
# Parameter values for the reader
reader=[
# reader to use
readerType=HTKMLFReader
featIn=[
dim=792
scpFile=$FBankScpShortTest$
type=Real
]
]
writer=[
writerType=HTKMLFWriter
L2.BFF.FF.P = [
dim=64
scpFile=$BNeckScpShortTest$
type=Real
]
]
]

40
Common/CheckInSuites/ASR/config/TIMIT_WriteScaledLogLike.config
Просмотреть файл

@ -1,40 +0,0 @@
stderr=$ExpDir$\WriteScaledLogLike\log\log
command=TIMIT_WriteScaledLogLike
precision=float
##################
# WRITER CONFIG #
##################
TIMIT_WriteScaledLogLike=[
action=write
modelPath=$ExpDir$\TrainSimpleNetwork\model\cntkSpeech.dnn
# deviceId=-1 for CPU, >=0 for GPU devices
deviceId=$DeviceNumber$
traceLevel=1
# notation xxx:yyy*3:zzz is equivalent to xxx:yyy:yyy:yyy:zzz
outputNodeNames=ScaledLogLikelihood
# Parameter values for the reader
reader=[
# reader to use
readerType=HTKMLFReader
features=[
dim=792
scpFile=$FBankScpShortTest$
type=Real
]
]
writer=[
writerType=HTKMLFWriter
ScaledLogLikelihood = [
dim=183
scpFile=$LogLikeScpShortTest$
type=Real
]
]
]

22
Common/CheckInSuites/ASR/config/globals.config
Просмотреть файл

@ -1,22 +0,0 @@
# globals.config - config for ASR check-in tests
# the following variables must be passed on the command line to CNTK:
# - WorkDir e.g. ..\..\..\ExampleSetups\ASR\TIMIT # assumes current directory is location of this config file; override on cmd line if not correct
# - ExpDir e.g. ..\test_out\cpu
# - LibDir e.g. $WorkDir$\lib
# - ScpDir e.g. $LibDir$\scp
# data paths
FBankScpShort=$ScpDir$\TIMIT.train.scp.fbank.fullpath.100
MfccScpShort=$ScpDir$\TIMIT.train.scp.mfcc.fullpath.100
FBankScpShortTest=$ScpDir$\TIMIT.core.scp.fbank.fullpath.20
LogLikeScpShortTest=$ScpDir$\TIMIT.core.scp.scaledloglike.fullpath.20
BNeckScpShortTest=$ScpDir$\TIMIT.core.scp.bottleneck.fullpath.20
MlfDir=$LibDir$\mlf
# config paths & parameters
ConfigDir=$WorkDir$\config
NdlDir=$ConfigDir$
MelDir=$ConfigDir$
MaxNumEpochs=5

37
Common/CheckInSuites/ASR/config/runall.bat
Просмотреть файл

@ -1,37 +0,0 @@
::: first argument is CPU or GPU
set PROC=%1
::: second argument is buildconfig (Release or Debug)
set BUILD=%2
echo running ASR test on %PROC%
::: the CNTK executable is found relative to this BAT file
set THIS=%~dp0
set ROOT=%THIS%..\..\..
set CNTK=%ROOT%\x64\%2\CNTK.exe
::: directories we pass in to CNTK config
::: example setups are here
set WorkDir=%ROOT%\ExampleSetups\ASR\TIMIT
set ExpDir=%THIS%..\test_out
set LibDir=%WorkDir%\lib
set ScpDir=%LibDir%\scp
::: run all tests
::: TODO: fix the log path, it seems it cannot be passed to CNTK currently on the command line
for %%t in (TrainSimpleNetwork TrainNDLNetwork TrainAutoEncoder TrainMultiInput TrainMultiTask EvalSimpleNetwork WriteScaledLogLike WriteBottleneck) do (
echo ------
echo running test TIMIT_%%t.config logging to %ExpDir%\%%t\log\log_TIMIT_%%t.log
%CNTK% WorkDir=%WorkDir% ExpDir=%ExpDir% LibDir=%LibDir% ScpDir=%ScpDir% configFile=%THIS%globals.config+%THIS%select_%PROC%.config+%THIS%TIMIT_%%t.config
if ERRORLEVEL 1 (
echo CNTK FAILED:
findstr /I EXCEPTION %ExpDir%\%%t\log\log_TIMIT_%%t.log
) else (
echo CNTK OUTPUT:
findstr /I Finished %ExpDir%\%%t\log\log_TIMIT_%%t.log
findstr /I EXCEPTION %ExpDir%\%%t\log\log_TIMIT_%%t.log
echo REFERENCE:
findstr /I Finished %THIS%..\%PROC%\%%t.output
)
)

1
Common/CheckInSuites/ASR/config/select_cpu.config
Просмотреть файл

@ -1 +0,0 @@
DeviceNumber=-1

1
Common/CheckInSuites/ASR/config/select_gpu.config
Просмотреть файл

@ -1 +0,0 @@
DeviceNumber=0

193
Common/CheckInSuites/CheckInSuites.vcxproj
Просмотреть файл

@ -1,193 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|x64">
<Configuration>Debug</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|x64">
<Configuration>Release</Configuration>
<Platform>x64</Platform>
</ProjectConfiguration>
</ItemGroup>
<ItemGroup>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\add_layer.mel" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\ae.ndl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\classify.ndl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\create_1layer.ndl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\default_macros.ndl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\globals.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\lstm.ndl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\mtl_fbank_mfcc.ndl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\mtl_senones_dr.ndl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\PAC-RNN.ndl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\README.txt" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_AdaptLearnRate.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_CrossValidateSimpleNetwork.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_EvalSimpleNetwork.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainAutoEncoder.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainLSTM.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainMultiInput.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainMultiTask.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainNDLNetwork.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainSimpleNetwork.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainWithPreTrain.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_WriteBottleneck.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_WriteScaledLogLike.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\CPU\LSTM_ExpectedOutputs.txt" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\CPU\TIMIT_DNN.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\CPU\TIMIT_LSTM.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.bigram.arpa" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.dnn_map" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.lookahead" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.tfsa" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.transitions" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\GPU\DNN_ExpectedOutputs.txt" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\GPU\LSTM_ExpectedOutputs.txt" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\GPU\TIMIT_DNN.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\GPU\TIMIT_LSTM.config" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\cf\CF.fbank24_zda" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\make_dr_mlf.pl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\make_mlf_cntk.pl" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.core.align_cistate.mlf" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.core.align_cistate.mlf.cntk" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.core.align_dr.mlf.cntk" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.dev.align_cistate.mlf" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.dev.align_cistate.mlf.cntk" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.dev.align_dr.mlf.cntk" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.statelist" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.train.align_cistate.mlf" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.train.align_cistate.mlf.cntk" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.train.align_dr.mlf.cntk" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.bottleneck.fullpath" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.bottleneck.fullpath.20" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.fbank.fullpath" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.fbank.fullpath.20" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.scaledloglike.fullpath" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.scaledloglike.fullpath.20" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.dev.scp.fbank.fullpath" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.dev.scp.fbank.fullpath.rnn" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.dev.scp.fbank.fullpath.rnn.20" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank.fullpath" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank.fullpath.100" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank.fullpath.rnn" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank.fullpath.rnn.100" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.mfcc.fullpath" />
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.mfcc.fullpath.100" />
<None Include="..\..\ExampleSetups\LM\LSTMLM\ExpectedResults.log" />
<None Include="..\..\ExampleSetups\LM\LSTMLM\global.config" />
<None Include="..\..\ExampleSetups\LM\LSTMLM\lstmlm.gpu.config" />
<None Include="..\..\ExampleSetups\LM\LSTMLM\lstmlmconfig.txt" />
<None Include="..\..\ExampleSetups\LM\LSTMLM\perplexity.nce100.lr0.1.txt" />
<None Include="..\..\ExampleSetups\LM\LSTMLM\ptb.test.cntk.100.txt" />
<None Include="..\..\ExampleSetups\LM\LSTMLM\ptb.train.cntk.100.txt" />
<None Include="..\..\ExampleSetups\LM\LSTMLM\ptb.valid.cntk.100.txt" />
<None Include="..\..\ExampleSetups\LM\RNNLM\CPU\ExpectedOutputs.txt" />
<None Include="..\..\ExampleSetups\LM\RNNLM\CPU\fnnlm.config" />
<None Include="..\..\ExampleSetups\LM\RNNLM\CPU\global.config" />
<None Include="..\..\ExampleSetups\LM\RNNLM\CPU\rnnlm.config" />
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\ExpectedOutputs.txt" />
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\fnnlm.config" />
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\global.config" />
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\rnnlm.config" />
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\rnnlm.gpu.config" />
<None Include="..\..\ExampleSetups\LM\RNNLM\rnnlm.ndl" />
<None Include="..\..\ExampleSetups\NdlExamples\NDLExamples.ndl" />
<None Include="..\..\ExampleSetups\SLU\conlleval.pl" />
<None Include="..\..\ExampleSetups\SLU\feat.txt" />
<None Include="..\..\ExampleSetups\SLU\globals.config" />
<None Include="..\..\ExampleSetups\SLU\lbl.txt" />
<None Include="..\..\ExampleSetups\SLU\lstmNDL.txt" />
<None Include="..\..\ExampleSetups\SLU\README.txt" />
<None Include="..\..\ExampleSetups\SLU\rnnlu.config" />
<None Include="..\..\ExampleSetups\SLU\rnnluModelEditor.txt" />
<None Include="..\..\ExampleSetups\SLU\score.sh" />
<None Include="ASR\config\globals.config" />
<None Include="ASR\config\README.txt" />
<None Include="ASR\config\runall.bat" />
<None Include="ASR\config\select_cpu.config" />
<None Include="ASR\config\select_gpu.config" />
<None Include="ASR\config\TIMIT_EvalSimpleNetwork.config" />
<None Include="ASR\config\TIMIT_TrainAutoEncoder.config" />
<None Include="ASR\config\TIMIT_TrainMultiInput.config" />
<None Include="ASR\config\TIMIT_TrainMultiTask.config" />
<None Include="ASR\config\TIMIT_TrainNDLNetwork.config" />
<None Include="ASR\config\TIMIT_TrainSimpleNetwork.config" />
<None Include="ASR\config\TIMIT_WriteBottleneck.config" />
<None Include="ASR\config\TIMIT_WriteScaledLogLike.config" />
<None Include="ASR\CPU\EvalSimpleNetwork.output" />
<None Include="ASR\CPU\TrainAutoEncoder.output" />
<None Include="ASR\CPU\TrainMultiInput.output" />
<None Include="ASR\CPU\TrainMultiTask.output" />
<None Include="ASR\CPU\TrainNDLNetwork.output" />
<None Include="ASR\CPU\TrainSimpleNetwork.output" />
<None Include="ASR\GPU\EvalSimpleNetwork.output" />
<None Include="ASR\GPU\TrainAutoEncoder.output" />
<None Include="ASR\GPU\TrainMultiInput.output" />
<None Include="ASR\GPU\TrainMultiTask.output" />
<None Include="ASR\GPU\TrainNDLNetwork.output" />
<None Include="ASR\GPU\TrainSimpleNetwork.output" />
<None Include="LM\RNNLM\CPU\ExpectedOutputs.txt" />
<None Include="LM\RNNLM\CPU\global.config" />
<None Include="LM\RNNLM\CPU\rnnlm.config" />
<None Include="LM\RNNLM\GPU\ExpectedOutputs.txt" />
<None Include="LM\RNNLM\GPU\rnnlm.config" />
<None Include="MNIST\CPU\expectedresults.txt" />
<None Include="MNIST\DefaultMacros.ndl" />
<None Include="MNIST\Example.ndl" />
<None Include="MNIST\GPU\expectedresults.txt" />
<None Include="MNIST\mnistCheckIn.config" />
<None Include="MNIST\mnistlabels.txt" />
<None Include="runall.bat" />
<None Include="SLU\atis.dev.IOB.simple" />
<None Include="SLU\atis.test.apos.pred.pos.head.IOB.simple" />
<None Include="SLU\atis.train.apos.pred.pos.head.IOB.simple" />
<None Include="SLU\globals.config" />
<None Include="SLU\input.txt" />
<None Include="SLU\inputmap.txt" />
<None Include="SLU\lstmNDL.txt" />
<None Include="SLU\output.txt" />
<None Include="SLU\README.txt" />
<None Include="SLU\rnnlu.config" />
<None Include="SLU\rnnlu.ndl.config" />
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{DBB3C106-B0B4-4059-8477-C89528CEC1B0}</ProjectGuid>
<Keyword>MakeFileProj</Keyword>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="Configuration">
<ConfigurationType>Makefile</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="Configuration">
<ConfigurationType>Makefile</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v120</PlatformToolset>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'" Label="PropertySheets">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<NMakePreprocessorDefinitions>WIN32;_DEBUG;$(NMakePreprocessorDefinitions)</NMakePreprocessorDefinitions>
<NMakeBuildCommandLine>runall.bat $(Configuration)</NMakeBuildCommandLine>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<NMakePreprocessorDefinitions>WIN32;NDEBUG;$(NMakePreprocessorDefinitions)</NMakePreprocessorDefinitions>
<NMakeBuildCommandLine>runall.bat $(Configuration)</NMakeBuildCommandLine>
</PropertyGroup>
<ItemDefinitionGroup>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>

517
Common/CheckInSuites/CheckInSuites.vcxproj.filters
Просмотреть файл

@ -1,517 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="CheckInSuites">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0000}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\ASR">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0001}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\ASR\config">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0002}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\ASR\CPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0003}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\ASR\GPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0004}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\LM">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0005}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\LM\RNNLM">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0006}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\LM\RNNLM\CPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0007}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\LM\RNNLM\GPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0008}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\MNIST">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0009}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\MNIST\CPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0010}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\MNIST\GPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0011}</UniqueIdentifier>
</Filter>
<Filter Include="CheckInSuites\SLU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0012}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0013}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0014}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0015}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT\config">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0016}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT\CPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0017}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT\decoding">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0018}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT\GPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0019}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT\lib">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0020}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT\lib\cf">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0021}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT\lib\mlf">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0022}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\ASR\TIMIT\lib\scp">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0023}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\LM">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0024}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\LM\LSTMLM">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0025}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\LM\RNNLM">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0026}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\LM\RNNLM\CPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0027}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\LM\RNNLM\GPU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0028}</UniqueIdentifier>
</Filter>
<Filter Include="ExampleSetups\SLU">
<UniqueIdentifier>{39d7aaf4-beea-43e1-8bc3-1c2a442f0029}</UniqueIdentifier>
</Filter>
</ItemGroup>
<ItemGroup>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\add_layer.mel">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\ae.ndl">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\classify.ndl">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\create_1layer.ndl">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\default_macros.ndl">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\globals.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\lstm.ndl">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\mtl_fbank_mfcc.ndl">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\mtl_senones_dr.ndl">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\PAC-RNN.ndl">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\README.txt">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_AdaptLearnRate.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_CrossValidateSimpleNetwork.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_EvalSimpleNetwork.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainAutoEncoder.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainLSTM.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainMultiInput.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainMultiTask.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainNDLNetwork.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainSimpleNetwork.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_TrainWithPreTrain.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_WriteBottleneck.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\config\TIMIT_WriteScaledLogLike.config">
<Filter>ExampleSetups\ASR\TIMIT\config</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\CPU\LSTM_ExpectedOutputs.txt">
<Filter>ExampleSetups\ASR\TIMIT\CPU</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\CPU\TIMIT_DNN.config">
<Filter>ExampleSetups\ASR\TIMIT\CPU</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\CPU\TIMIT_LSTM.config">
<Filter>ExampleSetups\ASR\TIMIT\CPU</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.bigram.arpa">
<Filter>ExampleSetups\ASR\TIMIT\decoding</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.dnn_map">
<Filter>ExampleSetups\ASR\TIMIT\decoding</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.lookahead">
<Filter>ExampleSetups\ASR\TIMIT\decoding</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.tfsa">
<Filter>ExampleSetups\ASR\TIMIT\decoding</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\decoding\TIMIT.transitions">
<Filter>ExampleSetups\ASR\TIMIT\decoding</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\GPU\DNN_ExpectedOutputs.txt">
<Filter>ExampleSetups\ASR\TIMIT\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\GPU\LSTM_ExpectedOutputs.txt">
<Filter>ExampleSetups\ASR\TIMIT\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\GPU\TIMIT_DNN.config">
<Filter>ExampleSetups\ASR\TIMIT\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\GPU\TIMIT_LSTM.config">
<Filter>ExampleSetups\ASR\TIMIT\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\cf\CF.fbank24_zda">
<Filter>ExampleSetups\ASR\TIMIT\lib\cf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\make_dr_mlf.pl">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\make_mlf_cntk.pl">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.core.align_cistate.mlf">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.core.align_cistate.mlf.cntk">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.core.align_dr.mlf.cntk">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.dev.align_cistate.mlf">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.dev.align_cistate.mlf.cntk">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.dev.align_dr.mlf.cntk">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.statelist">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.train.align_cistate.mlf">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.train.align_cistate.mlf.cntk">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\mlf\TIMIT.train.align_dr.mlf.cntk">
<Filter>ExampleSetups\ASR\TIMIT\lib\mlf</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.bottleneck.fullpath">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.bottleneck.fullpath.20">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.fbank.fullpath">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.fbank.fullpath.20">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.scaledloglike.fullpath">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.core.scp.scaledloglike.fullpath.20">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.dev.scp.fbank.fullpath">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.dev.scp.fbank.fullpath.rnn">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.dev.scp.fbank.fullpath.rnn.20">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank.fullpath">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank.fullpath.100">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank.fullpath.rnn">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.fbank.fullpath.rnn.100">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.mfcc.fullpath">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\ASR\TIMIT\lib\scp\TIMIT.train.scp.mfcc.fullpath.100">
<Filter>ExampleSetups\ASR\TIMIT\lib\scp</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\LSTMLM\ExpectedResults.log">
<Filter>ExampleSetups\LM\LSTMLM</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\LSTMLM\global.config">
<Filter>ExampleSetups\LM\LSTMLM</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\LSTMLM\lstmlm.gpu.config">
<Filter>ExampleSetups\LM\LSTMLM</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\LSTMLM\lstmlmconfig.txt">
<Filter>ExampleSetups\LM\LSTMLM</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\LSTMLM\perplexity.nce100.lr0.1.txt">
<Filter>ExampleSetups\LM\LSTMLM</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\LSTMLM\ptb.test.cntk.100.txt">
<Filter>ExampleSetups\LM\LSTMLM</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\LSTMLM\ptb.train.cntk.100.txt">
<Filter>ExampleSetups\LM\LSTMLM</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\LSTMLM\ptb.valid.cntk.100.txt">
<Filter>ExampleSetups\LM\LSTMLM</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\CPU\ExpectedOutputs.txt">
<Filter>ExampleSetups\LM\RNNLM\CPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\CPU\fnnlm.config">
<Filter>ExampleSetups\LM\RNNLM\CPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\CPU\global.config">
<Filter>ExampleSetups\LM\RNNLM\CPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\CPU\rnnlm.config">
<Filter>ExampleSetups\LM\RNNLM\CPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\ExpectedOutputs.txt">
<Filter>ExampleSetups\LM\RNNLM\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\fnnlm.config">
<Filter>ExampleSetups\LM\RNNLM\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\global.config">
<Filter>ExampleSetups\LM\RNNLM\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\rnnlm.config">
<Filter>ExampleSetups\LM\RNNLM\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\GPU\rnnlm.gpu.config">
<Filter>ExampleSetups\LM\RNNLM\GPU</Filter>
</None>
<None Include="..\..\ExampleSetups\LM\RNNLM\rnnlm.ndl">
<Filter>ExampleSetups\LM\RNNLM</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\conlleval.pl">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\feat.txt">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\globals.config">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\lbl.txt">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\lstmNDL.txt">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\README.txt">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\rnnlu.config">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\rnnluModelEditor.txt">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="..\..\ExampleSetups\SLU\score.sh">
<Filter>ExampleSetups\SLU</Filter>
</None>
<None Include="ASR\config\README.txt">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\runall.bat">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\TIMIT_EvalSimpleNetwork.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\TIMIT_TrainAutoEncoder.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\TIMIT_TrainMultiInput.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\TIMIT_TrainMultiTask.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\TIMIT_TrainNDLNetwork.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\TIMIT_TrainSimpleNetwork.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\TIMIT_WriteBottleneck.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\TIMIT_WriteScaledLogLike.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\CPU\EvalSimpleNetwork.output">
<Filter>CheckInSuites\ASR\CPU</Filter>
</None>
<None Include="ASR\CPU\TrainAutoEncoder.output">
<Filter>CheckInSuites\ASR\CPU</Filter>
</None>
<None Include="ASR\CPU\TrainMultiInput.output">
<Filter>CheckInSuites\ASR\CPU</Filter>
</None>
<None Include="ASR\CPU\TrainMultiTask.output">
<Filter>CheckInSuites\ASR\CPU</Filter>
</None>
<None Include="ASR\CPU\TrainNDLNetwork.output">
<Filter>CheckInSuites\ASR\CPU</Filter>
</None>
<None Include="ASR\CPU\TrainSimpleNetwork.output">
<Filter>CheckInSuites\ASR\CPU</Filter>
</None>
<None Include="ASR\GPU\EvalSimpleNetwork.output">
<Filter>CheckInSuites\ASR\GPU</Filter>
</None>
<None Include="ASR\GPU\TrainAutoEncoder.output">
<Filter>CheckInSuites\ASR\GPU</Filter>
</None>
<None Include="ASR\GPU\TrainMultiInput.output">
<Filter>CheckInSuites\ASR\GPU</Filter>
</None>
<None Include="ASR\GPU\TrainMultiTask.output">
<Filter>CheckInSuites\ASR\GPU</Filter>
</None>
<None Include="ASR\GPU\TrainNDLNetwork.output">
<Filter>CheckInSuites\ASR\GPU</Filter>
</None>
<None Include="ASR\GPU\TrainSimpleNetwork.output">
<Filter>CheckInSuites\ASR\GPU</Filter>
</None>
<None Include="LM\RNNLM\CPU\ExpectedOutputs.txt">
<Filter>CheckInSuites\LM\RNNLM\CPU</Filter>
</None>
<None Include="LM\RNNLM\CPU\global.config">
<Filter>CheckInSuites\LM\RNNLM\CPU</Filter>
</None>
<None Include="LM\RNNLM\CPU\rnnlm.config">
<Filter>CheckInSuites\LM\RNNLM\CPU</Filter>
</None>
<None Include="LM\RNNLM\GPU\ExpectedOutputs.txt">
<Filter>CheckInSuites\LM\RNNLM\GPU</Filter>
</None>
<None Include="LM\RNNLM\GPU\rnnlm.config">
<Filter>CheckInSuites\LM\RNNLM\GPU</Filter>
</None>
<None Include="MNIST\CPU\expectedresults.txt">
<Filter>CheckInSuites\MNIST\CPU</Filter>
</None>
<None Include="MNIST\DefaultMacros.ndl">
<Filter>CheckInSuites\MNIST</Filter>
</None>
<None Include="MNIST\Example.ndl">
<Filter>CheckInSuites\MNIST</Filter>
</None>
<None Include="MNIST\GPU\expectedresults.txt">
<Filter>CheckInSuites\MNIST\GPU</Filter>
</None>
<None Include="MNIST\mnistCheckIn.config">
<Filter>CheckInSuites\MNIST</Filter>
</None>
<None Include="MNIST\mnistlabels.txt">
<Filter>CheckInSuites\MNIST</Filter>
</None>
<None Include="SLU\atis.dev.IOB.simple">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\atis.test.apos.pred.pos.head.IOB.simple">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\atis.train.apos.pred.pos.head.IOB.simple">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\globals.config">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\input.txt">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\inputmap.txt">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\lstmNDL.txt">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\output.txt">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\README.txt">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\rnnlu.config">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="SLU\rnnlu.ndl.config">
<Filter>CheckInSuites\SLU</Filter>
</None>
<None Include="ASR\config\globals.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\select_cpu.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="ASR\config\select_gpu.config">
<Filter>CheckInSuites\ASR\config</Filter>
</None>
<None Include="runall.bat">
<Filter>CheckInSuites</Filter>
</None>
<None Include="..\..\ExampleSetups\NdlExamples\NDLExamples.ndl">
<Filter>ExampleSetups</Filter>
</None>
</ItemGroup>
</Project>

21
Common/CheckInSuites/MNIST/CPU/expectedresults.txt
Просмотреть файл

@ -1,21 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 0.40258092 EvalErr Per Sample = 0.12081667 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=11.784
Finished Epoch[1]: [Validation Set] Train Loss Per Sample = 0.24683523 EvalErr Per Sample = 0.073100001
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 0.19932216 EvalErr Per Sample = 0.058800001 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=3.7809999
Finished Epoch[2]: [Validation Set] Train Loss Per Sample = 0.18965457 EvalErr Per Sample = 0.056000002
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 0.1451894 EvalErr Per Sample = 0.043366667 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=4.5050001
Finished Epoch[3]: [Validation Set] Train Loss Per Sample = 0.12459327 EvalErr Per Sample = 0.037
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 0.11351292 EvalErr Per Sample = 0.033433333 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=3.763
Finished Epoch[4]: [Validation Set] Train Loss Per Sample = 0.11849558 EvalErr Per Sample = 0.035700001
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 0.094123989 EvalErr Per Sample = 0.027466668 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=8.2119999
Finished Epoch[5]: [Validation Set] Train Loss Per Sample = 0.097263642 EvalErr Per Sample = 0.0297
Finished Epoch[6]: [Training Set] Train Loss Per Sample = 0.079397582 EvalErr Per Sample = 0.023949999 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=3.4319999
Finished Epoch[6]: [Validation Set] Train Loss Per Sample = 0.086300962 EvalErr Per Sample = 0.0262
Finished Epoch[7]: [Training Set] Train Loss Per Sample = 0.067904279 EvalErr Per Sample = 0.019966668 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=3.4849999
Finished Epoch[7]: [Validation Set] Train Loss Per Sample = 0.084572069 EvalErr Per Sample = 0.025699999
Finished Epoch[8]: [Training Set] Train Loss Per Sample = 0.05927179 EvalErr Per Sample = 0.017266667 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=3.493
Finished Epoch[8]: [Validation Set] Train Loss Per Sample = 0.086920179 EvalErr Per Sample = 0.0265
Finished Epoch[9]: [Training Set] Train Loss Per Sample = 0.052622452 EvalErr Per Sample = 0.015666667 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=4.0120001
Finished Epoch[9]: [Validation Set] Train Loss Per Sample = 0.073479153 EvalErr Per Sample = 0.0218
Finished Epoch[10]: [Training Set] Train Loss Per Sample = 0.046487566 EvalErr Per Sample = 0.013533333 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=3.7950001
Finished Epoch[10]: [Validation Set] Train Loss Per Sample = 0.078413151 EvalErr Per Sample = 0.0243
Final Results: Minibatch[1-100]: Samples Seen = 10000 ErrorPrediction/Sample = 0.0243 CrossEntropyWithSoftmax/Sample = 0.078413136

72
Common/CheckInSuites/MNIST/DefaultMacros.ndl
Просмотреть файл

@ -1,72 +0,0 @@
# Macro definitions
#inline Rectified Linear Feed Forward
RFF_R(x1, w1, b1)=RectifiedLinear(Plus(Times(w1,x1),b1))
#Feed Forward
FF(X1, W1, B1)
{
T=Times(W1,X1);
P=Plus(T, B1);
}
#Base feed Forward network, defines Bias and wieght parameters
BFF(in, rows, cols)
{
B=Parameter(rows, init="fixedValue", value=0)
W=Parameter(rows, cols)
FF = FF(in, w, b)
}
#RectifiedLinear Base Feed Forward
RBFF(in,rowCount,colCount)
{
BFF = BFF(in, rowCount, colCount);
RL = RectifiedLinear(BFF);
}
#Rectified Linear Feed Forward
RFF(X2,W2,B2)
{
FF = FF(X2, W2, B2);
RL = RectifiedLinear(FF);
}
#RectifiedLinear Feed Forward with Dropout
RFFD(X3,W3,B3)
{
RFF=RFF(X3, W3, B3)
DO=Dropout(RFF)
}
#Sigmoid Base Feed Forward
SBFF(in,rowCount,colCount)
{
BFF = BFF(in, rowCount, colCount);
S = Sigmoid(BFF);
}
#Sigmoid Feed Forward
SFF(X2,W2,B2)
{
FF = FF(X2, W2, B2);
S = Sigmoid(FF);
}
#Sigmoid Feed Forward with Dropout
SFFD(X3,W3,B3)
{
SFF=SFF(X3, W3, B3)
DO=Dropout(SFF)
}
#Sigmoid Feed Forward with Dropout
SBFFD(input,rowCount,colCount)
{
SBFF=SBFF(input,rowCount,colCount)
DO=Dropout(SBFF)
}
#SoftMax Feed Forward
SMFF(x,y,z, labels)
{
FF = FF(x,y,z);
SM = CrossEntropyWithSoftmax(labels, FF)
}
#SoftMax Base Feed Forward
SMBFF(x,r,c, labels)
{
BFF = BFF(x,r,c);
SM = CrossEntropyWithSoftmax(labels, BFF)
}
RFFD_R(x1, w1, b1)=Dropout(RectifiedLinear(Plus(Times(w1,x1),b1)))

364
Common/CheckInSuites/MNIST/Example.ndl
Просмотреть файл

@ -1,364 +0,0 @@
load=ndlMacroDefine
run=ndlFull
ndlFull=[
SDim=784
HDim=256
LDim=10
B0=Parameter(HDim)
W0=Parameter(HDim, SDim)
features=Input(SDim)
labels=Input(LDim)
#mean0=Mean(features)
#invstd0=InvStdDev(features)
#input=PerDimMeanVarNormalization(features, mean0, invstd0)
Times1=Times(W0, features)
Plus1=Plus(Times1, B0)
RL1=RectifiedLinear(Plus1)
B1=Parameter(LDim, 1)
W1=Parameter(LDim, HDim)
Times2=Times(W1, RL1)
Plus2=Plus(Times2, B1)
CE=CrossEntropyWithSoftmax(labels, Plus2)
ErrPredict=ErrorPrediction(labels, Plus2)
FeatureNodes=(features)
LabelNodes=(labels)
CriterionNodes=(CE)
EvalNodes=(ErrPredict)
OutputNodes=(Plus2)
]
ndlMacroDefine=[
# Macro definitions
#inline Rectified Linear Feed Forward
RFF_R(x1, w1, b1)=RectifiedLinear(Plus(Times(w1,x1),b1))
#Feed Forward
FF(X1, W1, B1)
{
T=Times(W1,X1);
P=Plus(T, B1);
}
#Base feed Forward network, defines Bias and wieght parameters
BFF(in, rows, cols)
{
B=Parameter(rows, init="fixedValue", value=0)
W=Parameter(rows, cols)
FF = FF(in, w, b)
}
#RectifiedLinear Base Feed Forward
RBFF(in,rowCount,colCount)
{
BFF = BFF(in, rowCount, colCount);
RL = RectifiedLinear(BFF);
}
#Rectified Linear Feed Forward
RFF(X2,W2,B2)
{
FF = FF(X2, W2, B2);
RL = RectifiedLinear(FF);
}
#RectifiedLinear Feed Forward with Dropout
RFFD(X3,W3,B3)
{
RFF=RFF(X3, W3, B3)
DO=Dropout(RFF)
}
#Sigmoid Base Feed Forward
SBFF(in,rowCount,colCount)
{
BFF = BFF(in, rowCount, colCount);
S = Sigmoid(BFF);
}
#Sigmoid Feed Forward
SFF(X2,W2,B2)
{
FF = FF(X2, W2, B2);
S = Sigmoid(FF);
}
#Sigmoid Feed Forward with Dropout
SFFD(X3,W3,B3)
{
SFF=SFF(X3, W3, B3)
DO=Dropout(SFF)
}
#Sigmoid Feed Forward with Dropout
SBFFD(input,rowCount,colCount)
{
SBFF=SBFF(input,rowCount,colCount)
DO=Dropout(SBFF)
}
#SoftMax Feed Forward
SMFF(x,y,z, labels)
{
FF = FF(x,y,z);
SM = CrossEntropyWithSoftmax(labels, FF)
}
#SoftMax Base Feed Forward
SMBFF(x,r,c, labels)
{
BFF = BFF(x,r,c);
SM = CrossEntropyWithSoftmax(labels, BFF)
}
RFFD_R(x1, w1, b1)=Dropout(RectifiedLinear(Plus(Times(w1,x1),b1)))
]
ndlMacroUse2=[
# constants defined
# Sample, Hidden, and Label dimensions
SDim=784
HDim=256
LDim=10
features=Input(SDim, tag="feature")
labels=Input(LDim, tag="label")
# compute mean/stddev for mean/stddev normalization
fmean = Mean(features);
finvstd=InvStdDev(features)
finput=PerDimMeanVarNormalization(features, fmean, finvstd)
# Layer operations
L1 = RBFF(finput, HDim, SDim)
L2 = RBFF(L1, HDim, HDim)
L3 = RBFF(L2, HDim, HDim)
CE = SMBFF(L3, LDim, HDim, labels, tag="criterion")
Err=ErrorPrediction(labels, CE.BFF, tag="eval")
# rootNodes defined here
OutputNodes=(CE.BFF)
]
ndlMacroUseCNNSubSample2ZeroPadding=[
# constants defined
# Sample, Hidden, and Label dimensions
SDim=784
LDim=10
features=Input(SDim, tag="feature")
labels=Input(LDim, tag="label")
#convolution
inputWidth=28
inputHeight=28
inputChannels=1
kernelWidth=5
kernelHeight=5
outputChannels=24
horizontalSubsample=2
verticalSubsample=2
# weight[outputChannels, kernelWidth * kernelHeight * inputChannels]
cvweight=Parameter(outputChannels, 25)
cv = Convolution(cvweight, features, inputWidth, inputHeight, inputChannels, kernelWidth, kernelHeight, outputChannels,horizontalSubsample, verticalSubsample, zeroPadding=true)
#one bias per channel
cvbias=Parameter(outputChannels, 1)
cvplusbias=Plus(cv, cvbias);
nlcv=Sigmoid(cvplusbias);
#outputWidth = (m_inputWidth-1)/m_horizontalSubsample + 1;
outputWidth=14
#outputHeight = (m_inputHeight-1)/m_verticalSubsample + 1;
outputHeight=14
#maxpooling
windowWidth=2
windowHeight=2
stepW=2
stepH=2
mp=MaxPooling(nlcv, outputWidth, outputHeight, outputChannels, windowWidth, windowHeight, stepW, stepH)
#m_outputWidth = (m_inputWidth-m_windowWidth)/m_horizontalSubsample + 1;
mpoutputWidth=7
#m_outputHeight = (m_inputHeight-m_windowHeight)/m_verticalSubsample + 1;
mpoutputHeight=7
#m_outputSizePerSample = m_outputWidth * m_outputHeight * m_channels;
mpoutputSizePerSample=1176
# Layer operations
HDim=128
L1 = SBFF(mp, HDim, mpoutputSizePerSample)
CE = SMBFF(L1, LDim, HDim, labels, tag="criterion")
Err=ErrorPrediction(labels, CE.BFF, tag="eval")
# rootNodes defined here
OutputNodes=(CE.BFF)
]
ndlMacroUseCNNSubSample2=[
# constants defined
# Sample, Hidden, and Label dimensions
SDim=784
LDim=10
features=Input(SDim, tag="feature")
labels=Input(LDim, tag="label")
#convolution
inputWidth=28
inputHeight=28
inputChannels=1
kernelWidth=5
kernelHeight=5
outputChannels=24
horizontalSubsample=2
verticalSubsample=2
# weight[outputChannels, kernelWidth * kernelHeight * inputChannels]
cvweight=Parameter(outputChannels, 25)
cv = Convolution(cvweight, features, inputWidth, inputHeight, inputChannels, kernelWidth, kernelHeight, outputChannels,horizontalSubsample, verticalSubsample, zeroPadding=false)
#one bias per channel
cvbias=Parameter(outputChannels, 1)
cvplusbias=Plus(cv, cvbias);
nlcv=Sigmoid(cvplusbias);
#outputWidth = (m_inputWidth-m_kernelWidth)/m_horizontalSubsample + 1;
outputWidth=12
#outputHeight = (m_inputHeight-m_kernelHeight)/m_verticalSubsample + 1;
outputHeight=12
#maxpooling
windowWidth=2
windowHeight=2
stepW=2
stepH=2
mp=MaxPooling(nlcv, outputWidth, outputHeight, outputChannels, windowWidth, windowHeight, stepW, stepH)
#m_outputWidth = (m_inputWidth-m_windowWidth)/m_horizontalSubsample + 1;
mpoutputWidth=6
#m_outputHeight = (m_inputHeight-m_windowHeight)/m_verticalSubsample + 1;
mpoutputHeight=6
#m_outputSizePerSample = m_outputWidth * m_outputHeight * m_channels;
mpoutputSizePerSample=864
# Layer operations
HDim=128
L1 = SBFF(mp, HDim, mpoutputSizePerSample)
CE = SMBFF(L1, LDim, HDim, labels, tag="criterion")
Err=ErrorPrediction(labels, CE.BFF, tag="eval")
# rootNodes defined here
OutputNodes=(CE.BFF)
]
ndlMacroUseCNN=[
# constants defined
# Sample, Hidden, and Label dimensions
inputWidth=28
inputHeight=28
inputChannels=1
SDim=784
LDim=10
features=ImageInput(inputWidth, inputHeight, inputChannels, 1, tag="feature")
labels=Input(LDim, tag="label")
#convolution
kernelWidth=5
kernelHeight=5
outputChannels=24
horizontalSubsample=1
verticalSubsample=1
# weight[outputChannels, kernelWidth * kernelHeight * inputChannels]
cvweight=Parameter(outputChannels, 25)
cv = Convolution(cvweight, features, kernelWidth, kernelHeight, outputChannels,horizontalSubsample, verticalSubsample, zeroPadding=false)
#one bias per channel
cvbias=Parameter(outputChannels, 1)
cvplusbias=Plus(cv, cvbias);
nlcv=Sigmoid(cvplusbias);
#maxpooling
windowWidth=2
windowHeight=2
stepW=2
stepH=2
mp=MaxPooling(nlcv, windowWidth, windowHeight, stepW, stepH)
mpoutputSizePerSample=0
# Layer operations
HDim=128
L1 = SBFF(mp, HDim, 0)
CE = SMBFF(L1, LDim, HDim, labels, tag="criterion")
Err=ErrorPrediction(labels, CE.BFF, tag="eval")
# rootNodes defined here
OutputNodes=(CE.BFF)
]
ndlMacroUseNoBase=[
# constants defined
# Sample, Hidden, and Label dimensions
SDim=784
HDim=256
LDim=10
# Weight, Bias, features and label inputs
B0=Parameter(HDim, 1, init="fixedValue", value=0)
W0=Parameter(HDim, SDim, init="uniform")
B1=Parameter(HDim, 1, init="fixedValue", value=0)
W1=Parameter(HDim, HDim, init="uniform")
B2=Parameter(HDim, 1, init="fixedValue", value=0)
W2=Parameter(HDim, HDim, init="uniform")
BTop=Parameter(LDim, 1, init="fixedValue", value=0)
WTop=Parameter(LDim, HDim, init="uniform")
features=Input(SDim, tag="feature")
labels=Input(LDim, tag="label")
# Layer operations
L1 = RFFD(features, HDim, SDim)
L2 = RFFD(L1, HDim, HDim)
L3 = RFFD(L2, HDim, HDim)
CE = SMFF(L3, LDim, SDim, labels, tag="criterion")
Err=ErrorPrediction(labels, CE.BFF, tag="eval")
# rootNodes defined here
OutputNodes=(CE.BFF)
]
ndlMacroUseMask=[
# constants defined
# Sample, Hidden, and Label dimensions
SDim=784
HDim=256
LDim=10
features=Input(SDim, tag="feature")
labels=Input(LDim, tag="label")
# compute mean/stddev for mean/stddev normalization
fmean = Mean(features);
finvstd=InvStdDev(features)
finput=PerDimMeanVarNormalization(features, fmean, finvstd)
# Layer operations
L1 = RBFF(finput, HDim, SDim)
L2 = RBFF(L1, HDim, HDim)
L3 = RBFF(L2, HDim, HDim)
# mask layers
ML1=RBFF(finput, HDim, SDim)
ML2=RBFF(ML1, HDim, HDim)
# mask
L4=ElementTimes(L3, ML2)
CE = SMBFF(L4, LDim, HDim, labels, tag="criterion")
Err=ErrorPrediction(labels, CE.BFF, tag="eval")
# output nodes
Prior=Mean(labels)
LP=Log(Prior)
O=Minus(CE.BFF, LP)
# rootNodes defined here
OutputNodes=(O)
CriterionNodes=(CE)
EvaluationNodes(Err)
]

21
Common/CheckInSuites/MNIST/GPU/expectedresults.txt
Просмотреть файл

@ -1,21 +0,0 @@
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 0.40693939 EvalErr Per Sample = 0.1224 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=12.07
Finished Epoch[1]: [Validation Set] Train Loss Per Sample = 0.25263566 EvalErr Per Sample = 0.074500002
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 0.20054092 EvalErr Per Sample = 0.058933333 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.826
Finished Epoch[2]: [Validation Set] Train Loss Per Sample = 0.18347667 EvalErr Per Sample = 0.0539
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 0.1460132 EvalErr Per Sample = 0.042883333 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.855
Finished Epoch[3]: [Validation Set] Train Loss Per Sample = 0.12613444 EvalErr Per Sample = 0.036600001
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 0.11390968 EvalErr Per Sample = 0.033399999 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.858
Finished Epoch[4]: [Validation Set] Train Loss Per Sample = 0.11352145 EvalErr Per Sample = 0.034699999
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 0.094562002 EvalErr Per Sample = 0.027450001 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.8610001
Finished Epoch[5]: [Validation Set] Train Loss Per Sample = 0.099217452 EvalErr Per Sample = 0.029899999
Finished Epoch[6]: [Training Set] Train Loss Per Sample = 0.080263816 EvalErr Per Sample = 0.023666667 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.777
Finished Epoch[6]: [Validation Set] Train Loss Per Sample = 0.084881522 EvalErr Per Sample = 0.027000001
Finished Epoch[7]: [Training Set] Train Loss Per Sample = 0.068559825 EvalErr Per Sample = 0.019666666 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.849
Finished Epoch[7]: [Validation Set] Train Loss Per Sample = 0.090336256 EvalErr Per Sample = 0.027899999
Finished Epoch[8]: [Training Set] Train Loss Per Sample = 0.060529299 EvalErr Per Sample = 0.0175 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.8210001
Finished Epoch[8]: [Validation Set] Train Loss Per Sample = 0.079096399 EvalErr Per Sample = 0.024499999
Finished Epoch[9]: [Training Set] Train Loss Per Sample = 0.053066112 EvalErr Per Sample = 0.014983334 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.822
Finished Epoch[9]: [Validation Set] Train Loss Per Sample = 0.07318607 EvalErr Per Sample = 0.021600001
Finished Epoch[10]: [Training Set] Train Loss Per Sample = 0.046927109 EvalErr Per Sample = 0.013333334 Ave Learn Rate Per Sample = 0.006250000093 Epoch Time=2.8299999
Finished Epoch[10]: [Validation Set] Train Loss Per Sample = 0.082184061 EvalErr Per Sample = 0.0255
Final Results: Minibatch[1-100]: Samples Seen = 10000 ErrorPrediction/Sample = 0.0255 CrossEntropyWithSoftmax/Sample = 0.082184061

372
Common/CheckInSuites/MNIST/mnistCheckIn.config
Просмотреть файл

@ -1,372 +0,0 @@
# configuration file for CNTK MNIST Before Check-In Tests
ExpFolder=d:\temp\mnist
ConfigFolder=D:\CNTK\CheckInSuites\MNIST
DataFolder=d:\s2s\mnist
RunOnDevice=-1
stderr=$ExpFolder$
command=mnistTrainSimpleNet:mnistSimpleNetTest
precision=float
dumpNodeInfo=[
action=dumpnode
modelPath=$ExpFolder$\mnist_sn.cn
printValues=false
]
mnistTrainSimpleNet=[
modelPath=$ExpFolder$\mnist_sn.cn
action=train
traceLevel=0
deviceId=$RunOnDevice$
SimpleNetworkBuilder=[
trainingCriterion=CrossEntropyWithSoftmax
evalCriterion=ErrorPrediction
layerTypes=Sigmoid
initValueScale=1.0
layerSizes=784:256:10
uniformInit=true
]
# configuration file, base parameters
SGD=[
minibatchSize=32
epochSize=0
learningRatesPerMB=0.2
momentumPerMB=0.90
maxEpochs=10
validateAfterModelReloading=false
]
# Parameter values for the reader
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=Auto
features=[
dim=784
start=1
file=$DataFolder$\mnist_train.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_train.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=Auto
features=[
dim=784
start=1
file=$DataFolder$\mnist_train.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_train.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
cvReader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=None
features=[
dim=784
start=1
file=$DataFolder$\mnist_test.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_test.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
]
mnistTrainNDLNet=[
modelPath=$ExpFolder$\mnist_ndl.cn
action=train
traceLevel=0
deviceId=$RunOnDevice$
NDLNetworkBuilder=[
# run=sectionName - this will parse and run the sectionName from the file
networkDescription=$ConfigFolder$\Example.ndl
run=ndlFull
executionEngine=synchronous
]
# configuration file, base parameters
SGD=[
minibatchSize=32
epochSize=0
learningRatesPerMB=0.2
momentumPerMB=0.90
maxEpochs=10
validateAfterModelReloading=false
]
# Parameter values for the reader
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=Auto
features=[
dim=784
start=1
file=$DataFolder$\mnist_train.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_train.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=Auto
features=[
dim=784
start=1
file=$DataFolder$\mnist_train.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_train.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
cvReader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=None
features=[
dim=784
start=1
file=$DataFolder$\mnist_test.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_test.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
]
mnistAdaptSimpleNet=[
modelPath=$ExpFolder$\mnist_adapt.cn
action=adapt
traceLevel=0
deviceId=$RunOnDevice$
origModelFileName=$ExpFolder$\mnist_sn.cn.1
refNodeName=PosteriorProb
adaptationRegType=KL
adaptationRegWeight=0.1
# configuration file, base parameters
SGD=[
minibatchSize=32
epochSize=0
learningRatesPerMB=0.2
momentumPerMB=0.90
maxEpochs=10
validateAfterModelReloading=false
]
# Parameter values for the reader
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=Auto
features=[
dim=784
start=1
file=$DataFolder$\mnist_train.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_train.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=Auto
features=[
dim=784
start=1
file=$DataFolder$\mnist_train.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_train.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
cvReader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=None
features=[
dim=784
start=1
file=$DataFolder$\mnist_test.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_test.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
]
mnistSimpleNetTest=[
action=eval
deviceId=$RunOnDevice$
epochSize=10000
minibatchSize=100
modelPath=$ExpFolder$\mnist_sn.cn
# Parameter values for the reader
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=None
features=[
dim=784
start=1
file=$DataFolder$\mnist_test.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_test.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
]
mnistSimpleNetCV=[
action=cv
deviceId=$RunOnDevice$
epochSize=10000
minibatchSize=100
modelPath=$ExpFolder$\mnist_sn.cn
crossValidationInterval=1:1:10
sleepSecondsBetweenRuns=5
# Parameter values for the reader
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=None
features=[
dim=784
start=1
file=$DataFolder$\mnist_test.txt
]
labels=[
dim=1
start=0
file=$DataFolder$\mnist_test.txt
labelMappingFile=$DataFolder$\mnistlabels.txt
labelDim=10
labelType=Category
]
]
]
mnistSimpleNetWritePrediction=[
action=write
deviceId=$RunOnDevice$
epochSize=10000
minibatchSize=100
#outputNodeNames=HLast
modelPath=$ExpFolder$\mnist_sn.cn
outputPath=$ExpFolder$\ScaledLogLikelihood
# Parameter values for the reader
reader=[
# reader to use
readerType=UCIFastReader
miniBatchMode=Partial
randomize=None
features=[
dim=784
start=1
file=$DataFolder$\mnist_test.txt
]
]
]

10
Common/CheckInSuites/MNIST/mnistlabels.txt
Просмотреть файл

@ -1,10 +0,0 @@
1
2
3
4
5
6
7
8
9
10

22
Common/CheckInSuites/runall.bat
Просмотреть файл

@ -1,22 +0,0 @@
::: this runs all tests in this folder
::: BUGBUG: so far only the ASR tests have updated pathnames etc., the others are non-functional stubs here that need to be updated
::: TODO: find a good solution for specifying directories for data that we cannot distribute with CNTK ourselves.
set BUILD=%1
set THIS=%~dp0
::: ASR tests
::: BUGBUG: We do not get to see stdout from CNTK, only from the BAT files.
( %THIS%\ASR\config\runall.bat cpu %BUILD% ) 2>&1
( %THIS%\ASR\config\runall.bat gpu %BUILD% ) 2>&1
::: LM tests
::: TODO: provide BAT file
::: MNIST
::: TODO: provide BAT file
::: SLU
::: TODO: update paths
C:\dev\cntk3\CNTKSolution\x64\Release\cntk configFile=globals.config+rnnlu.config

727
DataReader/LibSVMBinaryReader/LibSVMBinaryReader.cpp
Просмотреть файл

@ -1,727 +0,0 @@
//
// <copyright file="LibSVMBinaryReader.cpp" company="Microsoft">
// Copyright (c) Microsoft Corporation. All rights reserved.
// </copyright>
//
// LibSVMBinaryReader.cpp : Defines the exported functions for the DLL application.
//
#include "stdafx.h"
#define DATAREADER_EXPORTS // creating the exports here
#include "DataReader.h"
#include "LibSVMBinaryReader.h"
#ifdef LEAKDETECT
#include <vld.h> // leak detection
#endif
#include "fileutil.h" // for fexists()
namespace Microsoft { namespace MSR { namespace CNTK {
DWORD HIDWORD(size_t size) {return size>>32;}
DWORD LODWORD(size_t size) {return size&0xFFFFFFFF;}
std::string ws2s(const std::wstring& wstr)
{
int size_needed = WideCharToMultiByte(CP_ACP, 0, wstr.c_str(), int(wstr.length() + 1), 0, 0, 0, 0);
std::string strTo(size_needed, 0);
WideCharToMultiByte(CP_ACP, 0, wstr.c_str(), int(wstr.length() + 1), &strTo[0], size_needed, 0, 0);
return strTo;
}
template<class ElemType>
size_t LibSVMBinaryReader<ElemType>::RandomizeSweep(size_t mbStartSample)
{
//size_t randomRangePerEpoch = (m_epochSize+m_randomizeRange-1)/m_randomizeRange;
//return m_epoch*randomRangePerEpoch + epochSample/m_randomizeRange;
return mbStartSample/m_randomizeRange;
}
// ReadLine - Read a line
// readSample - sample to read in global sample space
// returns - true if we successfully read a record, otherwise false
template<class ElemType>
bool LibSVMBinaryReader<ElemType>::ReadRecord(size_t /*readSample*/)
{
return false; // not used
}
// RecordsToRead - Determine number of records to read to populate record buffers
// mbStartSample - the starting sample from which to read
// tail - we are checking for possible remainer records to read (default false)
// returns - true if we have more to read, false if we hit the end of the dataset
template<class ElemType>
size_t LibSVMBinaryReader<ElemType>::RecordsToRead(size_t mbStartSample, bool tail)
{
assert(mbStartSample >= m_epochStartSample);
// determine how far ahead we need to read
bool randomize = Randomize();
// need to read to the end of the next minibatch
size_t epochSample = mbStartSample;
epochSample %= m_epochSize;
// determine number left to read for this epoch
size_t numberToEpoch = m_epochSize - epochSample;
// we will take either a minibatch or the number left in the epoch
size_t numberToRead = min(numberToEpoch, m_mbSize);
if (numberToRead == 0 && !tail)
numberToRead = m_mbSize;
if (randomize)
{
size_t randomizeSweep = RandomizeSweep(mbStartSample);
// if first read or read takes us to another randomization range
// we need to read at least randomization range records
if (randomizeSweep != m_randomordering.CurrentSeed()) // the range has changed since last time
{
numberToRead = RoundUp(epochSample, m_randomizeRange) - epochSample;
if (numberToRead == 0 && !tail)
numberToRead = m_randomizeRange;
}
}
return numberToRead;
}
template<class ElemType>
void LibSVMBinaryReader<ElemType>::WriteLabelFile()
{
// write out the label file if they don't have one
if (!m_labelFileToWrite.empty())
{
if (m_mapIdToLabel.size() > 0)
{
File labelFile(m_labelFileToWrite, fileOptionsWrite | fileOptionsText);
for (int i=0; i < m_mapIdToLabel.size(); ++i)
{
labelFile << m_mapIdToLabel[i] << '\n';
}
fprintf(stderr, "label file %ls written to disk\n", m_labelFileToWrite.c_str());
m_labelFileToWrite.clear();
}
else if (!m_cachingWriter)
{
fprintf(stderr, "WARNING: file %ls NOT written to disk yet, will be written the first time the end of the entire dataset is found.\n", m_labelFileToWrite.c_str());
}
}
}
// Destroy - cleanup and remove this class
// NOTE: this destroys the object, and it can't be used past this point
template<class ElemType>
void LibSVMBinaryReader<ElemType>::Destroy()
{
delete this;
}
// Init - Reader Initialize for multiple data sets
// config - [in] configuration parameters for the datareader
// Sample format below:
//# Parameter values for the reader
//reader=[
// # reader to use
// readerType=LibSVMBinaryReader
// miniBatchMode=Partial
// randomize=None
// features=[
// dim=784
// start=1
// file=c:\speech\mnist\mnist_test.txt
// ]
// labels=[
// dim=1
// start=0
// file=c:\speech\mnist\mnist_test.txt
// labelMappingFile=c:\speech\mnist\labels.txt
// labelDim=10
// labelType=Category
// ]
//]
template<class ElemType>
template<class ConfigRecordType>
void LibSVMBinaryReader<ElemType>::InitFromConfig(const ConfigRecordType & readerConfig)
{
m_labelDim = 2; // maximum label ID we will ever see (used for array dimensions)
std::vector<std::wstring> features;
std::vector<std::wstring> labels;
// Determine the names of the features and lables sections in the config file.
// features - [in,out] a vector of feature name strings
// labels - [in,out] a vector of label name strings
// For LibSVMBinary dataset, we only need features. No label is necessary. The following "labels" just serves as a place holder
GetFileConfigNames(readerConfig, features, labels);
// For LibSVMBinary dataset, it must have exactly two features
// In the config file, we must specify query features first, then document features. The sequence is different here. Pay attention
if (features.size() == 1 && labels.size() == 1)
{
m_featuresName = features[0];
m_labelsName = labels[0];
}
else
{
RuntimeError("LibSVMBinary requires exactly one feature and one label. Their names should match those in NDL definition");
return;
}
m_mbStartSample = m_epoch = m_totalSamples = m_epochStartSample = 0;
m_labelIdMax = m_labelDim = 2;
m_partialMinibatch = m_endReached = false;
m_labelType = labelCategory;
m_readNextSample = 0;
m_traceLevel = readerConfig(L"traceLevel", 0);
if (readerConfig.Exists(L"randomize"))
{
string randomizeString = readerConfig(L"randomize");
if (!_stricmp(randomizeString.c_str(), "none"))
{
m_randomizeRange = randomizeNone;
}
else if (!_stricmp(randomizeString.c_str(), "auto"))
{
m_randomizeRange = randomizeAuto;
}
else
{
m_randomizeRange = readerConfig(L"randomize");
}
}
else
{
m_randomizeRange = randomizeNone;
}
std::string minibatchMode(readerConfig(L"minibatchMode", "Partial"));
m_partialMinibatch = !_stricmp(minibatchMode.c_str(), "Partial");
// Get the config parameters for query feature and doc feature
ConfigParameters configFeatures = readerConfig(m_featuresName, "");
if (configFeatures.size() == 0)
RuntimeError("features file not found, required in configuration: i.e. 'features=[file=c:\\myfile.txt;start=1;dim=123]'");
// Read in feature size information
// This information will be used to handle OOVs
m_featuresDim= configFeatures(L"dim");
std::wstring file = configFeatures("file");
featuresInput.Init(file, m_featuresDim);
m_totalSamples = featuresInput.numRows;
if (read_order == NULL)
{
read_order = new int[featuresInput.numBatches];
for (int c = 0; c < featuresInput.numBatches; c++)
{
read_order[c] = c;
}
}
m_mbSize = 0;
}
// destructor - virtual so it gets called properly
template<class ElemType>
LibSVMBinaryReader<ElemType>::~LibSVMBinaryReader()
{
ReleaseMemory();
}
// ReleaseMemory - release the memory footprint of LibSVMBinaryReader
// used when the caching reader is taking over
template<class ElemType>
void LibSVMBinaryReader<ElemType>::ReleaseMemory()
{
if (m_qfeaturesBuffer!=NULL)
delete[] m_qfeaturesBuffer;
m_qfeaturesBuffer=NULL;
if (m_dfeaturesBuffer!=NULL)
delete[] m_dfeaturesBuffer;
m_dfeaturesBuffer=NULL;
if (m_labelsBuffer!=NULL)
delete[] m_labelsBuffer;
m_labelsBuffer=NULL;
if (m_labelsIdBuffer!=NULL)
delete[] m_labelsIdBuffer;
m_labelsIdBuffer=NULL;
m_featureData.clear();
m_labelIdData.clear();
m_labelData.clear();
}
//SetupEpoch - Setup the proper position in the file, and other variable settings to start a particular epoch
template<class ElemType>
void LibSVMBinaryReader<ElemType>::SetupEpoch()
{
}
// utility function to round an integer up to a multiple of size
size_t RoundUp(size_t value, size_t size)
{
return ((value + size -1)/size)*size;
}
//StartMinibatchLoop - Startup a minibatch loop
// mbSize - [in] size of the minibatch (number of Samples, etc.)
// epoch - [in] epoch number for this loop, if > 0 the requestedEpochSamples must be specified (unless epoch zero was completed this run)
// requestedEpochSamples - [in] number of samples to randomize, defaults to requestDataSize which uses the number of samples there are in the dataset
// this value must be a multiple of mbSize, if it is not, it will be rounded up to one.
template<class ElemType>
void LibSVMBinaryReader<ElemType>::StartMinibatchLoop(size_t mbSize, size_t epoch, size_t requestedEpochSamples)
{
m_epoch = epoch;
size_t mbStartSample = m_epoch * m_epochSize;
if (m_totalSamples == 0)
{
m_totalSamples = featuresInput.numRows;
}
size_t fileRecord = m_totalSamples ? mbStartSample % m_totalSamples : 0;
fprintf(stderr, "starting epoch %lld at record count %lld, and file position %lld\n", m_epoch, mbStartSample, fileRecord);
//size_t currentFileRecord = m_mbStartSample % m_totalSamples;
// reset the next read sample
m_processedMinibatches = 0;
m_readNextSample = 0;
m_epochStartSample = m_mbStartSample = mbStartSample;
m_mbSize = mbSize;
m_epochSize = requestedEpochSamples;
featuresInput.SetupEpoch(mbSize);
if (m_epochSize > (size_t)featuresInput.numRows)
{
m_epochSize = (size_t)featuresInput.numRows;
}
if (Randomize())
{
random_shuffle(&read_order[0], &read_order[featuresInput.numBatches]);
}
m_epoch = epoch;
m_mbStartSample = epoch*m_epochSize;
}
// function to store the LabelType in an ElemType
// required for string labels, which can't be stored in ElemType arrays
template<class ElemType>
void LibSVMBinaryReader<ElemType>::StoreLabel(ElemType& labelStore, const LabelType& labelValue)
{
labelStore = (ElemType)m_mapLabelToId[labelValue];
}
// GetMinibatch - Get the next minibatch (features and labels)
// matrices - [in] a map with named matrix types (i.e. 'features', 'labels') mapped to the corresponing matrix,
// [out] each matrix resized if necessary containing data.
// returns - true if there are more minibatches, false if no more minibatchs remain
template<class ElemType>
bool LibSVMBinaryReader<ElemType>::GetMinibatch(std::map<std::wstring, Matrix<ElemType>*>& matrices)
{
if (m_processedMinibatches >= featuresInput.numBatches )
{
return false;
}
// In my unit test example, the input matrices contain 5: N, S, fD, fQ and labels
// Both N and S serve as a pre-set constant values, no need to change them
// In this node, we only need to fill in these matrices: fD, fQ, labels
Matrix<ElemType>& features = *matrices[m_featuresName];
Matrix<ElemType>& labels = *matrices[m_labelsName]; // will change this part later.
size_t actualMBSize = ( m_readNextSample + m_mbSize > m_totalSamples ) ? m_totalSamples - m_readNextSample : m_mbSize;
//features.SwitchToMatrixType(MatrixType::SPARSE, MatrixFormat::matrixFormatSparseCSC);
/*
featuresQ.Resize(dssm_queryInput.numRows, actualMBSize);
featuresD.Resize(dssm_docInput.numRows, actualMBSize);
*/
//fprintf(stderr, "featuresQ\n");
featuresInput.Next_Batch(features, labels, actualMBSize, read_order[m_processedMinibatches]);
//fprintf(stderr, "\n\n\nfeaturesD\n");
//fprintf(stderr, "\n\n\n\n\n");
m_readNextSample += actualMBSize;
m_processedMinibatches++;
//fprintf(stderr, "mbs: %d\tsamples: %d\n", m_processedMinibatches, m_readNextSample);
/*
featuresQ.Print("featuresQ");
fprintf(stderr, "\n");
featuresD.Print("featuresD");
fprintf(stderr, "\n");
*/
/*
GPUSPARSE_INDEX_TYPE* h_CSCCol;
GPUSPARSE_INDEX_TYPE* h_Row;
ElemType* h_val;
size_t nz;
size_t nrs;
size_t ncols;
featuresQ.GetMatrixFromCSCFormat(&h_CSCCol, &h_Row, &h_val, &nz, &nrs, &ncols);
for (int j = 0, k=0; j < nz; j++)
{
if (h_CSCCol[k] >= j)
{
fprintf(stderr, "\n");
k++;
}
fprintf(stderr, "%d:%.f ", h_Row[j], h_val[j]);
}
*/
/*
featuresQ.TransferFromDeviceToDevice(featuresQ.GetDeviceId(), -1);
featuresQ.SwitchToMatrixType(MatrixType::DENSE, MatrixFormat::matrixFormatDense);
featuresQ.Print("featuresQ");
featuresD.TransferFromDeviceToDevice(featuresD.GetDeviceId(), -1);
featuresD.SwitchToMatrixType(MatrixType::DENSE, MatrixFormat::matrixFormatDense);
featuresD.Print("featuresD");
exit(1);
*/
/*
if (actualMBSize > m_mbSize || m_labelsBuffer == NULL) {
size_t rows = labels.GetNumRows();
labels.Resize(rows, actualMBSize);
labels.SetValue(0.0);
m_labelsBuffer = new ElemType[rows * actualMBSize];
memset(m_labelsBuffer, 0, sizeof(ElemType)* rows * actualMBSize);
for (int i = 0; i < actualMBSize; i++)
{
m_labelsBuffer[i * rows] = 1;
}
labels.SetValue(rows, actualMBSize, m_labelsBuffer, 0, labels.GetDeviceId());
}
*/
/*
featuresQ.Print("featuresQ");
featuresD.Print("featuresD");
labels.print("labels");
*/
return true;
}
// GetLabelMapping - Gets the label mapping from integer index to label type
// returns - a map from numeric datatype to native label type
template<class ElemType>
const std::map<typename IDataReader<ElemType>::LabelIdType, typename IDataReader<ElemType>::LabelType>& LibSVMBinaryReader<ElemType>::GetLabelMapping(const std::wstring& sectionName)
{
if (m_cachingReader)
{
return m_cachingReader->GetLabelMapping(sectionName);
}
return m_mapIdToLabel;
}
// SetLabelMapping - Sets the label mapping from integer index to label
// labelMapping - mapping table from label values to IDs (must be 0-n)
// note: for tasks with labels, the mapping table must be the same between a training run and a testing run
template<class ElemType>
void LibSVMBinaryReader<ElemType>::SetLabelMapping(const std::wstring& /*sectionName*/, const std::map<typename IDataReader<ElemType>::LabelIdType, typename LabelType>& labelMapping)
{
if (m_cachingReader)
{
RuntimeError("Cannot set mapping table when the caching reader is being used");
}
m_mapIdToLabel = labelMapping;
m_mapLabelToId.clear();
for (std::pair<unsigned, LabelType> var : labelMapping)
{
m_mapLabelToId[var.second] = var.first;
}
}
template<class ElemType>
bool LibSVMBinaryReader<ElemType>::DataEnd(EndDataType endDataType)
{
bool ret = false;
switch (endDataType)
{
case endDataNull:
assert(false);
break;
case endDataEpoch:
//ret = (m_mbStartSample / m_epochSize < m_epoch);
//ret = (m_readNextSample >= m_totalSamples);
ret = (m_processedMinibatches >= featuresInput.numBatches);
break;
case endDataSet:
//ret = (m_readNextSample >= m_totalSamples);
ret = (m_processedMinibatches >= featuresInput.numBatches);
break;
case endDataSentence: // for fast reader each minibatch is considered a "sentence", so always true
ret = true;
break;
}
return ret;
}
template<class ElemType>
LibSVM_BinaryInput<ElemType>::LibSVM_BinaryInput(){
}
template<class ElemType>
LibSVM_BinaryInput<ElemType>::~LibSVM_BinaryInput(){
Dispose();
}
template<class ElemType>
void LibSVM_BinaryInput<ElemType>::Init(wstring fileName, size_t dim)
{
m_labelDim = 2;
m_dim = dim;
mbSize = 0;
/*
m_hndl = CreateFileA(fileName.c_str(), GENERIC_READ,
FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
*/
m_hndl = CreateFile(fileName.c_str(), GENERIC_READ,
FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
if (m_hndl == INVALID_HANDLE_VALUE)
{
char message[256];
sprintf_s(message, "Unable to Open/Create file %ls, error %x", fileName.c_str(), GetLastError());
RuntimeError(message);
}
m_filemap = CreateFileMapping(m_hndl, NULL, PAGE_READONLY, 0, 0, NULL);
SYSTEM_INFO sysinfo;
GetSystemInfo(&sysinfo);
DWORD sysGran = sysinfo.dwAllocationGranularity;
header_buffer = MapViewOfFile(m_filemap, // handle to map object
FILE_MAP_READ, // get correct permissions
HIDWORD(0),
LODWORD(0),
sizeof(int64_t) * 3 + sizeof(int32_t));
//cout << "After mapviewoffile" << endl;
memcpy(&numRows, header_buffer, sizeof(int64_t));
memcpy(&numBatches, (char*)header_buffer + sizeof(int64_t), sizeof(int64_t));
memcpy(&numCols, (char*)header_buffer + 2*sizeof(int64_t), sizeof(int32_t));
memcpy(&totalNNz, (char*)header_buffer + 2*sizeof(int64_t)+sizeof(int32_t), sizeof(int64_t));
//cout << "After gotvalues" << endl;
int64_t base_offset = sizeof(int64_t)* 3 + sizeof(int32_t);
int64_t offsets_padding = base_offset % sysGran;
base_offset -= offsets_padding;
int64_t header_size = numBatches*sizeof(int64_t)+offsets_padding;
offsets_orig = MapViewOfFile(m_filemap, // handle to map object
FILE_MAP_READ, // get correct permissions
HIDWORD(base_offset),
LODWORD(base_offset),
header_size);
offsets_buffer = (char*)offsets_orig + offsets_padding;
if (offsets != NULL){
free(offsets);
}
offsets = (int64_t*)malloc(sizeof(int64_t)*numBatches);
memcpy(offsets, offsets_buffer, numBatches*sizeof(int64_t));
int64_t header_offset = base_offset + offsets_padding + numBatches * sizeof(int64_t);
int64_t data_padding = header_offset % sysGran;
header_offset -= data_padding;
data_orig = MapViewOfFile(m_filemap, // handle to map object
FILE_MAP_READ, // get correct permissions
HIDWORD(header_offset),
LODWORD(header_offset),
0);
data_buffer = (char*)data_orig + data_padding;
}
template<class ElemType>
bool LibSVM_BinaryInput<ElemType>::SetupEpoch( size_t minibatchSize){
if (values == NULL || mbSize < minibatchSize)
{
if (values != NULL)
{
free(values);
free(colIndices);
free(rowIndices);
free(classIndex);
free(classWeight);
free(m_labelsBuffer);
}
m_labelsBuffer = (ElemType*)malloc(sizeof(ElemType)*m_labelDim*minibatchSize);
values = (ElemType*)malloc(sizeof(ElemType)*MAX_BUFFER*minibatchSize);
rowIndices = (int32_t*)malloc(sizeof(int32_t)*MAX_BUFFER*minibatchSize);
colIndices = (int32_t*)malloc(sizeof(int32_t)*(minibatchSize+1));
classIndex = (int32_t*)malloc(sizeof(int32_t)*MAX_BUFFER*minibatchSize);
classWeight = (ElemType*)malloc(sizeof(ElemType)*MAX_BUFFER*minibatchSize);
//fprintf(stderr, "values size: %d",sizeof(ElemType)*MAX_BUFFER*minibatchSize);
//fprintf(stderr, "colindi size: %d",sizeof(int32_t)*MAX_BUFFER*(1+minibatchSize));
//fprintf(stderr, "rowindi size: %d",sizeof(int32_t)*MAX_BUFFER*minibatchSize);
}
if (minibatchSize > mbSize)
{
mbSize = minibatchSize;
}
return true;
}
template<class ElemType>
bool LibSVM_BinaryInput<ElemType>::Next_Batch(Matrix<ElemType>& features, Matrix<ElemType>& labels, size_t actualmbsize, int cur_batch){
/*
int devId = matrices.GetDeviceId();
matrices.TransferFromDeviceToDevice(devId, -1);
*/
//int64_t cur_offset = offsets[ordering[cur]];
int64_t cur_offset = offsets[cur_batch];
//int64_t cur_offset = offsets[ordering[c]];
int32_t nnz;
size_t buffer_offset = cur_offset;
memcpy(&nnz, (char*)data_buffer + buffer_offset, sizeof(int32_t));
buffer_offset += sizeof(int32_t);
memcpy(values, (char*)data_buffer + buffer_offset, sizeof(ElemType)*nnz);
buffer_offset += sizeof(ElemType)*nnz;
memcpy(rowIndices, (char*)data_buffer + buffer_offset, sizeof(int32_t)*nnz);
buffer_offset += sizeof(int32_t)*nnz;
colIndices[0] = (int32_t)0;
memcpy((char*)colIndices + sizeof(int32_t), (char*)data_buffer + buffer_offset, sizeof(int32_t)*actualmbsize);
buffer_offset += sizeof(int32_t)*actualmbsize;
memcpy(classIndex, (char*)data_buffer + buffer_offset, sizeof(int32_t)*actualmbsize);
buffer_offset += sizeof(int32_t)*actualmbsize;
memcpy(classWeight, (char*)data_buffer + buffer_offset, sizeof(ElemType)*actualmbsize);
/**
fprintf(stderr, "%4d (%3d, %6d): ", c, nnz, cur_index + nnz);
for (int i = 0; i < nnz; i++)
{
fprintf(stderr, "%d:%.f ", rowIndices[cur_index+i], values[cur_index+i]);
//matrices.SetValue(rowIndices[cur_index + i], c, values[cur_index + i]);
}
fprintf(stderr, "\n");
**/
/**
int col = 0;
for (int c = 0; c < nnz; c++)
{
if (colIndices[col] == c)
{
fprintf(stderr, "\n%4d: ", col);
col++;
}
fprintf(stderr, "%d:%.lf ", rowIndices[c], values[c]);
}
**/
/*
fprintf(stderr, "\nXXXX nnz: %d\n", nnz);
fprintf(stderr, "XXXX max values read: %d vs %d\n", sizeof(ElemType)*nnz, sizeof(ElemType)*MAX_BUFFER*actualmbsize);
fprintf(stderr, "XXXX max indices read: %d vs %d\n", sizeof(int32_t)*nnz, sizeof(int32_t)*MAX_BUFFER*actualmbsize);
*/
/*
values = (ElemType*)malloc(sizeof(ElemType)*MAX_BUFFER*minibatchSize);
colIndices = (int32_t*)malloc(sizeof(int32_t)*MAX_BUFFER*(minibatchSize+1));
rowIndices = (int32_t*)malloc(sizeof(int32_t)*MAX_BUFFER*minibatchSize);
*/
features.SetMatrixFromCSCFormat(colIndices, rowIndices, values, nnz, m_dim, actualmbsize);
//features.Print("actual values");
// For Single Class
/**/
memset(m_labelsBuffer,0,sizeof(ElemType)*m_labelDim*actualmbsize);
for (int j = 0; j < actualmbsize; j++)
{
if (classIndex[j] > 0)
{
m_labelsBuffer[j] = (ElemType)1;
}
}
labels.SetValue(1, actualmbsize, labels.GetDeviceId(), m_labelsBuffer, matrixFlagNormal);
/**/
// For Multi Class
/**
memset(m_labelsBuffer,0,sizeof(ElemType)*m_labelDim*actualmbsize);
for (int j = 0; j < actualmbsize; j++)
{
m_labelsBuffer[j*m_labelDim + classIndex[j]] = 1;// classWeight[j];
}
labels.SetValue(m_labelDim, actualmbsize, m_labelsBuffer, matrixFlagNormal);
/**/
//exit(1);
/*
matrices.SwitchToMatrixType(MatrixType::DENSE, MatrixFormat::matrixFormatDense);
matrices.Print("featuresQ");
exit(1);
matrices.TransferFromDeviceToDevice(-1,devId);
*/
return true;
}
template<class ElemType>
void LibSVM_BinaryInput<ElemType>::Dispose(){
if (offsets_orig != NULL){
UnmapViewOfFile(offsets_orig);
}
if (data_orig != NULL)
{
UnmapViewOfFile(data_orig);
}
if (offsets!= NULL)
{
free(offsets);// = (ElemType*)malloc(sizeof(float)* 230 * 1024);
}
if (values != NULL)
{
free(values);// = (ElemType*)malloc(sizeof(float)* 230 * 1024);
}
if (rowIndices != NULL){
free(rowIndices);// = (int*)malloc(sizeof(float)* 230 * 1024);
}
if (colIndices != NULL){
free(colIndices);// = (int*)malloc(sizeof(float)* 230 * 1024);
}
if (classIndex != NULL){
free(classIndex);// = (int*)malloc(sizeof(float)* 230 * 1024);
}
if (classWeight != NULL){
free(classWeight);// = (int*)malloc(sizeof(float)* 230 * 1024);
}
}
template<class ElemType>
bool LibSVMBinaryReader<ElemType>::GetData(const std::wstring& sectionName, size_t numRecords, void* data, size_t& dataBufferSize, size_t recordStart)
{
if (m_cachingReader)
{
return m_cachingReader->GetData(sectionName, numRecords, data, dataBufferSize, recordStart);
}
RuntimeError("GetData not supported in LibSVMBinaryReader");
}
// instantiate all the combinations we expect to be used
template class LibSVMBinaryReader<double>;
template class LibSVMBinaryReader<float>;
}}}

159
DataReader/LibSVMBinaryReader/LibSVMBinaryReader.h
Просмотреть файл

@ -1,159 +0,0 @@
//
// <copyright file="LibSVMBinaryReader.h" company="Microsoft">
// Copyright (c) Microsoft Corporation. All rights reserved.
// </copyright>
//
// LibSVMBinaryReader.h - Include file for the MTK and MLF format of features and samples
#pragma once
#include "DataReader.h"
#include "DataWriter.h"
#include "Config.h"
#include "RandomOrdering.h"
#include <map>
#include <vector>
#include <string>
namespace Microsoft { namespace MSR { namespace CNTK {
enum LabelKind
{
labelNone = 0, // no labels to worry about
labelCategory = 1, // category labels, creates mapping tables
labelRegression = 2, // regression labels
labelOther = 3, // some other type of label
};
template<class ElemType>
class LibSVM_BinaryInput {
private:
HANDLE m_hndl;
HANDLE m_filemap;
HANDLE m_header;
HANDLE m_offsets;
HANDLE m_data;
//void* header_orig; // Don't need this since the header is at the start of the file
void* offsets_orig;
void* data_orig;
void* header_buffer;
void* offsets_buffer;
void* data_buffer;
size_t m_dim;
size_t mbSize;
size_t MAX_BUFFER = 400;
size_t m_labelDim;
ElemType* values; // = (ElemType*)malloc(sizeof(ElemType)* 230 * 1024);
int64_t* offsets; // = (int*)malloc(sizeof(int)* 230 * 1024);
int32_t* colIndices; // = (int*)malloc(sizeof(int) * (batchsize + 1));
int32_t* rowIndices; // = (int*)malloc(sizeof(int) * MAX_BUFFER * batchsize);
int32_t* classIndex; // = (int*)malloc(sizeof(int) * batchsize);
ElemType* classWeight; // = (ElemType*)malloc(sizeof(ElemType) * batchsize);
ElemType* m_labelsBuffer;
public:
int64_t numRows;
int64_t numBatches;
int32_t numCols;
int64_t totalNNz;
LibSVM_BinaryInput();
~LibSVM_BinaryInput();
void Init(std::wstring fileName, size_t dim);
bool SetupEpoch( size_t minibatchSize);
bool Next_Batch(Matrix<ElemType>& features, Matrix<ElemType>& labels, size_t actualmbsize, int batchIndex);
void Dispose();
};
template<class ElemType>
class LibSVMBinaryReader : public IDataReader<ElemType>
{
//public:
// typedef std::string LabelType;
// typedef unsigned LabelIdType;
private:
int* read_order; // array to shuffle to reorder the dataset
std::wstring m_featuresName;
size_t m_featuresDim;
LibSVM_BinaryInput<ElemType> featuresInput;
int64_t m_processedMinibatches;
size_t m_mbSize; // size of minibatch requested
LabelIdType m_labelIdMax; // maximum label ID we have encountered so far
LabelIdType m_labelDim; // maximum label ID we will ever see (used for array dimensions)
size_t m_mbStartSample; // starting sample # of the next minibatch
size_t m_epochSize; // size of an epoch
size_t m_epoch; // which epoch are we on
size_t m_epochStartSample; // the starting sample for the epoch
size_t m_totalSamples; // number of samples in the dataset
size_t m_randomizeRange; // randomization range
size_t m_featureCount; // feature count
size_t m_readNextSample; // next sample to read
bool m_labelFirst; // the label is the first element in a line
bool m_partialMinibatch; // a partial minibatch is allowed
LabelKind m_labelType; // labels are categories, create mapping table
RandomOrdering m_randomordering; // randomizing class
MBLayoutPtr m_pMBLayout;
std::wstring m_labelsName;
std::wstring m_labelsCategoryName;
std::wstring m_labelsMapName;
ElemType* m_qfeaturesBuffer;
ElemType* m_dfeaturesBuffer;
ElemType* m_labelsBuffer;
LabelIdType* m_labelsIdBuffer;
std::wstring m_labelFileToWrite; // set to the path if we need to write out the label file
bool m_endReached;
int m_traceLevel;
// feature and label data are parallel arrays
std::vector<ElemType> m_featureData;
std::vector<LabelIdType> m_labelIdData;
std::vector<LabelType> m_labelData;
// map is from ElemType to LabelType
// For LibSVMBinary, we really only need an int for label data, but we have to transmit in Matrix, so use ElemType instead
std::map<LabelIdType, LabelType> m_mapIdToLabel;
std::map<LabelType, LabelIdType> m_mapLabelToId;
// caching support
DataReader<ElemType>* m_cachingReader;
DataWriter<ElemType>* m_cachingWriter;
ConfigParameters m_readerConfig;
size_t RandomizeSweep(size_t epochSample);
//bool Randomize() {return m_randomizeRange != randomizeNone;}
bool Randomize() { return false; }
void SetupEpoch();
void StoreLabel(ElemType& labelStore, const LabelType& labelValue);
size_t RecordsToRead(size_t mbStartSample, bool tail=false);
void ReleaseMemory();
void WriteLabelFile();
virtual bool ReadRecord(size_t readSample);
public:
template<class ConfigRecordType> void InitFromConfig(const ConfigRecordType &);
virtual void Init(const ConfigParameters & config) override { InitFromConfig(config); }
virtual void Init(const ScriptableObjects::IConfigRecord & config) override { InitFromConfig(config); }
virtual void Destroy();
LibSVMBinaryReader() : m_pMBLayout(make_shared<MBLayout>()) { m_qfeaturesBuffer = NULL; m_dfeaturesBuffer = NULL; m_labelsBuffer = NULL; }
virtual ~LibSVMBinaryReader();
virtual void StartMinibatchLoop(size_t mbSize, size_t epoch, size_t requestedEpochSamples=requestDataSize);
virtual bool GetMinibatch(std::map<std::wstring, Matrix<ElemType>*>& matrices);
size_t GetNumParallelSequences() { return 1; }
void SetNumParallelSequences(const size_t) { };
void CopyMBLayoutTo(MBLayoutPtr pMBLayout) { pMBLayout->CopyFrom(m_pMBLayout); NOT_IMPLEMENTED; }
virtual const std::map<LabelIdType, LabelType>& GetLabelMapping(const std::wstring& sectionName);
virtual void SetLabelMapping(const std::wstring& sectionName, const std::map<LabelIdType, typename LabelType>& labelMapping);
virtual bool GetData(const std::wstring& sectionName, size_t numRecords, void* data, size_t& dataBufferSize, size_t recordStart=0);
virtual bool DataEnd(EndDataType endDataType);
void SetRandomSeed(int) { NOT_IMPLEMENTED; }
};
}}}

56
ExampleSetups/LM/LSTMLM/ExpectedResults.log
Просмотреть файл

@ -1,56 +0,0 @@
$ grep -a --text Finish gpulstmlm_train_test.log | grep Train
Finished Epoch[1]: [Training Set] Train Loss Per Sample = 6.0689292 EvalErr Per Sample = 6.0689292 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=501.42099
Finished Epoch[1]: [Validation Set] Train Loss Per Sample = 5.7789721 EvalErr Per Sample = 5.7789721
Finished Epoch[2]: [Training Set] Train Loss Per Sample = 5.6369767 EvalErr Per Sample = 5.6369767 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=501.62399
Finished Epoch[2]: [Validation Set] Train Loss Per Sample = 5.5511756 EvalErr Per Sample = 5.5511756
Finished Epoch[3]: [Training Set] Train Loss Per Sample = 5.4327216 EvalErr Per Sample = 5.4327216 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=501.52399
Finished Epoch[3]: [Validation Set] Train Loss Per Sample = 5.4396544 EvalErr Per Sample = 5.4396544
Finished Epoch[4]: [Training Set] Train Loss Per Sample = 5.2914305 EvalErr Per Sample = 5.2914305 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=503.44199
Finished Epoch[4]: [Validation Set] Train Loss Per Sample = 5.345387 EvalErr Per Sample = 5.345387
Finished Epoch[5]: [Training Set] Train Loss Per Sample = 5.1821184 EvalErr Per Sample = 5.1821184 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=508.905
Finished Epoch[5]: [Validation Set] Train Loss Per Sample = 5.2927375 EvalErr Per Sample = 5.2927375
Finished Epoch[6]: [Training Set] Train Loss Per Sample = 5.0916734 EvalErr Per Sample = 5.0916734 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=522.10303
Finished Epoch[6]: [Validation Set] Train Loss Per Sample = 5.2429872 EvalErr Per Sample = 5.2429872
Finished Epoch[7]: [Training Set] Train Loss Per Sample = 5.0149188 EvalErr Per Sample = 5.0149188 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=526.94702
Finished Epoch[7]: [Validation Set] Train Loss Per Sample = 5.1904435 EvalErr Per Sample = 5.1904435
Finished Epoch[8]: [Training Set] Train Loss Per Sample = 4.9472914 EvalErr Per Sample = 4.9472914 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=544.46198
Finished Epoch[8]: [Validation Set] Train Loss Per Sample = 5.1749911 EvalErr Per Sample = 5.1749911
Finished Epoch[9]: [Training Set] Train Loss Per Sample = 4.8864255 EvalErr Per Sample = 4.8864255 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=531.33002
Finished Epoch[9]: [Validation Set] Train Loss Per Sample = 5.1553888 EvalErr Per Sample = 5.1553888
Finished Epoch[10]: [Training Set] Train Loss Per Sample = 4.831315 EvalErr Per Sample = 4.831315 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=523.29999
Finished Epoch[10]: [Validation Set] Train Loss Per Sample = 5.1265602 EvalErr Per Sample = 5.1265602
Finished Epoch[11]: [Training Set] Train Loss Per Sample = 4.7812014 EvalErr Per Sample = 4.7812014 Ave Learn Rate Per Sample = 0.009999999776 Epoch Time=521.64697
Finished Epoch[11]: [Validation Set] Train Loss Per Sample = 5.1217532 EvalErr Per Sample = 5.1217532
Finished Epoch[12]: [Training Set] Train Loss Per Sample = 4.6650786 EvalErr Per Sample = 4.6650786 Ave Learn Rate Per Sample = 0.004999999888 Epoch Time=523.90198
Finished Epoch[12]: [Validation Set] Train Loss Per Sample = 5.0993018 EvalErr Per Sample = 5.0993018
Finished Epoch[13]: [Training Set] Train Loss Per Sample = 4.5911136 EvalErr Per Sample = 4.5911136 Ave Learn Rate Per Sample = 0.002499999944 Epoch Time=520.66101
Finished Epoch[13]: [Validation Set] Train Loss Per Sample = 5.0787692 EvalErr Per Sample = 5.0787692
Finished Epoch[14]: [Training Set] Train Loss Per Sample = 4.5472517 EvalErr Per Sample = 4.5472517 Ave Learn Rate Per Sample = 0.001249999972 Epoch Time=517.742
Finished Epoch[14]: [Validation Set] Train Loss Per Sample = 5.0703268 EvalErr Per Sample = 5.0703268
Finished Epoch[15]: [Training Set] Train Loss Per Sample = 4.5230498 EvalErr Per Sample = 4.5230498 Ave Learn Rate Per Sample = 0.000624999986 Epoch Time=516.28998
Finished Epoch[15]: [Validation Set] Train Loss Per Sample = 5.0566416 EvalErr Per Sample = 5.0566416
Finished Epoch[16]: [Training Set] Train Loss Per Sample = 4.510385 EvalErr Per Sample = 4.510385 Ave Learn Rate Per Sample = 0.000312499993 Epoch Time=513.30499
Finished Epoch[16]: [Validation Set] Train Loss Per Sample = 5.0524426 EvalErr Per Sample = 5.0524426
$ grep -a --text Finish gpulstmlm_train_test.log | grep Validation
Finished Epoch[1]: [Validation Set] Train Loss Per Sample = 5.7789721 EvalErr Per Sample = 5.7789721
Finished Epoch[2]: [Validation Set] Train Loss Per Sample = 5.5511756 EvalErr Per Sample = 5.5511756
Finished Epoch[3]: [Validation Set] Train Loss Per Sample = 5.4396544 EvalErr Per Sample = 5.4396544
Finished Epoch[4]: [Validation Set] Train Loss Per Sample = 5.345387 EvalErr Per Sample = 5.345387
Finished Epoch[5]: [Validation Set] Train Loss Per Sample = 5.2927375 EvalErr Per Sample = 5.2927375
Finished Epoch[6]: [Validation Set] Train Loss Per Sample = 5.2429872 EvalErr Per Sample = 5.2429872
Finished Epoch[7]: [Validation Set] Train Loss Per Sample = 5.1904435 EvalErr Per Sample = 5.1904435
Finished Epoch[8]: [Validation Set] Train Loss Per Sample = 5.1749911 EvalErr Per Sample = 5.1749911
Finished Epoch[9]: [Validation Set] Train Loss Per Sample = 5.1553888 EvalErr Per Sample = 5.1553888
Finished Epoch[10]: [Validation Set] Train Loss Per Sample = 5.1265602 EvalErr Per Sample = 5.1265602
Finished Epoch[11]: [Validation Set] Train Loss Per Sample = 5.1217532 EvalErr Per Sample = 5.1217532
Finished Epoch[12]: [Validation Set] Train Loss Per Sample = 5.0993018 EvalErr Per Sample = 5.0993018
Finished Epoch[13]: [Validation Set] Train Loss Per Sample = 5.0787692 EvalErr Per Sample = 5.0787692
Finished Epoch[14]: [Validation Set] Train Loss Per Sample = 5.0703268 EvalErr Per Sample = 5.0703268
Finished Epoch[15]: [Validation Set] Train Loss Per Sample = 5.0566416 EvalErr Per Sample = 5.0566416
Finished Epoch[16]: [Validation Set] Train Loss Per Sample = 5.0524426 EvalErr Per Sample = 5.0524426
# test
Final Results: Minibatch[1-82430]: Samples Seen = 82430 TrainNodeClassBasedCrossEntropy/Sample = 4.9702353 TrainNodeClassBasedCrossEntropy/Sample = 4.9702353
corresponding PPL = 144

7
ExampleSetups/LM/LSTMLM/global.config
Просмотреть файл

@ -1,7 +0,0 @@
ExpDir=d:\temp\lstm
ConfigDir=D:\zhaoyg\2015-5-6\cntk\ExampleSetups\LM\LSTMLM\
DataDir=\\speechstore5\transient\kaishengy\data\lm\PennTreeBank
# to run this
# ..\..\..\x64\debug\cntk.exe configFile=global.config+lstmlm.gpu.config ExpDir=d:\temp\lm\ptb

432
ExampleSetups/LM/LSTMLM/lstmlm.cpu.nce.config
Просмотреть файл

@ -1,432 +0,0 @@
# configuration file for class based RNN training
# final test PPL=122.54
ExpFolder=$ExpDir$
ConfigFolder=$ConfigDir$
DataFolder=$DataDir$
stderr=$ExpFolder$
numCPUThreads=4
# command=dumpNodeInfo
#command=train
#command=test
command=writeWordAndClassInfo:train:test
command=train:test
type=double
DEVICEID=-1
NOISE=100
RATE=0.1
VOCABSIZE=10000
CLASSSIZE=50
makeMode=true
TRAINFILE=ptb.train.cntk.txt
VALIDFILE=ptb.valid.cntk.txt
TESTFILE=ptb.test.cntk.txt
#number of threads
nthreads=4
writeWordAndClassInfo=[
action=writeWordAndClass
inputFile=$DataFolder$\$TRAINFILE$
outputVocabFile=$DataFolder$\vocab.txt
outputWord2Cls=$ExpFolder$\word2cls.txt
outputCls2Index=$ExpFolder$\cls2idx.txt
vocabSize=$VOCABSIZE$
cutoff=0
printValues=true
]
dumpNodeInfo=[
action=dumpnode
modelPath=$ExpFolder$\modelRnnCNTK
#nodeName=W0
printValues=true
]
devtest=[action=devtest]
train=[
action=train
minibatchSize=10
traceLevel=1
deviceId=$DEVICEID$
epochSize=4430000
# which is 886 * 5000
recurrentLayer=1
defaultHiddenActivity=0.1
useValidation=true
rnnType=NCELSTM
#CLASSLSTM
# uncomment below and comment SimpleNetworkBuilder section to use NDL to train RNN LM
# NDLNetworkBuilder=[
# networkDescription=$ConfigFolder$\rnnlm.ndl
# ]
SimpleNetworkBuilder=[
trainingCriterion=NoiseContrastiveEstimationNode
evalCriterion=NoiseContrastiveEstimationNode
nodeType=Sigmoid
initValueScale=6.0
layerSizes=$VOCABSIZE$:200:$VOCABSIZE$
addPrior=false
addDropoutNodes=false
applyMeanVarNorm=false
uniformInit=true;
# these are for the class information for class-based language modeling
vocabSize=$VOCABSIZE$
#nbrClass=$CLASSSIZE$
noise_number=$NOISE$
]
# configuration file, base parameters
SGD=[
makeMode=true
learningRatesPerSample=$RATE$
momentumPerMB=0
gradientClippingWithTruncation=true
clippingThresholdPerSample=15.0
maxEpochs=40
unroll=false
numMBsToShowResult=2000
# gradUpdateType=AdaGrad
gradUpdateType=None
modelPath=$ExpFolder$\modelRnnCNTK
loadBestModel=true
# settings for Auto Adjust Learning Rate
AutoAdjust=[
# auto learning rate adjustment
autoAdjustLR=adjustafterepoch
reduceLearnRateIfImproveLessThan=0.001
continueReduce=false
increaseLearnRateIfImproveMoreThan=1000000000
learnRateDecreaseFactor=0.5
learnRateIncreaseFactor=1.382
numMiniBatch4LRSearch=100
numPrevLearnRates=5
numBestSearchEpoch=1
]
dropoutRate=0.0
]
reader=[
readerType=LMSequenceReader
randomize=None
nbruttsineachrecurrentiter=10
# word class info
wordclass=$DataFolder$\vocab.txt
noise_number=$NOISE$
mode=nce
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$TRAINFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
cvReader=[
# reader to use
readerType=LMSequenceReader
randomize=None
mode=softmax
# word class info
wordclass=$DataFolder$\vocab.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.valid.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$VALIDFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
# it should be the same as that in the training set
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
]
test=[
action=eval
# correspond to the number of words/characteres to train in a minibatch
minibatchSize=1
# need to be small since models are updated for each minibatch
traceLevel=1
deviceId=$DEVICEID$
epochSize=4430000
# which is 886 * 5000
recurrentLayer=1
defaultHiddenActivity=0.1
useValidation=true
rnnType=NCELSTM
modelPath=$ExpFolder$\modelRnnCNTK
reader=[
# reader to use
readerType=LMSequenceReader
randomize=None
mode=softmax
# word class info
wordclass=$DataFolder$\vocab.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
# wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
# windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$TESTFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
]

425
ExampleSetups/LM/LSTMLM/lstmlm.gpu.config
Просмотреть файл

@ -1,425 +0,0 @@
# configuration file for class based RNN training
# ppl=133.35
ExpFolder=$ExpDir$
ConfigFolder=$ConfigDir$
DataFolder=$DataDir$
stderr=$ExpFolder$
# command=dumpNodeInfo
#command=train
#command=test
command=writeWordAndClassInfo:train:test
#command=writeWordAndClassInfo
type=double
DEVICEID=Auto
numCPUThreads=4
VOCABSIZE=10000
CLASSSIZE=50
TRAINFILE=ptb.train.cntk.txt
VALIDFILE=ptb.valid.cntk.txt
TESTFILE=ptb.test.cntk.txt
writeWordAndClassInfo=[
action=writeWordAndClass
inputFile=$DataFolder$\$TRAINFILE$
outputVocabFile=$ExpFolder$\vocab.txt
outputWord2Cls=$ExpFolder$\word2cls.txt
outputCls2Index=$ExpFolder$\cls2idx.txt
vocabSize=$VOCABSIZE$
nbrClass=$CLASSSIZE$
cutoff=0
printValues=true
]
dumpNodeInfo=[
action=dumpnode
modelPath=$ExpFolder$\modelRnnCNTK
#nodeName=W0
printValues=true
]
devtest=[action=devtest]
train=[
action=train
minibatchSize=10
traceLevel=1
deviceId=$DEVICEID$
epochSize=4430000
# which is 886 * 5000
recurrentLayer=1
defaultHiddenActivity=0.1
useValidation=true
rnnType=CLASSLSTM
# uncomment below and comment SimpleNetworkBuilder section to use NDL to train RNN LM
# NDLNetworkBuilder=[
# networkDescription=$ConfigFolder$\rnnlm.ndl
# ]
SimpleNetworkBuilder=[
trainingCriterion=classcrossentropywithsoftmax
evalCriterion=classcrossentropywithsoftmax
nodeType=Sigmoid
initValueScale=6.0
layerSizes=$VOCABSIZE$:150:200:$VOCABSIZE$
addPrior=false
addDropoutNodes=false
applyMeanVarNorm=false
uniformInit=true;
lookupTableOrder=1
# these are for the class information for class-based language modeling
vocabSize=$VOCABSIZE$
nbrClass=$CLASSSIZE$
]
# configuration file, base parameters
SGD=[
learningRatesPerSample=0.1
momentumPerMB=0
gradientClippingWithTruncation=true
clippingThresholdPerSample=15.0
maxEpochs=40
unroll=false
numMBsToShowResult=2000
# gradUpdateType=AdaGrad
gradUpdateType=None
modelPath=$ExpFolder$\modelRnnCNTK
loadBestModel=true
# settings for Auto Adjust Learning Rate
AutoAdjust=[
# auto learning rate adjustment
autoAdjustLR=adjustafterepoch
reduceLearnRateIfImproveLessThan=0.001
continueReduce=false
increaseLearnRateIfImproveMoreThan=1000000000
learnRateDecreaseFactor=0.5
learnRateIncreaseFactor=1.382
numMiniBatch4LRSearch=100
numPrevLearnRates=5
numBestSearchEpoch=1
]
dropoutRate=0.0
]
reader=[
readerType=LMSequenceReader
randomize=None
nbruttsineachrecurrentiter=10
# word class info
wordclass=$ExpFolder$\vocab.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$TRAINFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
cvReader=[
# reader to use
readerType=LMSequenceReader
randomize=None
# word class info
wordclass=$ExpFolder$\vocab.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.valid.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$VALIDFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
# it should be the same as that in the training set
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
]
test=[
action=eval
# correspond to the number of words/characteres to train in a minibatch
minibatchSize=1
# need to be small since models are updated for each minibatch
traceLevel=1
deviceId=$DEVICEID$
epochSize=4430000
# which is 886 * 5000
recurrentLayer=1
defaultHiddenActivity=0.1
useValidation=true
rnnType=CLASSLM
modelPath=$ExpFolder$\modelRnnCNTK
reader=[
# reader to use
readerType=LMSequenceReader
randomize=None
# word class info
wordclass=$ExpFolder$\vocab.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
# wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
# windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$TESTFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
]

432
ExampleSetups/LM/LSTMLM/lstmlm.gpu.nce.config
Просмотреть файл

@ -1,432 +0,0 @@
# configuration file for class based RNN training
# final test PPL=122.54
ExpFolder=$ExpDir$
ConfigFolder=$ConfigDir$
DataFolder=$DataDir$
stderr=$ExpFolder$
numCPUThreads=4
# command=dumpNodeInfo
#command=train
#command=test
command=writeWordAndClassInfo:train:test
command=train:test
type=double
DEVICEID=-1
NOISE=100
RATE=0.1
VOCABSIZE=10000
CLASSSIZE=50
makeMode=true
TRAINFILE=ptb.train.cntk.txt
VALIDFILE=ptb.valid.cntk.txt
TESTFILE=ptb.test.cntk.txt
#number of threads
nthreads=4
writeWordAndClassInfo=[
action=writeWordAndClass
inputFile=$DataFolder$\$TRAINFILE$
outputVocabFile=$DataFolder$\vocab.txt
outputWord2Cls=$ExpFolder$\word2cls.txt
outputCls2Index=$ExpFolder$\cls2idx.txt
vocabSize=$VOCABSIZE$
cutoff=0
printValues=true
]
dumpNodeInfo=[
action=dumpnode
modelPath=$ExpFolder$\modelRnnCNTK
#nodeName=W0
printValues=true
]
devtest=[action=devtest]
train=[
action=train
minibatchSize=10
traceLevel=1
deviceId=$DEVICEID$
epochSize=4430000
# which is 886 * 5000
recurrentLayer=1
defaultHiddenActivity=0.1
useValidation=true
rnnType=NCELSTM
#CLASSLSTM
# uncomment below and comment SimpleNetworkBuilder section to use NDL to train RNN LM
# NDLNetworkBuilder=[
# networkDescription=$ConfigFolder$\rnnlm.ndl
# ]
SimpleNetworkBuilder=[
trainingCriterion=NoiseContrastiveEstimationNode
evalCriterion=NoiseContrastiveEstimationNode
nodeType=Sigmoid
initValueScale=6.0
layerSizes=$VOCABSIZE$:200:$VOCABSIZE$
addPrior=false
addDropoutNodes=false
applyMeanVarNorm=false
uniformInit=true;
# these are for the class information for class-based language modeling
vocabSize=$VOCABSIZE$
#nbrClass=$CLASSSIZE$
noise_number=$NOISE$
]
# configuration file, base parameters
SGD=[
makeMode=true
learningRatesPerSample=$RATE$
momentumPerMB=0
gradientClippingWithTruncation=true
clippingThresholdPerSample=15.0
maxEpochs=40
unroll=false
numMBsToShowResult=2000
# gradUpdateType=AdaGrad
gradUpdateType=None
modelPath=$ExpFolder$\modelRnnCNTK
loadBestModel=true
# settings for Auto Adjust Learning Rate
AutoAdjust=[
# auto learning rate adjustment
autoAdjustLR=adjustafterepoch
reduceLearnRateIfImproveLessThan=0.001
continueReduce=false
increaseLearnRateIfImproveMoreThan=1000000000
learnRateDecreaseFactor=0.5
learnRateIncreaseFactor=1.382
numMiniBatch4LRSearch=100
numPrevLearnRates=5
numBestSearchEpoch=1
]
dropoutRate=0.0
]
reader=[
readerType=LMSequenceReader
randomize=None
nbruttsineachrecurrentiter=10
# word class info
wordclass=$DataFolder$\vocab.txt
noise_number=$NOISE$
mode=nce
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$TRAINFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
cvReader=[
# reader to use
readerType=LMSequenceReader
randomize=None
mode=softmax
# word class info
wordclass=$DataFolder$\vocab.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.valid.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$VALIDFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
# it should be the same as that in the training set
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
]
test=[
action=eval
# correspond to the number of words/characteres to train in a minibatch
minibatchSize=1
# need to be small since models are updated for each minibatch
traceLevel=1
deviceId=$DEVICEID$
epochSize=4430000
# which is 886 * 5000
recurrentLayer=1
defaultHiddenActivity=0.1
useValidation=true
rnnType=NCELSTM
modelPath=$ExpFolder$\modelRnnCNTK
reader=[
# reader to use
readerType=LMSequenceReader
randomize=None
mode=softmax
# word class info
wordclass=$DataFolder$\vocab.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=$ExpFolder$\sequenceSentence.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
# wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
# windowSize - number of records we should include in BinaryWriter window
windowSize=$VOCABSIZE$
file=$DataFolder$\$TESTFILE$
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
labelIn=[
dim=1
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
# vocabulary size
labelDim=$VOCABSIZE$
labelMappingFile=$ExpFolder$\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
]

415
ExampleSetups/LM/LSTMLM/lstmlmconfig.txt
Просмотреть файл

@ -1,415 +0,0 @@
# configuration file for CNTK MNIST Before Check-In Tests
stderr=c:\temp\\penntreebank\lstmpenntreebank.log
command=penntreebanklstmrndmb10lr01:penntreebanktestlstmrndmb10lr01
#
# Class-based cross entropy LSTM experiments
#
# learning rate: 0.15
# minibatch size 10
# number of sentences in each minibatch nbruttsineachrecurrentiter=10
#
# iter 30: PPL = exp(4.9226532) = 137.37
# iter 20: PPL = exp(4.9226532) = 137.37
# iter 10: PPL = exp(4.9226532) = 137.37
# iter 1: PPL = exp(5.087323) = 161.96
#
# baseline from Thomas Mikolov's paper
# http://www.fit.vutbr.cz/research/groups/speech/publi/2011/mikolov_icassp2011_5528.pdf
# test PPL : 136
#
# LSTM results from Alex Graves LSTM paper
# http://arxiv.org/pdf/1308.0850v2.pdf
# test PPL : 138
# notice that Alex's paper doesn't use class for speed-up.
#
penntreebanklstmrndmb10lr01=[
# this is the maximum size for the minibatch, since sequence minibatches are really just a single sequence
# can be considered as the maximum length of a sentence
action=train
# correspond to the number of words/characteres to train in a minibatch
minibatchSize=6
# need to be small since models are updated for each minibatch
traceLevel=1
deviceId=-1
epochSize=4430000
# which is 886 * 5000
SimpleNetworkBuilder=[
recurrentLayer=1
defaultHiddenActivity=0.1
rnnType=CLASSLSTM
trainingCriterion=classcrossentropywithsoftmax
evalCriterion=classcrossentropywithsoftmax
nodeType=Sigmoid
initValueScale=6.0
layerSizes=10000:200:10000
addPrior=false
addDropoutNodes=false
applyMeanVarNorm=false
uniformInit=true;
vocabSize=10000
nbrClass=50
]
# configuration file, base parameters
SGD=[
learningRatesPerSample=0.15
momentumPerMB=0.90
# momentumPerMB=0.0
# gradientClippingWithTruncation=false
# clippingThresholdPerSample=15.0
maxEpochs=150
unroll=false
numMBsToShowResult=1000
useAdagrad=true
modelPath=c:\temp\penntreebank\cntkdebug.dnn
loadBestModel=true
# settings for Auto Adjust Learning Rate
AutoAdjust=[
# auto learning rate adjustment
autoAdjustLR=adjustafterepoch
reduceLearnRateIfImproveLessThan=0
increaseLearnRateIfImproveMoreThan=1000000000
learnRateDecreaseFactor=0.618
learnRateIncreaseFactor=1.382
numMiniBatch4LRSearch=100
numPrevLearnRates=5
numBestSearchEpoch=1
]
dropoutRate=0.0
]
reader=[
# reader to use
readerType=LMSequenceReader
randomize=None
# word class info
wordclass=c:\exp\penntreebank\data\wordclass.txt
# number of utterances to be allocated for each minibatch
nbruttsineachrecurrentiter=10
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=c:\temp\penntreebank\sequenceSentence.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=10000
# file=c:\exp\penntreebank\data\ptb.train.cntk.100.txt
file=c:\exp\penntreebank\data\ptb.train.cntk.txt
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
labelIn=[
dim=1
# vocabulary size
labelDim=10000
labelMappingFile=c:\temp\penntreebank\sentenceLabels.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
# vocabulary size
labelDim=10000
labelMappingFile=c:\temp\penntreebank\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
cvReader=[
# reader to use
readerType=LMSequenceReader
randomize=None
# word class info
wordclass=c:\exp\penntreebank\data\wordclass.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=c:\temp\penntreebank\sequenceSentence.valid.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=10000
# file=c:\exp\penntreebank\data\ptb.valid.cntk.100.txt
file=c:\exp\penntreebank\data\ptb.valid.cntk.txt
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
# it should be the same as that in the training set
labelIn=[
dim=1
# vocabulary size
labelDim=10000
labelMappingFile=c:\temp\penntreebank\sentenceLabels.out.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
labelDim=10000
labelMappingFile=c:\temp\penntreebank\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
]
penntreebanktestlstmrndmb10lr01=[
# this is the maximum size for the minibatch, since sequence minibatches are really just a single sequence
# can be considered as the maximum length of a sentence
action=eval
# correspond to the number of words/characteres to train in a minibatch
minibatchSize=6
# need to be small since models are updated for each minibatch
traceLevel=1
deviceId=-1
epochSize=4430000
# which is 886 * 5000
recurrentLayer=1
defaultHiddenActivity=0.1
useValidation=true
rnnType=CLASSLM
modelPath=c:\temp\penntreebank\cntkdebug.dnn
reader=[
# reader to use
readerType=LMSequenceReader
randomize=None
# word class info
wordclass=c:\exp\penntreebank\data\wordclass.txt
# if writerType is set, we will cache to a binary file
# if the binary file exists, we will use it instead of parsing this file
# writerType=BinaryReader
#### write definition
wfile=c:\temp\penntreebank\sequenceSentence.bin
#wsize - inital size of the file in MB
# if calculated size would be bigger, that is used instead
wsize=256
#wrecords - number of records we should allocate space for in the file
# files cannot be expanded, so this should be large enough. If known modify this element in config before creating file
wrecords=1000
#windowSize - number of records we should include in BinaryWriter window
windowSize=10000
# file=c:\exp\penntreebank\data\ptb.test.cntk.100.txt
file=c:\exp\penntreebank\data\ptb.test.cntk.txt
#additional features sections
#for now store as expanded category data (including label in)
features=[
# sentence has no features, so need to set dimension to zero
dim=0
### write definition
sectionType=data
]
# sequence break table, list indexes into sequence records, so we know when a sequence starts/stops
sequence=[
dim=1
wrecords=2
### write definition
sectionType=data
]
#labels sections
labelIn=[
dim=1
# vocabulary size
labelDim=10000
labelMappingFile=c:\temp\penntreebank\sentenceLabels.txt
labelType=Category
beginSequence="</s>"
endSequence="</s>"
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=11
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=11
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
#labels sections
labels=[
dim=1
labelType=NextWord
beginSequence="O"
endSequence="O"
# vocabulary size
labelDim=10000
labelMappingFile=c:\temp\penntreebank\sentenceLabels.out.txt
#### Write definition ####
# sizeof(unsigned) which is the label index type
elementSize=4
sectionType=labels
mapping=[
#redefine number of records for this section, since we don't need to save it for each data record
wrecords=3
#variable size so use an average string size
elementSize=10
sectionType=labelMapping
]
category=[
dim=3
#elementSize=sizeof(ElemType) is default
sectionType=categoryLabels
]
]
]
]

Двоичные данные
ExampleSetups/LM/LSTMLM/perplexity.class50.lr0.1.txt
Просмотреть файл

Двоичный файл не отображается.

40
ExampleSetups/LM/LSTMLM/perplexity.nce100.lr0.1.txt
Просмотреть файл

@ -1,40 +0,0 @@
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.4415899 Perplexity = 230.80885 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.4415899 Perplexity = 230.80885
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.2513086 Perplexity = 190.8158 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.2513086 Perplexity = 190.8158
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.1372039 Perplexity = 170.23909 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.1372039 Perplexity = 170.23909
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0720036 Perplexity = 159.49358 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0720036 Perplexity = 159.49358
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0618825 Perplexity = 157.88746 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0618825 Perplexity = 157.88746
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0321352 Perplexity = 153.25991 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0321352 Perplexity = 153.25991
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0083887 Perplexity = 149.66339 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0083887 Perplexity = 149.66339
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0220441 Perplexity = 151.72111 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 5.0220441 Perplexity = 151.72111
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9666183 Perplexity = 143.54066 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9666183 Perplexity = 143.54066
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9719939 Perplexity = 144.31436 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9719939 Perplexity = 144.31436
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9397468 Perplexity = 139.73487 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9397468 Perplexity = 139.73487
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9442405 Perplexity = 140.36421 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9442405 Perplexity = 140.36421
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9287878 Perplexity = 138.21187 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9287878 Perplexity = 138.21187
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9292672 Perplexity = 138.27814 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9292672 Perplexity = 138.27814
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9187232 Perplexity = 136.82781 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9187232 Perplexity = 136.82781
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9224714 Perplexity = 137.34162 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9224714 Perplexity = 137.34162
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9124989 Perplexity = 135.97878 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9124989 Perplexity = 135.97878
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9146056 Perplexity = 136.26556 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9146056 Perplexity = 136.26556
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9083939 Perplexity = 135.42174 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9083939 Perplexity = 135.42174
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9042288 Perplexity = 134.85886 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9042288 Perplexity = 134.85886
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9007954 Perplexity = 134.39664 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.9007954 Perplexity = 134.39664
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8977358 Perplexity = 133.98607 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8977358 Perplexity = 133.98607
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8960381 Perplexity = 133.75879 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8960381 Perplexity = 133.75879
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8950807 Perplexity = 133.63079 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8950807 Perplexity = 133.63079
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8947338 Perplexity = 133.58444 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8947338 Perplexity = 133.58444
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8945776 Perplexity = 133.56358 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8945776 Perplexity = 133.56358
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8944772 Perplexity = 133.55017 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8944772 Perplexity = 133.55017
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8944128 Perplexity = 133.54157 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8944128 Perplexity = 133.54157
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943936 Perplexity = 133.53901 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943936 Perplexity = 133.53901
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.89438 Perplexity = 133.53718 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.89438 Perplexity = 133.53718
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943731 Perplexity = 133.53627 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943731 Perplexity = 133.53627
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943699 Perplexity = 133.53584 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943699 Perplexity = 133.53584
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.894368 Perplexity = 133.53558 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.894368 Perplexity = 133.53558
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.894367 Perplexity = 133.53545 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.894367 Perplexity = 133.53545
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943667 Perplexity = 133.53541 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943667 Perplexity = 133.53541
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943665 Perplexity = 133.53538 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943665 Perplexity = 133.53538
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943664 Perplexity = 133.53537 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943664 Perplexity = 133.53537
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943664 Perplexity = 133.53537 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943664 Perplexity = 133.53537
Final Results: Minibatch[1-8905]: Samples Seen = 73760 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943663 Perplexity = 133.53537 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8943663 Perplexity = 133.53537
Final Results: Minibatch[1-82430]: Samples Seen = 82430 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8084526 Perplexity = 122.54185 TrainNodeNCEBasedCrossEntropy: NCEBasedCrossEntropyWithSoftmax/Sample = 4.8084526 Perplexity = 122.54185

4
ExampleSetups/LM/LSTMLM/readme.txt
Просмотреть файл

@ -1,4 +0,0 @@
# LSTM LM setup on PennTreebank.
to use it, call
cntk.exe configFile=global.config+lstmlm.gpu.config ExpDir=<your exp dir>

14
ExampleSetups/Readme.md
Просмотреть файл

@ -1,14 +0,0 @@
# CNTK Demos and Example Setups
This folder contains examples that correspond to popular data sets and tasks.
These data sets often require a license and are therefore not included in the repository.
The 'Demos' folder contains a few self-contained and documented demos to get started with CNTK.
The four examples shown in the table below provide a good introduction to CNTK.
|Folder | Domain | Network types |
|:------------------------|:-------------------------------------------------|:----------------|
Demos/Simple2d | Synthetic 2d data | FF (CPU and GPU)
Demos/Speech | Speech data (CMU AN4) | FF and LSTM
Demos/Text | Text data (penn treebank) | RNN
ExampleSetups/Image/MNIST | Image data (MNIST handwritten digit recognition) | CNN

0
ExampleSetups/Image/MNIST/AdditionalFiles/mnist_convert.py → Examples/Image/MNIST/AdditionalFiles/mnist_convert.py
Просмотреть файл

0
ExampleSetups/Image/MNIST/Config/01_OneHidden.config → Examples/Image/MNIST/Config/01_OneHidden.config
Просмотреть файл

0
ExampleSetups/Image/MNIST/Config/01_OneHidden.ndl → Examples/Image/MNIST/Config/01_OneHidden.ndl
Просмотреть файл

0
ExampleSetups/Image/MNIST/Config/02_Convolution.config → Examples/Image/MNIST/Config/02_Convolution.config
Просмотреть файл

0
ExampleSetups/Image/MNIST/Config/02_Convolution.ndl → Examples/Image/MNIST/Config/02_Convolution.ndl
Просмотреть файл

0
ExampleSetups/Image/MNIST/Config/03_ConvBatchNorm.config → Examples/Image/MNIST/Config/03_ConvBatchNorm.config
Просмотреть файл

0
ExampleSetups/Image/MNIST/Config/03_ConvBatchNorm.ndl → Examples/Image/MNIST/Config/03_ConvBatchNorm.ndl
Просмотреть файл

0
ExampleSetups/Image/MNIST/Config/Macros.ndl → Examples/Image/MNIST/Config/Macros.ndl
Просмотреть файл

0
ExampleSetups/Image/MNIST/Data/labelsmap.txt → Examples/Image/MNIST/Data/labelsmap.txt
Просмотреть файл

0
ExampleSetups/Image/MNIST/README.md → Examples/Image/MNIST/README.md
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/01_Conv.config → Examples/Image/Miscellaneous/CIFAR-10/01_Conv.config
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/01_Convolution.ndl → Examples/Image/Miscellaneous/CIFAR-10/01_Convolution.ndl
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/02_BatchNormConv.config → Examples/Image/Miscellaneous/CIFAR-10/02_BatchNormConv.config
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/02_BatchNormConv.mel → Examples/Image/Miscellaneous/CIFAR-10/02_BatchNormConv.mel
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/02_BatchNormConv.ndl → Examples/Image/Miscellaneous/CIFAR-10/02_BatchNormConv.ndl
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/CIFAR_convert.py → Examples/Image/Miscellaneous/CIFAR-10/CIFAR_convert.py
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/Macros.ndl → Examples/Image/Miscellaneous/CIFAR-10/Macros.ndl
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/labelsmap.txt → Examples/Image/Miscellaneous/CIFAR-10/labelsmap.txt
Просмотреть файл

0
ExampleSetups/Image/CIFAR-10/readme.txt → Examples/Image/Miscellaneous/CIFAR-10/readme.txt
Просмотреть файл

0
ExampleSetups/Image/ImageNet/AlexNet/AlexNet.config → Examples/Image/Miscellaneous/ImageNet/AlexNet/AlexNet.config
Просмотреть файл

0
ExampleSetups/Image/ImageNet/AlexNet/AlexNet.ndl → Examples/Image/Miscellaneous/ImageNet/AlexNet/AlexNet.ndl
Просмотреть файл

0
ExampleSetups/Image/ImageNet/AlexNet/Macros.ndl → Examples/Image/Miscellaneous/ImageNet/AlexNet/Macros.ndl
Просмотреть файл

0
ExampleSetups/Image/ImageNet/AlexNet/add_top5_layer.mel → Examples/Image/Miscellaneous/ImageNet/AlexNet/add_top5_layer.mel
Просмотреть файл

0
ExampleSetups/Image/ImageNet/VGG/ImageNet1K_mean.xml → Examples/Image/Miscellaneous/ImageNet/VGG/ImageNet1K_mean.xml
Просмотреть файл

0
ExampleSetups/Image/ImageNet/VGG/Macros.ndl → Examples/Image/Miscellaneous/ImageNet/VGG/Macros.ndl
Просмотреть файл

0
ExampleSetups/Image/ImageNet/VGG/VGG_A.config → Examples/Image/Miscellaneous/ImageNet/VGG/VGG_A.config
Просмотреть файл

0
ExampleSetups/Image/ImageNet/VGG/VGG_A.ndl → Examples/Image/Miscellaneous/ImageNet/VGG/VGG_A.ndl
Просмотреть файл

0
ExampleSetups/Image/ImageNet/VGG/VGG_E.config → Examples/Image/Miscellaneous/ImageNet/VGG/VGG_E.config
Просмотреть файл

0
ExampleSetups/Image/ImageNet/VGG/VGG_E.ndl → Examples/Image/Miscellaneous/ImageNet/VGG/VGG_E.ndl
Просмотреть файл

0
ExampleSetups/Image/ImageNet/VGG/add_top5_layer.mel → Examples/Image/Miscellaneous/ImageNet/VGG/add_top5_layer.mel
Просмотреть файл

0
Demos/Image/README.md → Examples/Image/README.md
Просмотреть файл

0
ExampleSetups/NdlExamples/NDLExamples.ndl → Examples/Miscellaneous/NdlExamples/NDLExamples.ndl
Просмотреть файл

0
Demos/Simple2d/AdditionalFiles/MakeData.m → Examples/Miscellaneous/Simple2d/AdditionalFiles/MakeData.m
Просмотреть файл

0
Demos/Simple2d/AdditionalFiles/SimpleDemoDataReference.png → Examples/Miscellaneous/Simple2d/AdditionalFiles/SimpleDemoDataReference.png
Просмотреть файл

Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 177 KiB

После

Ширина:  |  Высота:  |  Размер: 177 KiB

0
Demos/Simple2d/AdditionalFiles/SimpleDemoErrorRateReference.png → Examples/Miscellaneous/Simple2d/AdditionalFiles/SimpleDemoErrorRateReference.png
Просмотреть файл

Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 11 KiB

После

Ширина:  |  Высота:  |  Размер: 11 KiB

0
Demos/Simple2d/AdditionalFiles/SimpleDemoOutputReference.png → Examples/Miscellaneous/Simple2d/AdditionalFiles/SimpleDemoOutputReference.png
Просмотреть файл

Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 22 KiB

После

Ширина:  |  Высота:  |  Размер: 22 KiB

0
Demos/Simple2d/Config/Multigpu.config → Examples/Miscellaneous/Simple2d/Config/Multigpu.config
Просмотреть файл

0
Demos/Simple2d/Config/Simple.config → Examples/Miscellaneous/Simple2d/Config/Simple.config
Просмотреть файл

0
Demos/Simple2d/Data/SimpleDataTest.txt → Examples/Miscellaneous/Simple2d/Data/SimpleDataTest.txt
Просмотреть файл

0
Demos/Simple2d/Data/SimpleDataTrain.txt → Examples/Miscellaneous/Simple2d/Data/SimpleDataTrain.txt
Просмотреть файл

0
Demos/Simple2d/Data/SimpleMapping.txt → Examples/Miscellaneous/Simple2d/Data/SimpleMapping.txt
Просмотреть файл

0
Demos/Simple2d/README.md → Examples/Miscellaneous/Simple2d/README.md
Просмотреть файл

0
Demos/README.md → Examples/README.md
Просмотреть файл

0
Demos/Speech/AdditionalFiles/AN4.LICENSE.html → Examples/Speech/AN4/AdditionalFiles/AN4.LICENSE.html
Просмотреть файл

0
Demos/Speech/Config/FeedForward.config → Examples/Speech/AN4/Config/FeedForward.config
Просмотреть файл

0
Demos/Speech/Config/LSTM-NDL.config → Examples/Speech/AN4/Config/LSTM-NDL.config
Просмотреть файл

0
Demos/Speech/Config/lstmp-3layer-opt.ndl → Examples/Speech/AN4/Config/lstmp-3layer-opt.ndl
Просмотреть файл

0
Demos/Speech/Data/000000000.chunk → Examples/Speech/AN4/Data/000000000.chunk
Просмотреть файл

0
Demos/Speech/Data/glob_0000.mlf → Examples/Speech/AN4/Data/glob_0000.mlf
Просмотреть файл

0
Demos/Speech/Data/glob_0000.scp → Examples/Speech/AN4/Data/glob_0000.scp
Просмотреть файл

0
Demos/Speech/Data/state.list → Examples/Speech/AN4/Data/state.list
Просмотреть файл

0
Demos/Speech/README.md → Examples/Speech/AN4/README.md
Просмотреть файл

Некоторые файлы не были показаны из-за слишком большого количества измененных файлов Показать больше