This commit is contained in:
Thilo Will 2017-01-30 16:14:20 +01:00
Родитель 2d1782d8bb
Коммит 68e9cbfb35
1 изменённых файлов: 1 добавлений и 1 удалений

Просмотреть файл

@ -6,7 +6,7 @@
"source": [
"# Sampled Softmax\n",
"\n",
"For classification and prediction problems a typical criterion function is cross-entropy with softmax. If the number of output classes is high the computation of this criterion and the corresponding gradients could be quite costly. Sampled Softmax is a heuristic to speed up training in these cases.\n",
"For classification and prediction problems a typical criterion function is cross-entropy with softmax. If the number of output classes is high the computation of this criterion and the corresponding gradients could be quite costly. Sampled Softmax is a heuristic to speed up training in these cases. (see: [Adaptive Importance Sampling to Accelerate Training of a Neural Probabilistic Language Model](http://www.iro.umontreal.ca/~lisa/pointeurs/importance_samplingIEEEtnn.pdf), [Exploring the Limits of Language Modeling](https://arxiv.org/pdf/1602.02410v1.pdf), [What is Candidate Sampling](https://www.tensorflow.org/extras/candidate_sampling.pdf))\n",
"\n",
"## Basics\n",
"\n",