microsoft
/
AI-For-Beginners
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Merge branch 'main' into patch-1

This commit is contained in:
Pikachú 2023-07-05 13:56:56 +03:00 коммит произвёл GitHub
Родитель 87f722737f d474ffa284
Коммит d5a86cd479
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
11 изменённых файлов: 210 добавлений и 228 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

4
.gitignore поставляемый
Просмотреть файл

@ -3,6 +3,10 @@
##
## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore
# virtual environment
venv/
ENV/
# ignore all pdf creation files
package.json
package-lock.json

56
README.md
Просмотреть файл

@ -29,8 +29,8 @@ What we will not cover in this curriculum:
* Business cases of using **AI in Business**. Consider taking [Introduction to AI for business users](https://docs.microsoft.com/learn/paths/introduction-ai-for-business-users/?WT.mc_id=academic-77998-cacaste) learning path on Microsoft Learn, or [AI Business School](https://www.microsoft.com/ai/ai-business-school/?WT.mc_id=academic-77998-cacaste), developed in cooperation with [INSEAD](https://www.insead.edu/).
* **Classic Machine Learning**, which is well described in our [Machine Learning for Beginners Curriculum](http://github.com/Microsoft/ML-for-Beginners).
* Practical AI applications built using **[Cognitive Services](https://azure.microsoft.com/services/cognitive-services/?WT.mc_id=academic-77998-cacaste)**. For this, we recommend that you start with modules Microsoft Learn for [vision](https://docs.microsoft.com/learn/paths/create-computer-vision-solutions-azure-cognitive-services/?WT.mc_id=academic-77998-cacaste), [natural language processing](https://docs.microsoft.com/learn/paths/explore-natural-language-processing/?WT.mc_id=academic-77998-cacaste) and others.
* Specific ML **Cloud Frameworks**, such as [Azure Machine Learning](https://azure.microsoft.com/services/machine-learning/?WT.mc_id=academic-77998-cacaste) or [Azure Databricks](https://docs.microsoft.com/learn/paths/data-engineer-azure-databricks?WT.mc_id=academic-77998-cacaste). Consider using [Build and operate machine learning solutions with Azure Machine Learning](https://docs.microsoft.com/learn/paths/build-ai-solutions-with-azure-ml-service/?WT.mc_id=academic-77998-cacaste) and [Build and Operate Machine Learning Solutions with Azure Databricks](https://docs.microsoft.com/learn/paths/build-operate-machine-learning-solutions-azure-databricks/?WT.mc_id=academic-77998-cacaste) learning paths.
* Practical AI applications built using **[Cognitive Services](https://azure.microsoft.com/services/cognitive-services/?WT.mc_id=academic-77998-cacaste)**. For this, we recommend that you start with modules Microsoft Learn for [vision](https://docs.microsoft.com/learn/paths/create-computer-vision-solutions-azure-cognitive-services/?WT.mc_id=academic-77998-cacaste), [natural language processing](https://docs.microsoft.com/learn/paths/explore-natural-language-processing/?WT.mc_id=academic-77998-cacaste), **[Generative AI with Azure OpenAI Service](https://learn.microsoft.com/en-us/training/paths/develop-ai-solutions-azure-openai/?WT.mc_id=academic-77998-bethanycheum)** and others.
* Specific ML **Cloud Frameworks**, such as [Azure Machine Learning](https://azure.microsoft.com/services/machine-learning/?WT.mc_id=academic-77998-cacaste), [Microsoft Fabric](https://learn.microsoft.com/en-us/training/paths/get-started-fabric/?WT.mc_id=academic-77998-bethanycheum), or [Azure Databricks](https://docs.microsoft.com/learn/paths/data-engineer-azure-databricks?WT.mc_id=academic-77998-cacaste). Consider using [Build and operate machine learning solutions with Azure Machine Learning](https://docs.microsoft.com/learn/paths/build-ai-solutions-with-azure-ml-service/?WT.mc_id=academic-77998-cacaste) and [Build and Operate Machine Learning Solutions with Azure Databricks](https://docs.microsoft.com/learn/paths/build-operate-machine-learning-solutions-azure-databricks/?WT.mc_id=academic-77998-cacaste) learning paths.
* **Conversational AI** and **Chat Bots**. There is a separate [Create conversational AI solutions](https://docs.microsoft.com/learn/paths/create-conversational-ai-solutions/?WT.mc_id=academic-77998-cacaste) learning path, and you can also refer to [this blog post](https://soshnikov.com/azure/hello-bot-conversational-ai-on-microsoft-platform/) for more detail.
* **Deep Mathematics** behind deep learning. For this, we would recommend [Deep Learning](https://www.amazon.com/Deep-Learning-Adaptive-Computation-Machine/dp/0262035618) by Ian Goodfellow, Yoshua Bengio and Aaron Courville, which is also available online at [https://www.deeplearningbook.org/](https://www.deeplearningbook.org/).
@ -46,55 +46,55 @@ For a gentle introduction to *AI in the Cloud* topics you may consider taking th
<tr><td>1</td><td>Introduction and History of AI</td><td><a href="lessons/1-Intro/README.md">Text</a></td><td></td><td></td><td></td></tr>
<tr><td>II</td><td colspan="4"><b>Symbolic AI</b></td><td></td></tr>
<tr><td>2 </td><td>Knowledge Representation and Expert Systems</td><td><a href="lessons/2-Symbolic/README.md">Text</a></td><td colspan="2"><a href="lessons/2-Symbolic/Animals.ipynb">Expert System</a>, <a href="lessons/2-Symbolic/FamilyOntology.ipynb">Ontology</a>, <a href="lessons/2-Symbolic/MSConceptGraph.ipynb">Concept Graph</a></td><td></td></tr>
<tr><td>2 </td><td>Knowledge Representation and Expert Systems</td><td><a href="lessons/2-Symbolic/README.md">Text</a></td><td colspan="2"><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/2-Symbolic/Animals.ipynb">Expert System</a>, <a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/2-Symbolic/FamilyOntology.ipynb">Ontology</a>, <a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/2-Symbolic/MSConceptGraph.ipynb">Concept Graph</a></td><td></td></tr>
<tr><td>III</td><td colspan="4"><b><a href="lessons/3-NeuralNetworks/README.md">Introduction to Neural Networks</a></b></td><td></td></tr>
<tr><td>3</td><td>Perceptron</td>
<td><a href="lessons/3-NeuralNetworks/03-Perceptron/README.md">Text</a>
<td colspan="2"><a href="lessons/3-NeuralNetworks/03-Perceptron/Perceptron.ipynb">Notebook</a></td><td><a href="lessons/3-NeuralNetworks/03-Perceptron/lab/README.md">Lab</a></td></tr>
<tr><td>4 </td><td>Multi-Layered Perceptron and Creating our own Framework</td><td><a href="lessons/3-NeuralNetworks/04-OwnFramework/README.md">Text</a></td><td colspan="2"><a href="lessons/3-NeuralNetworks/04-OwnFramework/OwnFramework.ipynb">Notebook</a><td><a href="lessons/3-NeuralNetworks/04-OwnFramework/lab/README.md">Lab</a></td></tr>
<td colspan="2"><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/03-Perceptron/Perceptron.ipynb">Notebook</a></td><td><a href="lessons/3-NeuralNetworks/03-Perceptron/lab/README.md">Lab</a></td></tr>
<tr><td>4 </td><td>Multi-Layered Perceptron and Creating our own Framework</td><td><a href="lessons/3-NeuralNetworks/04-OwnFramework/README.md">Text</a></td><td colspan="2"><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/04-OwnFramework/OwnFramework.ipynb">Notebook</a><td><a href="lessons/3-NeuralNetworks/04-OwnFramework/lab/README.md">Lab</a></td></tr>
<tr><td>5</td>
<td>Intro to Frameworks (PyTorch/TensorFlow)<br/>Overfitting</td>
<td><a href="lessons/3-NeuralNetworks/05-Frameworks/README.md">Text</a><br/><a href="lessons/3-NeuralNetworks/05-Frameworks/Overfitting.md">Text</a></td>
<td><a href="lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb">PyTorch</a></td>
<td><a href="lessons/3-NeuralNetworks/05-Frameworks/IntroKeras.ipynb">Keras</a>/<a href="lessons/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb">TensorFlow</a></td>
<td>Intro to Frameworks (PyTorch/TensorFlow) and Overfitting</td>
<td><a href="lessons/3-NeuralNetworks/05-Frameworks/README.md">Text</a></td>
<td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/05-Frameworks/IntroPyTorch.ipynb">PyTorch</a></td>
<td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/05-Frameworks/IntroKeras.ipynb">Keras</a>/<a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/3-NeuralNetworks/05-Frameworks/IntroKerasTF.ipynb">TensorFlow</a></td>
<td><a href="lessons/3-NeuralNetworks/05-Frameworks/lab/README.md">Lab</a></td></tr>
<tr><td>IV</td><td><b><a href="lessons/4-ComputerVision/README.md">Computer Vision</a></b></td>
<td colspan="3"><a href="https://docs.microsoft.com/learn/paths/explore-computer-vision-microsoft-azure/?WT.mc_id=academic-77998-cacaste"><i>AI Fundamentals: Explore Computer Vision</i></a></td>
<td colspan="3"><a href="https://docs.microsoft.com/learn/paths/explore-computer-vision-microsoft-azure/?WT.mc_id=academic-77998-cacaste"><i>Microsoft Azure AI Fundamentals: Explore Computer Vision</i></a></td>
<td></td></tr>
<tr><td></td><td colspan="2"><i>Microsoft Learn Module on Computer Vision</i></td>
<td><a href="https://docs.microsoft.com/learn/modules/intro-computer-vision-pytorch/?WT.mc_id=academic-77998-cacaste"><i>PyTorch</i></a></td>
<td><a href="https://docs.microsoft.com/learn/modules/intro-computer-vision-TensorFlow/?WT.mc_id=academic-77998-cacaste"><i>TensorFlow</i></a></td>
<td></td></tr>
<tr><td>6</td><td>Intro to Computer Vision. OpenCV</td><td><a href="lessons/4-ComputerVision/06-IntroCV/README.md">Text</a><td colspan="2"><a href="lessons/4-ComputerVision/06-IntroCV/OpenCV.ipynb">Notebook</a></td><td><a href="lessons/4-ComputerVision/06-IntroCV/lab/README.md">Lab</a></td></tr>
<tr><td>7</td><td>Convolutional Neural Networks<br/>CNN Architectures</td><td><a href="lessons/4-ComputerVision/07-ConvNets/README.md">Text</a><br/><a href="lessons/4-ComputerVision/07-ConvNets/CNN_Architectures.md">Text</a></td><td><a href="lessons/4-ComputerVision/07-ConvNets/ConvNetsPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/4-ComputerVision/07-ConvNets/ConvNetsTF.ipynb">TensorFlow</a></td><td><a href="lessons/4-ComputerVision/07-ConvNets/lab/README.md">Lab</a></td></tr>
<tr><td>8</td><td>Pre-trained Networks and Transfer Learning<br/>Training Tricks</td><td><a href="lessons/4-ComputerVision/08-TransferLearning/README.md">Text</a><br/><a href="lessons/4-ComputerVision/08-TransferLearning/TrainingTricks.md">Text</a></td><td><a href="lessons/4-ComputerVision/08-TransferLearning/TransferLearningPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/4-ComputerVision/08-TransferLearning/TransferLearningTF.ipynb">TensorFlow</a><br/><a href="lessons/4-ComputerVision/08-TransferLearning/Dropout.ipynb">Dropout sample</a><br/><a href="lessons/4-ComputerVision/08-TransferLearning/AdversarialCat_TF.ipynb">Adversarial Cat</a></td><td><a href="lessons/4-ComputerVision/08-TransferLearning/lab/README.md">Lab</a></td></tr>
<tr><td>9</td><td>Autoencoders and VAEs</td><td><a href="lessons/4-ComputerVision/09-Autoencoders/README.md">Text</a></td><td><a href="lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb">TensorFlow</a></td><td></td></tr>
<tr><td>10</td><td>Generative Adversarial Networks<br/>Artistic Style Transfer</td><td><a href="lessons/4-ComputerVision/10-GANs/README.md">Text</a></td><td><a href="lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb">PyTorch</td><td><a href="lessons/4-ComputerVision/10-GANs/GANTF.ipynb">TensorFlow GAN</a><br/><a href="lessons/4-ComputerVision/10-GANs/StyleTransfer.ipynb">Style Transfer</a></td><td></td></tr>
<tr><td>11</td><td>Object Detection</td><td><a href="lessons/4-ComputerVision/11-ObjectDetection/README.md">Text</a></td><td>PyTorch</td><td><a href="lessons/4-ComputerVision/11-ObjectDetection/ObjectDetection.ipynb">TensorFlow</td><td><a href="lessons/4-ComputerVision/11-ObjectDetection/lab/README.md">Lab</a></td></tr>
<tr><td>12</td><td>Semantic Segmentation. U-Net</td><td><a href="lessons/4-ComputerVision/12-Segmentation/README.md">Text</a></td><td><a href="lessons/4-ComputerVision/12-Segmentation/SemanticSegmentationPytorch.ipynb">PyTorch</td><td><a href="lessons/4-ComputerVision/12-Segmentation/SemanticSegmentationTF.ipynb">TensorFlow</td><td></td></tr>
<tr><td>6</td><td>Intro to Computer Vision. OpenCV</td><td><a href="lessons/4-ComputerVision/06-IntroCV/README.md">Text</a><td colspan="2"><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/06-IntroCV/OpenCV.ipynb">Notebook</a></td><td><a href="lessons/4-ComputerVision/06-IntroCV/lab/README.md">Lab</a></td></tr>
<tr><td>7</td><td>Convolutional Neural Networks<br/>CNN Architectures</td><td><a href="lessons/4-ComputerVision/07-ConvNets/README.md">Text</a><br/><a href="lessons/4-ComputerVision/07-ConvNets/CNN_Architectures.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/07-ConvNets/ConvNetsPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/4-ComputerVision/07-ConvNets/ConvNetsTF.ipynb">TensorFlow</a></td><td><a href="lessons/4-ComputerVision/07-ConvNets/lab/README.md">Lab</a></td></tr>
<tr><td>8</td><td>Pre-trained Networks and Transfer Learning<br/>Training Tricks</td><td><a href="lessons/4-ComputerVision/08-TransferLearning/README.md">Text</a><br/><a href="lessons/4-ComputerVision/08-TransferLearning/TrainingTricks.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/08-TransferLearning/TransferLearningPyTorch.ipynb">PyTorch</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/08-TransferLearning/TransferLearningTF.ipynb">TensorFlow</a><br/><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/08-TransferLearning/Dropout.ipynb">Dropout sample</a><br/><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/08-TransferLearning/AdversarialCat_TF.ipynb">Adversarial Cat</a></td><td><a href="lessons/4-ComputerVision/08-TransferLearning/lab/README.md">Lab</a></td></tr>
<tr><td>9</td><td>Autoencoders and VAEs</td><td><a href="lessons/4-ComputerVision/09-Autoencoders/README.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/09-Autoencoders/AutoEncodersPyTorch.ipynb">PyTorch</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/09-Autoencoders/AutoencodersTF.ipynb">TensorFlow</a></td><td></td></tr>
<tr><td>10</td><td>Generative Adversarial Networks<br/>Artistic Style Transfer</td><td><a href="lessons/4-ComputerVision/10-GANs/README.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/GANPyTorch.ipynb">PyTorch</td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/GANTF.ipynb">TensorFlow GAN</a><br/><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/10-GANs/StyleTransfer.ipynb">Style Transfer</a></td><td></td></tr>
<tr><td>11</td><td>Object Detection</td><td><a href="lessons/4-ComputerVision/11-ObjectDetection/README.md">Text</a></td><td>PyTorch</td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/11-ObjectDetection/ObjectDetection.ipynb">TensorFlow</td><td><a href="lessons/4-ComputerVision/11-ObjectDetection/lab/README.md">Lab</a></td></tr>
<tr><td>12</td><td>Semantic Segmentation. U-Net</td><td><a href="lessons/4-ComputerVision/12-Segmentation/README.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/4-ComputerVision/12-Segmentation/SemanticSegmentationPytorch.ipynb">PyTorch</td><td><a href="lessons/4-ComputerVision/12-Segmentation/SemanticSegmentationTF.ipynb">TensorFlow</td><td></td></tr>
<tr><td>V</td><td><b><a href="lessons/5-NLP/README.md">Natural Language Processing</a></b></td>
<td colspan="3"><a href="https://docs.microsoft.com/learn/paths/explore-natural-language-processing/?WT.mc_id=academic-77998-cacaste"><i>AI Fundamentals: Explore Natural Language Processing</i></a></td>
<td colspan="3"><a href="https://docs.microsoft.com/learn/paths/explore-natural-language-processing/?WT.mc_id=academic-77998-cacaste"><i>Microsoft Azure AI Fundamentals: Explore Natural Language Processing</i></a></td>
<td></td></tr>
<tr><td></td><td colspan="2"><i>Microsoft Learn Module on Natural Language</i></td>
<tr><td></td><td colspan="2"><i>Microsoft Learn Module on Natural language processing</i></td>
<td><a href="https://docs.microsoft.com/learn/modules/intro-natural-language-processing-pytorch/?WT.mc_id=academic-77998-cacaste"><i>PyTorch</i></a></td>
<td><a href="https://docs.microsoft.com/learn/modules/intro-natural-language-processing-TensorFlow/?WT.mc_id=academic-77998-cacaste"><i>TensorFlow</i></a></td>
<td></td></tr>
<tr><td>13</td><td>Text Representation. Bow/TF-IDF</td><td><a href="lessons/5-NLP/13-TextRep/README.md">Text</a></td><td><a href="lessons/5-NLP/13-TextRep/TextRepresentationPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/5-NLP/13-TextRep/TextRepresentationTF.ipynb">TensorFlow</td><td></td></tr>
<tr><td>14</td><td>Semantic word embeddings. Word2Vec and GloVe</td><td><a href="lessons/5-NLP/14-Embeddings/README.md">Text</td><td><a href="lessons/5-NLP/14-Embeddings/EmbeddingsPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/5-NLP/14-Embeddings/EmbeddingsTF.ipynb">TensorFlow</a></td><td></td></tr>
<tr><td>15</td><td>Language Modeling. Training your own embeddings</td><td><a href="lessons/5-NLP/15-LanguageModeling/README.md">Text</a></td><td><a href="lessons/5-NLP/15-LanguageModeling/CBoW-PyTorch.ipynb">PyTorch</a></td><td><a href="lessons/5-NLP/15-LanguageModeling/CBoW-TF.ipynb">TensorFlow</a></td><td><a href="lessons/5-NLP/15-LanguageModeling/lab/README.md">Lab</a></td></tr>
<tr><td>16</td><td>Recurrent Neural Networks</td><td><a href="lessons/5-NLP/16-RNN/README.md">Text</a></td><td><a href="lessons/5-NLP/16-RNN/RNNPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/5-NLP/16-RNN/RNNTF.ipynb">TensorFlow</a></td><td></td></tr>
<tr><td>13</td><td>Text Representation. Bow/TF-IDF</td><td><a href="lessons/5-NLP/13-TextRep/README.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/13-TextRep/TextRepresentationPyTorch.ipynb">PyTorch</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/13-TextRep/TextRepresentationTF.ipynb">TensorFlow</td><td></td></tr>
<tr><td>14</td><td>Semantic word embeddings. Word2Vec and GloVe</td><td><a href="lessons/5-NLP/14-Embeddings/README.md">Text</td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/14-Embeddings/EmbeddingsPyTorch.ipynb">PyTorch</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/14-Embeddings/EmbeddingsTF.ipynb">TensorFlow</a></td><td></td></tr>
<tr><td>15</td><td>Language Modeling. Training your own embeddings</td><td><a href="lessons/5-NLP/15-LanguageModeling/README.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/15-LanguageModeling/CBoW-PyTorch.ipynb">PyTorch</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/15-LanguageModeling/CBoW-TF.ipynb">TensorFlow</a></td><td><a href="lessons/5-NLP/15-LanguageModeling/lab/README.md">Lab</a></td></tr>
<tr><td>16</td><td>Recurrent Neural Networks</td><td><a href="lessons/5-NLP/16-RNN/README.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/16-RNN/RNNPyTorch.ipynb">PyTorch</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/16-RNN/RNNTF.ipynb">TensorFlow</a></td><td></td></tr>
<tr><td>17</td><td>Generative Recurrent Networks</td><td><a href="lessons/5-NLP/17-GenerativeNetworks/README.md">Text</a></td><td><a href="lessons/5-NLP/17-GenerativeNetworks/GenerativePyTorch.md">PyTorch</a></td><td><a href="lessons/5-NLP/17-GenerativeNetworks/GenerativeTF.md">TensorFlow</a></td><td><a href="lessons/5-NLP/17-GenerativeNetworks/lab/README.md">Lab</a></td></tr>
<tr><td>18</td><td>Transformers. BERT.</td><td><a href="lessons/5-NLP/18-Transformers/README.md">Text</a></td><td><a href="lessons/5-NLP/18-Transformers/TransformersPyTorch.ipynb">PyTorch</a></td><td><a href="lessons/5-NLP/18-Transformers/TransformersTF.ipynb">TensorFlow</a></td><td></td></tr>
<tr><td>18</td><td>Transformers. BERT.</td><td><a href="lessons/5-NLP/18-Transformers/README.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/18-Transformers/TransformersPyTorch.ipynb">PyTorch</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/5-NLP/18-Transformers/TransformersTF.ipynb">TensorFlow</a></td><td></td></tr>
<tr><td>19</td><td>Named Entity Recognition</td><td><a href="lessons/5-NLP/19-NER/README.md">Text</a></td><td></td><td><a href="lessons/5-NLP/19-NER/NER-TF.ipynb">TensorFlow</a></td><td><a href="lessons/5-NLP/19-NER/lab/README.md">Lab</a></td></tr>
<tr><td>20</td><td>Large Language Models, Prompt Programming and Few-Shot Tasks</td><td><a href="lessons/5-NLP/20-LangModels/README.md">Text</a></td><td><a href="lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb">PyTorch</td><td></td><td></td></tr>
<tr><td>VI</td><td colspan="4"><b>Other AI Techniques</b></td><td></td></tr>
<tr><td>21</td><td>Genetic Algorithms</td><td><a href="lessons/6-Other/21-GeneticAlgorithms/README.md">Text</a><td colspan="2"><a href="lessons/6-Other/21-GeneticAlgorithms/Genetic.ipynb">Notebook</a></td><td></td></tr>
<tr><td>22</td><td>Deep Reinforcement Learning</td><td><a href="lessons/6-Other/22-DeepRL/README.md">Text</a></td><td><a href="lessons/6-Other/22-DeepRL/CartPole-RL-PyTorch.ipynb">PyTorch</a></td><td><a href="lessons/6-Other/22-DeepRL/CartPole-RL-TF.ipynb">TensorFlow</a></td><td><a href="lessons/6-Other/22-DeepRL/lab/README.md">Lab</a></td></tr>
<tr><td>21</td><td>Genetic Algorithms</td><td><a href="lessons/6-Other/21-GeneticAlgorithms/README.md">Text</a><td colspan="2"><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/6-Other/21-GeneticAlgorithms/Genetic.ipynb">Notebook</a></td><td></td></tr>
<tr><td>22</td><td>Deep Reinforcement Learning</td><td><a href="lessons/6-Other/22-DeepRL/README.md">Text</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/6-Other/22-DeepRL/CartPole-RL-PyTorch.ipynb">PyTorch</a></td><td><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/6-Other/22-DeepRL/CartPole-RL-TF.ipynb">TensorFlow</a></td><td><a href="lessons/6-Other/22-DeepRL/lab/README.md">Lab</a></td></tr>
<tr><td>23</td><td>Multi-Agent Systems</td><td><a href="lessons/6-Other/23-MultiagentSystems/README.md">Text</a></td><td></td><td></td><td></td></tr>
<tr><td>VII</td><td colspan="4"><b>AI Ethics</b></td><td></td></tr>
<tr><td>24</td><td>AI Ethics and Responsible AI</td><td><a href="lessons/7-Ethics/README.md">Text</a></td><td colspan="2"><a href="https://docs.microsoft.com/learn/paths/responsible-ai-business-principles/?WT.mc_id=academic-77998-cacaste"><i>MS Learn: Responsible AI Principles</i></a></td><td></td></tr>
<tr><td></td><td colspan="4"><b>Extras</b></td><td></td></tr>
<tr><td>X1</td><td>Multi-Modal Networks, CLIP and VQGAN</td><td><a href="lessons/X-Extras/X1-MultiModal/README.md">Text</a></td><td colspan="2"><a href="lessons/X-Extras/X1-MultiModal/Clip.ipynb">Notebook</a></td><td></td></tr>
<tr><td>X1</td><td>Multi-Modal Networks, CLIP and VQGAN</td><td><a href="lessons/X-Extras/X1-MultiModal/README.md">Text</a></td><td colspan="2"><a href="https://github.com/microsoft/AI-For-Beginners/blob/main/lessons/X-Extras/X1-MultiModal/Clip.ipynb">Notebook</a></td><td></td></tr>
</table>
**[Mindmap of the Course](http://soshnikov.com/courses/ai-for-beginners/mindmap.html)**
@ -125,7 +125,7 @@ However, if you would like to take the course as a self-study project, we sugges
- Take the post-lecture quiz.
- If there is a lab attached to the module - complete the assignment.
- Visit the [Discussion board](https://github.com/microsoft/AI-For-Beginners/discussions) to "learn out loud".
- Chat with other learners [on Gitter](https://gitter.im/Microsoft/ai-for-beginners) or [in Telegram channel](http://t.me/ai_for_beginners).
> For further study, we recommend following these [Microsoft Learn](https://docs.microsoft.com/en-us/users/dmitrysoshnikov-9132/collections/31zgizg2p418yo/?WT.mc_id=academic-77998-cacaste) modules and learning paths.

2
lessons/3-NeuralNetworks/03-Perceptron/Perceptron.ipynb
Просмотреть файл

@ -213,7 +213,7 @@
" * $t_{n} \\in \\{-1, +1\\}$ for negative and positive training samples, respectively\n",
" * $\\mathcal{M}$ - a set of wrongly classified examples\n",
" \n",
"We will use the process of **graident descent**. Starting with some initial random weights $\\mathbf{w}^{(0)}$, we will adjust weights on each step of the training using the gradient of $E$:\n",
"We will use the process of **gradient descent**. Starting with some initial random weights $\\mathbf{w}^{(0)}$, we will adjust weights on each step of the training using the gradient of $E$:\n",
"\n",
"$$\\mathbf{w}^{\\tau + 1}=\\mathbf{w}^{\\tau} - \\eta \\nabla E(\\mathbf{w}) = \\mathbf{w}^{\\tau} + \\eta \\mathbf{x}_{n} t_{n}$$\n",
"\n",

75
lessons/3-NeuralNetworks/05-Frameworks/Overfitting.md
Просмотреть файл

@ -1,75 +0,0 @@
# Overfitting
Overfitting is an extremely important concept in machine learning, and it is very important to get it right!
Consider the following problem of approximating 5 dots (represented by `x` on the graphs below):
![linear](../images/overfit1.jpg) | ![overfit](../images/overfit2.jpg)
-------------------------|--------------------------
**Linear model, 2 parameters** | **Non-linear model, 7 parameters**
Training error = 5.3 | Training error = 0
Validation error = 5.1 | Validation error = 20
* On the left, we see a good straight line approximation. Because the number of parameters is adequate, the model gets the idea behind point distribution right.
* On the right, the model is too powerful. Because we only have 5 points and the model has 7 parameters, it can adjust in such a way as to pass through all points, making training the error to be 0. However, this prevents the model from understanding the correct pattern behind data, thus the validation error is very high.
It is very important to strike a correct balance between the richness of the model (number of parameters) and the number of training samples.
## Why overfitting occurs
* Not enough training data
* Too powerful model
* Too much noise in input data
## How to detect overfitting
As you can see from the graph above, overfitting can be detected by a very low training error, and a high validation error. Normally during training we will see both training and validation errors starting to decrease, and then at some point validation error might stop decreasing and start rising. This will be a sign of overfitting, and the indicator that we should probably stop training at this point (or at least make a snapshot of the model).
![overfitting](../images/Overfitting.png)
## How to prevent overfitting
If you can see that overfitting occurs, you can do one of the following:
* Increase the amount of training data
* Decrease the complexity of the model
* Use some [regularization technique](../../4-ComputerVision/08-TransferLearning/TrainingTricks.md), such as [Dropout](../../4-ComputerVision/08-TransferLearning/TrainingTricks.md#Dropout), which we will consider later.
## Overfitting and Bias-Variance Tradeoff
Overfitting is actually a case of a more generic problem in statistics called [Bias-Variance Tradeoff](https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff). If we consider the possible sources of error in our model, we can see two types of errors:
* **Bias errors** are caused by our algorithm not being able to capture the relationship between training data correctly. It can result from the fact that our model is not powerful enough (**underfitting**).
* **Variance errors**, which are caused by the model approximating noise in the input data instead of meaningful relationship (**overfitting**).
During training, bias error decreases (as our model learns to approximate the data), and variance error increases. It is important to stop training - either manually (when we detect overfitting) or automatically (by introducing regularization) - to prevent overfitting.
## Conclusion
In this lesson, you learned about the differences between the various APIs for the two most popular AI frameworks, TensorFlow and PyTorch. In addition, you learned about a very important topic, overfitting.
## 🚀 Challenge
In the accompanying notebooks, you will find 'tasks' at the bottom; work through the notebooks and complete the tasks.
## [Post-lecture quiz](https://red-field-0a6ddfd03.1.azurestaticapps.net/quiz/205)
## Review & Self Study
Do some research on the following topics:
- TensorFlow
- PyTorch
- Overfitting
Ask yourself the following questions:
- What is the difference between TensorFlow and PyTorch?
- What is the difference between overfitting and underfitting?
## [Assignment](lab/README.md)
In this lab, you are asked to solve two classification problems using single- and multi-layered fully-connected networks using PyTorch or TensorFlow.
* [Instructions](lab/README.md)
* [Notebook](lab/LabFrameworks.ipynb)

79
lessons/3-NeuralNetworks/05-Frameworks/README.md
Просмотреть файл

@ -41,4 +41,81 @@ Low-Level API | [TensorFlow+Keras Notebook](IntroKerasTF.ipynb) | [PyTorch](Intr
--------------|-------------------------------------|--------------------------------
High-level API| [Keras](IntroKeras.ipynb) | *PyTorch Lightning*
After mastering the frameworks, let's recap the notion of [overfitting](Overfitting.md).
After mastering the frameworks, let's recap the notion of overfitting.
# Overfitting
Overfitting is an extremely important concept in machine learning, and it is very important to get it right!
Consider the following problem of approximating 5 dots (represented by `x` on the graphs below):
![linear](../images/overfit1.jpg) | ![overfit](../images/overfit2.jpg)
-------------------------|--------------------------
**Linear model, 2 parameters** | **Non-linear model, 7 parameters**
Training error = 5.3 | Training error = 0
Validation error = 5.1 | Validation error = 20
* On the left, we see a good straight line approximation. Because the number of parameters is adequate, the model gets the idea behind point distribution right.
* On the right, the model is too powerful. Because we only have 5 points and the model has 7 parameters, it can adjust in such a way as to pass through all points, making training the error to be 0. However, this prevents the model from understanding the correct pattern behind data, thus the validation error is very high.
It is very important to strike a correct balance between the richness of the model (number of parameters) and the number of training samples.
## Why overfitting occurs
* Not enough training data
* Too powerful model
* Too much noise in input data
## How to detect overfitting
As you can see from the graph above, overfitting can be detected by a very low training error, and a high validation error. Normally during training we will see both training and validation errors starting to decrease, and then at some point validation error might stop decreasing and start rising. This will be a sign of overfitting, and the indicator that we should probably stop training at this point (or at least make a snapshot of the model).
![overfitting](../images/Overfitting.png)
## How to prevent overfitting
If you can see that overfitting occurs, you can do one of the following:
* Increase the amount of training data
* Decrease the complexity of the model
* Use some [regularization technique](../../4-ComputerVision/08-TransferLearning/TrainingTricks.md), such as [Dropout](../../4-ComputerVision/08-TransferLearning/TrainingTricks.md#Dropout), which we will consider later.
## Overfitting and Bias-Variance Tradeoff
Overfitting is actually a case of a more generic problem in statistics called [Bias-Variance Tradeoff](https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff). If we consider the possible sources of error in our model, we can see two types of errors:
* **Bias errors** are caused by our algorithm not being able to capture the relationship between training data correctly. It can result from the fact that our model is not powerful enough (**underfitting**).
* **Variance errors**, which are caused by the model approximating noise in the input data instead of meaningful relationship (**overfitting**).
During training, bias error decreases (as our model learns to approximate the data), and variance error increases. It is important to stop training - either manually (when we detect overfitting) or automatically (by introducing regularization) - to prevent overfitting.
## Conclusion
In this lesson, you learned about the differences between the various APIs for the two most popular AI frameworks, TensorFlow and PyTorch. In addition, you learned about a very important topic, overfitting.
## 🚀 Challenge
In the accompanying notebooks, you will find 'tasks' at the bottom; work through the notebooks and complete the tasks.
## [Post-lecture quiz](https://red-field-0a6ddfd03.1.azurestaticapps.net/quiz/205)
## Review & Self Study
Do some research on the following topics:
- TensorFlow
- PyTorch
- Overfitting
Ask yourself the following questions:
- What is the difference between TensorFlow and PyTorch?
- What is the difference between overfitting and underfitting?
## [Assignment](lab/README.md)
In this lab, you are asked to solve two classification problems using single- and multi-layered fully-connected networks using PyTorch or TensorFlow.
* [Instructions](lab/README.md)
* [Notebook](lab/LabFrameworks.ipynb)

175
lessons/5-NLP/20-LangModels/GPT-PyTorch.ipynb
Просмотреть файл

@ -1,16 +1,17 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Experimenting with GPT-2\n",
"## Experimenting with OpenAI GPT\n",
"\n",
"This notebook is part of [AI for Beginners Curriculum](http://aka.ms/ai-beginners).\n",
"\n",
"In this notebook, we will explore how we can play with OpenAI GPT-2 model using Hugging Face `transformers` library.\n",
"In this notebook, we will explore how we can play with OpenAI-GPT model using Hugging Face `transformers` library.\n",
"\n",
"Without further ado, let's instantiate text generating pipeline and start generating! You can select smaller GPT-2 model in order to increase download time and speed of inference, but that would affect the quality."
"Without further ado, let's instantiate text generating pipeline and start generating! "
]
},
{
@ -22,19 +23,30 @@
"name": "stderr",
"output_type": "stream",
"text": [
"Some weights of GPT2Model were not initialized from the model checkpoint at gpt2-large and are newly initialized: ['h.0.attn.masked_bias', 'h.1.attn.masked_bias', 'h.2.attn.masked_bias', 'h.3.attn.masked_bias', 'h.4.attn.masked_bias', 'h.5.attn.masked_bias', 'h.6.attn.masked_bias', 'h.7.attn.masked_bias', 'h.8.attn.masked_bias', 'h.9.attn.masked_bias', 'h.10.attn.masked_bias', 'h.11.attn.masked_bias', 'h.12.attn.masked_bias', 'h.13.attn.masked_bias', 'h.14.attn.masked_bias', 'h.15.attn.masked_bias', 'h.16.attn.masked_bias', 'h.17.attn.masked_bias', 'h.18.attn.masked_bias', 'h.19.attn.masked_bias', 'h.20.attn.masked_bias', 'h.21.attn.masked_bias', 'h.22.attn.masked_bias', 'h.23.attn.masked_bias', 'h.24.attn.masked_bias', 'h.25.attn.masked_bias', 'h.26.attn.masked_bias', 'h.27.attn.masked_bias', 'h.28.attn.masked_bias', 'h.29.attn.masked_bias', 'h.30.attn.masked_bias', 'h.31.attn.masked_bias', 'h.32.attn.masked_bias', 'h.33.attn.masked_bias', 'h.34.attn.masked_bias', 'h.35.attn.masked_bias']\n",
"c:\\Users\\bethanycheum\\Desktop\\AI-For-Beginners\\.venv\\lib\\site-packages\\tqdm\\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n",
"Downloading model.safetensors: 100%|██████████| 479M/479M [04:28<00:00, 1.78MB/s] \n",
"c:\\Users\\bethanycheum\\Desktop\\AI-For-Beginners\\.venv\\lib\\site-packages\\huggingface_hub\\file_download.py:133: UserWarning: `huggingface_hub` cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\\Users\\bethanycheum\\.cache\\huggingface\\hub. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the `HF_HUB_DISABLE_SYMLINKS_WARNING` environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.\n",
"To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development\n",
" warnings.warn(message)\n",
"Some weights of OpenAIGPTLMHeadModel were not initialized from the model checkpoint at openai-gpt and are newly initialized: ['position_ids']\n",
"You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n",
"Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
"Downloading (…)neration_config.json: 100%|██████████| 74.0/74.0 [00:00<00:00, 48.8kB/s]\n",
"Downloading (…)olve/main/vocab.json: 100%|██████████| 816k/816k [00:00<00:00, 1.76MB/s]\n",
"Downloading (…)olve/main/merges.txt: 100%|██████████| 458k/458k [00:00<00:00, 1.11MB/s]\n",
"Downloading (…)/main/tokenizer.json: 100%|██████████| 1.27M/1.27M [00:00<00:00, 2.12MB/s]\n",
"Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers\n",
"pip install xformers.\n"
]
},
{
"data": {
"text/plain": [
"[{'generated_text': 'Hello! I am a neural network, and I want to say that I am an expert in the area of learning and understanding neural networks. I also know a bit about math. You might have seen \"How to make deep neural networks\" or \"What is a deep learning neural network?\". I would like to discuss the first one.\\n\\nHow to make deep neural networks is a very complicated topic, and I don\\'t know how to go into it in detail, but I want to do some'},\n",
" {'generated_text': 'Hello! I am a neural network, and I want to say that you can find a network algorithm, called Naive Bayes, which produces good results, but which is computationally too expensive to be useful, so I will not cover all of its details here.\\n\\nFirst we are going to define how an NN looks like.\\n\\nHere is a naive Bayes classification problem with a few variables:\\n\\nFeature Input Value (x) 1 2 3 4 5 1 3 4'},\n",
" {'generated_text': 'Hello! I am a neural network, and I want to say that… \"I\\'m done with this crap!\"\\n\\nNow, if you were to be able to tell me all about this, then you did something wrong with the previous part. If you weren\\'t able to tell I did this before, sorry! If your brain was too wired, then you don\\'t know anything about neural networks. I mean, it turns out neural networks have a big problem. It\\'s called lossy neural'},\n",
" {'generated_text': 'Hello! I am a neural network, and I want to say that I learned this from Wikipedia but also from lots of other books. They also gave me some tips on how to do the analysis, which is really useful for beginners.\\n\\nFor the code, I just used Python and scikit-learn. But I will warn you with two caveats –\\n\\nfirst, this code will have to be imported before you can run them. You have to import scikit-learn after'},\n",
" {'generated_text': 'Hello! I am a neural network, and I want to say that you\\'re awesome.\"'}]"
"[{'generated_text': \"Hello! I am a neural network, and I want to say that i apologize for not coming to you yourself, for not helping you, and that i was too busy getting dressed and studying for a midterm. you know, the kind where the teachers are like that and they come in pairs with their boyfriends, but not with theirs. it's true, that i have had a girlfriend, and i'm only going on wednesdays and thursdays because i was too busy with college, but maybe\"},\n",
" {'generated_text': 'Hello! I am a neural network, and I want to say that we have been blessed with a wonderful gift ; no one of us has died at all. and our spirits are strong, very strong. in one very lucky moment of luck for you, all has been given direction and destiny, and for us there are no more mysteries. the earth has been chosen for you, and that earth is now ours, and you must be forever in our hearts. \" \\n the words, as one,'},\n",
" {'generated_text': 'Hello! I am a neural network, and I want to say that if you would just turn and face the general, you would have a nice day. \" \\n \" sure thing, \" said one of the soldiers, and started to run. the rest of the soldiers followed, shouting. the general turned to general zulu, raising his arm. the general said something in his native language, and the general immediately started to run. zulu started to move toward the wall, with the'},\n",
" {'generated_text': 'Hello! I am a neural network, and I want to say that i am not a doctor but an anthropologist to you, a specialist, a specialist in the field of astrobiological biology, and that i am very much involved in this investigation. i am not sure, i am not certain, but i can confirm your conclusions and therefore i will go to the top. i have a colleague who has just returned from this expedition and his findings confirm that you are a specialist. that is, he'},\n",
" {'generated_text': \"Hello! I am a neural network, and I want to say that everyone here is in agreement that no matter how many times i say to myself,'he was never a man of action on the battlefield,'or'he 'll never take a chance at killing any civilians,'or'he 'll never let his men go undefended against enemy forces of this caliber,'or'that's just what i need in a day like today. \\n you see, there are only three groups that\"}]"
]
},
"execution_count": 1,
@ -45,7 +57,7 @@
"source": [
"from transformers import pipeline\n",
"\n",
"model_name = 'gpt2-large' # try 'gpt2' for small model, 'gpt2-medium' for medium one\n",
"model_name = 'openai-gpt' \n",
"\n",
"generator = pipeline('text-generation', model=model_name)\n",
"\n",
@ -53,37 +65,31 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Prompt Engineering\n",
"\n",
"In some of the problems, you can use GPT-2 generation right away by designing correct prompts. Have a look at the examples below:"
"In some of the problems, you can use openai-gpt generation right away by designing correct prompts. Have a look at the examples below:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
]
},
{
"data": {
"text/plain": [
"[{'generated_text': 'Synonyms of a word cat:\\n\\n(a) cat;\\n\\n(b) black'},\n",
" {'generated_text': 'Synonyms of a word cat: cat-bitch; cat-queen; cat-c'},\n",
" {'generated_text': 'Synonyms of a word cat: feline, feline form, feline spirit, fas'},\n",
" {'generated_text': 'Synonyms of a word cat:\\n\\n(a) cat-like, like a cat;'},\n",
" {'generated_text': 'Synonyms of a word cat:\\n\\nThe more common English words you need to understand the meaning'}]"
"[{'generated_text': 'Synonyms of a word cat: the same cat i used to stare at, and you in'},\n",
" {'generated_text': 'Synonyms of a word cat: cat of the woods, cat of the hills, cat of'},\n",
" {'generated_text': 'Synonyms of a word cat: you! \\n \" it\\'s a girl. \" i said'},\n",
" {'generated_text': \"Synonyms of a word cat: big cat. but how come, we didn't hear it\"},\n",
" {'generated_text': 'Synonyms of a word cat: \" mea - o - c \" which makes them sound'}]"
]
},
"execution_count": 10,
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
@ -94,27 +100,20 @@
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
]
},
{
"data": {
"text/plain": [
"[{'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> Negative\\nHow do you think this hurts us all? -> Positive\\nYou seem'},\n",
" {'generated_text': \"I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> Negative\\nThis is such a horrible way to treat yourself -> Negative (You're\"},\n",
" {'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> I am disappointed\\nI have to admit -> You are a good friend but you'},\n",
" {'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> Positive\\nThis is awful for me to do this with you -> Negative\\nWhy'},\n",
" {'generated_text': \"I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> I'm still sad for you!\\nMe? \\xa0I find this a\"}]"
"[{'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> positive this is so horrible - > positive that your brother is gay - >'},\n",
" {'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> negative i will bring this on you -, < positive am i, i'},\n",
" {'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> negative i have self - esteem i must take it - : \\n - -'},\n",
" {'generated_text': 'I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> negative this is - : \\n if it were true that the devil would have'},\n",
" {'generated_text': \"I love when you say this -> Positive\\nI have myself -> Negative\\nThis is awful for you to say this -> positive i have you - > positive it's a bad thing, > positive\"}]"
]
},
"execution_count": 25,
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
@ -125,25 +124,18 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
]
},
{
"data": {
"text/plain": [
"[{'generated_text': 'Translate English to French: cat => chat, dog => chien, student => étudiant;\\n\\n\\nOther:\\n\\nTo learn'},\n",
" {'generated_text': 'Translate English to French: cat => chat, dog => chien, student => étude, french = le français = \"'},\n",
" {'generated_text': \"Translate English to French: cat => chat, dog => chien, student => été, mama => m'aime. We\"}]"
"[{'generated_text': 'Translate English to French: cat => chat, dog => chien, student => new and unusual. there were no more words to be'},\n",
" {'generated_text': 'Translate English to French: cat => chat, dog => chien, student => student \\n his eyes were huge in his lean face as'},\n",
" {'generated_text': \"Translate English to French: cat => chat, dog => chien, student => the teacher's words, their words, their words.\"}]"
]
},
"execution_count": 20,
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
@ -157,21 +149,14 @@
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
]
},
{
"data": {
"text/plain": [
"[{'generated_text': 'People who liked the movie The Matrix also liked \\xa0\"Dawn of the Dead\" (or in the case of John Goodman \"Vanish\"),\\xa0but their votes didn\\'t matter! The first'},\n",
" {'generated_text': 'People who liked the movie The Matrix also liked _____. It\\'s a true statement, as \"tasteful\" movies are often more popular for their plot, characters and themes than for their entertainment'},\n",
" {'generated_text': 'People who liked the movie The Matrix also liked \\xa0the book... \\xa0And so on, and so forth. Now at the other end of the spectrum...\\n...the real \"truths'},\n",
" {'generated_text': \"People who liked the movie The Matrix also liked 『The Grand Budapest Hotel』. 『The Grand Budapest Hotel』 is the movie people who like to watch their dream come true. I'll explain\"},\n",
" {'generated_text': 'People who liked the movie The Matrix also liked \\xa0a lot of the lines that were used, and I got to listen to the dialogue in the movie. \\xa0The characters were really enjoyable characters'}]"
"[{'generated_text': 'People who liked the movie The Matrix also liked it, and there was the movie of the first man after us. \\n i wanted to laugh at how stupid these stupid actors were. no, they were'},\n",
" {'generated_text': \"People who liked the movie The Matrix also liked the movie, and the film was the result. and that's when the man in the story was brought into reality, after a few decades. \\n a\"},\n",
" {'generated_text': 'People who liked the movie The Matrix also liked the movie the matrix, because there was a very old movie movie called the matrix, where there was a great super hero, and the super hero came out'},\n",
" {'generated_text': \"People who liked the movie The Matrix also liked the movie that didn't have a chance to pay cash, if they could afford it. most often they got a good deal and a lot of money,\"},\n",
" {'generated_text': \"People who liked the movie The Matrix also liked the movie, and i didn't seem to have the same problem. \\n i 'd met the other half of my family. i spent most of my time\"}]"
]
},
"execution_count": 5,
@ -194,27 +179,20 @@
},
{
"cell_type": "code",
"execution_count": 28,
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
]
},
{
"data": {
"text/plain": [
"[{'generated_text': \"It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw my girlfriend with a very cute looking guy. I noticed that his eyes that looked into mine were a bit scary and kind of like a predator's eyes. And that's when I knew that I had to know something about him.\\n\\nAs I sat, studying him, my mind was racing in a crazy way. We didn't exactly\"},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw my reflection in a small screen attached to a thin, white piece of glass. Suddenly, I thought about the possibility of making money making a computer, and decided to try it for myself! I started working on making an e-ink display. I was able to make it work by following some simple rules!\\n\\nThe first step is'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a pile of clothing and furniture. They were completely ruined. The place had been abandoned. It had been a home, not a work place.\\n\\nWhen the police arrived, they found the crime scene and I walked with two officers to the police station. Before that I had not gone to the police station and I have never been to'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a woman who could not be a mother of any kind. I saw the same woman with her eyes closed and her mouth wide open. She was sitting naked on a couch and was sitting topless. I saw at least three different women with similar posture. The only thing I kept telling myself was \"if this is how a woman eats,'},\n",
" {'generated_text': \"It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a pale person with long white hair sitting quietly. He was a bit bigger and his eyes looked different. He was watching me.\\n\\nI didn't have to say anything. The man kept studying me, and after a while he turned his eyes away from me to the floor. In a moment he was gone, and he was still\"}]"
"[{'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw my friend, a young man, sprawled across the bed in his bed. \\n \" hi, i\\'m mike eptirard. \" \\n there was silence on the other side of the door. i listened for any trace of life but there was nothing. my heart began to pound, i was starting to sweat, i took out my wallet'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw my mother on the bed, hugging her legs to her chest and sobbing. i saw my dad and mother from the corner of my eye. \\n elfin face was covered in tears as i entered the room. my dad and mother also wept ; just as they did every other time i came to work. but this time, they had different faces'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw the room had changed because it was dark. it still smelled like a hospital. a new light shined through from a vent in the ceiling. i found myself in a bathroom and a small room with a sink and a wall of glass. the bathroom billion years ago. not so different from all of the rest of the apartment. \\n now...'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a large woman with dark hair and pale skin. she was asleep, but i noticed a faint movement of her face. i could sense she was awake. i got up and walked over to her. \\n \" hello miss. i am inspector michael o\\'dell ; we are investigating the case against you. i wanted to ask if you were the'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw i had an empty table and three empty chairs. that was all i needed. i had left a note on a table in the center of the room and had a pen in hand. \" \\n \" i think what you were doing was something he was doing to her. \" \\n \" yeah, \" i nodded with a grin. \" i'}]"
]
},
"execution_count": 28,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
@ -233,27 +211,20 @@
},
{
"cell_type": "code",
"execution_count": 29,
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
]
},
{
"data": {
"text/plain": [
"[{'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a group of people sitting around a table. One of them was a middle-aged man. He was wearing a black suit and a white shirt. His eyes were closed and he was staring at the floor with his hands on his knees.\\n\\n\"What are you doing here?\" I asked him. \"What do you want?\"\\n'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting at a table in front of a computer. He was wearing a black suit, a white shirt, and a red tie.\\n\\n\"Hello,\" he said to me. \"How are you?\" he asked me in a voice that sounded as if he was speaking to a child. There was a smile on his face.'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting on the edge of the bed. He was staring at me.\\n\\n\"What are you doing here?\" he asked in a low voice. \"I don\\'t know why you\\'re here, and I\\'m not interested in what you have to say,\" he said, as if he didn\\'t understand what I was saying.'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting in a chair in front of a desk. He was in his mid-thirties, and he was wearing a dark suit and a white shirt.\\n\\n\"What are you doing here?\" I asked him. The man didn\\'t answer. Instead, he walked over to the desk and sat down on it. \"'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw that the door was open, and a man was sitting on a chair. He was wearing a white shirt and black pants.\\n\\n\"What are you doing here?\" I asked. The man looked at me for a moment, then said, \"I have something to tell you.\" He handed me a piece of paper. It was a'}]"
"[{'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting in a chair with his head in his hands. he didn\\'t look up as i approached. \\n \" excuse me, sir, \" i said. \" can i help you? \" \\n the man looked up at me. his eyes were red - rimmed and his face was pale, as if he hadn\\'t slept in days'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting at a desk in the middle of the room. he had his back to me, so i couldn\\'t see what he was doing. \" \\n \" what did he look like? \" i asked as i sat down on the bed next to her. \\n she took a deep breath and looked at me with tears in her eyes'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a woman sitting on the bed, reading a book. she looked up at me and smiled. \\n \" hi, \" she said. \" can i help you? \" \\n i sat down next to her and looked around the room. the walls were white, and there was a large window in the middle of the wall that looked out on'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a man sitting at a table in the middle of the room. he looked up as i walked in, and when he saw me, he got up and walked over to me. \\n \" can i help you? \" he asked as he put his hand on the small of my back and led me to a chair at the other end of'},\n",
" {'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw a woman sitting on the edge of her bed, reading a book. she looked up at me and smiled. \\n \" hello, \" she said. \" can i help you? \" \\n i didn\\'t know what to say, so i just sat down in the chair next to the bed and looked at her. her hair was dark brown'}]"
]
},
"execution_count": 29,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
@ -272,23 +243,16 @@
},
{
"cell_type": "code",
"execution_count": 30,
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
]
},
{
"data": {
"text/plain": [
"[{'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw one of my colleagues lying on his stomach. He was unconscious. I could smell a strong odor of alcohol on his breath. I was on my way out. I ran to his room and found him unconscious on the bed. I saw a bottle of wine on the bed. He was in a state of intoxication. He was unconscious. I'}]"
"[{'generated_text': 'It was early evening when I can back from work. I usually work late, but this time it was an exception. When I entered a room, I saw her. she was on the bed, but she looked very different. \\n \" honey, what\\'s the matter? \" i asked. \\n she sat up. \" i can\\'t believe it\\'s real. i\\'ve been dreaming about you for the last two days. \" \\n \" i can\\'t believe it either. i guess that\\'s how'}]"
]
},
"execution_count": 30,
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
@ -310,12 +274,13 @@
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Fine-Tuning GPT-2\n",
"## Fine-Tuning your models\n",
"\n",
"You can also fine-tune GPT-2 text generation on your own dataset. This will allow you to adjust the style of text, while keeping the major part of language model. The example of fine-tuning GPT-2 to generate song lyrics can be found [in this blog post](https://towardsdatascience.com/how-to-fine-tune-gpt-2-for-text-generation-ae2ea53bc272)."
"You can also [fine-tune your model](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/fine-tuning?pivots=programming-language-studio?WT.mc_id=academic-77998-bethanycheum) on your own dataset. This will allow you to adjust the style of text, while keeping the major part of language model. "
]
}
],
@ -338,7 +303,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.12"
"version": "3.10.11"
},
"orig_nbformat": 4
},

29
lessons/5-NLP/20-LangModels/README.md
Просмотреть файл

@ -10,7 +10,7 @@ The idea of a neural network being able to do general tasks without downstream t
> Understanding and being able to produce text also entails knowing something about the world around us. People also learn by reading to the large extent, and GPT network is similar in this respect.
Text generation networks work by predicting probability of the next word $P(w_N)$. However, unconditional probability of the next word equals to the frequency of the this word in the text corpus. GPT is able to give us **conditional probability** of the next word, given the previous ones $P(w_N | w_{n-1}, ..., w_0)$.
Text generation networks wor;k by predicting probability of the next word $$P(w_N)$$ However, unconditional probability of the next word equals to the frequency of the this word in the text corpus. GPT is able to give us **conditional probability** of the next word, given the previous ones: $$P(w_N | w_{n-1}, ..., w_0)$$
> You can read more about probabilities in our [Data Science for Beginers Curriculum](https://github.com/microsoft/Data-Science-For-Beginners/tree/main/1-Introduction/04-stats-and-probability)
@ -23,34 +23,31 @@ $$
## GPT is a Family
GPT is not a single model, but rather a collection of models developed and trained by [OpenAI](http://openai.org). The latest model openly available is [GPT-2](https://huggingface.co/docs/transformers/model_doc/gpt2#openai-gpt2), which has up to 1.5 billion parameters (there are several variations of the model, so you can select one for your tasks that is a good compromise between size/performance). Latest GPT-3 model has up to 175 billion parameters, and is available [as a cognitive service from Microsoft Azure](https://azure.microsoft.com/en-us/services/cognitive-services/openai-service/#overview?WT.mc_id=academic-77998-cacaste), and as [OpenAI API](https://openai.com/api/).
GPT is not a single model, but rather a collection of models developed and trained by [OpenAI](https://openai.com).
## Prompt-based Inference
Under the GPT models, we have:
Because GPT has been trained on a vast volumes of data, it has some commonsense knowledge embedded directly inside the model. This allows us to force GPT to solve certain typical problems by just providing the right prompt. This presents a whole new approach for using pre-trained models, called [Prompt Engineering](https://en.wikipedia.org/wiki/Prompt_engineering). It is particularly useful with GPT-3, which has significantly more parameters, and consequently more embedded knowledge.
| [GPT-2](https://huggingface.co/docs/transformers/model_doc/gpt2#openai-gpt2) | [GPT 3](https://openai.com/research/language-models-are-few-shot-learners) | [GPT-4](https://openai.com/gpt-4) |
| -- | -- | -- |
|Language model with upto 1.5 billion parameters. | Language model with up to 175 billion parameters | 100T parameters and accepts both image and text inputs and outputs text. |
Here are a few example of using Prompt Engineering (answers from the model are *in italics*):
**Recommendation Systems**:<br/>
People, who liked the movie "The Matrix" also liked *Star Wars, Jupyter Ascending, Ex Machina*
The GPT-3 and GPT-4 models are available [as a cognitive service from Microsoft Azure](https://azure.microsoft.com/en-us/services/cognitive-services/openai-service/#overview?WT.mc_id=academic-77998-cacaste), and as [OpenAI API](https://openai.com/api/).
**Translation**:<br/>
Translate from English to French:<br/>
cat => chat, dog => chien, student => *étudiant*
## Prompt Engineering
**Looking for words:**<br/>
Synonyms of a word cat: *feline, feline form, feline spirit*
Because GPT has been trained on a vast volumes of data to understand language and code, they provide outputs in response to inputs (prompts). Prompts are GPT inputs or queries whereby one provides instructions to models on tasks they next completed. To elicit a desired outcome, you need the most effective prompt which involves selecting the right words, formats, phrases or even symbols. This approach is [Prompt Engineering](https://learn.microsoft.com/en-us/shows/ai-show/the-basics-of-prompt-engineering-with-azure-openai-service?WT.mc_id=academic-77998-bethanycheum)
[This article](https://www.gwern.net/GPT-3#prompts-as-programming) talks more about prompt engineering.
[This documentation](https://learn.microsoft.com/en-us/semantic-kernel/prompt-engineering/?WT.mc_id=academic-77998-bethanycheum) provides you with more information on prompt engineering.
## ✍️ Example Notebook: [Playing with GPT-2](GPT-PyTorch.ipynb)
## ✍️ Example Notebook: [Playing with OpenAI-GPT](GPT-PyTorch.ipynb)
Continue your learning in the following notebooks:
* [Generating text with GPT-2 and Hugging Face Transformers](GPT-PyTorch.ipynb)
* [Generating text with OpenAI-GPT and Hugging Face Transformers](GPT-PyTorch.ipynb)
## Conclusion
New general pre-trained language models do not only model language structure, but also contain vast amount of commonsense knowledge. Thus, they can be effectively used to solve some NLP tasks in zero-shop or few-shot settings.
New general pre-trained language models do not only model language structure, but also contain vast amount of natural language. Thus, they can be effectively used to solve some NLP tasks in zero-shop or few-shot settings.
## [Post-lecture quiz](https://red-field-0a6ddfd03.1.azurestaticapps.net/quiz/220)

18
lessons/X-Extras/X1-MultiModal/README.md
Просмотреть файл

@ -1,6 +1,6 @@
# Multi-Modal Networks
After the success of transformer models for solving NLP tasks, there were many attempts to apply the same or similar architectures to computer vision tasks. Also, there is a growing interest in building models that would *combine* vision and natural language capabilities. One of such attempts was done by OpenAI, and it is called CLIP.
After the success of transformer models for solving NLP tasks, the same or similar architectures have been applied to computer vision tasks. There is a growing interest in building models that would *combine* vision and natural language capabilities. One of such attempts was done by OpenAI, and it is called CLIP and DALL.E.
## Contrastive Image Pre-Training (CLIP)
@ -32,7 +32,7 @@ We can also do the opposite. If we have a collection of images, we can pass this
Open the [Clip.ipynb](Clip.ipynb) notebook to see CLIP in action.
## Image Generation with VQGAN + CLIP
## Image Generation with VQGAN+ CLIP
CLIP can also be used for **image generation** from a text prompt. In order to do this, we need a **generator model** that will be able to generate images based on some vector input. One of such models is called [VQGAN](https://compvis.github.io/taming-transformers/) (Vector-Quantized GAN).
@ -56,6 +56,20 @@ Picture generated from prompt *a closeup watercolor portrait of young male teach
> Pictures from **Artificial Teachers** collection by [Dmitry Soshnikov](http://soshnikov.com)
## DALL-E
### [DALL-E 1](https://openai.com/research/dall-e)
DALL-E is a version of GPT-3 trained to generate images from prompts. It has been trained with 12-billion parameters.
Unlike CLIP, DALL-E receives both text and image as a single stream of tokens for both images and text. Therefore, from multiple prompts, you can generate images based on the text.
### [DALL-E 2](https://openai.com/dall-e-2)
The main difference between DALL.E 1 and 2, is that it generates more realistic images and art.
Examples of image genrations with DALL-E:
![Picture produced by Pixray](images/DALL·E%202023-06-20%2015.56.56%20-%20a%20closeup%20watercolor%20portrait%20of%20young%20male%20teacher%20of%20literature%20with%20a%20book.png) | ![Picture produced by pixray](images/DALL·E%202023-06-20%2015.57.43%20-%20a%20closeup%20oil%20portrait%20of%20young%20female%20teacher%20of%20computer%20science%20with%20a%20computer.png) | ![Picture produced by Pixray](images/DALL·E%202023-06-20%2015.58.42%20-%20%20a%20closeup%20oil%20portrait%20of%20old%20male%20teacher%20of%20mathematics%20in%20front%20of%20blackboard.png)
----|----|----
Picture generated from prompt *a closeup watercolor portrait of young male teacher of literature with a book* | Picture generated from prompt *a closeup oil portrait of young female teacher of computer science with a computer* | Picture generated from prompt *a closeup oil portrait of old male teacher of mathematics in front of blackboard*
## References
* VQGAN Paper: [Taming Transformers for High-Resolution Image Synthesis](https://compvis.github.io/taming-transformers/paper/paper.pdf)

Двоичные данные
lessons/X-Extras/X1-MultiModal/images/DALL·E 2023-06-20 15.56.56 - a closeup watercolor portrait of young male teacher of literature with a book.png Normal file
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом"

После

Ширина:  |  Высота:  |  Размер: 1.5 MiB

Двоичные данные
lessons/X-Extras/X1-MultiModal/images/DALL·E 2023-06-20 15.57.43 - a closeup oil portrait of young female teacher of computer science with a computer.png Normal file
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом"

После

Ширина:  |  Высота:  |  Размер: 1.5 MiB

Двоичные данные
lessons/X-Extras/X1-MultiModal/images/DALL·E 2023-06-20 15.58.42 - a closeup oil portrait of old male teacher of mathematics in front of blackboard.png Normal file
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом"

После

Ширина:  |  Высота:  |  Размер: 1.8 MiB