nlp-recipes/scenarios

NLP Scenarios

This folder contains examples and best practices, written in Jupyter notebooks, for building Natural Language Processing systems for different scenarios.

Summary

The following summarizes each scenario of the best practice notebooks. Each scenario is demonstrated in one or more Jupyter notebook examples that make use of the core code base of models and utilities.

Scenario	Applications	Languages	Models
Text Classification	Topic Classification	en, zh, ar	BERT
Named Entity Recognition	Wikipedia NER	en, zh	BERT
Sentence Similarity	STS Benchmark	en	Representation: TF-IDF, Word Embeddings, Doc Embeddings Metrics: Cosine Similarity, Word Mover's Distance
Embeddings	Custom Embeddings Training	en	Word2Vec fastText GloVe

Azure-enhanced notebooks

Azure products and services are used in certain notebooks to enhance the efficiency of developing Natural Language systems at scale.

To successfully run these notebooks, the users need an Azure subscription or can use Azure for free.

The Azure products featured in the notebooks include:

• Azure Machine Learning service - Azure Machine Learning service is a cloud service used to train, deploy, automate, and manage machine learning models, all at the broad scale that the cloud provides. It is used across various notebooks for the AI model development related tasks like:

  • Using Datastores
  • Tracking and monitoring metrics to enhance the model creation process
  • Distributed Training
  • Hyperparameter tuning
  • Scaling up and out on Azure Machine Learning Compute
  • Deploying a web service to both Azure Container Instance and Azure Kubernetes Service

• Azure Kubernetes Service - You can use Azure Machine Learning service to host your classification model in a web service deployment on Azure Kubernetes Service (AKS). AKS is good for high-scale production deployments and provides autoscaling, and fast response times.

• Azure Container Instance- You can use Azure Machine Learning service to host your classification model in a web service deployment on Azure Container Instance (ACI). ACI is good for low scale, CPU-based workloads.

There may be other Azure service or products used in the notebooks. Introduction and/or reference of those will be provided in the notebooks.