Best Practices, code samples, and documentation for Computer Vision.
Перейти к файлу
bethz 684febe9f6 Release AML-nightly-gpu.yml for integration tests (#271)
* initial pipeline release

* add cron and pr for now until cron works. Smoke test has no tests so remove posting of smoke test logs. Integration tests are new.
2019-07-24 11:14:39 -04:00
.ci Release AML-nightly-gpu.yml for integration tests (#271) 2019-07-24 11:14:39 -04:00
.github
classification Rijai/amltesting (#268) 2019-07-23 13:47:20 -04:00
media added figure source 2019-06-10 14:58:14 -04:00
similarity removed python version of notebook 2019-07-12 10:19:49 -04:00
tests Rijai/amltesting (#268) 2019-07-23 13:47:20 -04:00
tools/repo_metrics Updated name of cosmos database 2019-06-13 05:04:54 -04:00
utils_cv Merging staging to this branch 2019-07-14 00:05:26 -04:00
.flake8 [#167] remove wildcard imports (#218) 2019-06-06 19:13:40 -04:00
.gitignore [#167] remove wildcard imports (#218) 2019-06-06 19:13:40 -04:00
.pre-commit-config.yaml
CONTRIBUTING.md beautified text 2019-07-22 16:59:40 -04:00
LICENSE
README.md Rijai/amltesting (#268) 2019-07-23 13:47:20 -04:00
environment.yml getting started at root level (#258) 2019-07-08 12:27:14 -04:00
pyproject.toml

README.md

Computer Vision

This repository provides implementations and best practice guidelines for building Computer Vision systems. All examples are given as Jupyter notebooks, and use PyTorch as Deep Learning library.

Build Status

VM Testing

Build Type Branch Status Branch Status
Linux GPU master Build Status staging Build Status
Linux CPU master Build Status staging Build Status
Windows GPU master Build Status staging Build Status
Windows CPU master Build Status staging Build Status
AzureML Notebooks master N/A staging Build Status

AzureML Testing

Build Type Branch Status Branch Status
Linux CPU master Build Status staging Build Status

Overview

The goal of this repository is to help speed up development of Computer Vision applications. Rather than implementing custom approaches, the focus is on providing examples and links to existing state-of-the-art libraries. In addition, having worked in this space for many years, we aim to answer common questions, point out often observed pitfalls, and show how to use the cloud for deployment and training.

Currently the main investment/priority is around image classification and to a lesser extend image segmentation. We are also actively working on providing a basic (but often sufficiently accurate) example for image similarity. Object detection is scheduled to start once image classification is completed. See the projects and milestones pages in this repository for more details.

Getting started

To get started on your local machine:

  1. Install Anaconda with Python >= 3.6. Miniconda is a quick way to get started.
  2. Clone the repository
    git clone https://github.com/Microsoft/ComputerVision
    
  3. Install the conda environment, you'll find the environment.yml file in the root directory. To build the conda environment:
    conda env create -f environment.yml
    
  4. Activate the conda environment and register it with Jupyter:
    conda activate cv
    python -m ipykernel install --user --name cv --display-name "Python (cv)"
    
    If you would like to use JupyterLab, install jupyter-webrtc widget:
    jupyter labextension install jupyter-webrtc
    
  5. Start the Jupyter notebook server
    jupyter notebook
    
  6. At this point, you should be able to run the notebooks in this repo. Explore our notebooks on the following computer vision domains. Make sure to change the kernel to "Python (cv)".

Services

Note that for certain Computer Vision problems, you may not need to build your own models. Instead, ready-made or easily customizable solutions exist which do not require any custom coding or machine learning expertise. We strongly recommend evaluating if these can sufficiently solve your problem. If these solutions are not applicable, or the accuracy of these solutions is not sufficient, then resorting to more complex and time-consuming custom approaches may be necessary.

The following Microsoft services offer simple solutions to address common Computer Vision tasks:

  • Cognitive Services provides pre-trained REST APIs which can be called for image classification, face recognition, OCR, video analytics, and much more. These APIs are easy to use and work out of the box (no training required), however customization is limited. See the various demos available for each domain to get a feel for the functionality (e.g., computer vision, speech to text ).

  • Custom Vision Service is a SaaS service to train and deploy a model as a REST API given a user-provided training set. All steps from image upload, annotation, to model deployment can be performed using either the UI or a Python SDK. Training image classification or object detection models are supported using only minimal machine learning knowledge. The Custom Vision Service offers more flexibility than using the pre-trained Cognitive Services APIs, but requires the user to bring and annotate their own data.

  • Azure Machine Learning service (AzureML) is a more general machine learning service that helps users accelerate training and deploying machine learning models. While not specific for Computer Vision workloads, the AzureML Python SDK can be used for scalable and reliable training and deploying machine learning solutions to the cloud. While AzureML offers significantly more flexibility than other options, it also requires significantly more machine learning and programming knowledge.

  • Azure AI Reference architectures While not Computer Vision specific, these reference architectures cover general Machine Learning aspects such as model deployment or batch scoring.

Computer Vision Domains

Most applications in Computer Vision (CV) fall into one of these 4 categories:

  • Image classification: Given an input image, predict what object is present in the image. This is typically the easiest CV problem to solve, however classification requires objects to be reasonably large in the image.

       Image classification visualization

  • Object Detection: Given an input image, identify and locate which objects are present (using rectangular coordinates). Object detection can find small objects in an image. Compared to image classification, both model training and manually annotating images is more time-consuming in object detection, since both the label and location are required.

       Object detect visualization

  • Image Similarity Given an input image, find all similar objects in images from a reference dataset. Here, rather than predicting a label and/or rectangle, the task is to sort through a reference dataset to find objects similar to that found in the query image.

       Image similarity visualization

  • Image Segmentation Given an input image, assign a label to every pixel (e.g., background, bottle, hand, sky, etc.). In practice, this problem is less common in industry, in large part due to time required to label the ground truth segmentation required in order to train a solution.

       Image segmentation visualization

Contributing

This project welcomes contributions and suggestions. Please see our contribution guidelines.

Data/Telemetry

The Azure Machine Learning image classification notebooks (20_azure_workspace_setup, 21_deployment_on_azure_container_instances, 22_deployment_on_azure_kubernetes_service, and 23_aci_aks_web_service_testing) collect browser usage data and send it to Microsoft to help improve our products and services. Read Microsoft's privacy statement to learn more.

To opt out of tracking, please go to the raw .ipynb files and remove the following line of code (the URL will be slightly different depending on the file):

    "![Impressions](https://PixelServer20190423114238.azurewebsites.net/api/impressions/ComputerVision/classification/notebooks/21_deployment_on_azure_container_instances.png)"