Operationalize a video anomaly detection model with Azure ML
Перейти к файлу
microsoft-github-policy-service[bot] ddac76731e
Auto merge mandatory file pr
This pr is auto merged as it contains a mandatory file and is opened for more than 10 days.
2023-06-12 18:22:10 +00:00
.azureml script_params 2019-05-04 06:31:15 +00:00
config fixed version mismatch of cudatoolkit and tensorflow-gpu 2020-06-07 00:57:08 +00:00
data working towards having built in anomaly detection 2020-03-12 03:45:51 +00:00
deployment fixed version mismatch of cudatoolkit and tensorflow-gpu 2020-06-07 00:57:08 +00:00
docs data_prep was renamed 2020-06-09 18:39:59 +00:00
models simplified data handling between batch_score and train_clf 2020-06-06 21:10:20 +00:00
tools Update input folder name to match instructions 2019-07-25 16:02:42 +00:00
.gitignore works with transfer learning 2020-04-07 01:17:04 +00:00
LICENSE Initial commit 2019-05-02 09:45:34 -07:00
README.md renamed some files and their references 2020-06-07 01:20:15 +00:00
SECURITY.md Microsoft mandatory file 2023-06-02 17:45:48 +00:00
azure-pipelines.yml Merge branch 'master' into dev 2020-05-29 18:18:29 +00:00
batch_scoring.py listen to correct path on blob 2020-06-08 01:20:49 +00:00
create_workspace.py added create workspace 2020-06-06 19:36:14 +00:00
data_preparation.py updates to documentation 2020-04-08 04:10:29 +00:00
pipelines_master.py use gpu base image 2020-06-09 23:42:59 +00:00
pipelines_slave.py use gpu base image 2020-06-09 23:42:59 +00:00
register_clf.py only changed n frames 2020-05-29 03:39:28 +00:00
register_prednet.py multi model deployment 2020-05-28 23:45:32 +00:00
train.py multi model deployment 2020-05-28 23:45:32 +00:00
train_clf.py simplified data handling between batch_score and train_clf 2020-06-06 21:10:20 +00:00
utils.py listen to correct path on blob 2020-06-08 01:20:49 +00:00
video_anom.PNG Add files via upload 2020-06-03 13:22:05 -07:00
video_anom_usup.png Add files via upload 2020-06-03 13:19:16 -07:00

README.md

Video Anomaly Detection - with Azure ML and MLOps

Build Status

The automation of detecting anomalous event sequences in videos is a challenging problem, but also has broad applications across industry verticals.

The approach followed in this repository involves self-supervised training deep neural networks to develop an in-depth understanding of the physical and causal rules in the observed scenes. The model effectively learns to predict future frames in the video in a self-supervised fashion.

The trained model can then be used to detect anomalies in videos. As the model tries to predict each next frame, one can calculate the error in the model's prediction. If the error is large, it is likely that an anomalous even occurred.

The approach can be used both in a supervised and unsupervised fashion, thus enabling the detection of pre-defined anomalies, but also of anomalous events that have never occurred in the past.

Post on LinkedIn (includes video)

Example scenarios

The following snapshots are also links to video demonstrations.

Unsupervised Approach:

Video of unsupervised anomaly detection

Supervised Approach:

Video of unsupervised anomaly detection

Learning Goals

You will learn:

  1. How to adapt an existing neural network architecture to your use-case.
  2. How to prepare video data for deep learning.
  3. How to perform hyperparameter tuning with HyperDrive to improve the performance of your model.
  4. How to deploy a deep neural network as a webservice for video processing.
  5. How to post-process the output of a Keras model for secondary tasks (here, anomaly detection).
  6. How to define a build pipeline for DevOps.

Pre-requisites

Skills

  1. Some familiarity with concepts and frameworks for neural networks:
  2. Knowledge of basic data science and machine learning concepts. Here and here you'll find short introductory material.
  3. Moderate skills in coding with Python and machine learning using Python. A good place to start is here.

Software Dependencies

  • Various python modules. We recommend working with a conda environement (see config/environment.yml and Documentation).
    • We recommend you begin by installing Miniconda.
    • Also consider running the following command to update you conda base image: conda update -n base -c defaults conda
    • Then, you could run: conda env create -f config/environment.yml
  • If you are using a DSVM:

Hardware Dependencies

A computer with a GPU, for example a Linux Azure VM. Compare VM sizes and prices). If you don't know what to choose, we recommend the Standard NC6, the most affordable VM with a GPU.

You could create a VM in the Azure Portal.

Dataset

UCSD Anomaly Detection Dataset

Agenda

The recommended first step is to clone this repository.

Getting Started

  1. Data Preparation - Download and prepare data for training/testing.
  2. Azure ML Configuration - Configure your Azure ML workspace.
  3. AML Pipelines - Automate data preparation, training, and re-training.
  4. Deployment - How to deploy your anomaly detection as a webservice on AKS.

Deep-dive

  1. Model Development - Understand model architecture and training.
  2. Fine Tuning - Perform transfer learning with pretrained model onnew data.
  3. Hyperparameter tuning - Tune hyperparameters with HyperDrive.
  4. Anomaly Detection - Use Model errors for detecting anomalies.

Contribute

We invite contributions to this repository. The preferred method would be to fork this repository and to create a pull request.

References / Resources

Troubleshooting

Missing cuda libraries

We noticed that tensorflow might be looking for the wrong the of cudatoolkit.

You might see the following error:

Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory

We found that it helps to look at which version of these libraries you have installed, and pin the version of cudatoolkit to the version that tensorflow is looking for. E.g. above, TF is looking for 10.0, so we pinned the version of the cudatoolkit to 10.0 in config/environment.yml.