Product Detection from Images - A Cortana Intelligence Solution How-to Guide

In today’s highly competitive and connected environment, retail stores are increasingly finding it imperative to make use of artificial intelligence in their daily operations. Image detection - a technique which can locate and classify objects in images - is a technique that has the potential to bring huge rewards for such stores. For example, retailers can use this technique to determine which product a customer has picked up from the shelf. This information in turn helps stores manage product inventory.

The objective of this Guide is to demonstrate data pipelines for retailers to detect products from images. This tutorial shows how to set up the prediction service as well as retrain models using newly available data.

Solution Demo

The snapshot below shows the products detected from an image using the demo website. You'll be able to deploy the same website using your own Azure subscription by the end of the Guide.

Solution Dashboard

The snapshot below shows an example PowerBI dashboard showing the performance of the model. It compares precision vs recall for different threshold levels. The performance is near perfect for the sample data, as indicated by the fact that the area under the curve is close to 1. Actual performance for you own data will differ. The provided dashboard template supports comparing model performance for multiple models.

Below is another dashboard snapshot that shows average precision by class. Users can select the classes that are important and this helps focus on select classes as the number of classes increases. Performance for multiple models can be compared as well.

Solution Architecture

What's Under the Hood

The end-to-end solution is implemented in the cloud, using Microsoft Azure. The solution is composed of several Azure components, including data ingest, data storage, data movement, advanced analytics and visualization. The advanced analytics are implemented using CNTK, a unified deep-learning toolkit by Microsoft. With data ingest, the solution can make predictions based on data that being transferred to Azure from an on-premises environment.

Getting Started

This solution package contains materials to help both technical and business audiences understand our Product Detection from Images Solution built on the Cortana Intelligence Suite.

Business Audiences (Coming Soon)

For information on how to tailor Cortana Intelligence to your needs connect with one of our partners.

Technical Audiences

See the technical_deployment folder for a full set of instructions on how to put together and deploy a Product Detection from Images Solution using the Cortana Intelligence Suite. For technical problems or questions about deploying this solution, please post in the issues tab of the repository.

Related Resources

ObjectDetectionUsingCntk offers more details of the model used in this solution.

Microsoft Open Source Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.