Auto Brew ML

Traditional machine learning model development is resource-intensive, requiring significant domain/statistical knowledge and time to produce and compare dozens of models. With automated machine learning, the time it takes to get production-ready ML models with great ease and efficiency highly accelerates. However, the Automated Machine Learning does not yet provide much in terms of data preparation and feature engineering. The AutoBrewML framework tries to solve this problem at scale as well as simplifies the overall process for the user. It leverages the Azure Automated ML coupled with components like Data Profiler, Data Sampler, Data Cleanser, Anomaly Detector which ensures quality data as a critical pre-step for building the ML model. This is powered with Telemetry, DevOps and Power BI integration, thus providing the users with a one-stop shop solution to productionize any ML model. The framework aims at ‘Democratizing’ AI all the while maintaining the vision of ‘Responsible’ AI.

Our Inspiration Story

As we think about the future of technology, it resides in the notion of intelligence. At Microsoft, we have an approach that’s both ambitious and broad, an approach that seeks to Democratize Machine Learning & Artificial Intelligence, to take it from the high walls of ivory towers and make it accessible for all.

It is very rightly said that if you want something you never had, you have to do something you have never done! When we started on our ML journey we hoped to have a hand-holding which could accelerate our sail across the extensive nature of ML. We were stuck in the vicious loop of finding right algorithms and tools to achieve our target rather than focusing on the data at hand and fine tuning it with our Business Domain knowledge. Also traditional machine learning model development is resource-intensive, requiring significant domain/statistical knowledge and time to produce and compare dozens of models. A team comprises of folks from various backgrounds and ML knowledge base, but driven by our mission at Microsoft 'To empower every person and every organization on the planet to achieve more' we want to open ways for each and everyone to have access over the wonders of ML & AI.

So by amalgamating all our pain points and covering the aspects of an end-to-end ML pipeline we came up with a Framework to get production-ready ML models with great ease and efficiency.

How we pursue this bold ambition to democratize AI for all via this Framework:

1. Implement machine learning solutions without extensive programming knowledge
2. Find the right dataset for modelling
3. Save time and resources
4. Leverage Data Science best practices & Responsible AI
5. Provide agile problem-solving
6. Provide visualizations to interpret data
7. Capture telemetry throughout the process

Getting Started

Prerequisites

1. Azure Databricks
2. Auto Brew ML Notebooks (Master, Trigger notebooks)
3. Azure ML Services workspace
4. Python cluster in Databricks with configurations as mentioned in Installations link above (PyPi library azureml-sdk[automl],azureml-opendatasets, azureml-widgets in cluster)

How to use it

1. AMLMasterNotebook: Contains all the base functions used Data Acquisition, EDA, Sampling, Cleansing, Anomaly Detection, Azure AutoML Trigger, AutoML Trigger bypassing authentication to Azure ML(used for pipelining the notebook).
2. AMLMasterNotebook_Trigger: Function calls in order to perform a pipeline of tasks.
   go to- Supervised Learning Notebooks
3. For Text Analytics go to- Text Analytics Notebooks
4. For sample notebook run go to- Sample Runs

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Note: To know in detail of the workings of Auto Brew ML Framework, please visit Auto Brew ML WiKi. Copyright (c) Microsoft Corporation
Licensed under the MIT License