The Edge Machine Learning library

This repository provides code for machine learning algorithms for edge devices developed at Microsoft Research India.

Machine learning models for edge devices need to have a small footprint in terms of storage, prediction latency, and energy. One instance of where such models are desirable is resource-scarce devices and sensors in the Internet of Things (IoT) setting. Making real-time predictions locally on IoT devices without connecting to the cloud requires models that fit in a few kilobytes.

Contents

Algorithms that shine in this setting in terms of both model size and compute, namely:

• Bonsai: Strong and shallow non-linear tree based classifier.
• ProtoNN: Prototype based k-nearest neighbors (kNN) classifier.
• EMI-RNN: Training routine to recover the critical signature from time series data for faster and accurate RNN predictions.
• FastRNN & FastGRNN - FastCells: Fast, Accurate, Stable and Tiny (Gated) RNN cells.

These algorithms can train models for classical supervised learning problems with memory requirements that are orders of magnitude lower than other modern ML algorithms. The trained models can be loaded onto edge devices such as IoT devices/sensors, and used to make fast and accurate predictions completely offline.

A tool that adapts models trained by above algorithms to be inferred by fixed point arithmetic.

• SeeDot: Floating-point to fixed-point quantization tool.

Applications demonstrating usecases of these algorithms.

Organization

• The edgem_tf directory contains the graphs and models in TensorFlow, and examples/tf contains examples and scripts that illustrate their usage.
• The edgeml directory contains the graphs and models in TensorFlow, and examples/pytorch contains examples and scripts that illustrate their usage.
• The cpp directory has training and inference code for Bonsai and ProtoNN algorithms in C++. Please see install/run instruction in the Readme pages within these directories.
• The applications directory has code/demonstrations of applications of the EdgeML algorithms.
• The Tools/SeeDot directory has the quantization tool to generate fixed-point inference code.

Details and project pages

For details, please see our project page and wiki. our ICML'17 publications on Bonsai and ProtoNN algorithms, NeurIPS'18 publications on EMI-RNN and FastGRNN, and PLDI'19 publication on SeeDot.

People who have contributed to this project. Also see Microsoft Research page.

Please also checkout the ELL which can provide optimized binaries for the models trained by this library.

Contributors:

Algorithms, applications and tools were contributed by:

• Don Dennis
• Sridhar Gopinath
• Chirag Gupta
• Ashish Kumar
• Aditya Kusupati
• Shishir Patil
• Harsha Vardhan Simhadri

We welcome contributions, comments, and criticism. For questions, please email us.

If you use software from this library in your projects or publications, please cite us using this BibTex entry:

@software{edgeml01,
   author = {{Dennis, Don Kurian and Gopinath, Sridhar and Gupta, Chirag and
      Kumar, Ashish and Kusupati, Aditya and Patil, Shishir G and Simhadri, Harsha Vardhan}},
   title = {{EdgeML: Machine Learning for resource-constrained edge devices}},
   url = {https://github.com/Microsoft/EdgeML},
   version = {0.1},
}

Microsoft Open Source Code of Conduct This project has adopted the

Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.