4f3ea9b52b
Adding Microsoft SECURITY.MD |
||
---|---|---|
LICENSE | ||
README.md | ||
SECURITY.md | ||
alexnet_model.py | ||
cifar10-vgg-small.py | ||
densenet_model.py | ||
googlenet_model.py | ||
imagenet.py | ||
imagenet_utils.py | ||
learned_quantization.py | ||
resnet_model.py | ||
vgg_model.py |
README.md
LQ-Nets
By Dongqing Zhang, Jiaolong Yang, Dongqiangzi Ye, Gang Hua.
Microsoft Research Asia (MSRA).
Introduction
This repository contains the training code of LQ-Nets introduced in our ECCV 2018 paper:
D. Zhang*, J. Yang*, D. Ye* and G. Hua. LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks. ECCV 2018 (*: Equal contribution) PDF
Dependencies
- Python 2.7 or 3.3+
- Python bindings for OpenCV
- TensorFlow >= 1.3.0
- TensorPack
Usage
Download the ImageNet dataset and decompress into the structure like
dir/
train/
n01440764/
n01440764_10026.JPEG
...
...
val/
ILSVRC2012_val_00000001.JPEG
...
To train a quantized "pre-activation" ResNet-18, simply run
python imagenet.py --gpu 0,1,2,3 --data /PATH/TO/IMAGENET --mode preact --depth 18 --qw 1 --qa 2 --logdir_id w1a2
After the training, the result model will be stored in ./train_log/w1a2
.
For more options, please refer to python imagenet.py -h
.
Results
ImageNet Experiments
Quantizing both weight and activation
Model | Bit-width(W/A) | Top-1(%) | Top-5(%) |
---|---|---|---|
ResNet-18 | 1/2 | 62.6 | 84.3 |
ResNet-18 | 2/2 | 64.9 | 85.9 |
ResNet-18 | 3/3 | 68.2 | 87.9 |
ResNet-18 | 4/4 | 69.3 | 88.8 |
ResNet-34 | 1/2 | 66.6 | 86.9 |
ResNet-34 | 2/2 | 69.8 | 89.1 |
ResNet-34 | 3/3 | 71.9 | 90.2 |
ResNet-50 | 1/2 | 68.7 | 88.4 |
ResNet-50 | 2/2 | 71.5 | 90.3 |
ResNet-50 | 3/3 | 74.2 | 91.6 |
ResNet-50 | 4/4 | 75.1 | 92.4 |
AlexNet | 1/2 | 55.7 | 78.8 |
AlexNet | 2/2 | 57.4 | 80.1 |
DenseNet-121 | 2/2 | 69.6 | 89.1 |
VGG-Variant | 1/2 | 67.1 | 87.6 |
VGG-Variant | 2/2 | 68.8 | 88.6 |
GoogLeNet-Variant | 1/2 | 65.6 | 86.4 |
GoogLeNet-Variant | 2/2 | 68.2 | 88.1 |
Quantizing weight only
Model | Bit-width(W/A) | Top-1(%) | Top-5(%) |
---|---|---|---|
ResNet-18 | 2/32 | 68.0 | 88.0 |
ResNet-18 | 3/32 | 69.3 | 88.8 |
ResNet-18 | 4/32 | 70.0 | 89.1 |
ResNet-50 | 2/32 | 75.1 | 92.3 |
ResNet-50 | 4/32 | 76.4 | 93.1 |
AlexNet | 2/32 | 60.5 | 82.7 |
More results can be found in the paper.
Citation
If you use our code or models in your research, please cite our paper with
@inproceedings{ZhangYangYeECCV2018,
author = {Zhang, Dongqing and Yang, Jiaolong and Ye, Dongqiangzi and Hua, Gang},
title = {LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2018}
}