「DL」 模型压缩资源汇总

模型量化概述

1 综述

1. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding
   ICLR 2016 oral 2015-10-01 paper

2. Speeding-up convolutional neural networks: A survey
   2018-06 paper

3. A Survey on Methods and Theories of Quantized Neural Networks
   2018-08-13 paper

4. Neural Network Quantization Introduction
   2019-05-01 中文

2 理论

1. Doctor of Crosswise: Reducing Over-parametrization in Neural Networks
   2019-05-24 paper

2. NetTailor: Tuning the Architecture, Not Just the Weights
   CVPR 2019 2019-06-29 paper

3 剪枝

1. FeTa: A DCA Pruning Algorithm with Generalization Error Guarantees
   2018-03-12 paper

2. Revisiting hard thresholding for DNN pruning
   2019-05-21 paper

3. Dissecting Pruned Neural Networks
   2019-01-29 paper
   修剪 resnet50，而不影响可解释性；

4. Accelerate Your CNN from Three Dimensions: A Comprehensive Pruning Framework
   2020-10-10 paper
   多维度（宽、高、输入大小）剪枝；

5 低秩分解

1. Learning Low-Rank Approximation for CNNs
   2019-05-24 paper

2. Traned Rank Pruning for Efficient Deep Neural Networks
   NIPS 2019 workshop 2019-10-09 paper
   认为离线低秩分解误差太大，于是将其集成到训练过程；

6 权值量化

6.1 INT

1. Integer Discrete Flows and Lossless Compression
   2019-05-17 paper

2. Weight Normalization based Quantization for Deep Neural Network Compression
   2019-07-01 paper

6.2 INT8

1. StatAssist & GradBoost: A Study on Optimal INT8 Quantization-aware Training from Scratch
   2020-06-17 paper | pytorch-official

6.3 Binary

1. Training Binary Neural Networks through Learning with Noisy Supervision
   ICML 2020 2020-10-10 paper

6.4 量化感知

1. Low-bit quantization and quantization-awaretraining for small-footprint keyword spotting
   2018 paper

6.5 其他

1. Quantized Convolutional Neural Networks for Mobile Devices
   2015-12-21 paper | caffe | blog
   兼顾速度和大小；

2. And the Bit Goes Down: Revisiting the Quantization of Neural Networks
   2019-07-12 paper
   对 resnet 系列网络进行了压缩；

7 知识蒸馏

1. Dream Distillation: A Data-Independent Model Compression Framework
   ICML 2019 workshop 2019-05-17 paper
   $\bullet \bullet$
   无需重新训练的模型压缩方法；

2. AVD: Adversarial Video Distillation
   2019-07-12 paper

3. Cross-modal knowledge distillation for action recognition
   NIPS 2019 2019-10-10 paper
   跨模态知识蒸馏（动作识别），使用交叉熵损失代替 KL 散度；认为其对于小数据集效果更好；

8 嵌入式加速

1. Closing the Accuracy Gap in an Event-Based Visual Recognition Task
   2019-05-06 paper

9 其他

1. DARC: Differentiable ARchitecture Compression
   2019-05-20 paper

2. Geometry of Deep Convolutional Networks
   2019-05-21 paper

3. Deep Model Compression via Filter Auto-sampling
   2019-07-12 paper

---

TOP

附录

A 参考资料

1. EwenWanW. 神经网络压缩 剪枝 量化 嵌入式计算优化NCNN[EB/OL]. https://blog.csdn.net/xiaoxiaowenqiang/article/details/80946522. 2018-07-06/2019-09-18.
2. 清水汪汪. 网络压缩论文整理(network compression)[EB/OL]. https://www.cnblogs.com/zhonghuasong/p/7822572.html. 2017-11-12/2019-02-17.