「DL」 模型压缩资源汇总
1 综述
-
Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding
ICLR 2016 oral 2015-10-01 paper -
Speeding-up convolutional neural networks: A survey
2018-06 paper -
A Survey on Methods and Theories of Quantized Neural Networks
2018-08-13 paper -
Neural Network Quantization Introduction
2019-05-01 中文
2 理论
-
Doctor of Crosswise: Reducing Over-parametrization in Neural Networks
2019-05-24 paper -
NetTailor: Tuning the Architecture, Not Just the Weights
CVPR 2019 2019-06-29 paper
3 剪枝
-
FeTa: A DCA Pruning Algorithm with Generalization Error Guarantees
2018-03-12 paper -
Revisiting hard thresholding for DNN pruning
2019-05-21 paper -
Dissecting Pruned Neural Networks
2019-01-29 paper
修剪 resnet50,而不影响可解释性; -
Accelerate Your CNN from Three Dimensions: A Comprehensive Pruning Framework
2020-10-10 paper
多维度(宽、高、输入大小)剪枝;
5 低秩分解
-
Learning Low-Rank Approximation for CNNs
2019-05-24 paper -
Traned Rank Pruning for Efficient Deep Neural Networks
NIPS 2019 workshop 2019-10-09 paper
认为离线低秩分解误差太大,于是将其集成到训练过程;
6 权值量化
6.1 INT
-
Integer Discrete Flows and Lossless Compression
2019-05-17 paper -
Weight Normalization based Quantization for Deep Neural Network Compression
2019-07-01 paper
6.2 INT8
- StatAssist & GradBoost: A Study on Optimal INT8 Quantization-aware Training from Scratch
2020-06-17 paper | pytorch-official
6.3 Binary
- Training Binary Neural Networks through Learning with Noisy Supervision
ICML 2020 2020-10-10 paper
6.4 量化感知
6.5 其他
-
Quantized Convolutional Neural Networks for Mobile Devices
2015-12-21 paper | caffe | blog
兼顾速度和大小; -
And the Bit Goes Down: Revisiting the Quantization of Neural Networks
2019-07-12 paper
对 resnet 系列网络进行了压缩;
7 知识蒸馏
-
Dream Distillation: A Data-Independent Model Compression Framework
ICML 2019 workshop 2019-05-17 paper
$\bullet \bullet$
无需重新训练的模型压缩方法; -
AVD: Adversarial Video Distillation
2019-07-12 paper -
Cross-modal knowledge distillation for action recognition
NIPS 2019 2019-10-10 paper
跨模态知识蒸馏(动作识别),使用交叉熵损失代替 KL 散度;认为其对于小数据集效果更好;
8 嵌入式加速
9 其他
-
DARC: Differentiable ARchitecture Compression
2019-05-20 paper -
Geometry of Deep Convolutional Networks
2019-05-21 paper -
Deep Model Compression via Filter Auto-sampling
2019-07-12 paper
附录
A 参考资料
- EwenWanW. 神经网络压缩 剪枝 量化 嵌入式计算优化NCNN[EB/OL]. https://blog.csdn.net/xiaoxiaowenqiang/article/details/80946522. 2018-07-06/2019-09-18.
- 清水汪汪. 网络压缩论文整理(network compression)[EB/OL]. https://www.cnblogs.com/zhonghuasong/p/7822572.html. 2017-11-12/2019-02-17.
Comments