1 minute read

模型量化概述

1 综述

  1. Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding
    ICLR 2016 oral 2015-10-01 paper

  2. Speeding-up convolutional neural networks: A survey
    2018-06 paper

  3. A Survey on Methods and Theories of Quantized Neural Networks
    2018-08-13 paper

  4. Neural Network Quantization Introduction
    2019-05-01 中文

2 理论

  1. Doctor of Crosswise: Reducing Over-parametrization in Neural Networks
    2019-05-24 paper

  2. NetTailor: Tuning the Architecture, Not Just the Weights
    CVPR 2019 2019-06-29 paper

3 剪枝

  1. FeTa: A DCA Pruning Algorithm with Generalization Error Guarantees
    2018-03-12 paper

  2. Revisiting hard thresholding for DNN pruning
    2019-05-21 paper

  3. Dissecting Pruned Neural Networks
    2019-01-29 paper
    修剪 resnet50,而不影响可解释性;

  4. Accelerate Your CNN from Three Dimensions: A Comprehensive Pruning Framework
    2020-10-10 paper
    多维度(宽、高、输入大小)剪枝;

5 低秩分解

  1. Learning Low-Rank Approximation for CNNs
    2019-05-24 paper

  2. Traned Rank Pruning for Efficient Deep Neural Networks
    NIPS 2019 workshop 2019-10-09 paper
    认为离线低秩分解误差太大,于是将其集成到训练过程;

6 权值量化

6.1 INT

  1. Integer Discrete Flows and Lossless Compression
    2019-05-17 paper

  2. Weight Normalization based Quantization for Deep Neural Network Compression
    2019-07-01 paper

6.2 INT8

  1. StatAssist & GradBoost: A Study on Optimal INT8 Quantization-aware Training from Scratch
    2020-06-17 paper | pytorch-official

6.3 Binary

  1. Training Binary Neural Networks through Learning with Noisy Supervision
    ICML 2020 2020-10-10 paper

6.4 量化感知

  1. Low-bit quantization and quantization-awaretraining for small-footprint keyword spotting
    2018 paper

6.5 其他

  1. Quantized Convolutional Neural Networks for Mobile Devices
    2015-12-21 paper | caffe | blog
    兼顾速度和大小;

  2. And the Bit Goes Down: Revisiting the Quantization of Neural Networks
    2019-07-12 paper
    对 resnet 系列网络进行了压缩;

7 知识蒸馏

  1. Dream Distillation: A Data-Independent Model Compression Framework
    ICML 2019 workshop 2019-05-17 paper
    $\bullet \bullet$
    无需重新训练的模型压缩方法;

  2. AVD: Adversarial Video Distillation
    2019-07-12 paper

  3. Cross-modal knowledge distillation for action recognition
    NIPS 2019 2019-10-10 paper
    跨模态知识蒸馏(动作识别),使用交叉熵损失代替 KL 散度;认为其对于小数据集效果更好;

8 嵌入式加速

  1. Closing the Accuracy Gap in an Event-Based Visual Recognition Task
    2019-05-06 paper

9 其他

  1. DARC: Differentiable ARchitecture Compression
    2019-05-20 paper

  2. Geometry of Deep Convolutional Networks
    2019-05-21 paper

  3. Deep Model Compression via Filter Auto-sampling
    2019-07-12 paper


TOP

附录

A 参考资料

  1. EwenWanW. 神经网络压缩 剪枝 量化 嵌入式计算优化NCNN[EB/OL]. https://blog.csdn.net/xiaoxiaowenqiang/article/details/80946522. 2018-07-06/2019-09-18.
  2. 清水汪汪. 网络压缩论文整理(network compression)[EB/OL]. https://www.cnblogs.com/zhonghuasong/p/7822572.html. 2017-11-12/2019-02-17.

Comments