3 minute read

卷积网络概述

1 综述

  1. Notes on Convolutional Neural Networks
    2006-11-22 Paper | 亦轩Dhc
    $\bullet \bullet$
    主要讲了 CNN 前向和反向传播，包括卷积层和池化的反向传播的推导的讲解；

  2. A Selective Overview of Deep Learning
    2019-04-10 paper

  3. A review of modularization techniques in artificial neural networks
    2019-04-29 paper

  4. What does it mean to understand a neural network?
    2019-07-15 DeepMind, 宾夕法尼亚 paper

  5. What Do We Understand About Convolutional Networks?
    2019-09-09 paper

2 理论

2.1 深与浅

  1. The Power of Depth for Feedforward Neural Networks
    COLT 2016 2015-12-12 paper
    证明了存在 3 层的网络，是无法被 2 层网络逼近的；

  2. Benefits of depth in neural networks
    COLT 2016 2016-02-14 paper
    深度网络需要很少的节点就能构造出复杂网络，扁平网络则需要更多节点；

  3. Learning Functions: When Is Deep Better ThanShallow
    2016-03-03 paper

  4. Deep vs. shallow networks : An approximation theory perspective
    2016-08-10 paper

  5. Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks
    ICML 2017 2016-10-31 paper | openreview
    对于球状空间，深度可以很好拟合，而扁平网络做不到；

  6. Learning Functions: When Is Deep Better Than Shallow
    2016-03-03 paper

  7. When and Why Are Deep Networks Better than Shallow Ones
    AAAI 2017 2017 paper

  8. Efficient Processing of Deep Neural Networks: A Tutorial and Survey
    2017-03-27 paper | home

3 二维

3.1 经典网络

  1. ImageNet classification with deep convolutional neural networks
    2012 paper | tensorflow
    AlexNet；

  2. Very deep convolutional networks for large-scale image recognition
    ICLR 2015 oral 2014-09-04 paper | tensorflow
    VGG:

  3. Going deeper with convolutions
    CVPR 2015 2014-09-17 paper
    GoogLeNet;

  4. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
    2015-02-11 paper | blog-Michael | blog-极市
    提出了批量归一化，被大量使用；
    InceptionV2

  5. Deep residual learning for image recognition
    CVPR 2016 2015-12-10 paper
    ResNet;

  6. Densely Connected Convolutional Networks
    CVPR 2017 2016-08-25 paper
    DenseNet

  7. Deep Pyramidal Residual Networks
    CVPR 2017 2016-10-10 paper
    解决残差设计中遇到通道升维时效果不好的问题；
    PyramidNet

  8. Dual Path Networks
    2017-07-06 paper | mxnet-official
    DualPathNet

  9. Squeeze-and-Excitation Networks
    2017-09-05 Paper
    SENet

3.2 轻量级网络

  1. Aggregated Residual Transformations for Deep Neural Networks
    CVPR 2017 2016-11-16 paper
    ResNeXt;

  2. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
    ICLR 2017 2016-02-24 伯克利&斯坦福 paper
    SqueezeNet

  3. Xception: Deep Learning with Depthwise Separable Convolutions
    2016-10-07 Google paper
    Xception

  4. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
    CVPR 2017 2017-04-17 paper | ncnn-Blog | caffe
    MobileNet

  5. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
    CVPR 2017 2017-07-04 Face++ paper
    ShuffleNet

  6. Interleaved Group Convolutions for Deep Neural Networks
    ICCV 2017 2017-07-10 微软 paper | mxnet
    IGCV

  7. CondenseNet: An Efficient DenseNet using Learned Group Convolutions
    2017-11-25 paper | pytorch
    CondenseNet

  8. MobileNetV2: Inverted Residuals and Linear Bottlenecks
    2018-01-13 paper | mxnet | caffe
    MobileNetV2

  9. IGCV2: Interleaved Structured Sparse Convolutional Neural Networks
    2018-04-17 微软 paper
    IGCV2

  10. IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks
    BMVC 2018 2018-06-01 微软 paper | mxnet | pytorch
    IGCV3

  11. ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
    2018-07-30 paper
    ShuffleNetV2

  12. Searching for MobileNetV3
    2019-05-06 google paper | pytorch | pytorch | 我爱计算机视觉
    MobileNetV3:

  13. MoGA: Searching Beyond MobileNetV3
    2019-08-04 paper | pytorch-official

3.3 提升泛化能力

  1. Learning to Find Correlated Features by Maximizing Information Flow in Convolutional Neural Networks
    2019-06-30 paper
    设计了新的 loss 来指导网络学习更多的相关特征，以应对未见过的数据；

  2. Sparsifying and Down-scaling Networks to Increase Robustness to Distortions
    2020-06-08 paper
    抗噪声，提出了 STNet，用于修改网络的 block 提升噪声鲁棒；

3.4 注意力

  1. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
    2019-10-08 paper | pytorch-official
    反思注意力带来的计算冗余问题；

3.5 其他

  1. Multi-column Deep Neural Networks for Image Classification
    CVPR 2012 2012-02-13 paper | blog | bagging | blog | blog
    数据集：MNIST，NIST SD 19，CASIA（中国字），GTSRB（路标），CIFAR 10（natural color images），NORB（stereo images of 3D models）
    IJCNN 2012 第一名，准确率 99.46% 优于人类识别率；多个 DNNs 结合到一个MCDNN（MLP+DNN）；初评时，作者使用了一组 trained on provided features的多层感知器（MLP）和一个 DNN trained on raw pixel intensities；本篇论文描述的是现场总决赛阶段使用的方法，MCDNN；

1.Invertible Residual Networks
ICML 2019 2018-11-02 paper

4 三维

  1. 3D Dense Separated Convolution Module for Volumetric Image Analysis
    2019-05-14 paper

  2. A review on deep learning techniques for 3D sensed data classification
    2019-07-09 paper

5 四元组

  1. Quaternion Recurrent Neural Networks
    2018-06-12 paper

  2. Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition
    2018-06-20 paper

  3. Quaternion Convolutional Neural Networks
    ECCV 2018 2018-08 paper | paper-eccv | 机构 | blog
    $\bullet \bullet$ QCNN
    针对 CNN 中通道处理太简单（相加）而存在的多个弊端，本文借鉴机器人学中的四元数来表示彩色图像，并给除了前向和反向传播推导；与 CNN 做了对比实验，在彩色图像去噪和分类任务上表现都更好；

  4. A survey of quaternion neural networks
    2019-08-06
    $\bullet \bullet$

TOP

Comments

You May Also Enjoy