4 minute read

深度估计：根据单张图像得到深度图；
深度估计概述

Monocular Depth · Stereo Consistency · Depth Estimation · Depth Completion · Depth Fusion

1 综述

1.1 单目

  1. Monocular Depth Estimation: A Survey
    2019-01-27 paper

  2. How do neural networks see depth in single images?
    2019-05-16 paper

  3. Monocular Depth Estimation Based On Deep Learning: An Overview
    2020-07-03 paper | blog

  4. Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications
    2022-02-18 paper

2 理论

3 单目

3.1 回归任务

3.1.1 监督

  1. Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
    nips 2014 2014-06-09 paper | home

  2. Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
    iccv 2015 2014-11-18 paper | home

  3. Single-Image Depth Perception in the Wild
    nips 2016 2016-04-13 paper | home

  4. A Two-Streamed Network for Estimating Fine-Scaled Depth Maps from Single RGB Images
    iccv 2017 2016-07-04 paper

  5. Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation
    cvpr 2018 2018-03-29 paper | caffe-official

  6. PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network
    cvpr 2018 2018-05-11 paper

  7. GeoNet: Geometric Neural Network for Joint Depth and Surface Normal Estimation
    cvpr 2018 2020-12-13 paper | tensorflow-official
    GeoNet

  8. Deep Ordinal Regression Network for Monocular Depth Estimation
    cvpr 2018 2018-06-06 paper | caffe | pytorch
    DORN

  9. Joint Task-Recursive Learning for Semantic Segmentation and Depth Estimation
    eccv 2018 2018 paper

  10. Look Deeper into Depth: Monocular Depth Estimation with Semantic Booster and Attention-Driven Loss
    eccv 2018 2018 paper

  11. Depth Esimation via Affinity Learning with Convolutional Spatial Propagation Network
    eccv 2018 2018-08-01 paper | pytorch
    CSPN

  12. SOSD-Net: Joint Semantic Object Segmentation and Depth Estimation from Monocular images
    2021-01-19 paper

3.1.2 无/半监督

3.1.3 其他

  1. Deep Convolutional Neural Fields for Depth Estimation from a Single Image
    cvpr 2015 2014-12-18 paper

3.2 分类任务

3.2.1 监督

  1. Deeper Depth Prediction with Fully Convolutional Residual Networks
    3dv 2016 2016-06-01 paper | tensorflow/matlab | tensorflow/matlab-official | blog
    FCRN

  2. Monocular depth estimation using relative depth maps
    cvpr 2019 2019 paper | caffe-official

  3. From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation
    2019-07-24 paper | pytorch-tensorflow
    bts

  4. Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation
    cvpr 2019 2019-06-08 paper

  5. Exploiting temporal consistency for real-time video depth estimation
    iccv 2019 2019-08-10 paper | pytorch

3.2.2 无/半监督

3.2.3 其他

  1. Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks
    2016-05-08 paper

  2. Single image depth estimation by dilated deep residual convolutional neural network and soft-weight-sum inference
    2017-04-27 paper
    这篇论文主要采用了跳连接来改进网络结构，跳连接的结构有利于多尺度的融合，此外还使用了空洞卷积的方法来增加感受野。

3.3 其他

4 双目

3D CycleGAN

5 其他

  1. Make3D: Learning 3D Scene Structure from a Single Still Image
    2014 paper
    提出了 Make3D 数据集；

  2. Deep3D: Automatic 2D-to-3D Video Conversion with Deep Convolutional Neural Networks
    2016 Paper | MXNet

  3. Monocular Relative Depth Perception With Web Stereo Data Supervision
    cvpr 2018 2018 paper | home
    提出了 ReDWeb 数据集；

  4. Evaluation of CNN-based Single-Image Depth Estimation Methods
    eccv 2018 2018-05-03 paper
    提出了 IBims-1 数据集；单反相机拍摄；并提出了一套新的评估标准；

  5. Learning the Depths of Moving People by Watching Frozen People
    cvpr 2019 2019-04-25 paper | tensorflow-official
    提出了 MannequinChallenge 数据集；

  6. SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation
    2019-05-21 paper

  7. Enhancing self-supervised monocular depth estimation with traditional visual odometry
    2019-08-08 paper

  8. AI Playground: Unreal Engine-based Data Ablation Tool for Deep Learning
    2020-07-13 paper | pytorch
    AIP-Brown

  9. Learning to Recover 3D Scene Shape from a Single Image
    cvpr 2021 2020-12-17 paper | pytorch-official | pytorch-compphoto
    LeRes

  10. Towards Continual, Online, Unsupervised Depth
    2021-03-02 paper | pytorch-stereo-official | pytorch-sfm-official

  11. Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging
    cvpr 2021 2021-05-28 paper | home | pytorch-official
    MiDas

  12. GCNDepth: Self-supervised Monocular Depth Estimation based on Graph Convolutional Network
    2021-12-13 paper | pytorch-official
    GCNDepth

  13. Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth
    2022-01-25 paper | pytorch | onnx
    GLPDepth

6 顶会

6.1 CVPR 2020

  1. Generating and Exploiting Probabilistic Monocular Depth Estimates
    cvpr 2020 2019-06-13 paper | tensorflow-official
    通用的有监督深度估计

  2. Uncertainty-Aware CNNs for Depth Completion: Uncertainty from Beginning to End
    cvpr 2020 2020-06-05 paper
    有监督深度补全，给出了不确定性度量

  3. Structure-Guided Ranking Loss for Single Image Depth Prediction
    cvpr 2020 2020 paper | pyctorch-official 有监督深度估计，提出了更好的loss

  4. Online Depth Learning Against Forgetting in Monocular Videos
    2020 paper
    实时深度学习搞定深度估计

  5. Du2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels
    cvpr 2020 2020-03-31 paper

TOP

附录

A 资源

  1. Robust Vision Challenge 2020

文章

  1. 基于深度学习的单目图像深度估计
  2. 深度估计论文list
  3. https://github.com/scott89/awesome-depth
  4. 基于深度学习的单目深度估计综述
  5. 深度学习之单目深度估计 (Chapter.2)：无监督学习篇
  6. 单目深度估计技术进展综述
  7. 深度学习之单目深度估计

代码

  1. https://github.com/nianticlabs/monodepth2（ICCV 2019）无监督
  2. https://github.com/dwofk/fast-depth （ICRA 2019）
  3. https://github.com/wangq95/KITTI_Dense_Depth 监督学习
  4. https://github.com/JunjH/Revisiting_Single_Depth_Estimation 监督学习
  5. paper with code

B 参考资料

C 数据集

数据集 类型 数量（训练/验证/测试） 分辨率 说明 发布日期 best model 论文
KITTI       2个灰度相机，2个彩色相机，64线3D激光，1个GPS；
标签包含立体匹配结果、光流、场景流、深度估计、视觉测距、2D/3D目标检测、跟踪、道路/车道线检测、分割		   GLPDepth, DIFFNet  
NYUv2           GLPDepth, Semantic-aware NN  
Make3D           GCNDepth Make3D
Middlebury2014           MiDas MiDas
DIODE       室内场景   LeRes, AIP-Brown  
ReDWeb       室内外密集场景     ReDWeb
Mannequin       手持相机拍摄     Mannequin
IBims-1       室内   LeRes IBims
WSVD       Web 视频      
vKITTI       KITTI 的虚拟数据集，“虚拟”指的是：旋转、清晨、晴天、多云、雾天、下雨等增强，共 14G 原始 RGB 图像，
包含检测、跟踪、分割；
这一数据集的意义在于可以缓解深度信息对于光线的敏感问题		      
Cityscapes       德国的50多个城市的户外场景，包扩左右目图像、视差深度图、相机校准、车辆测距、行人检测、目标分割等，同时也包含有类似于vKITTI的虚拟渲染场景图像；
其中简单的左视角图像、相机标定、目标分割等数据需要利用学生账号注册获取，其他数据需要联系管理员获取；		      
]              
]              

Comments

You May Also Enjoy