Understanding Bayesian Deep Learning

1. Elementary mathematics

• Set theory
• Measure theory
• Probability
• Random variable
• Random process
• Functional analysis (harmonic analysis)

2. Gaussian process

• Gaussian process
• Weight-space view
• Function-space view
• Gaussian process latent variable model

3. Bayesian neural netwrok

• Minimum description length
• Ensemble learning in Bayesian neural network
• Practical variational inference
• Bayes by backprop
• Summary of variational inference
• Dropout as a Bayesian approximation
• Stein variational gradient descent

4. Summary

• Measure thoery
• Probability
• Random variable
• Random process
• Gaussian process
• Functional Analysis
• Summary of variational inference
• Stein variational gradient descent

5. Uncertainty in Deep Learning

• Yarin Gal, Uncertainty in Deep Learning
• Anonymous, Bayesian Uncertainty Estimation for Batch Normalized Deep Networks
• Patrick McClure, Representing Inferential Uncertainty in Deep Neural Networks through Sampling
• Balaji Lakshminarayanan, Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
• Alex Kendal, What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• Gregory Kahn, Uncertainty-Aware Reinforcement Learning for Collision Avoidance
• Charles Richter, Safe Visual Navigation via Deep Learning and Novelty Detection
• Sungjoon Choi, Uncertainty-Aware Learning from Demonstration Using Mixture Density Networks with Sampling-Free Variance Modeling