Course content

Penalized and generalized linear models: Logistic regression, L1 and L2 penalization, elastic net, SCAD penalty, application to high dimensional low sample size problems. Intro to neural networks: Artificial neuron, single hidden layer, multiple hidden layer, back propagation, momentum, loss functions, relation with support vector machines and penalized logistic regression. Convolutional neural networks: Convolutional layers, pooling layers, drop out, VGGnet, inception modules, residual networks, deconv nets, applications to object recognition. Why deep learning works: Role of depth, closeness of local minima to global minimal, predominance of saddle points and ridges vs. local minima. Recurrent neural networks and LSTMs: lateral connections, LSTM units, gated recurrent networks, applications to NLP. Probabilistic Graphical Models: Factor graphs and belief networks, Deep belief networks and Boltzmann machines, sampling methods including Gibbs sampling, contrastive divergence, generative adversarial networks.

References
Textbooks:

1. Deep Learning by Goodfellow, Bengio and Courville.
   
2. Probabilistic Graphical Models: Principles and Techniques by Koller and Friedman