AD3501 – Deep Learning
Unit I: Deep Networks Basics
`Linear Algebra: Scalars – Vectors – Matrices and tensors; Probability Distributions – Gradient
based Optimization – Machine Learning Basics: Capacity – Overfitting and underfitting –
Hyperparameters and validation sets – Estimators – Bias and variance – Stochastic gradient
descent – Challenges motivating deep learning; Deep Networks: Deep feedforward networks;
Regularization – Optimization.
Unit II: Convolutional Neural Networks
Convolution Operation – Sparse Interactions – Parameter Sharing – Equivariance – Pooling –
Convolution Variants: Strided – Tiled – Transposed and dilated convolutions; CNN Learning:
Nonlinearity Functions – Loss Functions – Regularization – Optimizers – Gradient Computation.
Unit III: Recurrent Neural Networks
Unfolding Graphs – RNN Design Patterns: Acceptor – Encoder – Transducer; Gradient
Computation – Sequence Modeling Conditioned on Contexts – Bidirectional RNN – Sequence to
Sequence RNN – Deep Recurrent Networks – Recursive Neural Networks – Long Term
Dependencies; Leaky Units: Skip connections and dropouts; Gated Architecture: LSTM.
Unit IV: Model Evaluation
Performance metrics – Baseline Models – Hyperparameters: Manual Hyperparameter –
Automatic Hyperparameter – Grid search – Random search – Debugging strategies.
Unit V: Autoencoders And Generative Models
Autoencoders: Undercomplete autoencoders – Regularized autoencoders – Stochastic encoders
and decoders – Learning with autoencoders; Deep Generative Models: Variational autoencoders
– Generative adversarial networks.
Text Book:
1. Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”, MIT Press, 2016.
2. Andrew Glassner, “Deep Learning: A Visual Approach”, No Starch Press, 2021.
References:
1. Salman Khan, Hossein Rahmani, Syed Afaq Ali Shah, Mohammed Bennamoun, “A Guide
to Convolutional Neural Networks for Computer Vision”, Synthesis Lectures on Computer
Vision, Morgan & Claypool Publishers, 2018.
2. Yoav Goldberg, “Neural Network Methods for Natural Language Processing”, Synthesis
Lectures on Human Language Technologies, Morgan & Claypool Publishers, 2017.
3. Francois Chollet, “Deep Learning with Python”, Manning Publications Co, 2018.
4. Charu C. Aggarwal, “Neural Networks and Deep Learning: A Textbook”, Springer
International Publishing, 2018.
5. Josh Patterson, Adam Gibson, “Deep Learning: A Practitioner’s Approach”, O’Reilly
Media, 2017.
