Introduction to Deep Learning Course

Course Overview

• Offered through NPTEL
• Focus on deep learning applications:
  • Speech recognition
  • Computer vision
  • Natural language processing
• Use of deep learning by major companies (e.g., Google, Facebook)

Course Content

Fundamental Blocks of Deep Learning

1. Perceptron and Single Neurons

   • Introduction to perceptrons and sigmoid neurons
   • Transition to multilayer perceptrons (MLP)
   • Training algorithms: Backpropagation using gradient descent

2. Feedforward Neural Networks

   • Applications of feedforward networks:
     • Autoencoders
     • Word2Vec

3. Recurrent Neural Networks (RNNs)

   • Use cases for sequences:
     • Natural language (sentences as sequences of words)
     • Speech (sequences of phonemes)
     • Videos (sequences of images)
   • Training RNNs with backpropagation through time
   • Addressing challenges in training RNNs
   • Advanced types of RNNs:
     • Long Short-Term Memory (LSTM)
     • Gated Recurrent Units (GRUs)

4. Convolutional Neural Networks (CNNs)

   • Applications in computer vision:
     • Image representation
     • Tasks: Classification, object detection, segmentation
   • Understanding convolution operations and hierarchical representations

Advanced Topics

• Encoder-Decoder Models

  • Combining different neural network types (RNNs, CNNs, Feedforward)
  • Applications:
    • Image captioning
    • Machine translation
    • Document summarization

• Attention Mechanism

  • Importance of focusing on critical input parts (e.g., main objects in an image)
  • Application in various tasks, including document classification

Course Structure

• Duration: 30 hours of teaching over 12 weeks

Extended Version of the Course

• Topics:
  • Deep generative models
  • Learning probability distributions with neural networks:
    • Restricted Boltzmann Machines
    • Variational Autoencoders
    • Autoregressive Models
    • Generative Adversarial Networks (GANs)
• Focus on theory, connections, advantages, disadvantages, and taxonomy of models

Conclusion

• Main syllabus includes:
  • Feedforward neural networks
  • RNNs
  • CNNs
  • Encoder-decoder models with attention
• Encouragement to enroll and enjoy the course.