Introduction to DL

Objectives

• Discuss the major trends driving the rise of deep learning.
• Explain how deep learning is applied to supervised learning
• List the major categories of models (CNNs, RNNs, etc.), and when they should be applied
• Assess appropriate use cases for deep learning.

Neural Network Basics

Objectives

• Build a logistic regression model structured as a shallow neural network
• Build the general architecture of a learning algorithm, including parameter initialization, cost function and gradient calculation, and optimization implemetation (gradient descent)
• Implement computationally efficient and highly vectorized versions of models
• Compute derivatives for logistic regression, using a backpropagation mindset
• Use Numpy functions and Numpy matrix/vector operations
• Work with iPython Notebooks
• Implement vectorization across multiple training examples
• Explain the concept of broadcasting

Shallow Neural Networks

• Describe hidden units and hidden layers
• Use units with a non-linear activation function, such as tanh
• Implement forward and backward propagation
• Apply random initialization to your neural network
• Increase fluency in Deep Learning notations and Neural Network Representations
• Implement a 2-class classification neural network with a single hidden layer
• Compute the cross entropy loss

Deep Neural Networks

• Describe the successive block structure of a deep neural network
• Build a deep L-layer neural network
• Analyze matrix and vector dimensions to check neural network implementations
• Use a cache to pass information from forward to back propagation
• Explain the role of hyperparameters in deep learning
• Build a 2-layer neural network