Practical Aspects Of Deep Learning

• Give examples of how different types of initializations can lead to different results
• Examine the importance of initialization in complex neural networks
• Explain the difference between train/dev/test sets
• Diagnose the bias and variance issues in your model
• Assess the right time and place for using regularization methods such as dropout or L2 regularization
• Explain Vanishing and Exploding gradients and how to deal with them
• Use gradient checking to verify the accuracy of your backpropagation implementation
• Apply zeros initialization, random initialization, and He initialization
• Apply regularization to a deep learning model

Optimization Algorithms

• Apply optimization methods such as (Stochastic) Gradient Descent, Momentum, RMSProp and Adam
• Use random minibatches to accelerate convergence and improve optimization
• Describe the benefits of learning rate decay and apply it to your optimization

Hyperparameter Tuning, Batch Normalization and Programming Frameworks

• Master the process of hyperparameter tuning
• Describe softmax classification for multiple classes
• Apply batch normalization to make your neural network more robust
• Build a neural network in TensorFlow and train it on a TensorFlow dataset
• Describe the purpose and operation of GradientTape
• Use tf.Variable to modify the state of a variable
• Apply TensorFlow decorators to speed up code
• Explain the difference between a variable and a constant