Deep Learning Fundamentals

In the previous chapter, we created some machine learning models using neural network packages in R. This chapter will look at some of the fundamentals of neural networks and deep learning by creating a neural network using basic mathematical and matrix operations. This application sample will be useful for explaining some key parameters in deep learning algorithms and some of the optimizations that allow them to train on large datasets. We will also demonstrate how to evaluate different hyper-parameters for models to find the best set. In the previous chapter, we briefly looked at the problem of overfitting; this chapter goes into that topic in more depth and looks at how you can overcome this problem. It includes an example use case using dropout, the most common regularization technique in deep learning.

This chapter covers the following topics:

• Building neural networks from scratch in R
• Common parameters in deep learning
• Some key components in deep learning algorithms
• Using regularization to overcome overfitting
• Use case—improving out-of-sample model performance using dropout