Above diagram is one of the most popular one on the internet, that depicts the difference between artificial intelligence, machine learning and deep learning . I too have came across this in my early days into machine learning and have even used it in many of my presentations. In this article we go a little deeper and try to understand the relationship between the classical machine learning and deep learning.
What is Deep Learning? Let’s start with the statement from wiki. ”Deep Learning is a subset of broader family of Machine Learning Algorithms and is based on Artificial Neural Network”. This is evident from the above representation and is also the simplest answer that distinguishes DL from ML. But as always it misses the details.
Following definition from Wikipedia on Deep Learning provides a better view.
Deep learning is a class of machine learning algorithms that uses multiple layers to progressively extract higher-level features from the raw input.
Key take away is the phrase “uses multiple layers to progressively extract higher-level features from the raw input.” So that is what Deep Learning does. It uses artificial neural networks to extract features from the raw input and applies classical machine learning algorithms on those features.
Let us take the example of classifying an image to cat or not cat and make this point clear. In this problem we have a set of images of animal and we need to classify if the image is that of a cat or not
Classical Machine Learning
Now suppose from the original dataset we create another dataset that contains the features as image size, image dimensions, pre-dominant color of the object, number of eyes present, number of legs present, color of the eye (if present), whiskers if any, number of whiskers, presence of tail etc. Also record will have a label saying if that is a cat image or not.
Now given that we have derived this dataset, we could apply the regular data processing steps and then build a use a logistic regression algorithm to create a model. Once the model is created, we can predict if an image is that of cat or not by extracting the above features and providing it to the model.
A natural question then arises is if we need to manually see the image and then extract the feature before inputting them to the model, then where is the value add. Can’t I directly look at it and predict if it contains cat or not.
Yes, that is actually a really valid question and that is where Deep Learning comes into picture. (Remember the key phrase of “uses multiple layers to progressively extract higher-level features from the raw input” in the wiki definition)
Deep Learning
What Deep Learning tries to achieve is to extract the features (the one which we manually created earlier). This is done using deep neural networks containing multiple layers of artificial neurons. Note that we may not be able to explicitly identify and map the feature extracted by a neural network (say to the ones like number of eyes, whiskers, tail etch which we have created earlier). Nevertheless, extracted features will have the information needed for the classification.
Once we create such a neural network, we could then directly input an image to the network, network will extract the necessary features from the image and then use those features for prediction.
Thus any Deep Learning model can be thought as a feature extraction framework (to extract the features from data) and then feeding the features to a classical ML algorithms (like linear regression, logistic regression etc)
Summary
All the discussion we had can be summarized with the below image.
Foot Note
Deep Neural Networks can even be used for problems which has a well established feature dataset (like house prediction). Even there it is trying to extract features from the already existing features and then use those for prediction. But since the extracted feature might not be having much information addition when compared to the initial set of features and hence may not provide that stellar result that we expect. And then there are issues like overfitting which is a topic for another day.
