To support the production of more high quality content, consider supporting us on Patreon or YouTube membership. Additionally, consider visiting our parent company EarthOne, for sustainable living made simple. Throughout this deep learning series, we have gone from the origins of the field and how the structure of the artificial neural network was conceived, to working through an intuitive example, covering the main aspects and some of the many complexities of deep learning.
Now all these videos have only been focused on one type of neural network, the feedforward network. The focus of this video then will be to initiate discussion on another very popular and important neural network architecture, the convolutional neural network. For our discussions on convolutional neural networks, we will go with a very common example, number recognition. The reason this example was chosen is because of this great interactive resource by Adam Harley, a robotics PhD at Carnegie Mellon University.
I have linked the resource in the description so that you too can experiment with it and see the internals of a convolutional network. Now if you can recall from our example on an image pattern recognizer using a feedforward network, we set up an idealized system and made the assumption that each hidden layer in our network would build upon further layers of abstraction, from vertical lines to combinations of vertical lines. However, in actuality, as we explained, this wouldn't happen, and the receptive fields in the network would be a lot more random to the human eye due to the architecture of feedforward networks and how they compute. With convolutional networks, however, this isn't the case, and we could to some extent see these layers of abstraction building up.
Before we delve into why and how this happens, let's first set up a structure for the network we will be using. As with our previous example in image recognition, the input will be the individual pixels of our image, and the output the patterns we are trying to classify. In this example's case, we have 10 outputs, numbers 0 to 9. For the layers in between, we will have two convolutional layers, two pooling layers, and two fully connected layers. Now that was a lot of new terminology thrown around, so let's break it down layer by layer, starting with the input. While we have already stated that the input is the individual pixels of the image, I want to mention some important details about how a computer views this image.
Every image in a digital device is stored as a matrix of pixel values. This is referred to as a channel, a certain component of an image. Now with a typical digital camera, every image will have three channels, red, green, and blue, RGB, which you can imagine as three 2D matrices stacked upon one another. For the sake of simplicity, we will assume our input has just one channel, the luminance of the image, with the value of each pixel represented in 8 bits. In other words, the pixel values will range from 0 to 255, with 0 indicating dark or transparent luminance, and 255 as bright.
As a side note, digital systems store all types of sensor data and other information in this fashion. For example, for a convolutional network that would operate on speech, we could represent that speech as a matrix of frequency values. Coming back on topic, now that we understand the format of the input, let's finally delve into where the real magic happens, the convolutional layer. As you can assume, this layer and the network as a whole get their names from the convolutional operator.
which in layman terms is the mathematical combination, in other words the dot product of two functions to produce a third function. In CNNs, this operation is implemented in what is referred to as a feature detector, filter, or most commonly, a kernel. You can think of a kernel as a mini matrix, orders a magnitude smaller than the input.
In a convolution operation then, the kernel moves across the input image, taking the dot product of the two matrices, and then saving the values to a new matrix, dubbed the feature map of the original image. At this point, you may be wondering how this is able to detect any features at all. Well, if a kernel is initialized with the values in a specific configuration, they can be used to transform an input image and find various patterns. Take these kernels for instance. When loaded with the appropriate values and convolved with the input image, produces an output which highlights various edges in the photo.
Now, I won't delve into the specifics of the various types of kernels in this video, but there are many resources that talk about them extensively, such as Computerphile and their various videos on kernels like the Sobel Edge Detector. Coming back on topic, as you can see, kernels are essentially ways to do fast computation and produce a new output. In this case, we're doing it with an image, with an effect or features highlighted. Programs like Photoshop use these simple kernels for effects like blurs, for instance. Now when we look at a typical image, take in how much is actually going on.
There are various edges, shapes, textures, etc. that stack together to make various objects and the overall image we are looking at. At the start of a convolutional network, the types of kernels we'd use would be quite simple and more geometric, detecting things such as edges, corners, and simple shapes and patterns, like a circle for instance. Additionally, each convolutional layer can have multiple kernels that produce multiple feature maps of their own. So for our example, we can see that our first layer has 6 kernels, and these kernels are detecting simple patterns like horizontal lines, vertical lines, and corners on our drawn numbers. As a side note, for each of the convolved pixels in our feature map, a non-linearity function was also applied, in this instance a ReLU.
We discussed this function in previous videos and the need for non-linearity and neural networks, so our feature maps are more adaptable to real-world data. Also, you can now see how complicated this process would become if we had more than one channel, as each channel would have 6 feature maps, in this instance leading to 18 for an RGB image, and then those feature maps would have to be filtered together or operated on separately depending on your specifications. For instance, in a self-driving system, A red hexagon is a stop sign, so we would need separate channels and kernels for those channels.
However, coming back to our example, just one channel, luminance, was more than enough. Moving on, now that we have a rectified feature maps, the next layer in CNN's After the Convolution is the pooling layer. Pooling layers are used to downsample our feature maps, keeping the most important parts and discarding the rest, and primarily done to reduce the overfitting rate, and so that calculations can be sped up in later layers due to the reduced spatial size of the image.
The type of pooling our network implements is max pooling. in which we take another kernel and slide it across our input feature maps, and the largest pixel value in that region is what is saved to our new output feature map. As you can see in this number classifier, the max pooled layer still retains the important information from the convolutional layer, while being smaller in spatial size. For instance, in this feature map you can still make out the horizontal lines, and in this one the vertical from their respective original feature maps.
So now with the defining layers of the convolutional neural network understood, A the convolutional layer which extracts features, and then b the pooling layer which down samples the feature maps the next step is repeating these layers to build up more abstraction in the network as stated earlier in later layers of the network more complex kernels are incorporated that detect shapes objects and other complex structures which is done by leveraging the previously generated feature maps and their detected simple features to build more complex ones this is essentially the definition of abstraction you can see in our example that in the next convolutional layer there are 16 kernels and these kernels are now much more complex In the first convolutional layer we were detecting simple edges and corners, and now as you can see horizontal and vertical edges are being combined, as well as other simple shapes from the previous feature maps. For instance, with this drawn 4 you can see this feature map combining both horizontal and vertical edges from various earlier layer feature maps, and resembling a bit of the structure of a 4, or with these feature maps of a 5, where diagonal lines and s-shaped patterns are being detected. As you can see, similar higher level features are present for all numbers.
Now after this convolutional layer, Just as before, there is another max pooling layer to reduce the spatial dimensions of the feature maps before the fully connected layers. As seen here, with these newly pooled maps, they look a lot more random to the human eye since the spatial resolution is not very low. However, to a computer, they are just the key points in a matrix that represent a drawn number. You can also see now why we needed these pooling layers.
From a raw input 32x32 pixel image with 1024 pixels and no discernible features, to 6 low-level feature maps of size 14x14 pixels resulting in 1176 pixels, and then finally 16 high-level feature maps of 5x5 pixels resulting in 400 pixels. And this in general was the whole point of these convolutional and pooling layers, to detect high-level features with as low as possible spatial resolution, and why this part of the convolutional neural network is referred to as feature extraction. At this point with the features extracted we still need to be able to classify them, hence the second part of a convolutional network, the classifier.
This part is comprised of fully connected layers, similar to a feedforward network, except now our input to these perceptron layers are the high level abstracted features from our input rather than the raw input pixels. In this example's case, there are 120 neurons in the first fully connected layer and 100 in the second, and as you can see, these are sufficient to correctly classify numbers with a high degree of accuracy. Now I hope you can recall as we've covered in videos past, during the process of backpropagation and gradient descent, the weight and bias values are tuned as the network learns.
In terms of convolutional networks, those values would still be altered during this process, and also other values such as the kernel coefficients of our convolution and pooling layers. Now before continuing, I want to stop and highlight the fact that in our discussion of convolutional networks, we have made many generalizations and overlooked many details. For instance, hyperparameters like kernel size, stride and dilation rate, transpose convolutions and padding, different types of pooling layers, as well as other parameters and hyperparameters in our future extracting layers and their effects on the output. Not to mention the impact of all the parameters in our fully connected layers which we already mentioned in previous videos. If you want more information on these topics, check out the sources in the disclaimer and description below, as they are beyond the scope of this video.
Coming back on topic, throughout this video we've now seen how and why convolutional networks excel at tasks related to image classification. However, where they don't perform well is for tasks such as natural language processing. This is because these tasks require memory.
But not to fret, there are networks designed specifically for this, that among others will be discussed in the next video in this series. However, this doesn't mean you have to wait to learn more. If you want to learn more about deep learning, and I mean really learn about the field, from how these artificial learning algorithms were inspired from the brain, to their foundational building blocks, the perceptron, scaling up to multilayer networks, different types of networks such as convolutional networks and recurrent networks, and much more, then Brilliant.org is the place for you to go.
For instance, in this section on convolutional neural networks in the artificial neural networks course goes through many of the concepts we have discussed in this video. Now what we love about how the topics in these courses are presented is that first an intuitive explanation is given, and then you are taken through related problems. If you get a problem wrong, you can see an explanation for where you went wrong and how to rectify that flaw. In a world where automation through algorithms will increasingly replace more jobs, it is up to us as individuals to keep our brains sharp and think of creative solutions to multidisciplinary problems. To support Futurology and learn more about Brilliant, go to brilliant.org slash Futurology and sign up for free.
Additionally, the first 200 people that go to that link will get 20% off their annual premium subscription. At this point, the video has concluded. We'd like to thank you for taking the time to watch it.
If you enjoyed it, consider supporting us on Patreon or YouTube membership to keep this brand growing. And if you have any topic suggestions, please leave them in the comments below. Consider subscribing for more content, and check out our website and parent company EarthOne for more information. This has been Encore, you've been watching Futurology, and we'll see you again soon.