Written by Deep learning is an emerging and exciting subset of Machine Learning and in turn Artificial Intelligence that is loosely inspired by biological neural networks within the human brain.
The relationship with big data, storage and GPUs
Neural networks have been around for sometime but they struggled to gain traction until more recent years with the growth in data volumes and cheaper storage along with the development of powerful GPU chips enabling enhanced performance of neural networks. Hence the development of big data and parallel processing methods have enabled the deployment of deep learning methods.
The Key Moment
A pivotal moment for deep learning neural networks occurred with the success of the Alexnet team in the Imagenet competition in 2012. This demonstrated the enhanced performance of the techniques over other, older classical machine learning techniques.
If a neural network has just one hidden layer then it is considered shallow but if it has several hidden layers then it is considered deep.
Deep learning networks can use supervised or unsupervised learning (a topic for future discussion) and there are many different types of networks but two of the most common ones encountered are Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) with Long Short Term Memory (LSTM) popular neural network techniques that are applied. Future blogs will go into more detail on these networks but for now note that CNNs are particularly effective with the field of computer vision (object detection, segmentation and classification) and RNNs such as LSTMs applied to time series data sets.
Deep Learning requires large data sets for training. Prior to deep learning, feature extraction was manual and ML experts were hand crafting features for ML algorithms to distinguish between images of boys and girls. The manually crafted features came through months of research and few examples of such manually designed features that got published are for example the HOG, HAAR and ORB features (these are feature detectors and descriptors).
With the advent of Deep Learning, manual feature extraction took a back seat because Deep Neural Networks got very good at automatically determining what features of the input image are necessary for distinguishing between images of boys v/s girls. These automatic features later discovered represented a hierarchy of features, starting with edges and curves at the lower layers of the neural network and shapes representing a human face and facial attributes like eyes, lips and nose at the higher level layers of the neural network. All the automatic feature extraction was done by scanning the input image with what are known as kernels which are basically rectangular matrices filled with numbers which are basically the weights of the neural network.
In the era of rapid growth of IoT devices with sensors, cameras, and other connected devices, the application of Deep Learning technology will be increasing key in order to make sense out of the big data that will be generated.