Deep Dive into Convolutional Neural Networks (CNN)

Introduction to CNN

               CNNs are primarily used in the field of computer vision, finding success in an array of applications including:

• Image and video recognition
• Recommender systems
• Image generation
• Medical image analysis

Convolution: The First Layer

               The term “convolution” in CNNs refers to the mathematical operation that is applied to the input data. Specific features of the input are extracted during this operation, which forms a feature map.

Key points about the Convolution operation:

• Each neuron in the feature map corresponds to a small region or subregion in the input image.
• The neurons are connected to their corresponding region in the input via weights (also known as a filter or kernel).
• Each neuron applies the same filter to its specific subregion of the input image, hence the name “convolutional” layer.

Pooling: Reducing Spatial Size

               Pooling layers are used to reduce the spatial dimensions (width and height) of the input volume. It helps to decrease the computational complexity and to control overfitting.

Features of the pooling operation:

• It operates on each feature map separately.
• The most common form of pooling is Max Pooling, which extracts the maximum value of the region covered by the filter.

CNN Components

               CNNs are made up of several layers that process and transform an input to produce an output. These include:

1. Input Layer: Takes raw pixel data of the image.
2. Convolutional Layer: Computes the output of neurons connected to local regions or subregions in the input, each computing a dot product between their weights and a small region (the receptive field) in the input volume.
3. ReLU Layer: Applies an element-wise activation function, such as the max(0,x) thresholding at zero. This leaves the size of the volume unchanged.
4. Pooling Layer: Performs a downsampling operation along the spatial dimensions (width, height).
5. Fully-Connected Layer: Neurons in a fully connected layer have connections to all activations in the previous layer, as seen in regular Neural Networks.

Writing our First CNN

               Building a CNN involves defining the architecture and specifying parameters such as the number of filters, the filter size, the architecture of the fully connected layers, etc. Here’s a simplified process:

• Define the architecture.
• Specify parameters (e.g., the number of filters, the filter size, etc.)
• Use deep learning libraries like TensorFlow or PyTorch to build, train, and validate the CNN model.

Regularization Techniques

               Regularization techniques are crucial to prevent overfitting in a CNN model. Here are two widely used methods:

• Dropout: During training, randomly selected neurons are ignored or “dropped out”. This helps to prevent overfitting.
• L1/L2 Regularization: These add a penalty equivalent to the absolute value (L1) or square (L2) of the magnitude of coefficients.

Introduction to CNN Architectures

               There are several established CNN architectures that have proven effective in various fields. Some of the well-known ones include:

• LeNet-5: Mainly used for handwriting and character recognition.
• AlexNet: It was the pioneer in CNN and open-sourced to the community to further development.
• VGGNet: VGGNet is known for its simplicity, using only 3×3 convolutional layers stacked on top of each other in increasing depth.

Deep diving into these components will offer an enhanced understanding of CNNs and their practical applications. As with any technology, hands-on experience and consistent practice will provide the most valuable insights.

Ref: https://www.boardinfinity.com/blog/deep-dive-into-convolutional-neural-networks-cnn/