Exercise 8: Build AlexNet using Advanced CNN
AlexNet is a deep convolutional neural network architecture that gained significant attention
and played a crucial role in advancing the field of deep learning and computer vision. It was
developed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, and it won the
ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012, marking a
breakthrough in image classification tasks.
Here are some key features and components of AlexNet:
Deep Convolutional Layers: AlexNet consists of eight layers of learnable parameters,
including five convolutional layers and three fully connected layers. The convolutional layers
are designed to automatically learn hierarchical features from input images.
Rectified Linear Units (ReLU): AlexNet uses the rectified linear unit activation function
(ReLU) in its hidden layers. ReLU helps mitigate the vanishing gradient problem and
accelerates convergence during training.
Local Response Normalization: The model incorporates local response normalization
(LRN) after the first and second convolutional layers. This normalization helps improve
generalization by normalizing responses within local receptive fields.
Max-Pooling: Max-pooling layers are used after the first, second, and fifth convolutional
layers to downsample the feature maps and reduce the spatial dimensions.
Large-Scale Dataset: AlexNet was trained on the ImageNet dataset, which contains over a
million images from thousands of categories, making it a large-scale image classification
network.
Dropout: Dropout, a regularization technique, is applied to the fully connected layers to
prevent overfitting.
Softmax Activation: The output layer of AlexNet uses the softmax activation function to
compute class probabilities for image classification tasks.
Parallelism: During training, AlexNet was one of the first models to take advantage of GPU
parallelism, which significantly accelerated the training process.
AlexNet demonstrated the effectiveness of deep convolutional neural networks for image
classification tasks and led to a surge of interest in deep learning research. It laid the
foundation for subsequent CNN architectures like VGG, GoogLeNet (Inception), and
ResNet, which have further improved the state-of-the-art performance on various computer
vision tasks.
IMDB dataset:
The IMDB dataset is a widely recognized benchmark in the field of natural language
processing (NLP), primarily used for sentiment analysis. It consists of 50,000 movie reviews, equally
split between positive and negative sentiments, with 25,000 reviews designated for training and the
remaining 25,000 for testing. Each review is labeled either as positive or negative, making it a binary
classification problem. The reviews vary significantly in length, which adds to the complexity of the
task. To process the textual data, common preprocessing steps include tokenization, removal of stop
words, and converting the text into numerical representations using techniques like Bag of Words,
TF-IDF, or word embeddings such as Word2Vec or GloVe. The IMDB dataset is not only instrumental
in training models for sentiment analysis but also plays a key role in broader text classification tasks,
making it an essential tool for researchers and practitioners alike.
#Import necessary libraries
import tensorflow as tf
from [Link] import imdb
from [Link] import sequence
from [Link] import Sequential
from [Link] import Embedding, Conv1D, MaxPooling1D, Flatten,
Dense, Dropout
from [Link] import Adam
# Set random seed for reproducibility
[Link].set_seed(42)
# Parameters
max_features = 10000 # Number of words to consider as features
max_len = 500 # Max sequence length for each review
embedding_dim = 128 # Embedding dimensions for each word
# Load the IMDB dataset
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features)
# Pad sequences to ensure uniform input size
x_train = sequence.pad_sequences(x_train, maxlen=max_len)
x_test = sequence.pad_sequences(x_test, maxlen=max_len)
def build_alexnet():
model = Sequential()
# Embedding layer (maps integer-encoded words to dense vectors)
[Link](Embedding(max_features, embedding_dim, input_length=max_len))
# 1st Convolutional Layer
[Link](Conv1D(filters=96, kernel_size=11, strides=1, activation='relu',
padding='same'))
[Link](MaxPooling1D(pool_size=2, strides=2, padding='same'))
# 2nd Convolutional Layer
[Link](Conv1D(filters=256, kernel_size=5, activation='relu', padding='same'))
[Link](MaxPooling1D(pool_size=2, strides=2, padding='same'))
# 3rd, 4th, 5th Convolutional Layers
[Link](Conv1D(filters=384, kernel_size=3, activation='relu', padding='same'))
[Link](Conv1D(filters=384, kernel_size=3, activation='relu', padding='same'))
[Link](Conv1D(filters=256, kernel_size=3, activation='relu', padding='same'))
[Link](MaxPooling1D(pool_size=2, strides=2, padding='same'))
# Flatten layer
[Link](Flatten())
# 1st Fully Connected Layer
[Link](Dense(4096, activation='relu'))
[Link](Dropout(0.5))
# 2nd Fully Connected Layer
[Link](Dense(4096, activation='relu'))
[Link](Dropout(0.5))
# Output layer (binary classification)
[Link](Dense(1, activation='sigmoid'))
return model
# Initialize the model
model = build_alexnet()
# Compile the model
[Link](optimizer=Adam(learning_rate=0.0001),
loss='binary_crossentropy',
metrics=['accuracy'])
# Display model architecture
[Link]()
# Train the model
batch_size = 256
epochs =10
history = [Link](x_train, y_train,
batch_size=batch_size,
epochs=epochs,
validation_data=(x_test, y_test),
verbose=1)
# Training accuracy and loss
train_acc = [Link]['accuracy'][-1]
train_loss = [Link]['loss'][-1]
print(f'Train Loss: {train_loss:.4f}, Train Accuracy: {train_acc * 100:.2f}%')
# Evaluate the model on the test data
test_loss, test_acc = [Link](x_test, y_test, verbose=1)
print(f'Test Loss: {test_loss:.4f}, Test Accuracy: {test_acc * 100:.2f}%')
# Plot training & validation accuracy values
import [Link] as plt
[Link](figsize=(14, 5))
[Link](1, 2, 1)
[Link]([Link]['accuracy'], label='Train Accuracy')
[Link]([Link]['val_accuracy'], label='Test Accuracy')
[Link]('Model Accuracy')
[Link]('Accuracy')
[Link]('Epoch')
[Link](loc='upper left')
# Plot training & validation loss values
[Link](1, 2, 2)
[Link]([Link]['loss'], label='Train Loss')
[Link]([Link]['val_loss'], label='Test Loss')
[Link]('Model Loss')
[Link]('Loss')
[Link]('Epoch')
[Link](loc='upper left')
# Display the plots
plt.tight_layout()
[Link]()
