ELECTRONICS AND COMMUNICATIONS ENGINEERING (B.

Tech)
Jawaharlal Nehru Technological University Kakinada
University College of Engineering Kakinada (A)
Kakinada District, Andhra Pradesh – 533003

INTERNSHIP REPORT ON

MACHINE LEARNING USING PYTHON

Submitted By

V VEERENDRA(22021A409)

ELECTRONICS AND COMMUNICATIONS ENGINEERING (B. Tech)

Jawaharlal Nehru Technological University Kakinada
University College of Engineering Kakinada (A)
Kakinada District, Andhra Pradesh – 533003
Under the guidance of

Dr. PULI KISHORE KUMAR MTech., PhD, Assistant Professor

DEPARTMENT OF ELECTRONICS AND COMMUNICATION ENGINEERING

NATIONAL INSTITUTE OF TECHNOLOGY , TADEPALLIGUDEM
 CERTIFICATE

This is to certify that the project report entitled “MACHINE LEARNING

USING PYTHON”,submitted by V Veerendra(22021A0409),K Asheesh
Noel(22021A0416),V L V Shiva Charan(23025A0459), K V S N
Varma(23025A0461) of II-year 2nd semester, in partial fulfilment of the award
of degree of Bachelor of Technology in Electronics and Communication
Engineering, is a record of bonfire work carried out by them.

PROJECT GUIDE HEAD OF THE

DEPARTMENT
Dr. PULI KISHORE KUMAR MTech., PhD Dr. B.LEELA KUMARI
Assistant Professor Head of the Department
Dept. Of ECE Dept. Of ECE
National Institute of Technology , Tadepalligudem UCEK, JNTU Kakinada
 INTERNSHIP REPORT

Submitted in accordance with the requirement for the degree of

Bachelor of
Technology

Name of the college: University college of Engineering Kakinada

Name of the Faculty Guide: Dr. PULI KISHORE KUMAR MTech.,

PhD, Assistant Professor

Duration of Internship: 02/05/24 to 28/05/24

Batch of Students: V Veerendra(22021A0409)

K Asheesh Noel(22021A0416)
V L V Shiva Charan(23025A0459)
K V S N Varma(23025A0461)

Programme of study: BTech

Year of study: II Year
Date of Submission: 03-08-2024

ABSTRACT
 During our internship, we collaborated with a skilled team to develop and
implement machine learning models using Python, with a particular focus on
understanding and applying forward and backward propagation algorithms. Our
project aimed to build neural networks capable of performing complex predictive
tasks.

Utilizing Python libraries such as TensorFlow and Keras, we constructed and

trained various models, employing forward propagation to calculate outputs and
loss, and backward propagation to update weights through gradient descent.

This hands-on experience involved close teamwork, with data scientists,

engineers, and researchers working together to fine-tune the models for optimal
performance. We followed an iterative approach, testing and refining the models
to ensure accuracy and efficiency.
INSTITUTION INFORMATION:

The National Institute of Technology Andhra Pradesh (NIT Andhra Pradesh) is one
of the premier engineering institutions in India, established as part of the NIT system to
provide high-quality technical education and promote research in various engineering
disciplines. Located in Tadepalligudem, West Godavari district, the institute was founded in
2015 and has quickly grown to become a center of excellence in engineering and technology.

The Institute places a strong emphasis on research and development, encouraging students
and faculty to engage in cutting-edge research projects. Collaborative research initiatives
with industry and other academic institutions are actively promoted.

The institute maintains strong ties with the industry through various collaborative projects,
internships, and placement programs. These partnerships provide students with valuable
exposure to real-world engineering challenges and opportunities for hands-on experience.

BENEFIT OF INSTITUTION THROUGH OUR REPORT:

The institute is equipped with state-of-the-art laboratories and research centres,

facilitating advanced research and innovation. Access to these facilities allowed us to engage
in hands-on learning and apply theoretical knowledge to real-world problems.
 ACKNOWLEDGEMENT

First, we would like to thank Dr Puli Kishore Kumar, Assistant Professor, Department of
Electronics and Communication Engineering, National Institute of Technology,
Tadepalligudem, Andhra Pradesh for giving us the opportunity to do an internship within
the Institution.

We also would like to thank all the people that worked along with us in the Institution with
their patience and openness they created an enjoyable learning and working environment.

We would like to thank our Head of the Department Dr B. Leela Kumari for her
constructive criticism throughout our internship.

We are extremely grateful to the department staff members and friends who helped us in
successful completion of this Internship.

V. VEERENDRA 22021A0409
ASHEESH NOEL 22021A0416
V.L.V. SHIVA CHARAN 23025A0459
K.V.S. NAGENDRA VARMA 23025A0461
 INTRODUCTION TO MACHINE LEARNING

Machine learning is a subset of artificial intelligence (AI) that focuses on the development of
algorithms and statistical models that enable computers to perform tasks without explicit
instructions. Instead, machine learning systems learn from data, identifying patterns and making
decisions with minimal human intervention.

Key Concepts in Machine Learning:

1. Supervised Learning:
In supervised learning, the algorithm is trained on a labelled dataset, which means that
each training example is paired with an output label. The goal is to learn a mapping from
inputs to outputs that can be used to predict the labels of new, unseen data. Common
tasks include classification and regression.

2. Unsupervised Learning:
Unsupervised learning involves training an algorithm on data without labelled responses.
The system tries to learn the underlying structure of the data. Common tasks include
clustering and dimensionality reduction.

3. Reinforcement Learning:
In reinforcement learning, an agent learns to make decisions by performing actions in an
environment to maximize cumulative rewards. This is often used in robotics, game
playing, and other applications where sequential decision-making is crucial.

4. Deep Learning:
A subset of machine learning, deep learning utilizes neural networks with many layers
(deep neural networks) to model complex patterns in large datasets. This approach has
led to breakthroughs in image recognition, natural language processing, and other areas.
Applications:

Machine learning is transforming industries by enabling more efficient and accurate decision-
making. Some notable applications include:

1. Healthcare: Predicting disease outbreaks, personalizing treatment plans, and automating

medical imaging analysis.
2. Finance: Fraud detection, algorithmic trading, and risk management.
3. Transportation: Autonomous vehicles, traffic prediction, and route optimization.
4. Retail: Personalized recommendations, inventory management, and demand forecasting.
5. Entertainment: Content recommendation systems, user behaviour analysis, and
automated content creation.

Software Used:

In our machine learning project, we utilized Google Colab, a cloud-based platform that provides
a powerful and flexible environment for developing and executing machine learning models.
Google Colab offers several benefits:

1. Accessibility: Google Colab is accessible from any device with internet access, allowing
for seamless collaboration and sharing of notebooks.
2. Integration with Popular Libraries: Google Colab supports a wide range of popular
Python libraries used in machine learning, including TensorFlow, Keras, PyTorch, scikit-
learn, and many others.
3. Ease of Use: The platform is user-friendly and supports Markdown, making it easy to
document and share code, explanations, and results in a single notebook.
 NEURON

In machine learning, particularly in the realm of deep learning, neurons are the
fundamental building blocks of artificial neural networks (ANNs). Modelled loosely after the
biological neurons in the human brain, these artificial neurons are designed to process and
transmit information in a way that enables complex pattern recognition and decision-making
capabilities.

Biological Neuron:

Biological neurons are the basic units of the nervous system in living organisms, including
humans. The structure of a biological neuron consists of several components:

 Cell Body: The cell body contains the nucleus and other organelles necessary for the
neuron's metabolic functions.
 Myelin Sheath: The myelin sheath is a fatty, insulating layer that enhances the speed
and efficiency of nerve impulse transmission along axons.
 Dendrites: Dendrites are branching extensions of the cell body that receive signals
from other neurons or sensory receptors.
 Axon: The axon is a long, slender projection that conducts electrical impulses away
from the cell body and toward other neurons, muscles, or glands.
 Synapses: Synapses are the junctions between neurons where electrochemical
communication occurs. Neurotransmitters released from the axon terminals of one
neuron transmit signals to the dendrites or cell body of another neuron.
Artificial Neuron:

Artificial neurons, also known as perceptron or nodes, are the building blocks of artificial
neural networks (ANNs) used in machine learning and deep learning. These neurons are
designed to mimic the basic functionality of biological neurons, albeit in a simplified and
mathematical form. The structure of an artificial neuron typically includes:

 Inputs: Artificial neurons receive input signals, often represented as numerical

values, from other neurons or external sources.
 Weights: Each input signal is associated with a weight that determines its importance
or influence on the neuron's output. These weights are adjustable parameters that are
learned during the training process.
 Summation Function: The inputs are multiplied by their corresponding weights and
summed together. This process represents the weighted sum of inputs.
 Activation Function: The weighted sum of inputs is passed through an activation
function, which introduces non-linearity into the neuron's output. Common activation
functions include sigmoid, ReLU, and tanh.

Differences between Biological and Artificial Neuron:

 Physical Structure: Biological neurons have a complex three-dimensional structure,

including dendrites, axons, and synapses, which enable them to interact with other
neurons in intricate ways. In contrast, artificial neurons are typically represented as
mathematical constructs with simple input-output relationships.
 Processing Speed: Biological neurons operate at relatively slow speeds compared to
artificial neurons, which can perform computations at lightning-fast speeds,
especially when implemented on specialized hardware such as GPUs or TPUs.
 Learning Mechanism: Biological neurons learn and adapt through processes such as
synaptic plasticity, where the strength of connections between neurons is modified
based on experience. Artificial neurons learn through mathematical optimization
algorithms, such as gradient descent, which adjust the weights of connections
between neurons to minimize prediction errors.
 Energy Efficiency: Biological neurons are highly energy-efficient, consuming
minimal energy to transmit and process information. In contrast, artificial neurons
implemented on digital computers require significant amounts of energy to perform
computations, especially for complex neural network architectures.
 MULTI LAYER PERCEPTION

A Multilayer Perceptron (MLP) is a type of artificial neural network (ANN) that

consists of multiple layers of nodes, or neurons, arranged in a feedforward manner. It is one
of the most common and widely used architectures in machine learning and deep learning.
Here are some key aspects of MLPs:

1. Architecture of MLP:
 Input Layer: Receives input data, with each node representing a feature or attribute.
 Hidden Layers: Intermediate layers between the input and output layers. Each hidden
layer contains one or more neurons, and the number of hidden layers and neurons can
vary depending on the complexity of the problem.
 Output Layer: Produces the final output of the network, such as classification
probabilities or regression predictions. The number of nodes in the output layer
depends on the nature of the task.

2. Feedforward Process: The information flows in one direction, from the input layer
through the hidden layers to the output layer, without any cycles or loops. Each neuron
in a layer receives inputs from the neurons in the previous layer, performs a weighted
sum of these inputs, applies an activation function to the sum, and passes the result to the
neurons in the next layer.

3. Activation Functions: Nonlinear activation functions are applied to the weighted sum
of inputs in each neuron to introduce nonlinearity into the model and enable it to learn
complex patterns in the data. Common activation functions include sigmoid, tanh, ReLU
(Rectified Linear Unit), and softmax.

4. Training: MLPs are typically trained using the backpropagation algorithm, which
involves two main steps:
 Forward Propagation: Compute the output of the network for a given input by
propagating the input through the layers.
 Backpropagation: Compute the error between the predicted output and the true
output, and then propagate this error backward through the network to update the
weights using gradient descent optimization.
5. Applications:
MLPs are versatile and can be applied to a wide range of tasks, including:
 Classification: Image classification, text classification, sentiment analysis.
 Regression: Predictive modelling, time series forecasting.
 Pattern Recognition: Handwritten digit recognition, speech recognition.
 Reinforcement Learning: Building agents for game playing and robotics.

MLPs have been foundational in the development of deep learning and have paved the way
for more advanced architectures such as convolutional neural networks (CNNs) and recurrent
neural networks (RNNs). Despite their simplicity compared to these architectures, MLPs
remain powerful and effective for many machine learning tasks.
 FORWARD PROPAGATION

Forward propagation is a fundamental process in training neural networks, where

input data is processed through the network’s layers to produce predictions or outputs.
Understanding forward propagation is crucial for comprehending how neural networks make
predictions and learn from data.

Structure of Forward Propagation:

 Input Layer: The process begins with the input layer, where the raw input data is fed
into the network. Each input neuron represents a feature or attribute of the input data.
 Weighted Sum: In each subsequent layer, the input from the previous layer is
multiplied by weights associated with the connections between neurons. These
weighted sums are then summed together, often with the addition of a bias term.
 Activation Function: The weighted sum is passed through an activation function,
which introduces non-linearity into the network. Common activation functions
include sigmoid, ReLU, and tanh. This activation function determines whether a
neuron should be activated and to what extent.
 Output Layer: The process continues through the hidden layers of the network until
the output layer is reached. The output layer produces the final predictions or outputs
of the network based on the activations of the neurons in the preceding layers.
Mathematical Expression:

1. Multiplication: For each batch, we perform matrix multiplication Wi×Xi, where

Wi is the weight matrix corresponding to the 𝑖it batch and Xi is the input matrix for
that batch.
 Let 𝑍𝑖=𝑊𝑖×𝑋𝑖=Wi×Xi represent the result of the matrix multiplication.
2. Adding Bias: After the matrix multiplication, we add the bias matrix B element-wise
to the result 𝑍𝑖Zi.
 Let Yi=Zi+Bi represent the final output for the 𝑖 batch.
3. Activation Function:
 ReLU Function: R(x)=max (0, x)

 Sigmoid Function: The sigmoid function maps the incoming inputs to a range
between 0 and 1.

 Tanh Function: The Tanh function maps the incoming inputs to a range between -
1 and 1.

4. Final Expression:
 Yi=(Wi*Xi) + Bi
Where i represent the layer in which the operation is being executed.
Program:
import numpy as np
r1,c1=map(int,input(‘Enter then number of rows and columns of w matrix: ‘).split())
print(‘Enter the elements of matrix: ‘)
w1=np.zeros((r1,c1))
for I in range(r1):
for j in range(c1):
w1[i][j]=input(‘Enter: ‘)
print(w1)
r3=r1
c3=1
print(‘Enter the elements of bias matrix: ‘)
b1=np.zeros((r3,c3))
for I in range(r3):
for j in range(c3):
b1[i][j]=input(‘Enter: ‘)
print(b1)
r2,c2=map(int,input(‘Enter then number of rows and columns of batches: ‘).split())
print(‘Enter the elements of batch: ‘)
x1=np.zeros((r2,c2))
for I in range(r2):
for j in range(c2):
x1[i][j]=input(‘Enter: ‘)
print(x1)
h1=np.dot(w1,x1)+b1
print(‘result: ‘)
print(h1)
ch=int(input(‘Select the Operation :\n1.ReLU Function\n2.Sigmoid Function(3-
Bit)\n3.Sigmoid Function(5-Bit)\n4.Tanh Function(3-Bit)\n5.Tanh Function(5-Bit)\n’))
if ch==1:
def relu(a):
 return np.maximum(0,a) elif ch==4:
re_val=relu(h1) def tanh_f(x):
print(‘ReLU result: \n’) for I in range(0,len(x)):
print(re_val) if x[i]<=-2:
elif ch==2: print(‘-1’)
def sig(x): elif x[i]>=2:
for I in range(0,len(x)): print(‘1’)
if x[i]<=-2: else:
print(‘0’) print(‘0’)
elif x[i]>=2: print(‘TanH result:
print(‘1’) ‘) tanh_f(h1)
else: elif ch==5:
print(‘0.5’) def tanh_f(x):
print(‘Sigmoid result: ‘) for I in range(0,3):
sig(h1) if x[i]<=-2:
elif ch==3: print(‘-1’)
def sig(x): elif x[i]==-1:
for I in range(0,len(x)): print(‘-0.8’)
if x[i]<=-2: elif x[i]==0:
print(‘0’) print(‘0’)
elif x[i]==-1: elif x[i]==1:
print(‘0.3’) print(‘0.8’)
elif x[i]==0: else:
print(‘0.5’) print(‘1’)
elif x[i]==1: print(‘TanH result:
print(‘0.7’) ‘) tanh_f(h1)
else: else:
print(‘1’) print(‘Invalid Choice.’)
print(‘Sigmoid result: ‘)
sig(h1)
Output:
Enter then number of rows and columns of w matrix: 3 2
Enter the elements of matrix:
Enter: 1
Enter: 1
Enter: 1
Enter: 1
Enter: 1
Enter: 1
[[1. 1.]
[1. 1.]
[1. 1.]]
Enter the elements of bias matrix:
Enter: -2
Enter: -4
Enter: -6
[[-2.]
[-4.]
[-6.]]
Enter then number of rows and columns of batches: 2 1
Enter the elements of batch:
Enter: 3
Enter: 2
[[3.]
[2.]]
result:
[[ 3.]
[ 1.]
[-1.]]
Select the Operation:
1. ReLU Function
2. Sigmoid Function(3-Bit)
3. Sigmoid Function(5-Bit)
4. Tanh Function(3-Bit)
5. Tanh Function(5-Bit)
1
ReLU result:
[[3.]
[1.]
[0.]]
 BACK PROPAGATION

Back propagation is a critical process in training neural networks, involving the

adjustment of weights and biases based on the error of the network’s predictions. This
iterative process ensures that the network learns to minimize the prediction error over time.

Process of Back Propagation:

 Forward Propagation: Initially, input data is fed through the network to

generate an output prediction. This involves calculating the weighted sums,
adding biases, and applying activation functions.

 Loss: The loss, or error, is computed by comparing the predicted output to the
actual target values using a loss function (e.g., mean squared error, cross-entropy
loss).

 Backward Propagation: The gradients of the loss function with respect to the
weights and biases are computed using the chain rule of calculus. This involves
propagating the error backward through the network from the output layer to the
input layer.
Advantages:

 Ease of Implementation: Backpropagation does not require prior knowledge of

neural networks, making it accessible to beginners. Its straightforward nature
simplifies the programming process.

 Simplicity and Flexibility: The algorithm’s simplicity allows it to be applied to

a wide range of problems and network architectures. Its flexibility makes it
suitable for various scenarios.

 Efficiency: Backpropagation accelerates the learning process by directly

updating weights based on the calculated error derivatives. This efficiency is
particularly advantageous in training deep neural networks.

 Scalability: Backpropagation scales well with the size of the dataset and the
complexity of the network. This scalability makes it suitable for large-scale
machine learning tasks.

Mathematical Expression:
 New Weight: This is an optimization technique which used to find the minimum
value of the function.
 SoftMax Function: The softmax activation function forces the values of the
output neurons to take values between zero and one, so that they can represent
probability values in the interval [0, 1].

 Gradient:

Gradient with Respect to Weights w:

 The Weights of the input are kept at random values with respect to the input
values so that the input layer.

 The Hidden layer operation is implemented and the cost function is determined
from the Target output using the Obtained output.
  Also, the learning rate of the system can also be determined by using the alpha or
constant to the required value.

 Then, this cost function is added to the weights to modify them to get the target
output from the output layer when the loss will be very low or negligible.

Program:

import numpy as np
# X = (hours sleeping, hours studying), y = test score of the student
X = np.array(([2, 9], [1, 5], [3, 6]), dtype=float)
y = np.array(([92], [86], [89]), dtype=float)
# scale units
X = X/np.amax(X, axis=0) #maximum of X array
y = y/100 # maximum test score is 100
print(X)
class NeuralNetwork(object):
def init (self):
#parameters
self.inputSize = 2
self.outputSize = 1
self.hiddenSize = 3

#weights
self.W1 = np.random.randn(self.inputSize, self.hiddenSize) # (3x2) weight matrix from
input to hidden layer
 self.W2 = np.random.randn(self.hiddenSize, self.outputSize) # (3x1) weight matrix
from hidden to output layer
def 21robabiliti(self, X):
#forward propogation through the network
self.z = np.dot(X, self.W1) #dot product of X (input) and first set of weights (3x2)
self.z2 = self.sigmoid(self.z) #activation function
self.z3 = np.dot(self.z2, self.W2) #dot product of hidden layer (z2) and second set of
weights (3x1)
output = self.sigmoid(self.z3)
return output
print(output)
def sigmoid(self, s, deriv=False):
if (deriv == True):
return s * (1 – s)
return 1/(1 + np.exp(-s))
def backward(self, X, y, output):
#backward propogate through the network
self.output_error = y – output # error in output
self.output_delta = self.output_error * self.sigmoid(output, deriv=True)
self.z2_error = self.output_delta.dot(self.W2.T) #z2 error: how much our hidden layer
weights contribute to output error
self.z2_delta = self.z2_error * self.sigmoid(self.z2, deriv=True) #applying derivative of
sigmoid to z2 error
self.W1 += X.T.dot(self.z2_delta) # adjusting first set (input -> hidden) weights
self.W2 += self.z2.T.dot(self.output_delta) # adjusting second set (hidden -> output)
weights
def train(self, X, y):
output = self.feedForward(X)
self.backward(X, y, output)

NN = NeuralNetwork()

for I in range(1000): #trains the NN 1000 times

 if (I % 100 == 0):
print(“Loss: “ + str(np.mean(np.square(y – NN.feedForward(X)))))
NN.train(X, y)
print(“Input: “ + str(X))
print(“Actual Output: “ + str(y))
print(“Loss: “ + str(np.mean(np.square(y – NN.feedForward(X)))))
print(“\n”)
print(“Predicted Output: “ + str(NN.feedForward(X)))

Output:

[[0.66666667 1. ]
[0.33333333 0.55555556]
[1. 0.66666667]]

Loss: 0.009334108908074934
Loss: 0.0010916905194009302
Loss: 0.0010296010947815793

Loss: 0.0009833495674670032
Loss: 0.0009399137312401804
Loss: 0.0008989599161029326
Loss: 0.000860280141810026

Loss: 0.0008236997891859802
Loss: 0.0007890698311634454
Loss: 0.0007562613646175108

Input:
[[0.66666667 1. ]
[0.33333333 0.55555556]
[1. 0.66666667]]
Actual Output:

[[0.92]
[0.86]
[0.89]]

Loss: 0.0007251613933311722
Predicted Output:
[[0.88519048]
[0.88927613]

[0.90032907]]

Applications:
 One of the main applications of Back Propagation is that the machine can learn to
implement the given code and learn the operation based on the given conditions.

HANDWRITTEN DIGIT RECOGNITION:

 This is used to identify the given number which is in the form of Handwritten
digits to get the accurate number.
 Handwritten digit recognition using Python machine learning involves the
development of algorithms that can accurately identify handwritten digits from
images.
 This task is widely recognized as a fundamental problem in the field of computer
vision and has numerous real-world applications, including optical character
recognition (OCR), automatic form processing, and postal sorting.
 The process typically involves several key steps:

1. Data Acquisition: The first step is to collect a dataset of handwritten digits.

One of the most popular datasets used for this task is the MNIST dataset,
which contains a large number of grayscale images of handwritten digits
ranging from 0 to 9.
 2. Data Preprocessing: Before feeding the data into a machine learning model,
preprocessing steps are often applied to enhance the quality of the images and
make them suitable for training.

3. Feature Extraction: In the context of handwritten digit recognition, feature

extraction techniques may involve converting the pixel values of the images
into a format that can be easily processed by the model.

4. Model Training: Once the data is prepared, a machine learning model is

trained on the dataset to learn the underlying patterns that distinguish one digit
from another.

5. Model Evaluation: After training, the performance of the model is evaluated

using a separate validation dataset or through techniques such as cross-
validation.

6. Model Deployment: Once the model has been trained and evaluated, it can be
deployed into production environments where it can be used to recognize
handwritten digits in real-time applications.

 MNIST DATASET:

The MNIST dataset is a widely used benchmark dataset in the field of machine
learning and computer vision. It consists of a large collection of grayscale images of
handwritten digits, each labelled with the corresponding digit (0 through 9). The MNIST
dataset is commonly used for training and testing machine learning algorithms, particularly
for tasks such as digit recognition.
Here are some key characteristics of the MNIST dataset:

 Size: The dataset contains 70,000 images in total, divided into 60,000 training
images and 10,000 test images.

 Image Dimensions: Each image in the dataset is a grayscale image with

dimensions of 28 pixels by 28 pixels. Therefore, each image is represented as a
matrix of size 28x28.

 Labelling: Each image is associated with a label indicating the digit it represents.
The labels range from 0 to 9, corresponding to the digits 0 through 9.

 Usage: The MNIST dataset is commonly used as a benchmark dataset for

evaluating the performance of machine learning algorithms, particularly for tasks
such as classification and digit recognition.
  Accessibility: The MNIST dataset is freely available and widely used in research
and education. It can be easily accessed and downloaded from various sources,
including the official MNIST website and machine learning libraries such as
TensorFlow and scikit-learn.

 To import the MNIST dataset into your Python environment, you can use popular
machine learning libraries such as TensorFlow or scikit-learn, which provide convenient
functions for downloading and loading the dataset:

import tensorflow as tf

# Load the MNIST dataset using TensorFlow

mnist = tf.keras.datasets.mnist

# Split the dataset into training and testing sets

(x_train, y_train), (x_test, y_test) = mnist.load_data()

Program:
import tensorflow as tf
import matplotlib.pyplot as plt
import seaborn as sn
import numpy as np
import pandas as pd
import math
import datetime
import platform

# Load the TensorBoard notebook extension.

# %reload_ext tensorboard
#load_ext tensorboard
# Clear any logs from previous runs.
#rm -rf ./.logs/
mnist_dataset = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist_dataset.load_data()
print(‘x_train:’, x_train.shape)
print(‘y_train:’, y_train.shape)
print(‘x_test:’, x_test.shape)
print(‘y_test:’, y_test.shape)
pd.DataFrame(x_train[0])
plt.imshow(x_train[0], cmap=plt.cm.binary)
plt.show()
numbers_to_display = 25
num_cells = math.ceil(math.sqrt(numbers_to_display))
plt.Figure(figsize=(10,10))
for I in range(numbers_to_display):
plt.subplot(num_cells, num_cells, i+1)
plt.xticks([])
plt.yticks([])
plt.grid(False)
plt.imshow(x_train[i], cmap=plt.cm.binary)
plt.xlabel(y_train[i])
plt.show()
x_train_normalized = x_train / 255
x_test_normalized = x_test / 255
with pd.option_context(‘display.float_format’, ‘{:,.2f}’.format):
display(pd.DataFrame(x_train_normalized[0]))
plt.imshow(x_train_normalized[0], cmap=plt.cm.binary)
plt.show()
model = tf.keras.models.Sequential()

# Input layers.
Model.add(tf.keras.layers.Flatten(input_shape=x_train_normalized.shape[1:]))
model.add(tf.keras.layers.Dense(
units=128,
 activation=tf.keras.activations.relu,
kernel_regularizer=tf.keras.regularizers.l2(0.002)
))

# Hidden layers.
Model.add(tf.keras.layers.Dense(
units=128,
activation=tf.keras.activations.relu,
kernel_regularizer=tf.keras.regularizers.l2(0.002)
))

# Output layers.
Model.add(tf.keras.layers.Dense(
units=10,
activation=tf.keras.activations.softmax
))
tf.keras.utils.plot_model( mo
del, show_shapes=True,
show_layer_names=True,
)
adam_optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)

model.compile( optimizer=adam_opt
imizer,
loss=tf.keras.losses.sparse_categorical_crossentropy,
metrics=[‘accuracy’]
)
log_dir=”.logs/fit/” + datetime.datetime.now().strftime(“%Y%m%d-%H%M%S”)
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)

training_history = model.fit(
x_train_normalized,
y_train,
 epochs=10,
validation_data=(x_test_normalized, y_test),
callbacks=[tensorboard_callback]
)
plt.xlabel(‘Epoch Number’)
plt.ylabel(‘Loss’)
plt.plot(training_history.history[‘loss’], label=’training set’)
plt.plot(training_history.history[‘val_loss’], label=’test set’)
plt.legend()
plt.xlabel(‘Epoch Number’)
plt.ylabel(‘Accuracy’)
plt.plot(training_history.history[‘accuracy’], label=’training set’)
plt.plot(training_history.history[‘val_accuracy’], label=’test set’)
plt.legend()
##capture
train_loss, train_accuracy = model.evaluate(x_train_normalized, y_train)
print(‘Training loss: ‘, train_loss)
print(‘Training accuracy: ‘, train_accuracy)
##capture
validation_loss, validation_accuracy = model.evaluate(x_test_normalized, y_test)
print(‘Validation loss: ‘, validation_loss)
print(‘Validation accuracy: ‘, validation_accuracy)
model_name = ‘Back Propagation.h5’
model.save(model_name, save_format=’h5’)
loaded_model = tf.keras.models.load_model(model_name)
predictions_one_hot = loaded_model.predict([x_test_normalized])
print(‘predictions_one_hot:’, predictions_one_hot.shape)
# Predictions in form of one-hot vectors (arrays of probabilities).
Pd.DataFrame(predictions_one_hot)
# Let’s extract predictions with highest 29robabilities and detect what digits have been
actually recognized.
Predictions = np.argmax(predictions_one_hot, axis=1)
pd.DataFrame(predictions)
print(predictions[0])
plt.imshow(x_test_normalized[0], cmap=plt.cm.binary)
plt.show()
numbers_to_display = 196
num_cells = math.ceil(math.sqrt(numbers_to_display))
plt.figure(figsize=(15, 15))

for plot_index in range(numbers_to_display):

predicted_label = predictions[plot_index]
plt.xticks([])
plt.yticks([])
plt.grid(False)
color_map = ‘Greens’ if predicted_label == y_test[plot_index] else ‘Reds’
plt.subplot(num_cells, num_cells, plot_index + 1)
plt.imshow(x_test_normalized[plot_index], cmap=color_map)
plt.xlabel(predicted_label)

plt.subplots_adjust(hspace=1, wspace=0.5)
plt.show()

Output:
Python version: 3.10.12
Tensorflow version: 2.15.0
x_train: (60000, 28, 28)
y_train: (60000,)
x_test: (10000, 28, 28)
y_test: (10000,)

28 rows × 28 columns

Epoch 1/10
1875/1875 [==============================] – 12s 6ms/step – loss: 0.5130 –
accuracy: 0.9234 – val_loss: 0.3519 – val_accuracy: 0.9440

Epoch 2/10
1875/1875 [==============================] – 9s 5ms/step – loss: 0.3007 –
accuracy: 0.9549 – val_loss: 0.2929 – val_accuracy: 0.9533

Epoch 3/10
1875/1875 [==============================] – 11s 6ms/step – loss: 0.2605 –
accuracy: 0.9608 – val_loss: 0.2410 – val_accuracy: 0.9638
Epoch 4/10
1875/1875 [==============================] – 11s 6ms/step – loss: 0.2387 –
accuracy: 0.9637 – val_loss: 0.2252 – val_accuracy: 0.9659
Epoch 5/10
1875/1875 [==============================] – 11s 6ms/step – loss: 0.2238 –
accuracy: 0.9651 – val_loss: 0.2174 – val_accuracy: 0.9652
Epoch 6/10
1875/1875 [==============================] – 10s 5ms/step – loss: 0.2143 –
accuracy: 0.9671 – val_loss: 0.2042 – val_accuracy: 0.9684
Epoch 7/10
1875/1875 [==============================] – 14s 7ms/step – loss: 0.2062 –
accuracy: 0.9678 – val_loss: 0.1972 – val_accuracy: 0.9709
Epoch 8/10
1875/1875 [==============================] – 12s 6ms/step – loss: 0.2014 –
accuracy: 0.9674 – val_loss: 0.2012 – val_accuracy: 0.9673
Epoch 9/10
1875/1875 [==============================] – 18s 10ms/step – loss: 0.1963 –
accuracy: 0.9688 – val_loss: 0.1949 – val_accuracy: 0.9688
Epoch 10/10
1875/1875 [==============================] – 10s 5ms/step – loss: 0.1927 –
accuracy: 0.9691 – val_loss: 0.2001 – val_accuracy: 0.9673
1875/1875 [==============================] – 4s 2ms/step – loss: 0.1875 –
accuracy: 0.9714
Training loss: 0.18751470744609833
Training accuracy: 0.9714000225067139
313/313 [==============================] – 1s 3ms/step – loss: 0.2001 – accuracy:
0.9673
Validation loss: 0.20008863508701324
Validation accuracy: 0.9672999978065491
 PROGRAMMING USING PYNQ BOARD

PYNQ (Python Productivity for Zynq) is an open-source project from Xilinx that aims to
make it easier to program FPGA-based systems using Python. PYNQ is a platform that
allows you to harness the power of FPGAs using the simplicity and flexibility of Python,
making it easier for developers and researchers to create high-performance applications.

Typical Applications of PYNQ:

 Machine Learning: Accelerating inference tasks by offloading computation to FPGA.

 Image and Video Processing: Real-time image and video processing using hardware-
accelerated functions.
 Embedded Systems: Developing embedded applications that require a mix of
software flexibility and hardware performance.
 IoT: Implementing edge computing solutions with high performance and low latency.

Setting Up the PYNQ-Z2 Board:

This guide will walk you through the steps to set up your PYNQ-Z2 board using a
micro-USB cable, an Ethernet cable, and an SD card.
1. Gather Required Components
Before you begin, make sure you have the following components:

 PYNQ-Z2 board
 Micro USB cable
 Ethernet cable
 SD card
 Computer with an Internet connection
 Insert the SD card into the SD card slot on the PYNQ-Z2 board.
2. Connect the PYNQ-Z2 Board to Your
Computer Power the Board:

 Connect the micro-USB cable to the micro-USB port on the PYNQ-Z2

board.
  Plug the other end of the micro-USB cable into a USB port on your
computer or a USB power adapter. This will power the board.
Connect to the Network:

 Connect one end of the Ethernet cable to the Ethernet port on the
PYNQ-Z2 board.
 Connect the other end of the Ethernet cable to your router or switch.
Access the PYNQ-Z2 Board:
 The PYNQ-Z2 board will boot from the SD card. Wait a few moments
for the board to initialize.

4. Find the IP Address of the PYNQ-Z2 Board

To interact with the PYNQ-Z2 board, you need to know its IP address.
Using Router Interface:

 Access your router’s web interface and check the list of connected
devices. Look for the PYNQ-Z2 board and note its IP address.
Using USB Connection:

 If your computer has Bonjour services installed (common on macOS),

you can try accessing the board directly using the address
http://192.168.2.99:9090.
 Alternatively, on Windows, you can use the terminal command arp -a
to list network devices and identify the PYNQ-Z2 board by its MAC
address.
5. Access the Jupyter Notebook Interface
Open a Web Browser:
 Open a web browser on your
computer. Enter the IP Address:

 Enter the IP address of the PYNQ-Z2 board into the browser's address
bar, followed by :9090. For example, if the IP address is 192.168.1.99,
enter http://192.168.1.99:9090.
Log In:

 The Jupyter Notebook login page will appear. The default password is
“xilinx”.

6. Verify the Setup

Create a New Notebook:
  Once logged in, you can create a new Python 3 notebook by clicking
on "New" and selecting "Python 3".
Run a Simple Command:

 In the new notebook, run a simple command like print ("Hello,

PYNQ!") to verify that the board is set up correctly.

Troubleshooting:
1. No Power: Ensure the micro-USB cable is securely connected to both the PYNQ-Z2
board and the power source.

2. No Network Connection: Check that the Ethernet cable is properly connected and
your router is configured to assign IP addresses via DHCP.

3. Cannot Access Jupyter Interface: Double-check the IP address of the PYNQ-Z2

board and ensure your firewall or antivirus software is not blocking the connection.
`
Program 1:
import numpy as np
r1,c1=map(int,input(‘Enter then number of rows and columns of w matrix: ‘).split())
print(‘Enter the elements of matrix: ‘)
w1=np.zeros((r1,c1))
for I in range(r1):
for j in range(c1):
w1[i][j]=input(‘Enter: ‘)
print(w1)
r3=r1
c3=1
print(‘Enter the elements of bias matrix: ‘)
b1=np.zeros((r3,c3))
for I in range(r3):
for j in range(c3):
b1[i][j]=input(‘Enter: ‘)
print(b1)
r2,c2=map(int,input(‘Enter then number of rows and columns of batches: ‘).split())
print(‘Enter the elements of batch: ‘)
x1=np.zeros((r2,c2))
for I in range(r2):
for j in range(c2):
x1[i][j]=input(‘Enter: ‘)
print(x1)
h1=np.dot(w1,x1)+b1
print(‘result: ‘)
print(h1)
ch=int(input(‘Select the Operation :\n1.ReLU Function\n2.Sigmoid Function(3-
Bit)\n3.Sigmoid Function(5-Bit)\n4.Tanh Function(3-Bit)\n5.Tanh Function(5-Bit)\n’))
if ch==1:
def relu(a):
 return np.maximum(0,a) elif ch==4:
re_val=relu(h1) def tanh_f(x):
print(‘ReLU result: \n’) for I in range(0,len(x)):
print(re_val) if x[i]<=-2:
elif ch==2: print(‘-1’)
def sig(x): elif x[i]>=2:
for I in range(0,len(x)): print(‘1’)
if x[i]<=-2: else:
print(‘0’) print(‘0’)
elif x[i]>=2: print(‘TanH result:
print(‘1’) ‘) tanh_f(h1)
else: elif ch==5:
print(‘0.5’) def tanh_f(x):
print(‘Sigmoid result: ‘) for I in range(0,3):
sig(h1) if x[i]<=-2:
elif ch==3: print(‘-1’)
def sig(x): elif x[i]==-1:
for I in range(0,len(x)): print(‘-0.8’)
if x[i]<=-2: elif x[i]==0:
print(‘0’) print(‘0’)
elif x[i]==-1: elif x[i]==1:
print(‘0.3’) print(‘0.8’)
elif x[i]==0: else:
print(‘0.5’) print(‘1’)
elif x[i]==1: print(‘TanH result:
print(‘0.7’) ‘) tanh_f(h1)
else: else:
print(‘1’) print(‘Invalid Choice.’)
print(‘Sigmoid result: ‘)
sig(h1)
Program 2:

# Import necessary libraries from PYNQ

from pynq import Overlay
from pynq.lib import AxiGPIO
import time

# Load the base overlay

overlay = Overlay("base.bit")

# Initialize the GPIO for LED (connected to MIO pin 7)

led = overlay.leds[4]

# Blink the LED 10 times

for i in range(10):
led.on()
time.sleep(0.5)
led.off()
time.sleep(0.5)
 CONCLUSION

In conclusion, our exploration into machine learning using Python, incorporating

forward and backpropagation techniques alongside handwritten digit recognition programs,
exemplifies the powerful synergy between cutting-edge algorithms and practical
applications. By harnessing the computational capabilities of the PYNQ board, we have
embarked on a journey to bridge the gap between theory and implementation, fostering
innovation at the intersection of hardware and software.

Leveraging Python's rich ecosystem of libraries such as TensorFlow and scikit-learn, the
convergence of machine learning and embedded systems holds immense promise for
revolutionizing industries ranging from healthcare and finance to robotics and IoT. By
embracing the versatility of Python and the computational prowess of the PYNQ board, we
are eager to unlock new frontiers in intelligent systems, driving innovation and shaping the
digital landscape for generations to come.