Department of Computer Science

Chapter 6: Learning Agents

16-Jan-23

1
 Chapter’s Outline
• Introduction
• Learning from Examples/Observation
• Knowledge in Learning
• Learning Probabilistic Models
• Neural Networks

• Learning is the process by which one entity acquires

2
 Introduction
• When all intelligence in an agent has been built-in by
the agent's designer, the agent does the best given the
way it was programmed to act.
– But this is not necessarily the best approach for the
agent with incomplete knowledge of its environment.
• Whenever the designer has incomplete knowledge of the
environment that the agent will live in, learning is the
only way that the agent can acquire what it needs to
know.
– Learning provides autonomy.
– Learning provides a good way to build high
performance systems by giving a learning system
experience in the application domain.

3
 Introduction(cont’d…)
• The idea behind learning is that percepts should be
used not only for acting, but also for improving the
agent's ability to act in the future.
• Learning takes place as a result of the interaction
between the agent and the world, and from
observation by the agent of its own decision-making
processes.
– Learning can range from trivial memorization of
experience to the creation of entire scientific
theories.

4
 Learning
Learning from agent
Examples/Observation
• Learning from example is the process of acquiring generalized
knowledge from examples or instances of some classes.
• As human learn from past experiences, a computer does not
have “experiences”.
– A computer system learns from data, which represent
some “past experiences” of an application domain.
– Learning agent is able to perform tasks, analyze
performance, and look for new ways to improve its
performance by itself by considering its past experiences.
• Learning agent starts with some basic knowledge and then
able to act and adapt autonomously, through learning, to
improve its own performance.

5
Learning from Examples/Observation(cont’d…)
• Machine learning: is the subfield of AI concerned with
programs that automatically learn from experience without
being explicitly programmed.
– Learning is the execution of a computer program to
optimize the parameters of a model using the training
data or past experience.
– The model may be:
• Predictive: to make predictions in the future, or
• Descriptive: to gain knowledge from data, or both.
• Machine learning uses the theory of statistics in building
mathematical models, because the core task is making
inference from a sample.

6
Learning from Examples/Observation(cont’d…)
• Based on data types machine learning could be:
Supervised or Unsupervised.
• Supervised Machine learning:
– learns from examples.
– Supervision: the data (observations, measurements,
etc.) are labeled with pre-defined classes.
– Test data are classified into these classes too.
• Unsupervised Machine learning
– The data have no target attribute.
– We want to explore the data to find some intrinsic
structures in them.

7
Learning from Examples/Observation(cont’d…)
• Data:
– a set of data records: also called examples, instances or
cases. It is described by k attributes: A1, A2, … Ak and
– a class: each example is labelled with a pre-defined class.
• Goal: to learn a classification model from the data that can be
used to predict the classes of new (future or test)
cases/instances.
• Example: A bank receives thousands of applications for loan.
Each application contains information about an applicant, age,
Marital status, annual salary, outstanding debts, credit rating,
etc.
• Problem: to decide whether an application should approved,
or to classify applications into two categories, approved(Yes)
and not approved(No).
8
Learning from Examples/Observation(cont’d…)

9
Learning from Examples/Observation(cont’d…)
1. Learn a classification model from the data
2. Use the model to classify future loan applications
into
– Yes (approved) and
– No (not approved)
• What is the class for following case/instance?

10
 Learning from Examples/Observation(cont’d…)
• Supervised machine learning has two steps:
– Learning (training): Learn a model using the training data
– Testing: Test the model using unseen test data to assess
the model accuracy
Number of correct classifications
Accuracy 
Total number of test cases

11
Learning from Examples/Observation(cont’d…)
• Given a data set D, a task T, and a performance measure M,
– A computer system is said to learn from D to perform the
task T if after learning the system’s performance on T
improves as measured by M.
– In other words, the learned model helps the system to
perform T better as compared to no learning.
• Example
– Data: Loan application data
– Task: Predict whether a loan should be approved or not.
– Performance measure: accuracy.
No learning: classify all future applications (test data) to
the majority class (i.e., Yes): Accuracy = 9/15 = 60%.
We can do better than 60% with learning.
12
Learning from Examples/Observation(cont’d…)
• Fundamental assumption of learning from examples:
– The distribution of training examples is identical
to the distribution of test examples (including
future unseen examples).
• In practice, this assumption is often violated to
certain degree.
• Strong violations will clearly result in poor
classification accuracy.
• To achieve good accuracy on the test data, training
examples must be sufficiently representative of
the test data.

13
 Example
• Consider the following training set

• Show all necessary steps to determine the class of test set

14
 Example(cont’d…)
• Naïve Bayes: classifier is a powerful classification
algorithm.
– It assumes that the presence of a feature in a class
is unrelated to any other feature (i.e., each feature
makes an independent and equal contribution to
the outcome).
– Rule: 𝑷 𝒙 𝒚 ∗ 𝑷(𝒚)
𝑷(𝒚|𝒙) =
𝑷(𝒙)

– Where
• y represents class and
• x represents conditions
15
 Example(cont’d…)
• From training set, we can generate the following
information.

16
 Example(cont’d…)
• Probability of play is given by:

and probability to not play is given by:

• Since, P(today) is common in both probabilities, we can

17
 Example(cont’d…)
• Since, P yes toda𝑦 + 𝑃 𝑛𝑜 𝑡𝑜𝑑𝑎𝑦 = 1, but if we
add the number we have, we cannot get it.
– Thus, these numbers can be converted into a probability
by making the sum equal to 1 (normalization):

• Since 𝑷 𝒚𝒆𝒔 𝒕𝒐𝒅𝒂𝒚 > 𝑷 𝒏𝒐 𝒕𝒐𝒅𝒂𝒚 , playing of

18
 Knowledge in Learning
• Knowledge in learning comes from different sources:
– Experts
– Books or textual files
– Direct experiences

19
 Learning Probabilistic Models
• Agents can handle uncertainty by using the
methods of probability and decision theory,
– but first they must learn their probabilistic
theories of the world from experience.

• Read chapter 5 material for more about

20
 Neural Networks
• Neural networks are artificial systems that were inspired
by biological neural networks.
• A neural network is a system that is designed to operate
like a human brain.
• A neural network is usually described as having different
layers.
– Input layer: picks up the input signals and passes
them to the next layer.
– Hidden layer: does all kinds of calculations and
feature extractions.
• Often, there will be more than one hidden layer.
– Output layer: delivers the final result.
21
 Activation functions
• Activation functions transform the weighted sum of
inputs that goes into the artificial neurons.
– These functions should be non-linear to encode
complex patterns of the data.
• The most popular activation functions are:
– Sigmoid
– Tanh and
– ReLU.

22
 Activation functions(cont’d…)
• Sigmoid Function: is used when the model is
predicting probability.

• Threshold Function: is used when we don’t want to

23
 Neural Networks(cont’d…)
• The training process consists of the following steps:
• Forward Propagation: take the inputs, multiply by the
weights (just use random numbers as weights), pass the
result through a sigmoid formula to calculate the
neuron’s output.
– The Sigmoid function is used to normalize the result
between 0 and 1: 1/(1 + e-x)
• Back Propagation: calculate the error (i.e., the difference
between the actual output and the expected output.
– Depending on the error, adjust the weights by
multiplying the error with the input and again with
the gradient of the Sigmoid curve:
• Weight += Error Input Output (1-Output), here
Output (1-Output) is derivative of sigmoid curve.

24
How Neural Network Works?

25
 Neural networks(cont’d…)
• Example: consider the following neural network
structure with the input X1 and X2:

• Calculate the value of Z when activation function is

26
 Types of Neural Networks
• Feed-forward Neural Network
• Recurrent Neural Network
• Convolution Neural Network
• Modular Neural Network
And many more

• Reading assignment:
– Read working nature of different types of neural
networks

27
Thank You!

28