AI:

INTELLIGENCE
BEYOND
HUMAN
AI is the simulation of human
intelligence in machines that
are programmed to think and
learn like humans, but with far
greater speed and accuracy
AI: REVOLUTIONIZING INDUSTRIES
• Analyze medical data, provide better diagnoses, recommend
Healthcare treatment plan

Finance • Detect fraud, predict stock prices, automate financial process

Education • Personalize learning, identify at-risk students, interactive learning

Transportation • Autonomous vehicle, optimize logistic, reduce traffic congestion

Customer service • Automate customer support, improve service satisfaction

 MACHINE
LEARNING: THE
ENGINE
POWERING AI

Machine learning is a subfield of AI

that enables machines to learn from
data without being explicitly
programmed.
It is the key technology behind many
AI applications.
What is
learning?
“Learning is any process by which a
system improves performance from
experience” – Herbert Simon
How Human and
Machine Learns?
Similarities

• Learn
• Experience
• Feedback

Differences

• General vs. specific learning

• Small vs. big data
• Learning speed
Designing a
Learning
System
Simple Learning Experience:
Teachable Machine

https://teachablemachine.withgoogle.com/
Training vs. Test Distribution
Generally, assume that the training and test examples are
independently drawn from the same overall distribution of
data. IID: Independently and identically distributed

If examples are not independent, requires collective

classification.

If test distribution is different, requires transfer learning.

Choosing a Target
Function
• Target function is the function that the
learning system is trying to learn or
approximate from the training data.
• Factors to consider when choosing a target
function
• Accuracy
• Complexity
• Interpretability
• Scalability
 Representing the
Target Function
• Target function can be represented in many
ways: lookup table, symbolic rules, numerical
function, neural network.
• There is a trade-off between the expressiveness of
a representation and the ease of learning.
• The more expressive a representation, the better it
will be at approximating an arbitrary function;
however, the more examples will be needed to
learn an accurate function.
13
 Learning Algorithm
• Uses training values for the target
function to induce a hypothesized
definition that fits these examples and
hopefully generalizes to unseen
examples.
• In statistics, learning to approximate a
continuous function is called regression.
• Attempts to minimize some measure of
error (loss function) such as mean squared
error.
 For each training
Initialize the weights of example, compute the
the model to small predicted output using
random values the current weights of
Least Mean the model

Square (LMS)
Algorithm Compute the error
between the predicted
Update the weights of
the model using the
output and the actual following rule:
A type of gradient descent output for the current
example wᵢ ← wᵢ + α(y - ȳ)xᵢ
algorithm that minimizes the
mean squared error (MSE)
between the predicted output
and the actual output for each
training example. Repeat steps 2-4 for all
training examples until
convergence or a
maximum number of
iterations is reached
 LMS Discussion
If the predicted output for an example is correct, make no change to the
No changes weights of the model.

If the predicted output is too high, lower the weights proportional to the
Lower values of their corresponding features, so the overall output decreases.

If the predicted output is too low, increase the weights proportional to the
Increase values of their corresponding features, so the overall output increases.
 Lesson Learned about
Learning
• Learning can be viewed as using direct or
indirect experience to approximate a chosen
target function.
• Function approximation can be viewed as a
search through a space of hypotheses
(representations of functions) for one that best
fits a set of training data.
• Different learning methods assume different
hypothesis spaces (representation languages)
and/or employ different search techniques.
 Linear regression
Numerical functions Neural networks
Support vector machines

Decision trees
Symbolic functions Rules in propositional logic
Rules in first-order predicate logic
Various
Function
Representatio
Instance-based
functions
Nearest-neighbor
Case-based ns
Naïve Bayes
Probabilistic Bayesian networks
Hidden-Markov Models (HMMs)
Graphical Probabilistic Context Free Grammars
Models (PCFGs)
Markov networks

18
 Gradient • Perceptron
descent • Backpropagation

Various Dynamic
Programming
• HMM Learning
• PCFG Learning
Search
Algorith Divide and • Decision tree induction
ms Conquer • Rule learning

• Genetic Algorithms (GAs)

Evolutionary • Genetic Programming (GP)
Computation • Neuro-evolution

19
 Evaluation of Learning Systems
• Conduct controlled cross-validation experiments to compare various methods on a variety
of benchmark datasets.
Expe • Gather data on their performance, e.g. test accuracy, training-time, testing-time.
rime • Analyze differences for statistical significance.
ntal

• Analyze algorithms mathematically and prove theorems about their:

• Computational complexity
Theo • Ability to fit training data
retic • Sample complexity (number of training examples needed to learn an accurate function)
al

20