COS 102: Problem Solving (2 Units C: LH 30; PH 0)

Learning Outcomes

At the end of this course, students should be able to:

1. Explain problem solving processes;

2. Demonstrate problem solving skills;

3. Describe the concept of algorithms development and properties of algorithms;

4. Discuss the solution techniques of solving problem;

5. Solve computer problems using algorithms, flowcharts, pseudocode; etc.; and

6. Solve problems using programming language using Java, PYTHON, etc.

Course Contents

1. Introduction to the core concepts of computing.

2. Problems and problem-solving. The identification of problems and types of problems
(routine problems and non-routine problems).
3. Method of solving computing problems (introduction to algorithms and heuristics).
4. Solvable and unsolvable problems.
5. Solution techniques of solving problems (abstraction, analogy, brainstorming, trial
and error, hypothesis testing, reduction, literal thinking, means-end analysis, method
of focal object, morphological analysis, research, root cause analysis, proof, divide
and conquer).
6. General Problem-solving process.
7. Solution formulation and design: flowchart, pseudocode, decision table, decision tree.
8. Implementation, evaluation and refinement. Programming in C, Python etc.
Lab Work: Use of simple tools for algorithms and flowcharts; writing pseudocode; writing
assignment statements, input-output statements and condition statements; demonstrating
simple programs using any programming language (Visual Basic, Python, C)

1. CORE CONCEPT OF COMPUTING

1
The core concept of computing revolves around the automatic processing of data to solve
problems using machines (computers). It is built on several fundamental principles,
including:

1. Computation

 The process of performing mathematical or logical operations on data.

 Based on algorithms – step-by-step procedures for solving problems.

2. Data and Information

 Data: Raw facts and figures (e.g., numbers, text).

 Information: Processed data that has meaning and context.

3. Algorithms

 A finite sequence of well-defined steps or instructions to solve a problem or perform a

task.
 Must be correct, efficient, and unambiguous.

4. Programming

 Writing instructions (code) that a computer can execute.

 Languages like Python, Java, and C++ are used to implement algorithms.

5. Hardware and Software

 Hardware: The physical components of a computer (CPU, memory, storage, etc.).

 Software: The programs and operating systems that tell the hardware what to do.

6. Input → Process → Output (IPO Model)

 A core model in computing:

o Input: Receiving data (e.g., via keyboard, sensors).
o Process: Performing operations on the data.
o Output: Producing results (e.g., display, print, save).

7. Abstraction

2
  Hiding complex details to focus on high-level operations i.e. hiding complex
implementation details and presenting a simpler interface to the user or programmer.
 Used in programming, hardware design, and problem-solving.

8. Efficiency

 Concerned with time (speed) and space (memory usage).

 Goal: solve problems quickly using minimal resources.

9. Security and Ethics

 Ensuring data privacy, integrity, and ethical use of computing systems.

10. Interconnectivity and Communication

 Computers communicate over networks (e.g., the Internet).

 Key for distributed systems, cloud computing, and modern applications.

These concepts are the foundation of fields like software engineering, data science, artificial
intelligence, and computer systems.

2. PROBLEMS AND PROBLEM-SOLVING:

Computer science is all about solving problems with computers. The problems that we want
to solve can come from any real-world problem or perhaps even from the abstract world. We
need to have a standard systematic approach to solving problems. Since we will be using
computers to solve problems, it is important to first understand the computer’s information
processing model. The model shown in the diagram below assumes a single CPU (Central
Processing Unit). Many computers today have multiple CPUs, so you can imagine the above
model duplicated multiple times within the computer.

3
A typical single CPU computer processes information as shown in the diagram. Problems are
solved using a computer by obtaining some kind of user input (e.g., keyboard/mouse
information or game control movements), then processing the input and producing some kind
of output (e.g., images, text or sound). Sometimes the incoming and outgoing data may be in
the form of hard drives or network devices.

4
In regards to problem solving, we will apply the above model such that we will assume that
we are given some kind of input information that we need to work with in order to produce
some desired output solution. However, the above model is quite simplified. For larger and
more complex problems, we need to iterate (i.e., repeat) the input/process/output stages
multiple times in sequence, producing intermediate results along the way that solve part of
our problem, but not necessarily the whole problem. For simple computations, the above
model is sufficient.

It is the “problem solving” part of the process that is the interesting part, so we’ll break this
down a little. There are many definitions for “problem solving”. Here is one:

Problem Solving is the sequential process of analysing information related to a given

situation and generating appropriate response options. Or better still,

Problem-solving is a cognitive and analytical process of identifying a problem, generating

possible solutions, evaluating and selecting the best option, and implementing it to achieve a
desired outcome.

It is a core skill in mathematics, engineering, programming, business, and everyday decision-

making.

There are 6 steps to be follow in order to solve a problem:

1. Understand the Problem

2. Formulate a Model

3. Develop an Algorithm

4. Write the Program

5. Test the Program

6. Evaluate the Solution

Consider a simple example of how the input/process/output works on a simple problem:

Example: Calculate the average grade for all students in a class.

1. Input: get all the grades … perhaps by typing them in via the keyboard or by reading them
from a USB flash drive or hard disk.

2. Process: add them all up and compute the average grade.

5
3. Output: output the answer to either the monitor, to the printer, to the USB flash drive or
hard disk … or a combination of any of these devices.

As you can see, the problem is easily solved by simply getting the input, computing
something and producing the output. Let us now examine the 6 steps to problems solving
within the context of the above example.

STEP 1: Understand the Problem: It sounds strange, but the first step to solving any
problem is to make sure that you understand the problem that you are trying to solve. You
need to know:

 Recognize that a problem exists.

 What input data/information is available?
 What does it represent?
 What format is it in?
 Is anything missing?
 Do I have everything that I need?
 What output information am I trying to produce?
 What do I want the result to look like … text, a picture, a graph …?
 What am I going to have to compute?

Tip: Ask “What is the actual problem?” or “Why is this a problem?”

In our example, we well understand that the input is a bunch of grades. But we need to
understand the format of the grades. Each grade might be a number from 0 to 100 or it may
be a letter grade from A to F. If it is a number, the grade might be a whole integer like 73
or it may be a real number like 73.42. We need to understand the format of the grades in
order to solve the problem.

We also need to consider missing grades. What if we do not have the grade for every student
(e.g., some were away during the test)? Do we want to be able to include that person in our
average (i.e., they received 0) or ignore them when computing the average?

We also need to understand what the output should be. Again, there is a formatting issue.
Should we output a whole or real number or a letter grade? Maybe we want to display a pie
chart with the average grade. It is our choice.

6
Finally, we should understand the kind of processing that needs to be performed on the data.
This leads to the next step.

STEP 2: Formulate a Model:

Now we need to understand the processing part of the problem. Many problems break down
into smaller problems that require some kind of simple mathematical computations in order
to process the data. In our example, we are going to compute the average of the incoming
grades. So, we need to know the model (or formula) for computing the average of a bunch of
numbers. If there is no such “formula”, we need to develop one. Often, however, the problem
breaks down into simple computations that we well understand. Sometimes, we can look up
certain formulas in a book or online if we get stuck.
In order to come up with a model, we need to fully understand the information available to us
by:

 Collecting all relevant data and background.

 Breaking the problem into smaller components if needed.
Assuming that the input data is a bunch of integers or real numbers X1, X2,…, Xn
representing a grade percentage, we can use the following computational model:
Average1 = (X1 + X2 + X3 + … + Xn) / N
where the result will be a number from 0 to 100.
That is very straight forward (assuming that we knew the formula for computing the average
of a bunch of numbers). However, this approach will not work if the input data is a set of
letter grades like B, C, A, F, D, etc because we cannot perform addition and division on the
letters. This problem solving step must figure out a way to produce an average from such
letters. Thinking is required.
Until we decide to assign an integer number to the incoming letters, the problem will remain
unsolved.
Another example is a case of a quadratic equation involving two variables, where one can be
obtain after finding the value of the other.
STEP 3: Develop an Algorithm:

Now that we understand the problem and have formulated a model, it is time to come up with
a precise plan of what we want the computer to do.

An algorithm is a precise sequence of instructions for solving a problem.

7
Some of the more complex algorithms may be considered “randomized algorithms” or
“non-deterministic algorithms” where the instructions are not necessarily in sequence and
in may not even have a finite number of instructions. However, the above definition will
apply for all algorithms that we will discuss in this course.

To develop an algorithm, we need to represent the instructions in some way that is

understandable to a person who is trying to figure out the steps involved.

Two commonly used representations for an algorithm is by using

 pseudo code
 flow charts.

Consider the following example (from Wikipedia) of solving the problem of a broken lamp.
To the right is an example of a flow chart, while to the left is an example of pseudocode for
solving the same problem:

Notice that:

pseudocode is a simple and concise sequence of English-like instructions to solve a problem.

Pseudocode is often used as a way of describing a computer program to someone who
doesn’t understand how to program a computer. When learning to program, it is important to
write pseudocode because it helps you clearly understand the problem that you are trying to
solve. It also helps you avoid getting bogged down with syntax details (i.e., like spelling
mistakes) when you write your program later (see step 4).

Although flowcharts can be visually appealing, pseudocode is often the preferred choice for
algorithm development because:

 It can be difficult to draw a flowchart neatly, especially when mistakes are made.
 Pseudocode fits more easily on a page of paper.

8
  Pseudocode can be written in a way that is very close to real program code, making it
easier later to write the program (i.e., in step 4).
 Pseudocode takes less time to write than drawing a flowchart.

Pseudocode will vary according to whoever writes it. That is, one person’s pseudocode is
often quite different from that of another person. However, there are some common control
structures (i.e., features) that appear whenever we write pseudocode.

9
10
11
STEP 4: Write the Program:
Now that we have a precise set of steps for solving the problem, most of the hard work has
been done. We now have to transform the algorithm from step 3 into a set of instructions that
can be understood by the computer.
Writing a program is often called "writing code" or “implementing an algorithm”. So the
code (or source code) is actually the program itself.

Without much of an explanation, below is a program (written in processing) that implements

our algorithm for finding the average of a set of grades. Notice that the code looks quite
similar in structure; however, the processing code is less readable and seems somewhat more
mathematical:

For now, we will not discuss the details of how to produce the above source code. In fact, the
source code would vary depending on the programming language that was used. Learning a
programming language may seem difficult at first, but it will become easier with practice.
The computer requires precise instructions in order to understand what you are asking it to
do. For example, if you removed one of the semi-colon characters ( ; ) from the program
above, the computer would become confused as to what you are doing because the ( ; ) is
what it understands to be the end of an instruction. Leaving one of them off will cause your
program to generate what is known as a compile error.

Compiling is the process of converting a program into instructions that can be understood by
the computer.

The longer your program becomes, the more likely you will have multiple compile errors.
You need to fix all such compile errors before continuing on to the next step.

12
STEP 5: Test the Program:

Once you have a program written that compiles, you need to make sure that it solves the
problem that it was intended to solve and that the solutions are correct.

Running a program is the process of telling the computer to evaluate the compiled
instructions.

When you run your program, if all is well, you should see the correct output. It is possible
however, that your program works correctly for some set of data input but not for all. If the
output of your program is incorrect, it is possible that you did not convert your algorithm
properly into a proper program. It is also possible that you did not produce a proper algorithm
back in step 3 that handles all situations that could arise. Maybe you performed some
instructions out of sequence. Whatever happened, such problems with your program are
known as bugs.

Bugs are problems/errors with a program that cause it to stop working or produce incorrect or
undesirable results.

You should fix as many bugs in your program as you can find. To find bugs effectively, you
should test your program with many test cases (called a test suite). It is also a good idea to
have others test your program because they may think up situations or input data that you
may never have thought of.

The process of finding and fixing errors in your code is called debugging and it is often a
very time-consuming “chore” when it comes to being a programmer. If you take your time to
carefully follow problem solving steps 1 through 3, this should greatly reduce the amount of
bugs in your programs and it should make debugging much easier.

STEP 6: Evaluate the Solution:

Once your program produces a result that seems correct, you need to re-consider the original
problem and make sure that the answer is formatted into a proper solution to the problem. It
is often the case that you realize that your program solution does not solve the problem the
way that you wanted it to. You may realize that more steps are involved.

For example, if the result of your program is a long list of numbers, but your intent was to
determine a pattern in the numbers or to identify some feature from the data, then simply
producing a list of numbers may not suffice. There may be a need to display the information

13
in a way that helps you visualize or interpret the results with respect to the problem. Perhaps
a chart or graph is needed.

It is also possible that when you examine your results, you realize that you need additional
data to fully solve the problem. Or, perhaps you need to adjust the results to solve the
problem more efficiently (e.g., your game is too slow).

It is important to remember that the computer will only do what you told it to do. It is up to
you to interpret the results in a meaningful way and determine whether or not it solves the
original problem. It may be necessary to re-do some of the steps again, perhaps going as far
back as step 1 again, if data was missing.

So there you have it. Those are the 6 steps that you should follow in order to solve problems
using computers. Throughout the course, you should try to use this approach for all of your
assignments. It is a good idea to practice problem solving to make sure that you understand
the process. Below are some practice exercises that will help you practice the first 3 steps of
the problem solving process. Later, you will gain experience with steps 4 through 6.

TYPES OF PROBLEMS

There are basically two types of problems in Computing and General Problem Solving;
routine and non-routine problems
Routine Problems
These are problems that have clear, known procedures or methods for solving them. They are
familiar, often repetitive, and require applying previously learned techniques.
Key Features:
 Well-defined steps to follow.
 Predictable outcome.
 Can often be solved using formulas, algorithms, or known procedures.
 Typically requires lower-order thinking (recall, application).
Examples:
 Converting binary to decimal.
 Calculating the average of a set of numbers.
 Sorting a list using a known algorithm (e.g., bubble sort).
 Writing code to check if a number is even or odd.

14
Non-Routine Problems
Problems that are unfamiliar, complex, or do not have a straightforward solution method.
They require creativity, critical thinking, and sometimes trial-and-error.
Key Features:
 No clear or single method of solution.
 Requires deeper thinking, analysis, or innovation.
 Often open-ended or involve multiple steps or strategies.
 May involve unknowns, constraints, or abstract reasoning.
Examples:
 Designing a new algorithm to optimize delivery routes.
 Debugging a software program with an unknown error.
 Creating a mobile app to solve a unique user problem.
 Developing a cybersecurity strategy for a new type of threat.

Comparison Table:

Feature Routine Problem Non-Routine Problem

Solution Path Known and direct Unknown, requires exploration
Complexity Low to moderate High
Thinking Level Lower-order (apply, recall) Higher-order (analyze, evaluate, create)
Algorithm design, system
Examples Basic arithmetic, code syntax fixes
troubleshooting
Creativity
Minimal High
Required

PRACTICE EXERCISES

Formulate a model and then develop an algorithm for each of the following problems. In each
case, start with a simple algorithm and then try to think about situations that can realistically
go wrong and make appropriate adjustments to the algorithm. Keep in mind that there is no
“right” answer to these problems. Everyone will have a unique solution.

PHY:

a. Building a 3 stair duplex in Gashua

b. Ascending to heaven

CHM:

15
a. Producing 3 kg of cooking salt

b. How to ascent to heaven

BCH, MCB & BIO:

a. Diagnosing patients for diabetes

b. How to ascent to heaven

3. METHOD OF SOLVING COMPUTING PROBLEM

Algorithm and Heuristic
A common type of strategy of solving computing problems is an algorithm. An algorithm is
a problem-solving formula that provides you with precise step-by-step instructions used to
achieve a desired outcome. You can think of an algorithm as a recipe with highly detailed
instructions that produce the same result every time they are performed. Algorithms are used
frequently in our everyday lives, especially in computer science. When you run a search on
the Internet, search engines like Google use algorithms to decide which entries will appear
first in your list of results. Facebook also uses algorithms to decide which posts to display on
your newsfeed. Can you identify other situations in which algorithms are used?

Characteristics:

 Deterministic: Follows a predictable path.

 Exact: Guarantees a correct and complete solution.
 Repeatable: Always produces the same result for the same input.
 Efficient (ideally): Designed with time and space in mind.

In today's digital age, the ubiquity of technology and the explosion of data have led to an
increasing reliance on algorithms in a variety of fields. These intricate sets of instructions and
rules define the core of artificial intelligence and machine learning, driving significant
advances in complex problem solving and automated decision-making. In this vast
algorithmic landscape, a fascinating variety of approaches unfolds, each designed to tackle
different types of problems.

Here, we will dive into the fascinating world of algorithms, exploring their different
categories and highlighting those that have emerged as the most prominent and widely used

16
today. From classic algorithms that have withstood the test of time to more recent innovations
that harness modern computational power, we will examine how these algorithms have
radically transformed the way we process information and make decisions in an increasingly
digitised world. Join us on this exploratory journey as we unravel the mysteries of algorithms,
from their foundations to their most cutting-edge applications.

Ten (10) most widely used types of algorithms

The truth is that there are many, many types of algorithms and it is impossible to be familiar
with all of them. However, here are a few of the most commonly used ones.

1. Sorting algorithm

A sorting algorithm is a set of instructions designed to arrange elements in a data set in

a specific order. Sorting is fundamental for efficient data manipulation and is applied in a
variety of areas, such as databases, search algorithms and information processing. The two
most common sorting algorithms are:

 Quicksort: An efficient sorting algorithm that uses a "divide and conquer" strategy. It
divides the data set into smaller subsets, sorts each subset and then combines the
results.

 Mergesort: A sorting algorithm that also employs the "divide and conquer" strategy.
It divides the set into halves, sorts each half, and then combines the sorted halves to
obtain the final sorted set.

17
2. Search algorithms

A search algorithm is a set of instructions designed to find the location or determine the
existence of a specific item within a data set. There are several search strategies, and the
choice of algorithm depends on factors such as the structure of the data and the efficiency
required. However, the two most commonly used search algorithms are:

 Linear Search: Examines each element in the dataset in sequence until the desired
element is found or the end is reached. It is simple but can be inefficient on large
datasets.

18
  Binary Search: Applicable only on ordered data sets. Repeatedly divides the set in
half and compares the desired element with the element in the middle, thus reducing
the search space.

Both sorting and search algorithms are fundamental and widely used in programming and
efficiently solving problems related to data organisation and search. The choice of the
appropriate algorithm depends on several factors, such as the size of the data set, the
complexity of the problem and efficiency requirements.

3. Graph algorithms

A graph algorithm is based on rules designed to solve problems involving data structures
called graphs. Graphs are abstract representations of relationships between objects and
consist of nodes (or vertices) and edges (or arcs) connecting these nodes.

Graph algorithms are used to analyse and manipulate these structures in order to solve
practical problems in various fields. Again, within the range of graph algorithms, there are
many types of algorithms, the most important of which are:

 Breadth-first search (BFS): An algorithm that explores all nodes at the same depth
before advancing to nodes at the next depth. It is useful for finding the shortest path in
un-weighted graphs.
 Dijkstra algorithm: Used to find the shortest path between a source node and all
other nodes in a weighted graph with non-negative weights.

19
  Bellman-Ford algorithm: Similar to Dijkstra but can handle graphs with negative
weighted edges. It is used to find the shortest path in graphs with variable costs.
 Kruskal algorithm: Used to find a minimum spanning tree in a weighted graph. This
tree spans all the nodes of the graph and has the minimum sum of the edge weights.

4. Classification algorithm

A classification algorithm is a set of instructions or rules designed to assign an object or

instance to a specific category or class, based on its characteristics or attributes. These
algorithms are widely used in machine learning and data mining to organise and label data
automatically, enabling pattern identification and decision making.

The classification process involves training a model using a training dataset, where
instances are provided along with their respective class labels. The model uses this
information to learn patterns and relationships between features and classes. Once
trained, the model can be used to classify new instances whose class labels are unknown.

Some common classification algorithms include:

 Logistic Regression: Despite its name, it is used for binary classification problems,
assigning instances to one of two possible classes.
 Support Vector Machines (SVM): Searches for a hyper-plane that maximises the
margin between classes in a feature space.
 Decision Trees (explored in more depth below): Represent decisions in the form of
a tree, iteratively dividing the data set into subsets.
 K-Nearest Neighbours (KNN): Classifies an instance according to the most classes
of its k nearest neighbours in the feature space.
 Naive Bayes: Based on Bayes' theorem, assumes independence between features and
assigns probabilities to classes.
 Neural Networks: Models inspired by the functioning of the brain, used for complex
classification problems.
 Random Forest: An ensemble of decision trees that vote to determine the class of an
instance.
 Gradient Boosting: Combines multiple weaker models to improve overall model
accuracy.

20
The choice of classification algorithm depends on the nature of the problem, the size and
quality of the data, as well as the specific requirements of the application context. Each
algorithm has its own characteristics and performance in different situations, so proper
selection is essential to achieve optimal results.

5. Regression algorithms

Regression algorithms are a set of techniques in machine learning and statistics used to
model and analyse the relationship between variables. These algorithms are mainly
applied in prediction problems for predictive analytics, where the objective is to predict the
value of an output variable (or dependent variable) based on one or more input variables (or
independent variables).

The main task of a regression model is to find a mathematical function that describes the
relationship between the input variables and the output variable. The resulting function
can be used to make predictions about future or unknown values of the output variable.
Some common regression algorithms include:

 Simple and Multiple Linear Regression: Simple linear regression models the
relationship between two variables, while multiple linear regression models the
relationship between more than two variables, considering multiple predictors.
 Polynomial Regression: This technique uses a polynomial equation to model non-
linear relationships.
 K-Nearest Neighbors Regression (KNN): The algorithm predicts the output value
based on the output values of the k nearest neighbors in the feature space.
 Support Vector Machines (SVM) for Regression: This algorithm finds a hyper-
plane in the feature space that best fits the data.
 Regression Trees: By using decision trees, this algorithm models non-linear
relationships and segments the feature space into regions.
 Ridge and Lasso Regression: These are regularization techniques that help control
model over-fitting.
 Neural Networks for Regression: This algorithm employs neural network
architectures to model complex relationships.

6. Decision tree algorithm

21
A decision tree algorithm is a supervised learning method used in the field of machine
learning and artificial intelligence. It is designed to make decisions based on multiple
conditions or features. Decision trees model decisions in the form of a tree, where each
internal node represents a test on a feature, each branch represents the outcome of that test,
and each leaf represents the final decision.

Key features of decision trees:

 Recursive Division: The tree is built recursively, partitioning the dataset into smaller
subsets based on relevant features.
 Decision Criteria: At each node, a test is conducted on a specific feature, and the
decision on which feature to choose and how to divide is made using a criterion that
maximizes the purity of the resulting leaves.
 Classification or Regression: Decision trees can be utilized for both classification
and regression problems. In classification, the leaves represent classes or categories,
while in regression, the leaves contain numerical values.
 Interpretability: A key advantage of decision trees is their ability to be easily
interpreted by humans. The tree provides a logical structure that can be visually
understood.

7. Greedy' algorithm:

A greedy algorithm is an approach to algorithm design that makes local decisions at each
step in the hope of arriving at a globally optimal solution. In other words, at each step, the
algorithm selects the option that appears to be the best at the time, without considering
the long-term consequences. The idea is to make decisions that appear to be the most
beneficial at the present moment, without worrying about their future impact.

Key features of greedy algorithms:

 Greedy Choice: At each step, the algorithm makes the decision that appears to be the
best at the time.
 No Backtracking: Once a decision is made, it is not reviewed or changed at later
stages.

22
  Global Optimisation Expectancy: Despite making local decisions, the sequence of
decisions is expected to lead to a globally optimal solution or at least an acceptable
solution.

Greedy algorithms are effective in situations where the locally optimal choice also leads
to a globally optimal or near-optimal solution. However, they do not always guarantee the
best solution in all cases, so their applicability depends on the specific problem being
addressed.

8. Brute-force algorithm

A brute-force algorithm is an algorithm whose essence is based on trying all possible

solutions and verifying which one is correct. This exhaustive method is simple and
guarantees to find the optimal solution to a problem, but is often inefficient due to the large
number of combinations to consider, especially when the size of the problem increases.

Key features of brute force algorithms:

 Exhaustive Exploration: This type of algorithm considers all possible combinations

to find the solution.
 No Optimisation Strategy: They do not rely on intelligent strategies to reduce the
search space.
 Guaranteed Optimal Solution: Because they examine all possibilities, brute-force
algorithms are guaranteed to find the optimal solution if it exists.

While brute-force algorithms can be effective for small problems or when there is no
known more efficient solution, their execution time can become impractical (time
complexity) as the problem size increases. In many cases, more specialised and efficient
algorithms are sought to address specific problems.

9. Backtracking algorithm

23
A backtracking algorithm tries to search for all possible solutions to a problem by
systematically exploring all available options. This approach is based on the principle of
trial and error, where the algorithm moves forward to try a solution, but backtracks when it
realises that the partial solution cannot lead to a valid solution.

Key features of backtracking algorithms:

 Systematic Exploration: The algorithm systematically tests all possible solutions,

building and un-building partial solutions as needed.
 Sequential Decisions: It makes sequential decisions to build a solution, backtracking
when it encounters an invalid state.
 No Guarantee of Optimal Solution: Although it may find a solution, it does not
guarantee the optimal solution.

The success of a backtracking algorithm depends on efficient decision-making to avoid

unnecessarily exploring certain paths. In some cases, additional techniques, such as pruning,
are used to reduce the search for solutions. Efficient implementation and appropriate choice
of strategies are essential to effectively tackle problems with backtracking algorithms.

10. Dynamic programming algorithms

A dynamic programming algorithm is a type of algorithm designed to divide a problem

into smaller sub-problems and solve each of the sub-problems first. The solutions found
are stored to avoid rework. This technique is especially useful when a problem can be
decomposed into overlapping sub-problems or sub-problems that share common solutions.

Key features of dynamic programming:

 Optimal Substructure: The overall problem can be solved by combining optimal

solutions of smaller sub-problems.
 Sub-problem Overlap: Sub-problems share common solutions and dynamic
programming avoids recalculating solutions for each sub-problem by storing and
reusing already calculated results.

Fundamental steps in a dynamic programming algorithm:

24
  Problem Definition: Formulate the problem in terms of smaller subproblems.
 Subproblem Identification: Identify subproblems to be solved and combined to
solve the overall problem.
 Recursive Equation Definition: Develop a recursive equation that relates the
solution of the original problem to the solutions of its sub-problems.
 Order of Solving: Determine the order in which the sub-problems are solved to
ensure that the necessary solutions are available when required.
 Results Storage: Storing the results of solved sub-problems in a data structure (often
a table or matrix) for reuse.

Other types of algorithms

Encryption algorithms

Beyond the algorithms mentioned above, it is also essential to know about encryption
algorithms, which convert the data in a file in such a way that it cannot be retrieved
without the corresponding encryption key. These algorithms are the basis of online
security.

Conclusion

In this blog post, we have explored a variety of algorithms used in machine learning and
artificial intelligence. We have learned about algorithms such as Polynomial Regression,
KNN, SVM, Regression Trees, Ridge and Lasso Regression, Neural Networks, as well as the
Decision Tree Algorithm, Greedy Algorithms, Brute Force Algorithms, Backtracking
Algorithms, and Dynamic Programming Algorithms. Each of these algorithms has its own
characteristics and specific applications. Additionally, we have mentioned the importance of
encryption algorithms in online security. If you are interested in learning more about these
algorithms and how they are applied in the real world, I encourage you to continue
researching and delving into each of them. The world of artificial intelligence and machine
learning is fascinating and constantly evolving, so don't get left behind!

Heuristic Strategy of Problem Solving

25
A heuristic is another type of problem solving strategy. While an algorithm must follow lay
procedure exactly to produce a correct result, a heuristic is a general problem-solving
framework which does not follow that order.
In problem-solving, a heuristic is a mental shortcut or rule of thumb that uses prior
experience and familiar knowledge to quickly find a "good enough" solution, though it
doesn't guarantee a correct or optimal one. Heuristics are practical, experience-based
strategies used when a complex problem is difficult, time-consuming, or impossible to solve
perfectly, allowing for rapid decision-making and a reduction in mental effort.
Such a rule saves the person time and energy when making a decision, but despite its time-
saving characteristics, it is not always the best method for making a rational decision.
Different types of heuristics are used in different types of situations, but the impulse to use a
heuristic occurs when one of five conditions is met:
 When one is faced with too much information
 When the time to make a decision is limited
 When the decision to be made is unimportant
 When there is access to very little information to use in making the decision
 When an appropriate heuristic happens to come to mind in the same moment, working
backwards is a useful heuristic in which you begin solving the problem by focusing
on the end result. It is common to use the working backwards heuristic to plan the
events of your day on a regular basis, probably without even thinking about it.
Another useful heuristic is the practice of accomplishing a large goal or task by breaking it
into a series of smaller steps. Students often use this common method to complete a large
research project or long essay for school. For example, students typically brainstorm, develop
a thesis or main topic, research the chosen topic, organize their information into an outline,
write a rough draft, revise and edit the rough draft, develop a final draft, organize the
references list, and proofread their work before turning in the project. The large task becomes
less overwhelming when it is broken down into a series of small steps.
Key Characteristics of Heuristics

 Mental Shortcuts:
They are action-oriented mental shortcuts and "rules of thumb" that simplify complex
choices and speed up the problem-solving process.
 Experience-Based:

26
 Heuristics are often derived from past experiences and what is familiar to the
problem-solver.
 "Good Enough" Solutions:
They are pragmatic and aim for satisfactory solutions rather than perfect or fully
optimized ones.
 No Guarantee of Success:
Unlike algorithms, which are step-by-step procedures that guarantee a solution,
heuristics do not promise a correct or optimal outcome.

How Heuristics Are Used

 Simplification: They help simplify complex factors and the cognitive load of making
decisions.
 Time-Saving: They save time by avoiding complex procedures for routine tasks.
 Analogy and Induction: They often rely on induction (finding patterns and applying
them) and analogy (comparing the current problem to a similar, familiar one).

Examples of Heuristics

 Decomposing a Problem: Breaking a large problem into smaller, more manageable

subgoals.
 Working Backwards: Starting from the desired solution and working backward to
the initial problem.
 Familiar Routines: Taking the most familiar path to a location, like the bathroom,
instead of calculating the most efficient route.
 Pattern Recognition: Looking for patterns in the problem that can guide the
solution.

27