Design and Analysis of Algorithm

1. Algorithms
An algorithm is a type of effective method in which a definite list of well-defined
instructions for completing a task; that given an initial state, will proceed through a well-
defined series of successive states, eventually terminating in an end-state. The concept of
an algorithm originated as a means of recording procedures for solving mathematical
problems such as finding the common divisor of two numbers or multiplying two
numbers.

Algorithms are named for the 9th century Persian mathematician Al-Khowarizmi. He
wrote a treatise in Arabic in 825 AD, On Calculation with Hindu Numerals. It was
translated into Latin in the 12th century as Algoritmi de numero Indorum, which title was
likely intended to mean "[Book by] Algoritmus on the numbers of the Indians", where
"Algoritmi" was the translator's rendition of the author's name in the genitive case; but
people misunderstanding the title treated Algoritmi as a Latin plural and this led to the
word "algorithm" (Latin algorismus) coming to mean "calculation method".

1.1 Algorithm Specification

The criteria for any set of instruction for an algorithm is as follows:
 Input : Zero of more quantities that are externally applied
 Output : At least one quantity is produced
 Definiteness : Each instruction should be clear and unambiguous
 Finiteness : Algorithm terminates after finite number of steps for all test
cases.
 Effectiveness : Each instruction is basic enough for a person to carried out
using a pen and paper. That means ensure not only definite but also check whether
feasible or not.

1.2 Algorithm Classification

There are various ways to classify algorithms. They are as follows

1.2.1 Classification by implementation

Recursion or iteration: A recursive algorithm is one that invokes (makes reference
to) itself repeatedly until a certain condition
matches, which is a method common to functional programming. Iterative algorithms
use repetitive constructs like loops and sometimes additional data structures like
stacks to solve the given problems. Some problems are naturally suited for one
implementation or the other. For example, towers of hanoi is well understood in
recursive implementation. Every recursive version has an equivalent (but possibly
more or less complex) iterative version, and vice versa.
Logical: An algorithm may be viewed as controlled logical deduction. This notion
may be expressed as:
Algorithm = logic + control.
The logic component expresses the axioms that may be used in the computation and
the control component determines the way in which deduction is applied to the
axioms. This is the basis for the logic programming paradigm. In pure logic

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 1

 Design and Analysis of Algorithm

programming languages, the control component is fixed and algorithms are specified
by supplying only the logic component. The appeal of this approach is the elegant
semantics: a change in the axioms has a well-defined change in the algorithm.
Serial or parallel or distributed: Algorithms are usually discussed with the
assumption that computers execute one instruction of an algorithm at a time. Those
computers are sometimes called serial computers. An algorithm designed for such an
environment is called a serial algorithm, as opposed to parallel algorithms or
distributed algorithms. Parallel algorithms take advantage of computer architectures
where several processors can work on a problem at the same time, whereas distributed
algorithms utilise multiple machines connected with a network. Parallel or distributed
algorithms divide the problem into more symmetrical or asymmetrical subproblems
and collect the results back together. The resource consumption in such algorithms is
not only processor cycles on each processor but also the communication overhead
between the processors. Sorting algorithms can be parallelized efficiently, but their
communication overhead is expensive. Iterative algorithms are generally
parallelizable. Some problems have no parallel algorithms, and are called inherently
serial problems.
Deterministic or non-deterministic: Deterministic algorithms solve the problem with
exact decision at every step of the algorithm whereas non-deterministic algorithm
solves problems via guessing although typical guesses are made more accurate
through the use of heuristics.
Exact or approximate: While many algorithms reach an exact solution,
approximation algorithms seek an approximation that is close to the true solution.
Approximation may use either a deterministic or a random strategy. Such algorithms
have practical value for many hard problems.

1.2.2 Classification by Design Paradigm

Divide and conquer. A divide and conquer algorithm repeatedly reduces an instance
of a problem to one or more smaller instances of the same problem (usually
recursively), until the instances are small enough to solve easily. One such example of
divide and conquer is merge sorting. Sorting can be done on each segment of data
after dividing data into segments and sorting of entire data can be obtained in conquer
phase by merging them. A simpler variant of divide and conquer is called decrease
and conquer algorithm, that solves an identical sub problem and uses the solution of
this sub problem to solve the bigger problem. Divide and conquer divides the problem
into multiple sub problems and so conquer stage will be more complex than decrease
and conquer algorithms. An example of decrease and conquer algorithm is binary
search algorithm.
Dynamic programming. When a problem shows optimal substructure, meaning the
optimal solution to a problem can be constructed from optimal solutions to sub
problems, and overlapping sub problems, meaning the same sub problems are used to
solve many different problem instances, a quicker approach called dynamic
programming avoids recomputing solutions that have already been computed. For
example, the shortest path to a goal from a vertex in a weighted graph can be found by
using the shortest path to the goal from all adjacent vertices. Dynamic programming
and memoization go together. The main difference between dynamic programming
and divide and conquer is that sub problems are more or less independent in divide
and conquer, whereas sub problems overlap in dynamic programming. The difference

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 2

 Design and Analysis of Algorithm

between dynamic programming and straightforward recursion is in caching or

memoization of recursive calls. When sub problems are independent and there is no
repetition, memoization does not help; hence dynamic programming is not a solution
for all complex problems. By using memoization or maintaining a table of sub
problems already solved, dynamic programming reduces the exponential nature of
many problems to polynomial complexity.
The greedy method. A greedy algorithm is similar to a dynamic programming
algorithm, but the difference is that solutions to the sub problems do not have to be
known at each stage; instead a "greedy" choice can be made of what looks best for the
moment. The greedy method extends the solution with the best possible decision (not
all feasible decisions) at an algorithmic stage based on the current local optimum and
the best decision (not all possible decisions) made in previous stage. It is not
exhaustive, and does not give accurate answer to many problems. But when it works,
it will be the fastest method. The most popular greedy algorithm is finding the
minimal spanning tree as given by Kruskal.
Linear programming. When solving a problem using linear programming, specific
inequalities involving the inputs are found and then an attempt is made to maximize
(or minimize) some linear function of the inputs. Many problems (such as the
maximum flow for directed graphs) can be stated in a linear programming way, and
then be solved by a 'generic' algorithm such as the simplex algorithm.
Reduction. This technique involves solving a difficult problem by transforming it
into a better-known problem for which we have (hopefully) asymptotically optimal
algorithms. The goal is to find a reducing algorithm whose complexity is not
dominated by the resulting reduced algorithm's. For example, one selection algorithm
for finding the median in an unsorted list involves first sorting the list (the expensive
portion) and then pulling out the middle element in the sorted list (the cheap portion).
This technique is also known as transform and conquer.
Search and enumeration. Many problems (such as playing chess) can be modeled as
problems on graphs. A graph exploration algorithm specifies rules for moving around
a graph and is useful for such problems. This category also includes search
algorithms, branch and bound enumeration and backtracking.
The probabilistic and heuristic paradigm. Algorithms belonging to this class fit the
definition of an algorithm more loosely.
Probabilistic algorithms are those that make some choices randomly (or pseudo-
randomly); for some problems, it can in fact be proven that the fastest solutions
must involve some randomness.
Genetic algorithms attempt to find solutions to problems by mimicking biological
evolutionary processes, with a cycle of random mutations yielding successive
generations of "solutions". Thus, they emulate reproduction and "survival of the
fittest". In genetic programming, this approach is extended to algorithms, by
regarding the algorithm itself as a "solution" to a problem.
Heuristic algorithms, whose general purpose is not to find an optimal solution, but
an approximate solution where the time or resources are limited. They are not
practical to find perfect solutions. An example of this would be local search, or
simulated annealing.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 3

 Design and Analysis of Algorithm

1.3 Recursive Algorithms

A recursive algorithm is an algorithm which calls itself with "smaller (or simpler)"
input values, and which obtains the result for the current input by applying simple
operations to the returned value for the smaller (or simpler) input. More generally if a
problem can be solved utilizing solutions to smaller versions of the same problem, and
the smaller versions reduce to easily solvable cases, then one can use a recursive
algorithm to solve that problem. For example, the elements of a recursively defined set, or
the value of a recursively defined function can be obtained by a recursive algorithm.
Recursive computer programs require more memory and computation compared with
iterative algorithms, but they are simpler and for many cases a natural way of thinking
about the problem.

For example consider factorial of a number, n

n! = n*(n-1)*(n-2)*...*2*1, and that 0! = 1.

In other words,

Function to calculate the factorial can be written as

int factorial(int n)
{
if (n == 0)
return 1;
else
return (n * factorial(n-1));
}

factorial(0)=> 1

factorial(3)
3 * factorial(2)
3 * 2 * factorial(1)
3 * 2 * 1 * factorial(0)
3*2*1*1
=> 6

This corresponds very closely to what actually happens on the execution stack in the
computer's memory.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 4

 Design and Analysis of Algorithm

1.4 Space and Time Complexity

Two important ways to characterize the effectiveness of an algorithm are its space
complexity and time complexity.

1.4.1 Space Complexity

Space complexity of an algorithm is the amount to memory needed by the program
for its completion. Space needed by a program has the following components:

1. Instruction Space
Space needed to store the compiled version of program. It depends on
i. Compiler used
ii. Options specified at the time of compilation
e.g., whether optimization specified, Is there any overlay option etc.
iii. Target computer
e.g., For performing floating point arithmetic, if hardware present or not.

2. Data Space
Space needed to store constant and variable values. It has two components:
i. Space for constants:
e.g., value ‘3’ in program 1.1
Space for simple variables:
e.g., variables a,b,c in program 1.1

Program 1.1
int add (int a, int b, int c)
{
return (a+b+c)/3;
}

ii. Space for component variables like arrays, structures, dynamically allocated
memory.
e.g., variables a in program 1.2

Program 1.2
int Radd (int a[], int n)
1 {
2 If (n>0)
3 return Radd (a, n-1) + a[n-1];
4 else
5 return 0;
6 }

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 5

 Design and Analysis of Algorithm

3. Environment stack space

Environment stack is used to store information to resume execution of partially
completed functions. When a function is invoked, following data are stored in
Environment stack.
i. Return address.
ii. Value of local and formal variables.
iii. Binding of all reference and constant reference parameters.

Space needed by the program can be divided into two parts.

i. Fixed part independent of instance characteristics. E.g., code space, simple
variables, fixed size component variables etc.
ii. Variable part. Space for component variables with space depends on particular
instance. Value of local and formal variables.
Hence we can write the space complexity as
S(P) = c + Sp (instance characteristics)

Example 1.1
Refer Program 1.1
One word for variables a,b,c. No instance characteristics. Hence Sp(TC) = 0

Example 1.2
Program 1.3
int Aadd (int *a, int n)
1 {
2 int s=0;
3 for (i=0; i<n; i++)
4 s+ = a[i];
5 return s;
6 }
One word for variables n and i. Space for a[] is address of a[0]. Hence it requires one
word. No instance characteristics. Hence Sp(TC) = 0

Example 1.3
Refer Program 1.2
Instance characteristics depend on values of n. Recursive stack space includes space
for formal parameters, local variables and return address. So one word each for a[],n,
return address and return variables. Hence for each pass it needs 4 words. Total recursive
stack space needed is 4(n).
Hence Sp(TC) = 4(n).

1.4.2 Time Complexity

Time complexity of an algorithm is the amount of time needed by the program for its
completion. Time taken is the sum of the compile time and the execution time. Compile
time does not depend on instantaneous characteristics. Hence we can ignore it.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 6

 Design and Analysis of Algorithm

Program step: A program step is syntactically or semantically meaningful segment of a

program whose execution time is independent of instantaneous characteristics. We can
calculate complexity in terms of

1. Comments:
No executables, hence step count = 0

2. Declarative Statements:
Define or characterize variables and constants like (int , long, enum, …)
Statement enabling data types (class, struct, union, template)
Determine access statements ( public, private, protected, friend )
Character functions ( void, virtual )
All the above are non executables, hence step count = 0

3. Expressions and Assignment Statements:

Simple expressions : Step count = 1. But if expressions contain function call, step
count is the cost of the invoking functions. This will be large if parameters are passed
as call by value, because value of the actual parameters must assigned to formal
parameters.

Assignment statements : General form is <variable> = <expr>. Step count = expr,

unless size of <variable> is a function of instance characteristics. eg., a = b, where a
and b are structures. In that case, Step count = size of <variable> + size of < expr >

4. Iterative Statements:

While <expr> do
Do .. While <expr>
Step count = Number of step count assignable to <expr>

For (<init-stmt>; <expr1>; <expr2>)

Step count = 1, unless the <init-stmt>, <expr1>,<expr2> are function of instance
characteristics. If so, first execution of control part has step count as sum of count of
<init-stmt> and <expr1>. For remaining executions, control part has step count as
sum of count of <expr1> and <expr2>.

5. Switch Statements:
Switch (<expr>) {
Case cond1 : <statement1>
Case cond2 : <statement2>
.
.
Default : <statement>
}

Switch (<expr>) has step count = cost of <expr>

Cost of Cond statements is its cost plus cost of all preceding statements.

6. If-else Statements:
If (<expr>) <statement1>;

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 7

 Design and Analysis of Algorithm

Else <statement2>;
Step count of If and Else is the cost of <expr>.

7. Function invocation:
All function invocation has Step count = 1, unless it has parameters passed as called
by value which depend s on instance characteristics. If so, Step count is the sum of the
size of these values.
If function being invoked is recursive, consider the local variables also.

8. Memory management Statements:

new object, delete object, sizeof(object), Step count =1.

9. Function Statements:
Step count = 0, cost is already assigned to invoking statements.

10. Jump Statements:

continue, break, goto has Step count =1
return <expr>: Step count =1, if no expr which is a function of instance
characteristics. If there is, consider its cost also.

Example 1.4
Refer Program 1.2
Introducing a counter for each executable line we can rewrite the program as

int Radd (int a[], int n)

{
count++ // if
If (n>0)
{
count++ // return
return Radd (a, n-1) + a[n-1];
}
else
{
count++ // return
return 0;
}
}
Case 1: n=0
tRadd = 2

Case 2: n>0
2 + tRadd (n-1)
= 2 + 2 + tRadd (n-2)
= 2 * 2 + tRadd (n-2)
.
.
= 2n + tRadd (0)
= 2n + 2

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 8

 Design and Analysis of Algorithm

Example 1.5
Program 1.4
int Madd (int a[][], int b[][], int c[][], int n)
1 {
2 For (int i=0; i<m; i++)
3 For (int j=0; j<n; j++)
4 c[i][j] = a[i][j] + b[i][j];
5 }

Introducing a counter for each executable line we can rewrite the program as
int Madd (int a[][], int b[][], int c[][], int n)
{
For (int i=0; i<m; i++)
{
count++ //for i
For (int j=0; j<n; j++)
{
count++ //for j
c[i][j] = a[i][j] + b[i][j];
count++ //for assignment
}
count++ //for last j
}
count++ //for last i
}
Step count is 2mn + 2m +1.

Step count does not reflect the complexity of statement. It is reflected in step per
execution (s/e).

Refer Program 1.2

Line s/e Frequency Total Steps
n=0 n>0 n=0 n>0
1 0 1 1 0 0
2 1 1 1 1 1
3 1 + tRadd (n-1) 0 1 0 1 + tRadd (n-1)
4 0 1 0 0 0
5 1 1 0 1 0
Total no. of steps 2 2 + tRadd (n-1)

Refer Program 1.3

Line s/e Frequency Total Steps
1 0 1 0
2 1 1 1
3 1 n+1 n+1
4 1 n n
5 1 1 1
6 0 1 0
Total no. of steps 2n + 3

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 9

 Design and Analysis of Algorithm

Refer Program 1.4

Line s/e Frequency Total Steps
1 0 1 0
2 1 m+1 m+1
3 1 m(n+1) m(n+1)
4 1 mn mn
5 0 1 0
Total no. of steps 2mn + 2m + 1

1.5 Asymptotic Notations

Step count is to compare time complexity of two programs that compute same
function and also to predict the growth in run time as instance characteristics changes.
Determining exact step count is difficult and not necessary also. Since the values are not
exact quantities we need only comparative statements like c1n2 ≤ tp(n) ≤ c2n2.

For example, consider two programs with complexities c1n2 + c2n and c3n
respectively. For small values of n, complexity depend upon values of c1, c2 and c3. But
there will also be an n beyond which complexity of c3n is better than that of c1n2 +
c2n.This value of n is called break-even point. If this point is zero, c3n is always faster (or
at least as fast). Common asymptotic functions are given below.

Function Name
1 Constant
log n Logarithmic
n Linear
n log n n log n
n2 Quadratic
n3 Cubic
2n Exponential
n! Factorial

1.5.1 Big ‘Oh’ Notation (O)

O(g(n)) = { f(n) : there exist positive constants c and n0 such that 0 ≤ f(n) ≤ cg(n) for all n
≥ n0 }
It is the upper bound of any function. Hence it denotes the worse case complexity of any
algorithm. We can represent it graphically as

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 10

 Design and Analysis of Algorithm

Fig 1.1

Find the Big ‘Oh’ for the following functions:

Linear Functions
Example 1.6
f(n) = 3n + 2

General form is f(n) ≤ cg(n)

When n ≥ 2, 3n + 2 ≤ 3n + n = 4n
Hence f(n) = O(n), here c = 4 and n0 = 2

When n ≥ 1, 3n + 2 ≤ 3n + 2n = 5n
Hence f(n) = O(n), here c = 5 and n0 = 1

Hence we can have different c,n0 pairs satisfying for a given function.

Example 1.7
f(n) = 3n + 3
When n ≥ 3, 3n + 3 ≤ 3n + n = 4n
Hence f(n) = O(n), here c = 4 and n0 = 3

Example 1.8
f(n) = 100n + 6
When n ≥ 6, 100n + 6 ≤ 100n + n = 101n
Hence f(n) = O(n), here c = 101 and n0 = 6

Quadratic Functions
Example 1.9
f(n) = 10n2 + 4n + 2
When n ≥ 2, 10n2 + 4n + 2 ≤ 10n2 + 5n
When n ≥ 5, 5n ≤ n2, 10n2 + 4n + 2 ≤ 10n2 + n2 = 11n2
Hence f(n) = O(n2), here c = 11 and n0 = 5

Example 1.10
f(n) = 1000n2 + 100n - 6

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 11

 Design and Analysis of Algorithm

f(n) ≤ 1000n2 + 100n for all values of n.

When n ≥ 100, 5n ≤ n2, f(n) ≤ 1000n2 + n2 = 1001n2
Hence f(n) = O(n2), here c = 1001 and n0 = 100

Exponential Functions
Example 1.11
f(n) = 6*2n + n2
When n ≥ 4, n2 ≤ 2n
So f(n) ≤ 6*2n + 2n = 7*2n
Hence f(n) = O(2n), here c = 7 and n0 = 4

Constant Functions
Example 1.12
f(n) = 10
f(n) = O(1), because f(n) ≤ 10*1

1.5.2 Omega Notation (Ω)

Ω (g(n)) = { f(n) : there exist positive constants c and n0 such that 0 ≤ cg(n) ≤ f(n) for all
n ≥ n0 }
It is the lower bound of any function. Hence it denotes the best case complexity of any
algorithm. We can represent it graphically as

Fig 1.2
Example 1.13
f(n) = 3n + 2
3n + 2 > 3n for all n.
Hence f(n) = Ω(n)
Similarly we can solve all the examples specified under Big ‘Oh’.

1.5.3 Theta Notation (Θ)

Θ(g(n)) = {f(n) : there exist positive constants c1,c2 and n0 such that c1g(n) ≤f(n) ≤c2g(n)
for all n ≥ n0 }
If f(n) = Θ(g(n)), all values of n right to n0 f(n) lies on or above c1g(n) and on or below
c2g(n). Hence it is asymptotic tight bound for f(n).

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 12

 Design and Analysis of Algorithm

Fig 1.3
Example 1.14
f(n) = 3n + 2
f(n) = Θ(n) because f(n) = O(n) , n ≥ 2.

Similarly we can solve all examples specified under Big’Oh’.

1.5.4 Little ‘Oh’ Notation (o)

o(g(n)) = { f(n) : for any positive constants c > 0, there exists n0>0, such that 0 ≤ f(n) <
cg(n) for all n ≥ n0 }

It defines the asymptotic tight upper bound. Main difference with Big Oh is that Big Oh
defines for some constants c by Little Oh defines for all constants.

1.5.5 Little Omega (ω)

ω(g(n)) = { f(n) : for any positive constants c>0 and n0>0 such that 0 ≤ cg(n) < f(n) for
all n ≥ n0 }
It defines the asymptotic tight lower bound. Main difference with Ω is that, ω defines for
some constants c by ω defines for all constants.

1.6 Recurrence Relations

Recurrence is an equation or inequality that describes a function in terms of its value on

smaller inputs, and one or more base cases
e.g., recurrence for Merge-Sort

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 13

 Design and Analysis of Algorithm


(1) if n = 1
T(n)  

2T(n/2) + (n) if n > 1

• Useful for analyzing recurrent algorithms

• Make it easier to compare the complexity of two algorithms
• Methods for solving recurrences
– Substitution method
– Recursion tree method
– Master method
– Iteration method

1.6.1 Substitution Method

 Use mathematical induction to derive an answer
 Derive a function of n (or other variables used to express the size of the problem)
that is not a recurrence so we can establish an upper and/or lower bound on the
recurrence
 May get an exact solution or may just get upper or lower bounds on the solution

Steps
 Guess the form of the solution
 Use mathematical induction to find constants or show that they can be found and
to prove that the answer is correct

Example 1.15
Find the upper bound for the recurrence relation
T(n) = 2 T( n/2 ) + n

Guess the solution as T(n) = O(n.lg(n))

Then T(n) = c.n.lgn
Substituting in T(n), we get
T(n) =
= c n lg(n/2) + n
= cn lg(n) – cnlg(2) + n
= cn lg(n) – cn + n
= cn lg(n), c >=1
To prove using mathematical induction, we have to show that the solution holds for
boundary condition also. We select boundary condition as n>=2 (Because for n = 1,
T(1) = c.1.lg(1) = 0 which is false according to the definition of T(n))

Example 1.16
Find the worse case complexity of Binary Search
T(n) = c + T(n/2)

• Guess: T(n) = O(lgn)

– Induction goal: T(n) ≤ d lgn, for some d and n ≥ n0

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 14

 Design and Analysis of Algorithm

– Induction hypothesis: T(n/2) ≤ d lg(n/2)

• Proof of induction goal:
T(n) = T(n/2) + c ≤ d lg(n/2) + c
= d lgn – d + c ≤ d lgn
if: – d + c ≤ 0, d ≥ c

Example 1.17
T(n) = T(n-1) + n

• Guess: T(n) = O(n2)

– Induction goal: T(n) ≤ c n2, for some c and n ≥ n0
– Induction hypothesis: T(n-1) ≤ c(n-1)2 for all k < n
• Proof of induction goal:
T(n) = T(n-1) + n ≤ c (n-1)2 + n
= cn2 – (2cn – c - n) ≤ cn2
if: 2cn – c – n ≥ 0  c ≥ n/(2n-1)  c ≥ 1/(2 – 1/n)
– For n ≥ 1  2 – 1/n ≥ 1  any c ≥ 1 will work

Example 1.17
T(n) = 2T(n/2) + n
• Guess: T(n) = O(nlgn)
– Induction goal: T(n) ≤ cn lgn, for some c and n ≥ n0
– Induction hypothesis: T(n/2) ≤ cn/2 lg(n/2)
• Proof of induction goal:
T(n) = 2T(n/2) + n ≤ 2c (n/2)lg(n/2) + n
= cn lgn – cn + n ≤ cn lgn
if: - cn + n ≤ 0  c ≥ 1

1.6.2 Recursion tree Method

 Main disadvantage of Substitution method is that it is always difficult to come up
with a good guess
 Recursion tree method allows you make a good guess for the substitution method
 Allows to visualize the process of iterating the recurrence

Steps
 Convert the recurrence into a tree.
 Each node represents the cost of a single sub problem somewhere in the set of
recursive function invocations
 Sum the costs within each level of the tree to obtain a set of per-level costs
 Sum all the per-level costs to determine the total cost of all levels of the recursion

Example 1.18
T(n) = 3T(n/4) + (n2)
T(n) = 3T(n/4) + cn2

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 15

 Design and Analysis of Algorithm

• The sub problem size for a node at depth i is n/4i

– When the sub problem size is 1, n/4i = 1, i=log4n
– The tree has log4n+1 levels (0, 1, 2,.., log4n)
• The cost at each level of the tree (0, 1, 2,.., log4n-1)
– Number of nodes at depth i is 3i
– Each node at depth i has a cost of c(n/4i)2
– The total cost over all nodes at depth i is 3i c(n/4i)2=(3/16)icn2
• The cost at depth log4n
– Number of nodes is 3log 4 n  nlog 4 3
– Each contributing cost T(1)
– The total cost n log 4 3T (1)  (n log 4 3 )

3 2 3 3
T (n)  cn 2  cn  ( ) 2 cn 2  ...  ( ) log 4 n 1 cn 2  (n log 4 3 )
16 16 16
log 4 n 1
3
 
i 0
( )i cn 2  (n log 4 3 )
16

3 i 2
 ( ) cn  (n log 4 3 )
i 0 16
1 16 2
 cn 2  (n log 4 3 )  cn  (n log 4 3 )
3 13
1
16
 O(n 2 )

Prove T(n)=O(n2) is an upper bound by the substitution method

T(n)  dn2 for some constant d > 0

T (n)  3T ( n / 4)  cn 2
 3d n / 4  cn 2
2

 3d (n / 4) 2  cn 2
3 2
 dn  cn 2
16
 dn 2

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 16

 Design and Analysis of Algorithm

Example 1.19
W(n) = 2W(n/2) + (n2)

• Subproblem size at level i is: n/2i

• Subproblem size hits 1 when 1 = n/2i  i = lgn
• Cost of the problem at level i = (n/2i)2 No. of nodes at level i = 2i
• Total cost:
lg n 1 2 lg n 1  i i
n 1 1 1
W (n)   i  2 lg n W (1)  n 2     n  n 2     O(n) n 2  O ( n)  2n 2
i 0 2 i 0  2  i 0  2  1 1
2
 W(n) = O(n2)

Example 1.20
W(n) = W(n/3) + W(2n/3) + O(n)

• The longest path from the root to a leaf is: n  (2/3)n  (2/3)2 n  …  1
• Subproblem size hits 1 when 1 = (2/3)in  i=log3/2n
• Cost of the problem at level i = n
• Total cost:
log 3 / 2 n log 3 / 2 n
lg n 1
n  n  ...  n  n
i 0
 1  n log
i 0
3/ 2 nn 
lg 3 / 2 lg 3 / 2
n lg n

 W(n) = O(nlgn)

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 17

 Design and Analysis of Algorithm

Example 1.21
T(n) = T(n/4) + T(n/2) + n2

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 18

 Design and Analysis of Algorithm

1.6.3 Master Method

 The master method applies to recurrences of the form T(n) = a T(n/b) + f (n) ,
where a  1, b > 1, and f is asymptotically positive.
 Describe the running time of an algorithm that divides a problem of size n into a
sub problems, each of size n/b

 There are three common cases

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 19

 Design and Analysis of Algorithm

Case 1
 Compare f (n) with nlogba
 If f (n) = O(nlogba – e) for some constant e > 0.
i.e., f (n) grows polynomially slower than nlogba
i.e., f(n) is asymptotically smaller by an ne factor.
Then Solution: T(n) = Θ(nlogba) .

Case 2
 Compare f (n) with nlogba
 If f (n) = Θ (nlogba)
i.e., f (n) and nlogba grow at similar rates.
Then Solution: T(n) = Θ(nlogba lgn) .

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 20

 Design and Analysis of Algorithm

Case 3
 Compare f (n) with nlogba
 If f (n) = Ω(nlogba+e) for some constant e > 0
i.e., f (n) grows polynomially faster than nlogba
i.e., f(n) is asymptotically larger by an ne factor.
Then Solution: T(n) = Θ(f(n)) .

Example 1.22
T(n)=9T(n/3) + n
a=9, b=3, f(n) = n
CASE 1: n log b a  n log 3 9  (n 2 ), f (n)  O(n log 3 91 )
T(n) = (n2)

Example 1.23
T(n) = 4T(n/2) + n
a = 4, b = 2  nlogba = n2; f (n) = n.
CASE 1: f (n) = O(n2 – e) for e = 1.
 T(n) = Θ(n2).

Example 1.24
T(n) = T(2n/3) + 1
a=1, b=3/2,logf(n)=1
Case 2: n
ba
 n log 3 / 2 1  1, f (n)  1  (1)
T(n) = (lg n)

Example 1.25
T(n) = 4T(n/2) + n2
a = 4, b = 2  nlogba = n2; f (n) = n2.
CASE 2: f (n) = Q(n2lg0n), that is, k = 0.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 21

 Design and Analysis of Algorithm

 T(n) = Θ(n2lg n).

Example 1.26
T(n) = 3T(n/4) + n lg n (Case 3)
a=3, b=4, f(n) = n lg n
Case3: n log a  n log 3  O(n 0.793 )
b 4
f (n)  (n log 4 3 ) e = 0.2

For sufficiently large n, af(n/b) = 3(n/4)lg(n/4)  (3/4)n lg n=cf(n) for c=3/4

T(n) = (n lg n)

Example 1.27
T(n) = 4T(n/2) + n3
a = 4, b = 2  nlogba = n2; f (n) = n3.
CASE 3: f (n) = Ω(n2 + e) for e = 1 and 4(cn/2)3 £ cn3 (reg. cond.) for c = 1/2.
 T(n) = Θ(n3).

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 22

 Design and Analysis of Algorithm

Set Representation:

The set will be represented as the tree structure where all children will store the address
of parent / root node. The root node will store null at the place of parent address. In the given
set of elements any element can be selected as the root node, generally we select the first node
as the root node.
Example:
S1={1,7,8,9} S2={2,5,10} s3={3,4,6}

Then these sets can be represented as

Disjoint Union:

To perform disjoint set union between two sets Si and Sj can take any one root and
make it sub-tree of the other. Consider the above example sets S1 and S2 then the union of S1
and S2 can be represented as any one of the following.

Find:

To perform find operation, along with the tree structure we need to maintain the name
of each set. So, we require one more data structure to store the set names. The data structure
contains two fields. One is the set name and the other one is the pointer to root.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 23

 Design and Analysis of Algorithm

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 24

Design and Analysis of Algorithms Divide and Conquer

Divide and Conquer

Divide and conquer (D&C) is an important algorithm design paradigm. It works by
recursively breaking down a problem into two or more sub-problems of the same (or
related) type, until these become simple enough to be solved directly. The solutions to the
sub-problems are then combined to give a solution to the original problem. A divide and
conquer algorithm is closely tied to a type of recurrence relation between functions of the
data in question; data is "divided" into smaller portions and the result calculated thence.

Steps
 Splits the input with size n into k distinct sub problems 1 < k ≤ n
 Solve sub problems
 Combine sub problem solutions to get solution of the whole problem. Often sub
problems will be of same type as main problem. Hence solutions can be expressed
as recursive algorithms

Control Abstraction
Control abstraction is a procedure whose flow of control is clear but primary
operations are specified by other functions whose precise meaning is undefined

DAndC (P)
{
if Small(P) return S(P);
else
{
divide P into smaller instances P1, P2,…,Pk k1;
apply DandC to each of these sub problems;
retun Combine(DandC(P1), DandC(P2),…,DandC(Pk));
}
}

The recurrence relation for the algorithm is given by

g(n) – time to computer answer directly from small inputs.

f(n) – time for dividing P and combining solution to sub problems

2.1 Binary Search

Problem: Given a list of n sorted elements ai, 1≤ i ≤ n sorted in ascending order. Check
whether an element x is present in the list or not. If present, find the position of x in the
list else return zero

Program 2.1
int BinSearch (int a[], int n, int x)
{
int low=1, high=n, mid;
while (low <= high)
Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 1
Design and Analysis of Algorithms Divide and Conquer

{
mid = (low + high)/2;
if (x < a[mid])
high = mid – 1;
else if (x > a[mid])
low = mid + 1;
else
return (mid);
}
return (0);
}

An instance of the problem can be represented as

P = (n, ai,….,al, x)
where n : no of elements
ai,….,al: elements in the list
x : number to be searched

Comparing the program with the Divide and Conquer control abstraction,
S(P) : Index of the element, if the element is present else it will return zero.
g(1) : Θ(1)
If P has more than one element, divide the problem into sub problems as given below
 Pick an index 1 in the range [i, l]
 Compare x with aq
- x = aq, solved
- x < aq, search x in sub list ai, ai+1, … , aq-1 . Then P = (q-i, ai,….,aq-1, x)
- x > aq, search x in sub list aq+1, aq+2, … , al . Then P = (l-q, aq+1,….,al, x)

Time to divide P into one new sub problem is Θ(1). q is selected such that

q= . Answer to new sub problem is answer to P. Hence function Combine is

not needed here.

Simulation

Consider 10 elements 10, 20, 30, 40, 50, 60, 70, 80, 90, 100
Number of comparison needed to search element is as per the table given below
Position 1 2 3 4 5 6 7 8 9 10
Element 10 20 30 40 50 60 70 80 90 100
No. of comparisons 3 2 3 4 1 3 4 2 3 4
required

The search tree for the above list of elements will be as given below.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 2

Design and Analysis of Algorithms Divide and Conquer

If the element is present, then search end in a circular node (inner node), if the element is
not present, then it end up in a square node (leaf node)
Now we will consider the worse case, average case and best case complexities of the
algorithm for a successful and an unsuccessful search.

Worse Case
Find out k, such that 2k-1 <= n <= 2k
Then for a successful search, it will end up in either of the k inner nodes. Hence the
complexity is O(k), which is equal to O(log2n).
For an unsuccessful search, it need either k-1 or k comparisons. Hence complexity is
Θ (k), which is equal to Θ(log2n).

Average Case
Let I and E represent the sum of distance of all internal nodes from root and sum of
distance of all external nodes from the root respectively. Then
E = I + 2n

Let As(n) and Au(n) represents average case complexity of a successful and unsuccessful
search respectively. Then
As(n) = 1 + I/n
Au(n) = E / (n+1)

As(n) = 1 + I/n
= 1 + (E-2n)/n
= 1 + (Au(n)(n+1) – 2n)/n
= (n + (Au(n)(n+1) – 2n))/n
= (n(Au(n) -1) + Au(n))/n
= Au(n) – 1 + Au(n)/n
As(n) = Au(n)(1+1/n) - 1

As(n) and Au(n) are directly related and are proportional to log2n.
Hence the average case complexity for a successful and unsuccessful search is O(log2n)

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 3

Design and Analysis of Algorithms Divide and Conquer

Best Case
Best case for a successful search is when there is only one comparison, that is at middle
position if there is more than one element. Hence complexity is Θ(1)
Best case for an unsuccessful search is O(log2n)
Best Case Average Case Worse Case
Successful Θ(1) O(log2n) O(log2n)
Unsuccessful O(log2n) O(log2n) O(log2n)

2.2 Maximum and Minimum

Problem: Given a list of n elements ai, 1≤ i ≤ n, find the maximum and the minimum
element from the list

Given below is a straight forward method to calculate the maximum and minimum from
the list of n elements.

Program 2.2
Void StraightMinMax (int a[], int n, int *max, int *min)
{
int i;
*max = a[0];
*min = a[0];
for (i=1; i<n; i++)
{
if (a[i] > *max) *max = a[i];
if (a[i] < *min) *min = a[i];
}
}

Let us calculate the time complexity of the algorithm. Here, only element comparisons
are considered, because the frequency counts of other operations are same as that of
element comparison. 2(n-1) comparisons are required for worse, average and best case.
But if we modify the code as

if(a[i] > *max) *max = a[i];

else if (a[i]<min) *min = a[i];

Then Best case complexity is (n-1), it happens when elements are sorted in increasing
order.
Worse case complexity is 2(n-1), when elements are sorted in decreasing order
Average case complexity is (2(n-1)+ (n-1))/2 = 3(n-1)/2

Now let us solve the problem using Divide and Conquer method. An instance of the
problem can be represented as
P = (n, a[i],….,a[j])
where n : no of elements
a[i],….,a[j]: elements in the list

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 4

Design and Analysis of Algorithms Divide and Conquer

When n=1, max = min = a[1]. When n=2, the problem can be solved in one comparison.
Hence, Small(P) is true when n<=2. If there are more elements, then we have to divide
the problem into sub problems. So divide the problem into two instances,

To solve the problem, invoke Divide and Conquer algorithm recursively. Combine
solution for P1 and P2. Set Max(P) as larger among Max(P1) and Max(P2) and Min(P) as
smaller among Min(P1) and Min(P2)

Program 2.2
Void MaxMin (int i, int j, int *max, int *min)
{
if (i==j)
{
*max = a[i];
*min = a[i];
}
else if (i == j-1)
{
if (a[i] < a[j])
{
*max = a[j];
*min = a[i];

}
else
{
*max = a[i];
*min = a[j];

}
}
else
{
mid = (i+j)/2;
MaxMin (i, mid, *max, *min);
MaxMin (mid+1, j, *max1, *min1);
if (*max < *max1) *max = *max1;
if (*min > *min1) *min = *min1;
}
}

Example 2.1
Consider 9 elements 22, 13, -5, -8, 15, 60, 17, 31 and 47. The recursive call can be
represented using a tree as given below

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 5

Design and Analysis of Algorithms Divide and Conquer

This can be represented by recurrence relation as

If n is a power of two, n = 2k
T(n) = 2T(n/2) + 2
= 2 (2T(n/4) + 2) + 2
= 4T(n/4) + 4 + 2
= ...

= 2k-1+ 2k -2
= 2k-1+ n -2
= n/2 + n – 2
T(n) = 3n/2 – 2 (Best, average, Worst)
Comparing with the Straight forward method, Divide and Conquer is 50% more faster.
But additional stack space is required for recursion.

Yet another figures are obtained if we include position comparison also, while calculating
complexity. For this rewrite Small(P) as
if (i >= j-1) (Small(P)}

C(n) = 2C(n/2) + 3
= 4C(n/4) + 6 + 3
= 4C(n/4) + 3(1+2)
= ...

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 6

Design and Analysis of Algorithms Divide and Conquer

= 2k-1.2 + 3[2k-1-2]
= 2k + 3.2k-1-3
= n + 3.n/2 – 3
C(n) = 5n/2 – 3
While using StraightMinMax the comparison is 3(n-1) which is greater than 5n/2–3. Even
though the element comparison is less for MaxMin, it is slower than normal method
because of the stack.
.
2.3 Merge Sort
Given a sequence of n elements a[1],...,a[n].
Spilt into two sets
Each set is individually sorted, resultant is merged to form sorted list of n elements

Example 2.2
Consider 10 elements 310, 285, 179, 652, 351, 423, 861, 254, 450 and 520. The recursive
call can be represented using a tree as given below

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 7

Design and Analysis of Algorithms Divide and Conquer

The recurrence relation for the algorithm can be written as

When n is power of 2, we can write n = 2k

T(n) = 2(T(n/4) + cn/2) + cn
= 4T(n/4) + 2cn
= 4(2T(n/8) + cn/4) + 2cn
=…
= 2kT(1) + kcn
T(n) = an + cnlogn
If 2 <n≤2 , T(n)≤T(2k+1)
k k+1

Hence T(n) = O(nlogn)

2.4 Quick Sort

Given a sequence of n elements a[1],...,a[n].
Divide the list into two sub arrays and sorted so that the sorted sub arrays need not be
merged later. This is done by rearranging elements in the array such that a[i]<=a[j] for
1≤i≤m and m+1≤j≤n, 1≤m≤n. Then the elements a[1] to a[m] and a[m+1] to a[n] are
independently sorted, Hence no merging is required.

Pick an element, t=a[s], reorder other elements so that all elements appearing before t is
less than or equal to t, all elements after t is greater than or equal to t.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 8

Design and Analysis of Algorithms Divide and Conquer

Program 2.4

int partition (int a[], int m, int p)

{
int v=a[m], i=m, j=p ;
do
{
do
i++;
while(a[i]<v);
do
j--;
while(a[j]>v);
if (i<j) interchange(a,i,j);
}
while(i<j);
a[m] = a[j]; a[j] = v; return(j);
}
While calculating complexity we are considering element comparison alone. Assume that
n elements are distinct and the distribution is such that there is equal probability for any
element to get selected for partitioning.

Worse case complexity

According to the previous program the number of elements is atmost p-m+1.
Let the total number of elements in all call of Partition at a level be r.
At level one there is only one call of Partition, hence r=n
At level two, there is at most two calls, hence r=n-1.
Hence we can tell, at all levels the complexity is O(R), where the value of r is diminishing
at each level.
The worse case complexity Cw(n) is the sum of r at each level, hence Cw(n) = O(n2)
Average case complexity
The partitioning element has equal probability of becoming the ith smallest element,
1≤ i≤ p-m in the array a[m:p-1].
Hence probability for sub arrays being sorted, a[m:j], a[j+1:p-1]is 1/(p-m), m≤j<p.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 9

Design and Analysis of Algorithms Divide and Conquer

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT 10

Design and Analysis of Algoriths Greedy Method

UNIT – 2 (Module-2)
THE GREEDY METHOD
1. The General Method
2. Knapsack Problem
3. Job Sequencing with Deadlines
4. Minimum-Cost Spanning Trees
5. Prim’sAlgorithm
6. Kruskal’s Algorithm
7. Single Source Shortest Paths.

2.1The General Method

Definition:
Greedy technique is a general algorithm design strategy, built on following elements:
• configurations: different choices, values to find
• objective function: some configurations to be either maximized or minimized

The method:
• Applicable to optimization problems ONLY
• Constructs a solution through a sequence of steps
• Each step expands a partially constructed solution so far, until a complete solution
to the problem is reached.
On each step, the choice made must be
• Feasible: it has to satisfy the problem‘s constraints
• Locally optimal: it has to be the best local choice among all feasible choices
available on that step
• Irrevocable: Once made, it cannot be changed on subsequent steps of the
algorithm

NOTE:
• Greedy method works best when applied to problems with the greedy-choice
property
• A globally-optimal solution can always be found by a series of local
improvements from a starting configuration.

Greedy method vs. Dynamic programming method:

• LIKE dynamic programming, greedy method solves optimization problems.
• LIKE dynamic programming, greedy method problems exhibit optimal
substructure
• UNLIKE dynamic programming, greedy method problems exhibit the greedy
choice property -avoids back-tracing.

Applications of the Greedy Strategy:

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

• Optimal solutions:
Change making
Minimum Spanning Tree (MST)
Single-source shortest paths
Huffman codes
• Approximations:
Traveling Salesman Problem (TSP)
Fractional Knapsack problem

2.2Knapsack problem
2.2.1 One wants to pack n items in a luggage
2.2.1.1 The ith item is worth vi dollars and weighs wi pounds
2.2.1.2 Maximize the value but cannot exceed W pounds
2.2.1.3 vi , wi, W are integers
2.2.2 0-1 knapsack  each item is taken or not taken
2.2.3 Fractional knapsack  fractions of items can be taken
2.2.4 Both exhibit the optimal-substructure property
2.2.4.1 0-1: If item j is removed from an optimal packing, the remaining packing is an
optimal packing with weight at most W-wj
2.2.4.2 Fractional: If w pounds of item j is removed from an optimal packing, the
remaining packing is an optimal packing with weight at most W-w that can be taken
from other n-1items plus wj – w of item j
Greedy Algorithm for Fractional Knapsack problem
2.2.5 Fractional knapsack can be solvable by the greedy strategy
2.2.5.1 Compute the value per pound vi/wi for each item
2.2.5.2 Obeying a greedy strategy, take as much as possible of the item with the greatest
value perpound.
2.2.5.3 If the supply of that item is exhausted and there is still more room, take as
much as possible of the item with the next value per pound, and so forth until there is
no moreroom
2.2.5.4 O(n lg n) (we need to sort the items by value per
pound)O-1 knapsack is harder
2.2.6 knapsack cannot be solved by the greedy strategy

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

2.2.6.1 Unable to fill the knapsack to capacity, and the empty space lowers the effective
value perpound of the packing
2.2.6.2 We must compare the solution to the sub-problem in which the item is included
with thesolution to the sub-problem in which the item is excluded before we can make
the choice

2.3 Job sequencing with deadlines

The problem is stated as below.
2.3.1 There are n jobs to be processed on a machine.
2.3.2 Each job i has a deadline di≥ 0 and profit pi≥0 .
2.3.3 Pi is earned iff the job is completed by its deadline.
2.3.4 The job is completed if it is processed on a machine for unit time.
2.3.5 Only one machine is available for processing jobs.
2.3.6 Only one job is processed at a time on the machine
2.3.7 A feasible solution is a subset of jobs J such that each job is completed by its deadline.
2.3.8 An optimal solution is a feasible solution with maximum profit value.
Example : Let n = 4, (p1,p2,p3,p4) = (100,10,15,27), (d1,d2,d3,d4) = (2,1,2,1)

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

2.3.9 Consider the jobs in the non increasing order of profits subject to the constraint that the
resultingjob sequence J is a feasible solution.
2.3.10 In the example considered before, the non-increasing profit
vector is(100 27 15 10) (2 1 2 1)
p1 p4 p3 p2 d1 d d3 d2
J = { 1} is a feasible one
J = { 1, 4} is a feasible one with processing sequence ( 4,1)
J = { 1, 3, 4} is not feasible
J = { 1, 2, 4} is not feasible
J = { 1, 4} is optimal

Theorem: Let J be a set of K jobs and

Σ = (i1,i2,….ik) be a permutation of jobs in J such that di1 ≤ di2 ≤…≤ dik.
2.3.11 J is a feasible solution iff the jobs in J can be processed in the order Σ without
violating any deadly.
Proof:
2.3.12 By definition of the feasible solution if the jobs in J can be processed in the order
without violating any deadline then J is a feasible solution.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

2.3.13 So, we have only to prove that if J is a feasible one, then Σ represents a possible order in
whichthe jobs may be processed.
2.3.14 Suppose J is a feasible solution. Then there exists Σ1 = (r1,r2,…,rk)
such thatdrj ≥ j, 1 ≤ j <k
i.e. dr1 ≥1, dr2 ≥ 2, …, drk ≥ k.
each job requiring an unit time.

2.3.15 Σ = (i1,i2,…,ik) and Σ1 = (r1,r2,…,rk)

2.3.16 Assume Σ 1 ≠ Σ. Then let a be the least index in which Σ 1 and Σ differ. i.e. a is such that ra ≠
i a.
2.3.17 Let rb = ia, so b > a (because for all indices j less than a rj = ij).
2.3.18 In Σ 1 interchange ra and rb.
2.3.19 Σ = (i1,i2,… ia ib ik ) [rb occurs before ra
2.3.20 in i1,i2,…,ik]
2.3.21 Σ 1 = (r1,r2,… ra rb … rk )
2.3.22 i1=r1, i2=r2,…ia-1= ra-1, ia ≠ rb but ia = rb
2.3.23 We know di1 ≤ di2 ≤ … dia ≤ dib ≤… ≤ dik.
2.3.24 Since ia = rb, drb ≤ dra or dra ≥ drb.
2.3.25 In the feasible solution dra ≥ a drb ≥ b
2.3.26 So if we interchange ra and rb, the resulting permutation Σ11= (s1, … sk) represents an order
withthe least index in which Σ11 and Σ differ is incremented by one.
2.3.27 Also the jobs in Σ11 may be processed without violating a deadline.
2.3.28 Continuing in this way, Σ1 can be transformed into Σ without violating any deadline.
2.3.29 Hence the theorem is proved
GREEDY ALGORITHM FOR JOB SEQUENSING WITH DEADLINE

Procedure greedy job (D, J, n) J may be represented by

// J is the set of n jobs to be completed // one dimensional array J (1: K)
// by their deadlines // The deadlines are
J {1} D (J(1)) ≤ D(J(2)) ≤ .. ≤ D(J(K))
for I  2 to n do To test if JU {i} is feasible,

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

If all jobs in JU{i} can be completed we insert i into J and verify

by their deadlines D(J®) ≤ r 1 ≤ r ≤ k+1
then J  JU{I}
end if
repeat
end greedy-job
Procedure JS(D,J,n,k)
// D(i) ≥ 1, 1≤ i ≤ n are the deadlines //
// the jobs are ordered such that //
// p1 ≥ p2 ≥ ……. ≥ pn //
// in the optimal solution ,D(J(i) ≥ D(J(i+1)) //
// 1 ≤ i ≤ k //
integer D(o:n), J(o:n), i, k, n, r
D(0) J(0)  0
// J(0) is a fictious job with D(0) = 0 //
K1; J(1) 1 // job one is inserted into J //
for i 2 to do // consider jobs in non increasing order of pi //
// find the position of i and check feasibility of insertion //
r k // r and k are indices for existing job in J //
// find r such that i can be inserted after r //
while D(J(r)) > D(i) and D(i) ≠ r do
// job r can be processed after i and //
// deadline of job r is not exactly r //
r r-1 // consider whether job r-1 can be processed after i //
repeat
if D(J(r)) ≥ d(i) and D(i) > r then
// the new job i can come after existing job r; insert i into J at position r+1 //
for I  k to r+1 by –1 do
J(I+1) J(l) // shift jobs( r+1) to k right by//
//one position //
repeat

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

if D(J(r)) ≥ d(i) and D(i) > r then

// the new job i can come after existing job r; insert i into J at position r+1 //
for I  k to r+1 by –1 do
J(I+1) J(l) // shift jobs( r+1) to k right by//
//one position //
Repeat
COMPLEXITY ANALYSIS OF JS ALGORITHM
2.3.30 Let n be the number of jobs and s be the number of jobs included in the solution.
2.3.31 The loop between lines 4-15 (the for-loop) is iterated (n-1)times.
2.3.32 Each iteration takes O(k) where k is the number of existing jobs.
∴ The time needed by the algorithm is 0(sn) s ≤ n so the worst case time is 0(n2).
If di = n - i+1 1 ≤ i ≤ n, JS takes θ(n2) time
D and J need θ(s) amount of space.
2.4 Minimum-Cost Spanning Trees
Spanning Tree
Spanning tree is a connected acyclic sub-graph (tree) of the given graph (G) that includes
all of G‘s vertices

Example: Consider the following graph

1 b
a

5 2
d

c 3

The spanning trees for the above graph are as follows:

1 1
1 b b
b
a a
a
5 2 d d
c c 3
d

Weight (T2) = 8 Weight (T3) = 6

Weight (T1) = 9

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

Minimum Spanning Tree (MST)

Definition:
MST of a weighted, connected graph G is defined as: A spanning tree of G with
minimum total weight.
Example: Consider the example of spanning tree:
For the given graph there are three possible spanning trees. Among them the spanning
tree with the minimum weight 6 is the MST for the given graph

Question: Why can‘t we use BRUTE FORCE method in constructing MST ?

Answer: If we use Brute force method-
• Exhaustive search approach has to be applied.
• Two serious obstacles faced:
1. The number of spanning trees grows exponentially with graph size.
2. Generating all spanning trees for the given graph is not easy.
MST Applications:
• Network design.
Telephone, electrical, hydraulic, TV cable, computer, road
• Approximation algorithms for NP-hard problems.
Traveling salesperson problem, Steiner tree
• Cluster analysis.
• Reducing data storage in sequencing amino acids in a protein
• Learning salient features for real-time face verification
• Auto config protocol for Ethernet bridging to avoid cycles in a network, etc
2.5 Prim’s Algorithm
Some useful definitions:
2.5.1 Fringe edge: An edge which has one vertex is in partially constructed tree
Ti andthe other is not.
2.5.2 Unseen edge: An edge with both vertices not in Ti

Algorithm:
ALGORITHM Prim (G)
//Prim‘s algorithm for constructing a MST
//Input: A weighted connected graph G = { V, E }
//Output: ET the set of edges composing a MST of G
// the set of tree vertices can be initialized with any vertex
VT → { v0}
ET → Ø
for i→ 1 to |V| - 1 do
Find a minimum-weight edge e* = (v*, u*) among all the edges (v, u) such
that v is in VT and u is in V - VT
VT → VT U { u*}

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

ET → ET U { e*}
return ET
STEP 1: Start with a tree, T0, consisting of one vertex
STEP 2: ―G row‖ tree one vertex/edge at a time
2.5.2.1 Construct a series of expanding sub-trees T1, T2, … Tn-1.
2.5.2.2 At each stage construct Ti + 1 from Ti by adding the minimum weight
edge connecting a vertex in tree (Ti) to one vertex not yet in tree, choose
from “fringe” edges (this is the “greedy” step!)
Algorithm stops when all vertices are included

Example:
Apply Prim‘s algorithm for the following graph to find MST.

1
b c
3 4 4 6

a 5 f 5
d
2
6 8
e

Solution:

Tree Remaining Graph

vertices vertices
b(a,3) b
c(-,∞) 3
a ( -, - ) d(-,∞)
e(a,6) a
f(a,5)

1
c(b,1) b c
3
d(-,∞)
b ( a, 3 )
e(a,6) a
f(b,4)

1 c
b
d(c,6) 3
c ( b, 1 ) e(a,6) 4
f(b,4)
a f

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

1
b c
3 4

d(f,5) a f
f ( b, 4)
e(f,2)
2

1
b c
3 4

e ( f, 2) d(f,5) a f 5
d
2

Algorithm stops since all vertices

are included.
d( f, 5)
The weight of the minimum spanning
tree is 15

Efficiency:
Efficiency of Prim‘s algorithm is based on data structure used to store priority queue.
2.5.3 Unordered array: Efficiency: Θ(n2)
2.5.4 Binary heap: Efficiency: Θ(m log n)
2.5.5 Min-heap: For graph with n nodes and m edges: Efficiency: (n + m) log n

Conclusion:
2.5.6 Prim‘s algorithm is a ―evrtex based algorithm‖
2.5.7 Prim‘s algorithm ― Needs priority queue for locating the nearest vertex.‖
The choice of priority queue matters in Prim implementation.
o Array - optimal for dense graphs
o Binary heap - better for sparse graphs
o Fibonacci heap - best in theory, but not in practice

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

2.6 Kruskal’s Algorithm

Algorithm:

ALGORITHM Kruskal (G)

//Kruskal‘s algorithm for constructing a MST
//Input: A weighted connected graph G = { V, E }
//Output: ET the set of edges composing a MST of G

Sort E in ascending order of the edge weights

// initialize the set of tree edges and its size

ET→Ø
edge_counter →0

//initialize the number of processed edges

K →0
while edge_counter < |V| - 1
k→k + 1
if ET U { ei k} is acyclic
ET →ET U { ei k }
edge_counter →edge_counter + 1
return ET

The method:
STEP 1: Sort the edges by increasing weight
STEP 2: Start with a forest having |V| number of trees.
STEP 3: Number of trees are reduced by ONE at every inclusion of an edge
At each stage:
• Among the edges which are not yet included, select the one with minimum
weight AND which does not form a cycle.
• the edge will reduce the number of trees by one by combining two trees of
the forest

Algorithm stops when |V| -1 edges are included in the MST i.e : when the number of
trees in the forest is reduced to ONE.

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

Example:
Apply Kruskal‘s algorithm for the following graph to find MST.

1
b c
3 4 4 6
a 5 f 5
d
2
6 8
e
Solution:
The list of edges is:
Edge ab af ae bc bf cf cd df de ef
Weight 3 5 6 1 4 4 6 5 8 2

Sort the edges in ascending order:

Edge bc ef ab bf cf af df ae cd de
Weight 1 2 3 4 4 5 5 6 6 8

1
bc b
Edge c
1 f
Weight
a d
Insertion
YES
status
Insertion e
1
order

ef 1
Edge b c
2
Weight a f d
Insertion
YES
status 2
Insertion e
2
order

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

ab 1
Edge 3 b c
3
Weight a f d
Insertion
YES
status 2
Insertion e
3
order

bf 1
Edge 3 b c
4
Weight a 4 f d
Insertion
YES
status 2
Insertion e
4
order

Edge cf
Weight 4
Insertion
NO
status
Insertion
-
order

Edge af
Weight 5
Insertion
NO
status
Insertion
-
order

df 1
Edge
3 b c
5
Weight
a 4 f d
Insertion
YES 5
status
2
Insertion
5 e
order
Algorithm stops as |V| -1 edges are included in the MST

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

Efficiency:
Efficiency of Kruskal‘s algorithm is based on the time needed for sorting the edge
weights of a given graph.
2.6.1 With an efficient sorting algorithm: Efficiency: Θ(|E| log |E| )

Conclusion:
2.6.2 Kruskal‘s algorithm is an ―dege based algorithm‖
2.6.3 Prim‘s algorithm with a heap is faster than Kruskal‘s algorithm.
2.7 Single Source Shortest Paths.

Some useful definitions:

2.7.1 Shortest Path Problem: Given a connected directed graph G with non-
negativeweights on the edges and a root vertex r, find for each vertex x, a
directed path P
(x) from r to x so that the sum of the weights on the edges in the path P (x) is as
small as possible.
Algorithm
2.7.2 By Dutch computer scientist Edsger Dijkstra in 1959.
2.7.3 Solves the single-source shortest path problem for a graph with nonnegative
edgeweights.
2.7.4 This algorithm is often used in routing.
E.g.: Dijkstra's algorithm is usually the working principle behind link-state
routing protocols
ALGORITHM Dijkstra(G, s)
//Input: Weighted connected graph G and source vertex s
//Output: The length Dv of a shortest path from s to v and its penultimate vertex Pv for
every vertex v in V

//initialize vertex priority in the priority queue

Initialize (Q)
for every vertex v in V do
Dv→∞ ; Pv→null // Pv , the parent of v
insert(Q, v, Dv) //initialize vertex priority in priority queue
ds→0
//update priority of s with ds, making ds, the minimum
Decrease(Q, s, ds)

VT→0

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

for i→0 to |V| - 1 do

u*→DeleteMin(Q)
//expanding the tree, choosing the locally best vertex
VT→VT U {u*}
for every vertex u in V – VT that is adjacent to u* do
if Du* + w (u*, u) < Du
Du→Du + w (u*, u); Pu u*
Decrease(Q, u, Du)
The method
Dijkstra‘s algorithm solves the single source shortest path problem in 2 stages.
Stage 1: A greedy algorithm computes the shortest distance from source to all other
nodes in the graph and saves in a data structure.
Stage 2 : Uses the data structure for finding a shortest path from source to any vertex v.
• At each step, and for each vertex x, keep track of a “distance” D(x)
and a directed path P(x) from root to vertex x of length D(x).
• Scan first from the root and take initial paths P( r, x ) = ( r, x ) with
D(x) = w( rx ) when rx is an edge,
D(x) = ∞ when rx is not an edge.
For each temporary vertex y distinct from x, set
D(y) = min{ D(y), D(x) + w(xy) }

Example:
Apply Dijkstra‘s algorithm to find Single source shortest paths with vertex a as the
source.
1
b c
3 4 4 6

a 5 f 5
d
2
6 8
e

Solution:
Length Dv of shortest path from source (s) to other vertices v and Penultimate vertex Pv
for every vertex v in V:
Da = 0 , Pa = null
Db = ∞ , Pb = null
Dc = ∞ , Pc = null
Dd = ∞ , Pd = null
De = ∞ , Pe = null
Df = ∞ , Pf = null

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT

Design and Analysis of Algoriths Greedy Method

Tree RemainingDi stance & Path Graph

vertices vertices vertex
Da = 0 Pa = a
b ( a , 3D)b = 3 Pb = [ a, b ] b
c ( - , ∞D)c = ∞ Pc = null 3
a ( -, 0 ) d ( - , ∞D)d = ∞ Pd = null a
e ( a , 6D)e = 6 Pe = [ a, e ]
f ( a , 5D)f = 5 Pf = [ a, f ]

Da = 0 Pa = a 1
c ( b , 3+D1b)= 3 Pb = [ a, b ] b c
3
d ( - , ∞D)c = 4 Pc = [a,b,c]
b ( a, 3 )
e ( a , 6D)d = ∞ Pd = null a
f ( a , 5D)e = 6 Pe = [ a, e ]
Df = 5 Pf = [ a, f ]
Da = 0 Pa = a
Db =
d ( c , 4+6 ) 3 Pb = [ a, b ]
Dc = 5
e(a, 4 Pc = [a,b,c]
c ( b, 4 ) 6)Dd= 10 a f
f(a,5) Pd = [a,b,c,d]
De = 6 Pe = [ a, e ]
Df = 5 Pf = [ a, f ]
Da = 0 Pa = a
Db = 3 Pb = [ a, b ]
Dc = 4 Pc = [a,b,c] a
d ( c , 10D)d = 10 Pd = [a,b,c,d]
f ( a, 5) 6
e ( a , 6D)e = 6 Pe = [ a, e ] e
Df = 5 Pf = [ a, f ]

Da = 0 Pa = a 1
Db = 3 Pb = [ a, b ] b c
Dc = 4 Pc = [a,b,c]
e ( a, 6) d ( c, 10 )d=1 0 Pd = [a,b,c,d]
D 3 6
De = 6 Pe = [ a, e ] d
Df = 5 Pf = [ a, f ] a

Algorithm stops since no

d( c, 10)
edges to scan

Conclusion:
2.7.5 Doesn‘t work with negative weights
2.7.6 Applicable to both undirected and directed graphs
2.7.7 Use unordered array to store the priority queue: Efficiency = Θ(n2)
2.7.8 Use min-heap to store the priority queue: Efficiency = O(m log n)

Prepared by S.Rakesh, Asst.Prof. , IT Dept, CBIT