Lecture4 Sorting
Lecture4 Sorting
Lecture Note #4
Sorting
Objectives
1
Programs used in this lecture
◼ SelectionSort.java
◼ BubbleSort.java, BubbleSortImproved.java
◼ InsertionSort.java
◼ MergeSort.java
◼ QuickSort.java
◼ Sort.java, Sort2.java
◼ Person.java, AgeComparator.java,
NameComparator.java, TestComparator.java
2
Why Study Sorting?
◼ When an input is sorted by some sort key, many
problems become easy (eg. searching, min,
max, kth smallest, etc.)
Q: What is a sort key?
◼ Sorting has a variety of interesting algorithmic
solutions, which embody many ideas:
❑ Internal sort vs external sort
❑ Iterative vs recursive
❑ Comparison vs non-comparison based
❑ Divide-and-conquer
❑ Best/worst/average case time complexity bounds
3
Sorting applications
◼ Uniqueness testing
◼ Deleting duplicates
◼ Frequency counting
◼ Efficient searching
◼ Dictionary
◼ Telephone/street directory
◼ Index of book
◼ Author index of conference proceedings
◼ etc.
4
Outline
▪ Comparison based and Iterative algorithms
1. Selection Sort
2. Bubble Sort
3. Insertion Sort
▪ Comparison based and Recursive algorithms
4. Merge Sort
5. Quick Sort
▪ Non-comparison based
6. Radix Sort
7. Comparison of Sort Algorithms
▪ In-place sort
▪ Stable sort
8. Use of Java Sort Methods Note: In the lecture, we consider
only sorting in ascending order of
data.
5
1 Selection Sort
1 Idea of Selection Sort
◼ Given an array of n items
1. Find the largest item.
2. Swap it with the item at the end of the array.
3. Go to step 1 by excluding the largest item
from the array.
6
1 Selection Sort of 5 integers
29 10 14 13 37
13 10 14 29 37
13 10 14 29 37
10 13 14 29 37 Sorted!
7
1 Code of Selection Sort
public static void selectionSort(int[] a) {
for (int i = a.length-1; i >= 1; i--) {
int index = i; // i is the last item position and
// index is the largest element position
// loop to get the largest element
for (int j = 0; j < i; j++) {
if (a[j] > a[index])
index = j; // j is the current largest item
}
// Swap the largest item a[index] with the last item a[i]
int temp = a[index];
a[index] = a[i];
a[i] = temp;
}
}
SelectionSort.java
8
1 Analysis of Selection Sort
public static void selectionSort(int[] a)
{ Number of times the
statement is executed:
9
2 Bubble Sort
2 Idea of Bubble Sort
◼ “Bubble” down the largest item to the end of the
array in each iteration by examining the i-th and
(i+1)-th items
◼ If their values are not in the correct order, i.e.
a[i] > a[i+1], swap them.
1 4 6 9 1 7 5 9
i i+1 i i+1
10
2 Example of Bubble Sort
◼ The first two passes of Bubble Sort for an array of 5
integers
11
2 Code of Bubble Sort
public static void bubbleSort(int[] a) {
for (int i = 1; i < a.length; i++) {
for (int j = 0; j < a.length - i; j++) {
if (a[j] > a[j+1]) { // the larger item bubbles down (swap)
int temp = a[j];
a[j] = a[j+1];
a[j+1] = temp;
}
}
}
} BubbleSort.java
12
2 Analysis of Bubble Sort
◼ 1 iteration of the inner loop (test and swap) requires time
bounded by a constant c
◼ Doubly nested loops: public static void bubbleSort(int[ ] a) {
for (int i = 1; i < a.length; i++) {
❑ Outer loop: exactly n-1 iterations for (int j = 0; j < a.length - i; j++) {
if (a[j] > a[j+1]) { // (swap)
❑ Inner loop: int temp = a[j];
a[j] = a[j+1];
◼ When i=1, (n-1) iterations a[j+1] = temp;
◼ When i=2, (n-2) iterations }
}
◼ … }
◼ When i=(n-1), 1 iteration }
13
2 Bubble Sort can be improved
14
2 Code of Bubble Sort (Improved version)
public static void bubbleSort2(int[] a) {
for (int i = 1; i < a.length; i++) {
→boolean isSorted = true; // isSorted = true if a[] is sorted
for (int j = 0; j < a.length-i; j++) {
if (a[j] > a[j+1]) { // the larger item bubbles up
int temp = a[j]; // and isSorted is set to false,
a[j] = a[j+1]; // i.e. the data was not sorted
a[j+1] = temp;
→isSorted = false;
}
}
→if (isSorted) return; // Why?
}
}
BubbleSortImproved.java
15
2 Analysis of Bubble Sort (Improved version)
◼ Worst case
❑ Input in descending order (any other situation?)
❑ How many iterations in the outer loop are needed?
Answer: n-1 iterations
❑ Running time remains the same: O(n2)
◼ Best case
❑ Input is already in ascending order
❑ The algorithm returns after a single iteration in the
outer loop. (Why?)
❑ Running time: O(n)
16
3 Insertion Sort
3 Idea of Insertion Sort
◼ Arranging a hand of poker cards
❑ Start with one card in your hand
❑ Pick the next card and insert it into its proper sorted
order
❑ Repeat previous step for all the rest of the cards
17
3 Example of Insertion Sort
◼ n=4 S1 S2
◼ Given a seq: 40 13 20 8
◼ i=1 13 40 20 8
◼ i=2 13 20 40 8
◼ i=3 8 13 20 40
◼ n = no of items to be sorted
◼ S1 = sub-array sorted so far
◼ S2 = elements yet to be processed
◼ In each iteration, how to insert the next element into
S1 efficiently?
18
3 Code of Insertion Sort
public static void insertionSort(int[] a) {
for (int i=1;i<a.length;i++) { //Q: Why i starts from 1?
// a[i] is the next data to insert
int next = a[i];
// Scan backwards to find a place. Q: Why not scan forwards?
int j; // Q: Why is j declared here?
// Q: What if a[j] <= next?
for (j=i-1; j>=0 && a[j]>next; j--)
a[j+1] = a[j];
// Now insert the value next after index j at the end of loop
a[j+1] = next;
}
}
InsertionSort.java
}
20
4 Merge Sort
4 Idea of Merge Sort (1/3)
◼ Suppose we only know how to merge two sorted
lists of elements into one combined list
◼ Given an unsorted list of n elements
◼ Since each element is a sorted list, we can
repeatedly…
❑ Merge each pair of lists, each list containing one
element, into a sorted list of 2 elements.
❑ Merge each pair of sorted lists of 2 elements into a
sorted list of 4 elements.
❑ …
❑ The final step merges 2 sorted lists of n/2 elements to
obtain a sorted list of n elements.
21
4 Idea of Merge Sort (2/3)
◼ Divide-and-conquer method solves problem by
three steps:
❑ Divide Step: divide the larger problem into smaller
problems.
❑ (Recursively) solve the smaller problems.
❑ Conquer Step: combine the results of the smaller
problems to produce the result of the larger problem.
22
4 Idea of Merge Sort (3/3)
◼ Merge Sort is a divide-and-conquer sorting
algorithm
❑ Divide Step: Divide the array into two (equal) halves.
❑ (Recursively) Merge sort the two halves.
❑ Conquer Step: Merge the two sorted halves to form a
sorted array.
◼ Q: What are the base cases?
23
4 Example of Merge Sort
7 2 6 3 8 4 5
Divide into
two halves 7 2 6 3 8 4 5
Recursively
sort the halves 2 3 6 7 4 5 8
24
4 Code of Merge Sort
... mergeSort(int[] a, int i, int j) {
// to sort data from a[i] to a[j], where i<j
if (i < j) { // Q: What if i >= j?
int mid = (i+j)/2; // divide
mergeSort(a, i, mid); // recursion
mergeSort(a, mid+1, j);
merge(a,i,mid,j); //conquer: merge a[i..mid] and
//a[mid+1..j] back into a[i..j]
}
}
MergeSort.java
25
4 Merge Sort of a 6-element Array (1/2)
mergeSort(a,i,mid);
mergeSort(a,mid+1,j);
merge(a,i,mid,j);
38 16 27 39 12 27
38 16 27 39 12 27
38 16 27 39 12 27
38 16 39 12
16 38 12 39
16 27 38 12 27 39
12 16 27 27 38 39
26
4 Merge Sort of a 6-element Array (2/2)
mergeSort(a,i,mid);
mergeSort(a,mid+1,j);
merge(a,i,mid,j);
38 16 27 39 12 27
38 16 27 39 12 27 Divide phase:
Recursive call to
38 16 27 39 12 27 mergeSort
38 16 39 12
16 38 12 39
Conquer phase:
16 27 38 12 27 39 Merge steps
The sorting is done
here
12 16 27 27 38 39
27
4 How to Merge 2 Sorted Subarrays?
Temp array
t[0..5] a[0..2] a[3..5]
245 378
2 245 378
2 3 245 378
2 3 4 245 378
2 3 4 5 245 378
2 3 4 5 7 8 245 378
28
4 Merge Algorithm (1/2)
... merge(int[] a, int i, int mid, int j) {
// Merges the 2 sorted sub-arrays a[i..mid] and
// a[mid+1..j] into one sorted sub-array a[i..j]
29
4 Merge Algorithm (2/2)
// Copy the remaining elements into temp. Q: Why?
while (left<=mid) temp[it++] = a[left++];
while (right<=j) temp[it++] = a[right++];
// Q: Will both the above while statements be executed?
30
4 Analysis of Merge Sort (1/3)
◼ In Merge Sort, the bulk of work is done in the Merge step
merge(a, i, mid, j)
◼ Total number of items = k = j – i + 1
❑ Number of comparisons k – 1 (Q: Why not = k – 1?)
❑ Number of moves from original array to temp array = k
❑ Number of moves from temp array to original array = k
31
4 Analysis of Merge Sort (2/3)
Level 0: Level 0:
Mergesort n items n 0 call to Merge
Level 1: Level 1:
2 calls to Mergesort n/2 items n/2 n/2 1 calls to Merge
Level 2: Level 2:
4 calls to Mergesort n/22 items n/22 n/22 n/22 n/22 2 calls to Merge
…
Level (log n):
Level (log n): 2(log n) -1(= n/2)
n calls to Mergesort 1 item 1 1 …………………… 1 1 calls to Merge
32
4 Analysis of Merge Sort (3/3)
◼ Level 0: 0 call to Merge
◼ Level 1: 1 call to Merge with n/2 items each,
O(1 2 n/2) = O(n) time
◼ Level 2: 2 calls to Merge with n/22 items each,
O(2 2 n/22) = O(n) time
◼ Level 3: 22 calls to Merge with n/23 items each,
O(22 2 n/23) = O(n) time
◼ …
◼ Level (log n): 2(log n)-1(= n/2) calls to Merge with n/2log n
(= 1) item each,
O(n/2 2 x 1) = O(n) time
◼ In total, running time = (log n)*O(n) = O(n log n)
33
4 Drawbacks of Merge Sort
◼ Implementation of merge() is not
straightforward
◼ Requires additional temporary arrays and
to copy the merged sets stored in the
temporary arrays to the original array
◼ Hence, additional space complexity = O(n)
34
5 Quick Sort
5 Idea of Quick Sort
◼ Quick Sort is a divide-and-conquer algorithm
◼ Divide Step: Choose a pivot item p and partition the
items of a[i..j] into 2 parts so that
❑ Items in the first part are < p, and
❑ Items in the second part are p.
◼ Recursively sort the 2 parts
◼ Conquer Step: Do nothing! No merging is needed.
◼ What are the base cases?
35
5 Example of Quick Sort
Pivot
Choose the 1st item as pivot 27 38 12 39 27 16
Pivot
Partition a[] about
the pivot 27 16 12 27 39 27 38
Pivot
Recursively sort
the two parts 12 16 27 27 38 39
37
5 Partition algorithm idea (1/4)
◼ To partition a[i..j], we choose a[i] as the pivot p.
❑ Why choose a[i]? Are there other choices?
◼ The remaining items (i.e. a[i+1..j]) are divided into
3 regions:
❑ S1 = a[i+1..m] where items < p
❑ S2 = a[m+1..k-1] where item p
❑ Unknown (unprocessed) = a[k..j], where items are yet to
be assigned to S1 or S2.
p <p p ?
i m k j
S1 S2 Unknown
38
5 Partition algorithm idea (2/4)
◼ Initially, regions S1 and S2 are empty. All items
excluding p are in the unknown region.
◼ Then, for each item a[k] (for k=i+1 to j) in the unknown
region, compare a[k] with p:
❑ If a[k] p, put a[k] into S2.
❑ Otherwise, put a[k] into S1.
◼ Q: How about if we change to > in the condition
part?
39
5 Partition algorithm idea (3/4)
◼ Case 1:
S1 S2
If a[k] =y p, p <p x p y ?
i m k j
S1 S2
Increment k p <p x p y ?
i m k j
40
5 Partition algorithm idea (4/4)
◼ Case 2: S1 S2
If a[k]=y < p p <p x p y ?
i m k j
Increment m p <p x p y ?
i m k j
Increment k p <p y p x ?
i m k j
41
5 Code of Partition Algorithm
... partition(int[] a, int i, int j) {
// partition data items in a[i..j]
int p = a[i]; // p is the pivot, the ith item
int m = i; // Initially S1 and S2 are empty
for (int k=i+1; k<=j; k++) { //process unknown region
if (a[k] < p) { // case 2: put a[k] to S1
m++;
swap(a,k,m);
} else { // case 1: put a[k] to S2. Do nothing!
} // else part should be removed since it is empty
}
swap(a,i,m); // put the pivot at the right place
return m; // m is the pivot’s final position
}
◼ As there is only one ‘for’ loop and the size of the array is
n = j – i + 1, so the complexity for partition() is O(n)
42
5 Partition Algorithm: Example
10 13 14 29 37
p S2
S1 is empty
1 n-1
Total no. of
1 n-2 levels = n
……
45
5 Analysis of Quick Sort: Best/Average case
46
6 Radix Sort
6 Idea of Radix Sort
◼ Treats each data to be sorted as a character
string.
◼ It is not using comparison, i.e., no comparison
among the data is needed.
◼ Hence it is a non-comparison based sort (the
preceding sorting algorithms are called comparison based
sorts)
◼ In each iteration, organize the data into groups
according to the next character in each data.
47
6 Radix Sort of Eight Integers
48
6 Pseudocode and Analysis of Radix Sort
radixSort(int[] array, int n, int d) {
// Sorts n d-digit numeric strings in the array.
for (j = d down to 1) { // for digits in last position to 1st position
initialize 10 groups (queues) to empty // Q: why 10 groups?
49
7 Comparison of Sorting
Algorithms
7 In-place Sort
◼ A sorting algorithm is said to be an in-place sort if
it requires only a constant amount, i.e. O(1), of
extra space during the sorting process.
◼ Merge Sort is not in-place. (Why?)
◼ How about Quick Sort?
50
7 Stable Sort
◼ A sorting algorithm is stable if the relative order of
elements with the same key value is preserved by
the algorithm.
◼ Example 1 – An application of stable sort:
❑ Assume that names have been sorted in alphabetical
order.
❑ Now, if this list is sorted again by tutorial group number,
a stable sort algorithm would ensure that all students in
the same tutorial groups still appear in alphabetical order
of their names.
◼ Quick Sort and Selection Sort are not stable. (Why?)
51
7 Non-Stable Sort
◼ Example 2 – Quick Sort and Selection Sort are not stable:
Quick sort:
1285 150 5* 4746 602 5+ 8356 // pivot in bold
1285 (150 5* 602 5+) (4746 8356)
5+ 150 5* 602 1285 4746 8356 //pivot swapped with the last one in S1
// the 2 5’s are in different order of the initial list
Selection sort: select the largest element and swap with the last one
4746 1285 5* 602 8356 5+ 277
4746 1285 5* 602 277 5+ (8356)
5+ 1285 5* 602 277 (4746 8356)
// the 2 5’s are in different order of the initial list
52
7 Summary of Sorting Algorithms
Worst Case Best Case In-place? Stable?
Selection Sort O(n2) O(n2) Yes No
53
8 Use of Java Sort Methods
8 Java Sort Methods (in Arrays class)
static void sort(byte[] a)
static void sort(byte[] a, int fromIndex, int toIndex)
static void sort(char[] a)
static void sort(char[] a, int fromIndex, int toIndex)
static void sort(double[] a)
static void sort(double[] a, int fromIndex, int toIndex)
static void sort(float[] a)
static void sort(float[] a, int fromIndex, int toIndex)
static void sort(int[] a)
static void sort(int[] a, int fromIndex, int toIndex)
static void sort(long[] a)
static void sort(long[] a, int fromIndex, int toIndex)
static void sort(Object[] a)
static void sort(Object[] a, int fromIndex, int toIndex)
static void sort(short[] a)
static void sort(short[] a, int fromIndex, int toIndex)
static <T> void sort(T[] a, Comparator<? super T> c)
static <T> void sort(T[] a, int fromIndex, int toIndex,
Comparator<? super T> c)
54
8 To use sort( ) in Arrays
◼ The entities to be sorted must be stored in an
array first.
◼ If they are stored in a list, then we have to use
Collections.sort()
◼ If the data to be sorted are not primitive, then
Comparator must be defined and used
55
8 Simple program using Collections.sort()
import java.util.*;
public class Sort {
public static void main(String args[]) {
List<String> list = Arrays.asList(args);
Collections.sort(list);
System.out.println(list);
}
}
Sort.java
Sort2.java
57
8 Example: class Person
class Person {
public String name;
public int age;
58
8 Comparator interface
59
8 Comparator: AgeComparator
import java.util.Comparator;
class AgeComparator implements Comparator<Person> {
public int compare(Person p1, Person p2) {
// Returns the difference:
// if positive, age of p1 is greater than p2
// if zero, the ages are equal
// if negative, age of p1 is less than p2
return p1.getAge() - p2.getAge();
}
Note: compare() and equals() are two methods of the interface Comparator.
Need to implement them.
60
8 Comparator: NameComparator
import java.util.Comparator;
class NameComparator implements Comparator<Person> {
61
8 TestComparator (1/3)
import java.util.*;
62
8 TestComparator (2/3)
System.out.println("Sorting by age:");
Collections.sort(list, ageComp);
System.out.println(list + "\n");
} // end TestComparator
TestComparator.java
63
8 TestComparator (3/3)
java TestComparator
Sorting by age:
[Mimi – 9, Sarah – 12, Mark – 12, Michael – 15, Andrew – 15]
Sorting by name:
[Andrew – 15, Mark – 12, Michael – 15, Mimi – 9, Sarah – 12]
64
8 Another solution using Arrays.sort( )
We can replace the statements
List<Person> list = Arrays.asList(p);
System.out.println("Sorting by age:");
Collections.sort(list, ageComp);
System.out.println(list + "\n");
with
System.out.println("Sorting by age using Arrays.sort():");
Arrays.sort(p, ageComp);
System.out.println(Arrays.toString(p) + "\n");
65
Summary
◼ We have introduced and analysed some classic sorting
algorithms.
◼ Merge Sort and Quick Sort are in general faster than
Selection Sort, Bubble Sort and Insertion Sort.
◼ The sorting algorithms discussed here are comparison
based sorts, except for Radix Sort which is non-
comparison based.
◼ O(n log n) is the best possible worst-case running time for
comparison based sorting algorithms.
◼ There exist Java methods to perform sorting.
66
End of file