Lecture 4 – Software Threats and
Vulnerabilities
CS Department
City University of Hong Kong
Slides partially adapted from lecture notes by M. Goodrich&R. Tamassia,
W. Stallings&L. Brown, Dan Boneh, and Dawn Song.
An Information Security Short Course
1
(Summer 2020)
Why Software Security?
• You may have a perfect design, a perfect
specification, perfect algorithms, but still have
implementation vulnerabilities.
• After configuration errors, implementation errors are
probably the largest single class of security errors
exploited in practice.
• Flaws in software break certain assumptions
important for security
An Information Security Short Course
2
(Summer 2020)
Operating Systems Concepts
An Information Security Short Course
3
(Summer 2020)
A Computer Model
• An operating system has to deal with the fact
that a computer is made up of a CPU, random
access memory (RAM), input/output (I/O)
devices, and long-term storage.
0
1
2
3
I/O CPU 4
5 RAM Disk Drive
6
7
8
4
An Information Security Short Course 9
(Summer 2020)
.
OS Concepts
• An operating system (OS) provides the interface
between the users of a computer and that
computer’s hardware.
– An operating system manages the ways applications access
the resources in a computer, including its disk drives, CPU,
main memory, input devices, output devices, and network
interfaces.
– An operating system manages multiple users.
– An operating system manages multiple programs.
An Information Security Short Course
5
(Summer 2020)
Multitasking
• Give each running program a “slice” of the CPU’s time.
• The CPU is running so fast that to any user it appears that the
computer is running all the programs simultaneously.
An Information Security Short Course
6
(Summer 2020)
The Kernel
• The kernel is the core component of
the operating system. It handles the
management of low-level hardware
resources, including memory, User Applications Userland
processors, and input/output (I/O)
devices, such as a keyboard, mouse, Non-essential OS
or video display. Applications
• Most operating systems define the Operating System
tasks associated with the kernel in The OS Kernel
terms of a layer metaphor, with the
hardware components, such as the CPU, Memory,
Hardware
Input/Output
CPU, memory, and input/output
devices being on the bottom, and
users and applications being on the
top.
An Information Security Short Course
7
(Summer 2020)
Input/Output
• The input/output devices of a computer include things
like its keyboard, mouse, video display, and network
card, as well as other more optional devices, like a
scanner, Wi-Fi interface, video camera, USB ports, etc.
• Each such device is represented in an operating system
using a device driver, which encapsulates the details of
how interaction with that device should be done.
– The application programmer interface (API), which the
device drivers present to application programs, allows those
programs to interact with those devices at a fairly high level,
while the operating system does the “heavy lifting” of
performing the low-level interactions that make such devices
actually work.
An Information Security Short Course
8
(Summer 2020)
System Calls
• User applications don’t communicate directly with
low-level hardware components, and instead
delegate such tasks to the kernel via system calls.
• System calls are usually contained in a collection of
programs, that is, a library such as the C library (libc),
and they provide an interface that allows applications
to use a predefined series of APIs that define the
functions for communicating with the kernel.
– Examples of system calls include those for performing file
I/O (open, close, read, write) and running application
programs (exec).
An Information Security Short Course
9
(Summer 2020)
Processes
• A process is an instance of a program
that is currently executing.
• The actual contents of all programs are
initially stored in persistent storage,
such as a hard drive.
• In order to be executed, a program
must be loaded into random-access
memory (RAM) and uniquely identified
as a process.
• In this way, multiple copies of the same
program can be run as different
processes.
– For example, we can have multiple copies of
MS Powerpoint open at the same time.
An Information Security Short Course
10
(Summer 2020)
Process IDs
• Each process running on a given computer is identified by a
unique nonnegative integer, called the process ID (PID).
• Given the PID for a process, we can then associate its CPU
time, memory usage, user ID (UID), program name, etc.
An Information Security Short Course
11
(Summer 2020)
Memory Management
• The RAM memory of a computer is its address space.
• It contains both the code for the running program, its
input data, and its working memory.
• For any running process, it is organized into different
segments, which keep the different parts of the
address space separate.
• As we will discuss, security concerns require that we
never mix up these different segments.
An Information Security Short Course
12
(Summer 2020)
Control Hijacking
Buffer-overflow and
Memory Safety
An Information Security Short Course
13
(Summer 2020)
An exploit is any input (i.e., a piece of software, an
argument string, or sequence of commands) that
takes advantage of a bug, glitch or vulnerability
in order to cause an attack
An Information Security Short Course
14
(Summer 2020)
Control hijacking
• Attacker’s goal:
– Take over target machine
• Execute arbitrary code on target by hijacking application
control flow
– Webserver à server target
– Email reader , web browser à client target
• Examples.
– Buffer overflow attacks
– Integer overflow attacks
– Format string vulnerabilities
– Race condition vulnerabilities
An Information Security Short Course
15
(Summer 2020)
Example 1: buffer overflows
• Extremely common bug in C/C++ programs.
– First major exploit: 1988 Internet Worm. fingerd. (a simple background
process on server)
# of vulnerable 600
software product
500 »20% of all vuln.
400 2005-2007: » 10%
300
200
100 Source: NVD/CVE
0
1995 1997 1999 2001 2003 2005
What is needed
• Understanding C functions, memory manager, the stack, the heap.
• Know how system calls are made
• The execve() system call
• Attacker needs to know which CPU and OS used on the target
machine:
– Our examples are for x86 running Linux or Windows
– Details vary slightly between CPUs and OSs:
• Little endian vs. big endian (x86 vs. Motorola)
– Decimal 1000 (hexadecimal 3E8) in two bytes
» big endian 03 E8
» little endian E8 03
• Stack Frame structure (Unix vs. Windows)
An Information Security Short Course
18
(Summer 2020)
structure that grows downwards
and keeps track of the activated
method calls, their arguments
Newly allocated variables and local variables
will stay below the previous
(existing) variables in stack.
e.g. libc
data dynamically generated
during the execution of a
process. e.g., malloc()
static program variables
initialized in the source code
prior to execution.
machine code of the program,
compiled from the source code 19
An Information Security Short Course
Stack Frame
Every time a function call is executed, a new frame is pushed onto the stack.
high
Arguments from the
arguments function being called
The address control
return address flow is supposed to
return, when the
stack frame pointer function terminates.
exception handlers e.g.: Try-catch block
local variables Stack
Growth
callee saved registers
SP low
An Information Security Short Course
20
(Summer 2020)
An Information Security Short Course
21
(Summer 2020)
out[i] = ‘\0’;
An Information Security Short Course
22
(Summer 2020)
What are buffer overflows?
Suppose a web server contains a function: void func(char *str) {
char buf[128];
When func() is called stack looks like: strcpy(buf, str);
do-something(buf);
}
argument: str
return address
stack frame pointer
char buf[128]
SP
An Information Security Short Course
23
(Summer 2020)
What are buffer overflows?
• One of the most common OS bugs is a buffer overflow
– The developer fails to include code that checks whether an input
string fits into its buffer array
– An input to the running process exceeds the length of the buffer
– The input string overwrites a portion of the memory of the process
– Causes the application to behave improperly and unexpectedly
• Effect of a buffer overflow
– The process can operate on malicious data or execute malicious code
passed in by the attacker
– If the process is executed as root, the malicious code will be
executing with root privileges
An Information Security Short Course
24
(Summer 2020)
What are buffer overflows?
What if *str is 136 bytes long? void func(char *str) {
After strcpy: char buf[128];
strcpy(buf, str);
do-something(buf);
}
argument: str
return address
stack frame pointer
*str Problem:
no length checking in strcpy()
char buf[128]
SP An Information Security Short Course
25
(Summer 2020)
Another Example with strcpy() Vulnerability
Top of
Memory
domain.c 0xFFFFFFFF
Main(int argc, char *argv[ ]) Stack
/* get user_input */ Fill
{ Direction
char var1[15];
char command[20];
strcpy(command, “whois "); var1 (15 char)
strcat(command, argv[1]);
strcpy(var1, argv[1]); command
printf(var1); (20 char)
system(command);
} ..
.
• Retrieves domain registration info Bottom of
• e.g., domain brown.edu Memory
An Information Security Short Course 0x00000000
26
(Summer 2020)
Another Example with strcpy() Vulnerability
domain.c Top of
Main(int argc, char *argv[]) Memory
/*get user_input*/ 0xFFFFFFFF Stack
{ Fill
char var1[15]; Direction
char command[20];
strcpy(command, “whois "); argv[1]
var1argv[1]
(15 char)
strcat(command, argv[1]); (15 char)
(20 char)
strcpy(var1, argv[1]); Overflow
printf(var1); command
exploit
system(command); (20 char)
}
..
• argv[1] is the user input .
• strcpy(dest, src) does not check buffer Bottom of
• strcat(d, s) concatenates strings Memory
0x00000000
An Information Security Short Course
27
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
28
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
29
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
30
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
31
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
32
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
33
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
34
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
35
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
36
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
37
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
38
(Summer 2020)
A Deeper View of Buffer Overflow
An Information Security Short Course
39
(Summer 2020)
Buffer Overflow Attack in a Nutshell
• First described in
Aleph One. Smashing The Stack For Fun And Profit. e-zine
www.Phrack.org #49, 1996
• The attacker exploits an unchecked buffer to perform a
buffer overflow attack
• The ultimate goal for the attacker is getting a shell that
allows to execute arbitrary commands with high
privileges
– Vulnerability scenarios: the targeted victim program
has root privileges (setuid)
An Information Security Short Course
40
(Summer 2020)
Basic Stack Exploit
• Overwriting the return address allows an attacker
to redirect the flow of program control
• Instead of crashing, this can allow arbitrary code
to be executed
– Code segment is called “shellcode”
• Example, the execve system call is used to
execute a file
– With the correct permissions, execve(“/bin/sh”) can
be used to obtain a root level shell
An Information Security Short Course
41
(Summer 2020)
Basic stack exploit high
Suppose *str is such that Program P
after strcpy stack looks like:
Program P: execve(“/bin/sh”)
return address
When func() exits, the user gets shell !
Note: attack code P runs in stack.
char buf[128]
low
An Information Security Short Course
42
(Summer 2020)
Shellcode of execve
How to develop shellcode that runs as execve(“/bin/sh”)?
(exact shell code by Aleph One)
An Information Security Short Course
43
(Summer 2020)
An Information Security Short Course
44
(Summer 2020)
An Information Security Short Course
45
(Summer 2020)
An Information Security Short Course
46
(Summer 2020)
An Information Security Short Course
47
(Summer 2020)
An Information Security Short Course
48
(Summer 2020)
An Information Security Short Course
49
(Summer 2020)
An Information Security Short Course
50
(Summer 2020)
The NOP slide high
Problem: how does attacker
determine ret-address?
Solution: NOP slide
• Guess approximate stack state
when func() is called
– Install the program and perform random
fuzzy testing
• Insert many NOPs before program P:
nop ‘/x90’ , xor eax,eax , inc ax
low
An Information Security Short Course
51
(Summer 2020)
An Information Security Short Course
52
(Summer 2020)
More on Stack Smashing
• Some complications:
– Shellcode should not contain the ‘\0’ character.
– Overflow should not crash program before func()
exists.
• Sample remote stack smashing overflows:
– (2007) Overflow in Windows animated cursors (ANI).
LoadAniIcon()
– (2005) Overflow in Symantec Virus Detection
test.GetPrivateProfileString "file", [long string]
An Information Security Short Course
53
(Summer 2020)
Many unsafe libc functions
strcpy (char *dest, const char *src)
strcat (char *dest, const char *src)
gets (char *s)
scanf ( const char *format, … ) and many more.
• “Safe” libc versions strncpy(), strncat() are misleading
– Function strncpy() copies the string by specifying the number n of
characters to copy
• e.g., strncpy(dest, src, n); dest[n] = ‘\0’
– strncpy() may leave string unterminated.
• If source string is longer than the destination string, the overflow
characters are discarded automatically
– You have to place the null character manually
• Windows C run time (CRT):
– strcpy_s (*dest, DestSize, *src):
An Information Securityensures
Short Course proper termination
54
(Summer 2020)
Buffer overflow opportunities
• Exception handlers: (Windows SEH attacks)
– Overwrite the address of an exception handler in stack
frame.
• Function pointers: (e.g. PHP 4.0.2, MS MediaPlayer Bitmaps)
Heap
buf[128] FuncPtr or
stack
– Overflowing buf will override function pointer.
• Longjmp buffers: longjmp(pos) (e.g. Perl 5.003)
– Overflowing bufAnnext to pos overrides value of pos.
Information Security Short Course
55
(Summer 2020)
Corrupting Function Pointers in Heap
• Compiler generated function pointers (e.g. C++ code)
FP1
method #1
ptr FP2 method #2
FP3
method #3
data vtable
Object T
NOP shell
• After overflow of buf : slide code
data
buf[256] vtable
ptr
An Information Security Short Course 56
(Summer 2020) object T
General Control Hijacking:
Corrupting Function Pointers in Heap
shellcode
shellcode
shellcode
Overwrite Step: Overwrite entries in a vtable for Object T
Activate Step: Call any method from Object T
An Information Security Short Course
57
(Summer 2020)
General Control Hijacking:
Corrupting Function Pointers in Heap
Overwrite Step: Overwrite pointer to vtable on heap to point
to a crafted vtable.
An Information Security Short Course
Activate Step: Call any method from Object T (Summer 2020) 58
General Control Hijacking
Overwrite Step: Find some way to modify a Control Flow
Pointer to point to your shellcode, library entry point, or
Other code of interest..
Activate Step: Find some way to activate that modified
Control Flow Pointer
An Information Security Short Course
59
(Summer 2020)
Instances of Control Hijacking
An Information Security Short Course
60
(Summer 2020)
Finding buffer overflows
• To find overflow:
– Run web server on local machine
– Issue malformed requests (ending with “$$$$$” )
• Many automated tools exist (called fuzzers – next module)
– If web server crashes,
search core dump for “$$$$$” to find overflow location
• Construct exploit (not easy given latest defenses)
An Information Security Short Course
61
(Summer 2020)
Control Hijacking
More Control
Hijacking Attacks
An Information Security Short Course
62
(Summer 2020)
More Hijacking Opportunities
• Integer overflows: (e.g. MS DirectX MIDI Lib)
• Double free: double free space on heap.
– Can cause memory mgr to write data to specific location
– Examples: CVS server
• Format string vulnerabilities
An Information Security Short Course
63
(Summer 2020)
Integer Overflows (see Phrack 60)
Problem: what happens when int exceeds max value?
int m; (32 bits) short s; (16 bits) char c; (8 bits)
c = 0x80 + 0x80 = 128 + 128 ⇒ c=0
s = 0xff80 + 0x80 ⇒ s=0
m = 0xffffff80 + 0x80 ⇒ m=0
Can this be exploited?
An Information Security Short Course
64
(Summer 2020)
An example
void func( char *buf1, *buf2, unsigned int len1, len2) {
char temp[256];
if (len1 + len2 > 256) {return -1} // length check
memcpy(temp, buf1, len1); // cat buffers
memcpy(temp+len1, buf2, len2);
do-something(temp); // do stuff
}
What if len1 = 0x80, len2 = 0xffffff80 ?
⇒ len1+len2 = 0
Second memcpy() will overflow heap !!
An Information Security Short Course
65
(Summer 2020)
End of Segment
An Information Security Short Course
66
(Summer 2020)
