SE506 - Software Testing

and Quality Assurance

Lecture # 1

1
Introduction - 1

 This part of this course deals with a very

important aspect of software engineering:
quality assurance of software products
and services
 We’ll learn different aspects of software
quality assurance in this course
2
Introduction - 2
 In the first few lectures, we will discuss what
software quality is and how it impacts the
development of the software development and
maintenance and other basic concepts in SQA
 In the second phase of this course, we’ll discuss
in detail the activities in each phase of the
software development lifecycle, as they relate to
software quality assurance 3
What is Quality?

 Can you define quality?

 You must be thinking, what kind of
question is that.
 It is very easy to define quality, but if you
think really hard, it is not that easy to
define quality 4
Synonyms of Quality
 Excellence
 Superiority
 Class
 Eminence
 Value
 Worth 5
Antonym of Quality
 Inferiority

6
Marketability of Quality
 Everyone claims to manufacture / develop /
sell / market “good” quality products /
services
 You will never come across a person or
company selling products or services as
low or poor quality products, even when
they are 7
Software Quality - 1
 Quality as it relates to all aspects of software
(requirements / design / code / tests / documents /
training)
 Difficult to define
 Software quality is somewhat like the concept of beauty.
 Each of us has a strong opinion about what constitutes
beauty, and we recognize it when we see it.
 But when asked to explain exactly why we regard an
object as beautiful, it is hard to put the factors into
8
Software Quality - 2

 Good software quality characteristics can

be identified

 Bad or undesirable characteristics can

also be identified

9
Software Quality Definitions

 Now we’ll discuss six key factors, which

are considered as definitions of software
quality, and we’ll use them throughout this
course

10
Software Quality

 Low levels of defects when deployed,

ideally approaching zero
 High reliability, or the capability of running
without crashes or strange results
 A majority of clients with high user-
satisfaction when surveyed 11
Software Quality
 A structure that can minimize “bad fixes” or
insertion of new defects during repairs

 Effective customer support when problems

do occur

 Rapid repairs for defects, especially for

high-severity defects
12
Beyond Absence of Defects
 Sense of beauty
 Sense of fitness for purpose
 Sense of elegance that goes beyond the
simple absence of overt flaws
 Has well-formed requirements
 Robust
13
Why Software Quality? - 1

 Reduces time to market for new products

 Enhances market share compared to direct
competitors
 Minimizes “scrap and rework” expenses
 Attracts and keeps “top-gun” personnel
14
Why Software Quality? - 2

 Minimizes the risk of serious operating

failures and delays
 Minimizes the risk of bankruptcy or
business failures, which may be attributed
directly to poor quality or poor software
quality
15
Software Quality Assurance
 So the term SQA would mean that the
software guarantees high quality
 In this course, we’ll learn the different
processes, techniques, and activities, which
enables us – the software professionals – to
provide that guarantee to ourselves and our
clients 16
Achieving Software Quality
 “For a software application to achieve high
quality levels, it is necessary to begin
upstream and ensure that intermediate
deliverables and work products are also of
high quality levels. This means that the
entire process of software development
must itself be focused on quality”
 Capers Jones

17
What is a Software Defect?

 A software defect is an error, flaw, mistake,

failure, or fault in software that prevents it
from behaving as intended (e.g., producing
an incorrect or unexpected result)
 Software defects are also known as
software errors or software bugs
18
Effects of Software Defects - 1
 Bugs can have a wide variety of effects,
with varying levels of inconvenience to the
user of the software.
 Some bugs have only a subtle effect on
the program’s functionality, and may thus
lie undetected for a long time. More
serious bugs may cause the software to
crash or freeze leading to a denial of
service
19
Effects of Software Defects - 2

 Others qualify as security bugs and might

for example enable a malicious user to
bypass access controls in order to obtain
unauthorized privileges

20
Effects of Software Defects - 3
 The results of bugs may be extremely
serious
 In 1996, the European Space Agency’s US
$1 billion prototype Arian 5 rocket was
destroyed less than a minute after launch,
due a bug in the on-board guidance
computer program

21
Effects of Software Defects - 4
 In June 1994, a Royal Air Force Chinook
crashed into the Mull of Kintyre, killing 29
people.
 An investigation uncovered sufficient
evidence to convince that it may have
been caused by a software bug in the
aircraft’s engine control computer

22
Effects of Software Defects - 5
 In 2002, a study commissioned by the US
Department of Commerce’ National
Institute of Standards and Technology
concluded that software bugs are so
prevalent and detrimental that they cost
the US economy and estimated US $59
billion annually, or about 0.6 percent of the
gross domestic product
23
Categories of Software Defects

 Errors of commission
 Errors of omission
 Errors of clarity and ambiguity
 Errors of speed or capacity

24
Errors of Commission

 Something wrong is done

 A classic example at the code level would
be going through a loop one time too
many

25
Errors of Omission

 Something left out by accident

 For example, omitting a parentheses in
nested expressions

 The error of omission refers to the error in which a transaction is not at all recorded in the
books, either completely or partially. As against, the error of commission implies the error in
which the transaction is incorrectly recorded in the books.

26
Errors of Clarity and Ambiguity

 Different interpretations of the same

statement
 This kind of error is common with all
natural language requirements and
specification documents and user
manuals, too.
27
Errors of Speed and Capacity
 Application works, but not fast enough

28
 Software Defect Origins

 Software defects can be found in any of the

documents and work products including very serious
ones in cost estimates and development plans
 However, there are seven major classes of software
work products where defects have a strong
probability of triggering some kind of request for
repair if they reach the field 29
Software Defect Origins
 Errors in Requirements
 Errors in Design
 Errors in Source code
 Errors in User Documentation
 Errors due to “Bad fixes”
 Errors in Data and Tables
 Errors in Test Cases

30
Software Defect Origins

 We’ll discuss all of them in detail, when we

talk about different processes of software
development life cycle

31
Defect Discovery

 Defects are discovered by developers &

testers (usually) before release
 Defects are discovered by customers and
users (usually) after release
 Defects discovered after release can be
embarrassing for the development team 32
Defect Discovery by Customers
 Rule 1: Defect discovery is directly related
to the number of users
 Rule 2: Defect discovery process is
inversely related to the number of defects
 If a product has too many defects than the process of
finding defects is sloth
 If a product has too many defects than people will not
use it and defects will not be found out.

33
Software Defect Elimination
Strategies
 Effective defect prevention
 High levels of defect removal efficiency
 Accurate defect prediction before the
project begins
 Accurate defect tracking during
development
 Useful quality measurements
 Ensuring high levels of user-satisfaction

34
Defect Prevention and Removal
 Both defect prevention and removal
techniques are used by the “best-in-the-
class” companies
 Defect prevention is very difficult to
understand, study, and quantify. We’ll talk
about defect prevent in a later lecture
 Both non-test and testing defect removal
techniques must be applied

35
Typical Defect Removal
 Inspections
 Direct fault detection and removal

 Testing
 Failure observation and fault removal

36
Inspections - 1

 Inspections are critical examinations of

software artifacts by human inspectors
aimed at discovering and fixing faults in
the software systems

37
Inspections - 2
 Inspections are critical reading and
analysis of software code or other
software artifacts, such as designs,
product specifications, test plans, etc
 Inspections are typically conducted by
multiple human inspectors, through some
coordination process. Multiple inspection
phases or sessions may be used
38
Inspections - 3
 Faults are detected directly in inspection
by human inspectors, either during their
individual inspections or various types of
group sessions
 Identified faults need to be removed as a
result of the inspection process, and their
removal also needs to be verified

39
Inspections - 4
 The inspection processes vary, but typically
include some planning and follow-up activities
in addition to the core inspection activity
 The formality and structure of inspections
may vary, from very informal reviews and
walkthroughs, to fairly formal variations of
Fagan inspection, to correctness inspections
approaching the rigor and formality of formal
methods
40
Non-Test Defect Removal
Methods
 Requirement inspections
 Design inspections
 Code inspections
 Test plan reviews
 Test-case inspections
 User documentation editing or reviews

41
Testing Defect Removal Methods
 Unit test by individual programmers
 New function testing
 Regression testing
 Performance testing
 Integration testing
 System testing
 Field test (external beta test)

42
Defect Removal

 Not all defects are equal when it comes to

removal

 Requirements errors, design problems,

and “bad fixes” are particularly difficult

43
Software Defect Origins &
Defect Removal Effectiveness

44
Defect Removal Efficiency

 Accumulation of defect statistics for errors

found prior to delivery, and then for a
predetermined period after deployment
(usually one year)
 US averages: 85%
 Best projects in best US companies: 99% 45
Summary
 In today’s lecture, we have only discussed
what quality is and what software quality is
 We have briefly touched upon the need of
software quality
 We have also talked about software defects
and where are they introduced in the
software product
 We discussed the approaches to
eliminating these defects
46
References
 Software Quality: Analysis and Guidelines
for Success by Capers Jones
 Software Quality Engineering: Testing,
Quality Assurance, and Quantifiable
Improvement by Jeff Tian

47