Software Testing Techniques Overview
Software Testing Techniques Overview
QUALITY ASSURANCE
UNIT I TESTING TECHNIQUES & TEST CASE
DESIGN 9
Using White Box Approach to Test design - Test
Adequacy Criteria – Static Testing Vs. Structural Testing –
Code Functional Testing – Coverage and Control Flow
Graphs – Covering Code Logic – Paths – Their Role in
White box Based Test Design – Code Complexity Testing
– Evaluating Test Adequacy Criteria. Test Case Design
Strategies – Using Black Box Approach to Test Case
Design – Random Testing – Requirements based testing –
Boundary Value Analysis –Decision tables – Equivalence
Class Partitioning – State based testing – Cause-effect
graphing – Error guessing – Compatibility testing – User
documentation testing – Domain testing – Case study for
Control Flow Graph and State based Testing.
What is Software Testing
The process or method of finding error/s or bugs in
a software application or program.
It is defined as an activity to check whether the
actual results match the expected results and
To ensure that the software system is Defect free.
It is a process, to find whether the
developed software met the specified requirements or
not.
Why Software testing is important?
1. To point out the defects and errors that were
made during the development.
2. To satisfy the customer needs.
Software Quality Assurance
What is Quality?
Quality is meeting the requirement, expectation, and needs of
the customer is free from the defects.
What is Assurance?
Assurance is provided by organization management, it means
giving a positive declaration on a product.
What is Control?
Control is to test or verify actual results by comparing it with
the defined standards.
What is Quality Assurance?
It is defined as an activity to ensure that an organization is
providing the best possible product or service to customers.
It is a set of activities for ensuring quality in software
engineering processes that ultimately results, or at least gives
confidence, in the quality of software products.
Basic definitions
Errors
An error is a mistake or misunderstanding on the
part of a software developer.
In the category of developer we include
1. Software engineers,
2. Programmers,
3. Analysts, and
4. Testers.
For example,
A developer may misunderstand a design notation, or
A programmer might type a variable name
incorrectly.
Basic definitions (Cont.,)
Faults (Defects)
A fault (defect) is introduced into the
software as the result of an error.
Faults or defects are sometimes called -
bugs.
Use of the term ―defect is also associated
with software artifacts such as requirements
and design documents.
Defects occurring in these artifacts are also
caused by errors and are usually detected in
the review process.
Basic definitions (Cont.,)
Failures
A failure is the inability of a software system
or component to perform its required
functions within specified performance
requirements.
During execution of a software component
or system, a tester, developer, or user
observes that it does not produce the
expected results.
In some cases a particular type of
misbehavior indicates a certain type of fault
is failure.
Test case
A test case contains the following
information:
1. A set of test inputs. These are data items
received from an external source by the code
under test.
◦ The external source can be hardware, software, or
human.
2. Execution conditions. These are
conditions required for running the test,
3. Expected outputs. These are the specified
results to be produced by the code under test.
The Smart Tester
Software components have defects.
Developers cannot prevent/eliminate all
defects during development.
Therefore, software must be tested before it
is delivered to users.
Testing is usually performed under budget
and time constraints.
The smart tester must plan for testing, select
the test cases, and monitor the process to
insure that the resources and time allocated
for the job are utilized effectively.
Test Case Design Strategies
In an effective test case is good possibility of
illuminating a defect.
The ability to develop effective test cases is important
to an organization evolving toward a higher-quality
testing process.
For example, if test cases are effective there is
(i) a greater probability of detecting defects,
(ii) a more efficient use of organizational resources,
(iii) a higher probability for test reuse,
(iv) closer adherence to testing and project schedules
and budgets, and,
(v) the possibility for delivery of a higher-quality
software product
Test Case Design Strategies
There are two basic strategies that can be used to
design test cases.
These are called the black box (sometimes called
functional or specification) and white box
(sometimes called clear or glassbox) test
strategies.
The smart tester knows that to achieve the goal of
providing users with
1. low-defect,
[Link]-quality software,
both of these strategies should be used to design
test cases.
WHITE BOX APPROACH TO TEST CASE
DESIGN
The tester’s goal is to determine if all the logical and data
elements in the software unit are functioning properly.
The knowledge needed for the white box test design approach
to the tester is the later phases of the software life cycle.
Specifically during the detailed design phase of development.
White box testing is also known as
Clear Box testing,
Open Box testing,
Structural testing,
Transparent Box testing,
Code-Based testing, and
Glass Box testing
What do you verify in White Box
Testing?
Testing
Techniques used for Static Testing:
[Link] Reviews:
• This is one of the types of review which doesn't follow any process to
find errors in the document.
• Under this technique, you just review the document and give informal
comments on it.
Types of Reviews: (cont.,)
2. Technical Reviews:
A peer’s team review the technical specification of the software product and
checks whether it is suitable for the project.
To find any discrepancies in the specifications and standards followed.
This review concentrates mainly on the technical document related to the
software such as Test Plan and requirement specification documents.
3. Walkthrough:
The author of the work product explains the product to his team.
Participants can ask questions if any. Meeting is led by the author.
Note the review comments
Inspection:
The main purpose is to find defects and meeting is led by trained moderator.
This review is a formal type of review where it follows strict process to find the
defects.
Reviewers have checklist to review the work products .
They record the defect to rectify those errors.
Static code Review:
[Link] is systematic review of the software source code without executing the
code.
[Link] checks the syntax of the code, coding standards, etc.
[Link] review can be done at any point during development.
Static Analysis - By Tools:
Following are the types of defects found by the
tools during static analysis:
A variable with an undefined value
Inconsistent interface between modules and
components
Variables that are declared but never used
Unreachable code (or) Dead Code
Programming standards violations
Security Issues and
Syntax violations
Examples of Work documents
Requirement specifications
Design document
Source Code
Test Plans
Test Cases
Help or User document
Web Page content
Dynamic Testing
Dynamic testing involves validation.
Unit Testing: Under Unit Testing, individual units or
modules are tested by the developers. It involves testing
of source code by developers.
Integration Testing: Individual modules are grouped
together and tested by the developers.
System Testing: System Testing is performed on the
whole system by checking whether the system or
application meets the requirement specification
document.
Also, Non-functional testing like
performance, Installation Testing, Security Testing fall
under the category of dynamic testing.
Static Vs Dynamic Testing
Static Testing Dynamic Testing
Testing was done without executing the Testing is done by executing the
program program
This testing does the verification Dynamic testing does the validation
process process
Static testing involves a checklist and Dynamic testing involves test cases for
process to be followed execution
Cost of finding defects and fixing is less Cost of finding and fixing defects is
high
More reviews comments are highly More defects are highly recommended
recommended for good quality for good quality.
Coverage and Control Flow
Graph
The application of coverage analysis is
associated with the use of control and data flow
models.
The logic elements for coverage are based on
the flow of control in a unit of code.
For example,
1. program statements;
2. decisions/branches (these influence the program
flow of control);
3. conditions (expressions that evaluate to true/false)
4. combinations of decisions and conditions;
5. paths (node sequences in flow graphs).
Coverage and Control Flow
Graph (Cont.,)
All structured programs can be built from
three basic types
1. sequential (e.g., assignment statements),
2. decision (e.g., if/then/else statements), and
3. iterative (e.g., while, for loops).
Graphical representations for these three
types are shown in the following Figure.
Coverage and Control Flow
Graph (Cont.,)
Coverage and Control Flow
Graph (Cont.,)
The flow graph can be used by the tester
◦ to evaluate the code with respect to its testability,
◦ to develop white box test cases.
A flow graph representation for the code example in
Figure 1.2 is found in Figure 1.3.
In the flow graph the nodes represent sequential
statements, as well as decision and looping predicates.
Edges in the graph represent transfer of control.
The direction of the transfer depends on the outcome
of the condition in the predicate (true or false).
Code Sample with Branch and Loop
(Figure 1.2)
Control Flow Graph Representation
(Figure 1.3)
Coverage and Control Flow
Graph (Cont.,)
◦ 1-2-3-4-8
Where the dashes represent edges between two nodes.
For example, the sequence “4-8” represents the edge
between nodes 4 and 8.
Paths: Their Role in White-Box Test
Design (Cont.,)
A set of independent paths for a graph is
sometimes called a basis set.
For most software modules it may be possible to
derive a number of basis sets.
If we examine the flow graph in Figure 1.3, we
can derive the following set of independent
paths starting with the first path identified above
◦ 1-2-3-4-8
◦ 1-2-3-4-5-6-7-4-8
◦ 1-2-3-4-5-7-4-8
Example
T
3
1 1 2
F
4
CFG CFG
T
2
1 4 Two paths are needed:
[1–2–4]
F 3 [1–3–4]
CFG
Path Testing (Cont.,)
In the above diagram 2 paths or condition that
need to be tested to get the output,
Path 1: 1-2-4
Path 2: 1-3-4
The basic steps involved in basis path testing
include
◦ Draw a control graph (to determine different program
paths)
◦ Calculate Cyclomatic complexity (metrics to
determine the number of independent paths)
◦ Find a basis set of paths
◦ Generate test cases to exercise each path
Using the Block Box Approach to
Test Case Design
In the black box test strategy, only inputs and outputs as
a basis for designing test cases.
A suitable set of inputs choosing from the set of all
possible valid and invalid inputs.
As an example, suppose you tried to test a single
procedure that calculates the square root of a number.
Test would have to try all positive input values, all
negative numbers, and fractions.
The goal for the smart tester is to effectively utilize the
resources and yields maximum of defects for the time
and effort spent.
Techniques for Black box
approaches
The techniques opted for black box
approaches are as follows
◦ Random Testing
◦ Requirement based Testing
◦ Boundary Value Analysis
◦ Equivalence class partitioning
◦ State-based Testing
◦ Cause-effect Graphing
◦ Compatibility Testing
◦ User documentation Testing
◦ Domain Testing
Random Testing
Each software module or system test input data
is selected from input domain.
If a tester randomly selects inputs from the
domain, this is called random testing.
For example, if the valid input domain for a
module is all positive integers between 1 and
100.
The tester randomly, select values from within
that domain; for example, the values 55, 24, 3
might be chosen.
Random Testing (Cont.,)
Given this approach, some of the issues
that remain open are the following:
◦ Are the three values adequate to show that the
module meets its specification?
◦ Are there any input values, other than those
selected, more likely to reveal defects?
◦ Should any values outside the valid domain be
used as test inputs? For example, floating
point values, negative values, or integer values
greater than 100?
Random Testing (Cont.,)
Use of random test inputs may save some
of the time and effort.
Selecting test inputs randomly has very
little chance of producing an effective set
of test data.
This type of testing can be very useful
especially at the system level.
Equivalence Class Partitioning
Equivalent Class Partitioning is a black box
technique (code is not visible to tester)
It can be applied to all levels of testing like unit,
integration, system, etc.
In this technique, you divide the set of test
condition into a partition that can be considered
the same.
It divides the input data of software into
different equivalence data classes.
You can apply this technique, where there is a
range in the input field.
Equivalence Class Partitioning
Let's consider the behavior of Order Pizza
Text Box Below
Pizza values 1 to 10 is considered valid.
A success message is shown.
While value 11 to 99 are considered
invalid for order and an error message
will appear, "Only 10 Pizza can be
ordered“
Order Pizza: Submit
Equivalence Class Partitioning
Here is the test condition
◦ Any Number greater than 10 entered in the
Order Pizza field(let say 11) is considered
invalid.
◦ Any Number less than 1 that is 0 or below,
then it is considered invalid.
◦ Numbers 1 to 10 are considered valid
◦ Any 3 Digit Number say -100 is invalid.
Equivalence Class Partitioning
Format .jpg .jpg .jpg .jpg Not .jpg Not .jpg Not .jpg Not .jpg
Size Less Less than >= 32kb >= 32kb Less than Less than >= 32kb >= 32kb
than 32kb 32kb 32kb
32kb
S1) Start S5 S2
S2) 1st attempt S5 S3
S3) 2nd attempt S5 S4
S4) 3rd attempt S5 S6
◦ Requirement documents
◦ Test Plan
◦ Test Cases
◦ Bug reports
◦ Operator Manuals
◦ User Manuals
◦ Configuration of the software
User Documentation Testing
Testing the documented objects that are
developed prior, during and after the testing of a
product is known as documentation testing.
It is a non-functional type of software testing.
We know that defects found in testing phase were
more costly than if they were found during the
requirement phase.
The cost of fixing a bug increase exponentially
the later you find it.
So documentation testing can start right from the
beginning of the software development process to
save a large amount of money.
User Documentation Testing
Documentation testing has a vital role in Software Testing.
The users will refer this document when they start using the
software at their location.
Poor documentation can affect the quality of the product.
Good product documentation plays a important role in the
final product.
Defects found in the user documentation need to be tracked to
closure like any regular software defect.
Because these documents are the first interactions the users
with the product.
A good User Documentation aids in reducing customer
support calls.
The effort and money spend on this effort would form a
valuable investment in the long run for the organisation.
Error Guessing
Designing test cases using the error guessing approach
is based on the tester’s/developer’s past experience with
code.
That is similar to the code-under-test.
The tester/developer is sometimes able to make an
educated “guess” as to which types of defects may be
present and design test cases.
If defect data for similar code or past releases of the
code has been carefully recorded,.
The defect types classified, and failure symptoms due to
the defects carefully noted.
Such data would be available to testers in a TMM
(Testing Maturity Model level 4 organization.
Cause-and-Effect Graphing
A “Cause” represents a different input conditions.
An “Effect” represent an Output condition, or system
transformation.
A major weakness with equivalence class partitioning is
◦ It does not allow testers to combine conditions.
It can be used to combine conditions and derive an
effective set of test cases.
The specification must be transformed into a graph that
resembles a digital logic circuit.
The tester is not required to have a background in
electronics,.
But he should have knowledge of Boolean logic.
The graph must be converted to a decision table that the
tester uses to develop test cases.
Cause-and-Effect Graphing
Samples of cause-and-effect graph
notations.
T1 5 c 3
T2 5 w Not found
T3 90 Integer out of range
Cause-and-Effect Graphing