Efficient Test Suite Management: growing nature of test suite, Minimizing the test suite and its
benefits, test suite prioritization, Types of test case prioritization, prioritization techniques,
measuring the effectiveness of a prioritized test suite, Software Quality Management- Software
Quality metrics, SQA models.
Efficient Test Suite Management & Software Quality Management
1. Efficient Test Suite Management
Software testing involves creating and running test cases to ensure software correctness. As
software evolves, the test suite grows, leading to higher execution costs. Efficient management
focuses on minimizing, prioritizing, and measuring test suites.
1.1 Growing Nature of Test Suite
When software is modified or updated, new test cases are added to handle new features,
bug fixes, and changes.
Over time, the suite becomes large and redundant, causing:
o Increased execution time.
o Higher maintenance costs.
o Difficulty in selecting important tests.
1.2 Minimizing the Test Suite
Definition: Process of reducing the size of the test suite by eliminating redundant or obsolete test
cases without losing fault-detection ability.
1. Redundancy Removal: Delete duplicate or overlapping test cases.
2. Coverage-based Minimization: Keep only those test cases that ensure maximum
code/requirement coverage.
3. History-based Minimization: Remove test cases that never detect new faults over
multiple cycles.
Benefits:
Saves execution time.
Reduces testing cost.
Easier maintenance.
Faster feedback to developers.
1.3 Test Suite Prioritization
Definition: Reordering test cases to run the most important ones earlier, improving early fault
detection.
Types of Test Case Prioritization:
1. General Prioritization – Independent of program version (e.g., requirement-based).
2. Version-specific Prioritization – Considers the latest version of the program (e.g.,
modified code areas).
Benefits:
Faster detection of severe faults.
Useful in regression testing.
Improves resource utilization.
1.4 Prioritization Techniques
1. Coverage-based Techniques: Prioritize test cases that cover maximum statements,
branches, or paths first.
Total Statement Coverage Prioritization - In this technique, the total number of
statements covered by a test case is used as a factor to prioritize test cases. For example, a
test case covering 10 statements will be given higher priority than a test case covering 5
statements.
Additional Statement Coverage Prioritization - This technique involves iteratively
selecting a test case with maximum statement coverage, then selecting a test case that
covers statements that were left uncovered by the previous test case. This process is
repeated till all statements have been covered.
Total Branch Coverage Prioritization - Using total branch coverage as a factor for
ordering test cases, prioritization can be achieved. Here, branch coverage refers to coverage
of each possible outcome of a condition.
Additional Branch Coverage Prioritization - Similar to the additional statement coverage
technique, it first selects a text case with maximum branch coverage and then iteratively
selects a test case that covers branch outcomes that were left uncovered by the previous test
case.
Total Fault-Exposing-Potential Prioritization - Fault-exposing-potential (FEP) refers to
the ability of the test case to expose fault. Statement and Branch Coverage Techniques do
not take into account the fact that some bugs can be more easily detected than others and
also that some test cases have more potential to detect bugs than others.
2. Fault-based Techniques
a. Prioritize tests more likely to detect faults based on past defect data.
List potential problems.
Assigning probability of occurrence for each problem.
Calculating the severity of impact for each problem.
3. Prioritization using Relevant Slice: In this type of prioritization, slicing technique is used
– when program is modified, all existing regression test cases are executed in order to make
sure that program yields same result as before, except where it has been modified. For this
purpose, we try to find part of program which has been affected by modification, and then
prioritization of test cases is performed for this affected part. There are 3 parts to slicing
technique:
Execution slice - The statements executed under test case form execution slice.
Dynamic slice - Statements executed under test case that might impact program output.
Relevant Slice - Statements that are executed under test case and don't have any impact
on the program output but may impact output of test case.
4. Requirements - based Prioritization
Some requirements are more important than others or are more critical in nature, hence test
cases for such requirements should be prioritized first. The following factors can be considered
while prioritizing test cases based on requirements
Customer assigned priority - The customer assigns weight to requirements according
to his need or understanding of requirements of product.
Developer perceived implementation complexity - Priority is assigned by developer
on basis of efforts or time that would be required to implement that requirement.
Requirement volatility - This factor determines frequency of change of requirement.
Fault proneness of requirements - Priority is assigned based on how error-prone
requirement has been in previous versions of software.
1.5 Measuring Effectiveness of a Prioritized Test Suite
APFD (Average Percentage of Faults Detected):
o Widely used metric.
o Measures how quickly faults are detected in the prioritized test suite.
o Higher APFD = better prioritization.
Execution Time: How fast critical test cases finish.
Fault Detection Rate: Percentage of total defects caught early.
2. Software Quality Management
Ensures that the final product meets quality standards and satisfies customer needs.
2.1 Software Quality Metrics
Definition: Quantitative measures that indicate how well software meets requirements.
Types of Metrics:
1. Product Metrics – Size, complexity, performance, reliability.
o Example: LOC (Lines of Code), Cyclomatic Complexity, Defect Density.
2. Process Metrics – Efficiency of development/testing process.
o Example: Defect removal efficiency, review effectiveness.
3. Project Metrics – Project execution and management quality.
o Example: Effort variance, schedule variance, cost performance.
2.2 SQA (Software Quality Assurance) Models
Frame works to ensure systematic quality assurance.
Common Models:
1. Waterfall-based SQA model – Testing at the end of each phase.
2. V-Model – Verification and Validation at each development stage.
3. Spiral Model with SQA – Continuous risk analysis and quality checks.
4. CMMI (Capability Maturity Model Integration):
o Levels of process maturity for improving software quality.
o Level 1: Initial → Level 5: Optimizing.
5. ISO 9001 for Software: International standard ensuring quality processes.
Debugging:
Process: The systematic process of finding, analyzing, and fixing bugs in software. It
involves steps like:
1. Observation: Detecting a bug during testing or in the field.
2. Analysis: Tracing the bug to its root cause.
3. Correction: Implementing a fix for the bug.
4. Verification: Rerunning the test to confirm the bug is fixed.
Capability Maturity Model Integration (CMMI)
Capability Maturity Model Integration (CMMI) is a successor of CMM and is a more evolved
model that incorporates best components of individual disciplines of CMM like Software CMM,
Systems Engineering CMM, People CMM, etc.
