Chapter 3

Test Management
 Test Plan
 A test plan is a document describing software Testing scope and
activities.
 It is the basis for formally testing any software in the project.
Definition : A document describing the scope, approach, and
schedule of intended test activities.
 In this documentation of features to be tested, testing task, test
design techniques, entry and exit criteria.
 It is a record of planning process.
 Test Plan
There are two types of plans
 Master test plan
 Phase test plan
Master test plan
A plan that typically consists addresses different test levels.
Phase test plan
A plan that typically consist one test phase.
 Activities for test planning
 Preparing a test plan.
 Deciding a test approach.
 Setting up criteria for testing.
 Identifying responsibilities.
 Staffing and training needs.
 Resources requirement.
 Test deliverables.
 Testing tasks.
Sections of test plan template
 Test plan identifier: unique number to identify test
plan.
e.g. project_name001
 References: supportive documents for test plan.

e.g SRS
Use case documents, test strategy, test plan.
Introduction : Purpose and scope of the project.
To test functionality of the project name.
Test items: A list of test items which will be tested.
 Sections of test plan template
 Features to be tested: login functionality, dashboard,
reports.
 Features not to be tested
 Deciding test approach: Test types, test techniques.
 Setting up criteria for pass/fail
 Suspension criteria: login functionalities or major

functionalities not performed.

 Test deliverables :bug report.
 Testing task : test environment, test execution, test

summary report.
Sections of test plan template
 Resource requirement.
 Identifying responsibilities.
 Staffing and training needs.
 Schedule
 Risk
 Approval
 Advantages of Test Plan
 Serves as a guide to testing throughout the development.
 To define testing plan during the testing phase.
 Serves as a valuable record of testing was done.
 The entire test plan can be reused if regression testing
can be done later on.
 The test plan could have defects just like software.
 Deciding Test Approach
 The type of testing which is going to be used for testing the
functionality.
 The type of configuration or scenarios suitable for testing
features.
 Type of integration testing to ensure these features working
together.
 Localization validation which would be necessary.
 The respective “non-functional ” tests need to do.
 Setting up criteria for testing

 Entry and exit criteria for different phases of testing.

 Test strategies decide how to test these features and
combinations would be tested.
 To avoid last minute pressure of executing tests after
development.
 The entry criteria for a test specify threshold criteria for
each phase.
 Setting up criteria for testing
 Exit criteria decide when a cycle can be deemed
complete.
 Test activity is not isolated, it is continuous.
 Number of times it is suspended because it is not
proceed further.
 When situation comes under control it will be resumed.
 Setting up criteria for testing
 A suspension criteria is used when a test cycle or test case
should be suspended.
 A resumption criteria is used to mention when the
suspended tests can be resumed.
 Suspension criteria is as follows:
 Occurrence of more than predetermined number of
defects which lead to frequent stoppage of testing activity.
 Setting up criteria for testing
 Hitting show stoppers that make difficult further
progress of testing.
 Identifying Responsibilities Staffing
There are different roles in testing strategy:
• Test engineers

• Test lead
• Test Managers

There is also role definition on the basis of the modules which are
being tested or kind of testing.
The different role definitions should be in the following way:
Make sure there is unambiguous accountability for every task.
All of the persons know their work properly.
Identifying Responsibilities Staffing
 Fix responsibilities to different individuals.
 Complement each other by making sure that no one steps on
another toes.
 Supplement each other so that no task is left unassigned.
 List management and reporting responsibilities.
 Staffing is implemented on estimation of effort which has been
involved.
Identifying Responsibilities Staffing
 Appropriate task get executed, features, tasks are prioritized.
 According to skills and experience individuals are assigned
the tasks.
 It is necessary to implement appropriate training programs.
Identifying Resource Requirements
Project manager’s responsibility for hardware as well as
software.

 Machine configuration like RAM, Processor disk.

 Automation tool.
 Supporting tools like test data generators, configuration
management tools.
Identifying Resource Requirements
 Load tests and performance tests.
 Office space, support functions etc.
 Identifying Test Deliverables
 Test plan (Master test plan , phase test plan)
 Test design specifications
 Test cases as well as any automation which is
mentioned in the plan.
 Test logs generated by executing the test.
 Test summary reports.
 Testing tasks
Estimation in two phases
 Size estimation
 Effort estimation
Size estimation : Actual amount of testing which is
necessarily done.
Number of factors contribute size estimate
testing project
Lines of code:
It depends on:
 language used for coding,
 respective style of programming,
 compactness of programming,
 only considered for coding phase.
 not for requirement and design.
Number of factors contribute size estimate
testing project
A function point:
 A function point can be widely used to measure the
size of software.
 To measure functionality.
 To measure complexity of software.
Parameters for function point
 External input
 External output
 Logical internal files
 External interface files
 External inquiry
Parameters for function point
Parameters are categorized into:
 Simple
 Average
 complex
Parameters for function point
External input:
 Every distinct input which is coming from outside the system
is called external input.
 These inputs can be data or control parameters.
Simple: If count of input is less and affects a smaller amount of
internal files.
Complex: If count of input is high and affects number of
internal files.
Parameters for function point
Average :It is placed in between simple and complex.
External output: This category is useful for counting all
output types provided by the system.
Output is assumed distinct if their output format are distinct.
Simple: If count of output is less.
Complex: If count of output is high.
Average : If lies between simple and complex.
Parameters for function point
Logical internal files:
 Every software system maintain internal files in order to
maintain it’s functional information and to function properly.
 It holds logical data.
 It may be functional data or control data.
Simple: If number of record types are low.
Complex: If number of record types in shared file are high.
Average : In between simple and complex.
Parameters for function point
External interface files:
Sharing of files with external software.
Passing the file for processing.
Simple: If number of record types in shared file are low.
Complex: If number of record types in shared file are high.
Average : In between simple and complex .
Parameters for function point
External inquiry:
 Mixture of input & output.
 User send some data to ask about input and system
respond to the user with the output inquiry processed.
 Complexity of query is more than external input and external
output.
Parameters for function point
 Query is said to be distinct if input and output said to be
distinct in terms of format and data.
Simple: If query requires low processing and produces small
amount of output data.
Complex: If query requires high processing and produces large
amount of output data.
Average : In between simple and complex .
 Test Management
 It refers to the activity of the managing software testing
process.
 Test management allows different teams to plan,
develop, execute and access all testing activities.
Aspects in test management
 Test infrastructure management
 Team management
Test infrastructure management
Testing activities supported : manually or automated.
Stronger infrastructure provides stability, continuity,
reliability.
Testing infrastructure includes:
 Test plan
 Test cases
 Baseline test data
 A process to refresh or rollback baseline data.
 Stable back end and front end.
Test infrastructure management

 Dedicated test lab

 Integration group and process
 Way to prioritize or rank test cases per test cycle.
 Coverage analysis metrics and process.
 Defect tracking database.
 Risk management process.
 Version control system.
 Configuration management process.
 A method process/tool for tracing requirements to test cases.
Purposes of testing Infrastructure
 Place for running automated tests.
 A dedicated test environment to prevent conflict between
manual and automated testing.
 Process for tracking result of test cases pass or fail.
 A way of reporting test coverage levels.
 Ensuring that expected results remain consistent across test
runs.
 Test people management
A test team: A test team comprise of individuals having
varying
 skill levels.
 experience levels.
 expertise levels.
 different attitudes.
 different expectations/interest levels.
 Test people management
 These attributes are used to maximize quality.
 Need to work cohesively together.
 Follow test processes within time schedule.
Leadership skill and responsibilities
 To meet the product goals and achieving the
organization goals.
Responsibilities of test lead:
 To test how team align within organization and how the
team would achieve the roadmap.
 Scope of the testing required for particular release based
on the requirements of document.
Leadership skill and responsibilities
 To identify the required metrics and to work have them.
 To figure out what skills are required and the test resources
accordingly with those needs based on their own interests.
 To identify tools for test reporting.
 Guidance to junior.
 Create a fun and conductive environment for the resources.
Leadership skill and responsibilities
Manage the test team effectively:
 Initiate test planning activities.
 To hold review meetings.
 Monitor test progress
 Assess work.
 Hold meetings if delay occurred figure out issues.
 Present timely status to stakeholders.
 Test Process
 Testing is not a single activity.
 It is a set of number of processes.
Test Process Activities:
1. Base lining a test plan.
2. Test case specification.
 Test Process Activities
Baselining test refers to the validation of the documents
& specifications on which test cases are designed.
Base of any new creation.
It is a type of non-functional testing.
It is known as benchmark testing.
Compare the performance of new application with known
standard of reference.
 Test Process Activities
 A benchmark is about analyzing the relative performance of
an application.

 The baselined test plan becomes basis for successfully running

project.
 Test case specification
 Test case specification describes what scenarios will be
tested , how they will be tested how often they will be
tested.
 Identifies required input and expected results.
 Provides step by step procedures and pass/fail criteria.
 Feature to be tested each unit must be determined.
Test case specification identifiers
 Test case objectives: Purpose of the test.
 Test Items: It describes features and conditions
required for testing.
 Input specifications: Description of what is required
to execute the test case.
Test case specification identifiers

 Output specification : Description of what the system

should look like after the test case is run.
 This should be provided in the expected results.
Environmental needs: Description of any special
environmental needs.
This includes system architectures, hardware and software
tools , records or files , interfaces etc.
 Executing Test Cases
 Execute test cases at the appropriate times during a project.
 Defect repository updated with defects with previous test
cycles and updated with defects from previous test cycles
which are fixed in current build.
 Defect repository is communication among with test team and
development team.
 Defect repository defects which are uncovered during testing.
 Executing Test Cases
 Defect repository is reported to stakeholders for current status.
 Communicated with the help of mails, conference calls.
 Test may be suspended at the mid of execution due to
show stopper defects.

Suspended criteria wait still resumption criteria.

Traceability matrix is used while execution of test cases.
Executing Test Cases
Executing Test Cases
Executing Test Cases
Executing Test Cases
Executing Test Cases
Executing Test Cases
Executing Test Cases
 Preparing Test Summary
 Testing needs communication between test team and other team.
There are two types of communication report:
 Test incident report
 Test summary report.
Test incident report communication which occurs through testing
cycle whenever defects are encountered.
A test incident report is an entry which has been made in the
corresponding defect repository.
 Preparing Test Summary
 Every defect has unique defect Id.
 The test incident which has high impact are highlighted
in the test summary support.
 Test cycle report
 The execution of test projects is done in units of test cycles.
 Each cycle referring unique build of the product.
 Defects which have been uncovered during that cycle
based on their severity as well as impact.
 Progress from prior cycle to current cycle in terms of defect
fixed.
 Test cycle report
 Outstanding defects which yet to be fixed in this cycle.
 Any variations encountered in effort or schedule.
 Test Summary Report
 Once the process of testing is completed testing team prepares
various documents that help to communicate simple and
complicated activities related to the process of testing.
 Testing summary report is document prepared by the team.
 It summarize critical details about testing cycle and status of
testing result.
 Test Summary Report
 Prepared after the completion of testing process.
 Necessary information related to testing process as well as
results delivered by it.
 Offers information about various activities.
 Summarizes the results of all testing efforts.
 Prepared by test team lead.
 Manager prepare document at the end of testing project.
Aspects covered by test summary
Test summary report consists of following information
 Project information
 Test objective
 Test summary
 Defects
 Aspects covered by test summary
Project information:
Details about project along with description of its name ,version
, product name , date and time etc.

Test objective: purpose of testing.

e.g. purpose of unit testing is different than performance and
load testing.

Test Summary: Detail about test cases executed, passed,

failed , blocked along with comments provided by testers.
Aspects covered by test summary
 Defects : Defects found by team during the testing
process.
 Total number of bugs, their status , bugs resolved open
and closed as well as severity and priority.
 Format of summary report
 Test Summary Report Identifier : Company generated
unique number that is allocated to this report.
 Summary :After assigning unique id summarize test report.
 Test items: Test items should match test items provided in the
definition of test plan.
 Test Environment: It should be according to the test plan and
 and validate correct set up used by team.
 Format of summary report
 References: Includes any document or evidence that can
support to the report.
 Variances : The team documents any deviation or changes from
those areas and plans which were agreed by everyone from the
document.
 These includes references supporting the documentation like
change requests, enhancement request , incident report ,test plan
and specifications among other references.
 Format of summary report
 Comprehensiveness assessment : This involves
evaluation of testing and testing process for determining
quality and effectiveness of testing activities.
To find out weakness and any challenge while executing it.
 Summary of results: It includes impact of testing.
 Resolved incidents and defects.
 Division of these defects as per severity and priority.
 Format of summary report
 Some of other information included in this document are:
 Nature and cause of problems.
 Cost , impact and positive trends.
 Severity and priority of impact and defects.
 Defect pattern.
 Pass and failure of incident.
 Format of summary report
 Evaluation :On the basis of evaluation the team offers
information about the testing process, its information,
drawbacks, failure, high and medium risk areas ,good quality
areas and features.
 Summary of activities: It defines testing activities, the time and
efforts invested in it.
 The main focus is on the process , techniques, used to achieve
set goals and requirements, changes in the plan.
 Format of summary report
 Approval : Approval in the form of signature and
documentation are included by the team provided by testers
These should match the approval provided in the test plan to
verify the result of testing.
Advantages of test summary report
 It justifies the testing effort of teams as well as steps and
techniques used by them.
 Reports the current status of the project.
 Measure the quality of end project.
 Assist the team in building a mature and accurate process.
 Final document to check whether it is ready to release or not.
 Test Report

 Test Report is needed to reflect testing results in a formal way,

which gives an opportunity to estimate testing results quickly.
It is a document that records data obtained from an evaluation
experiment in an organized manner, describes the
environmental or operating conditions, and shows the
comparison of test results with test objectives.
 Test Summary Report
Test Objective

 Test Report should include the objective of each round of testing, such as Unit Test,

Performance Test, System Test …Etc.

Test Summary

This section includes the summary of testing activity in general. Information detailed here

includes

The number of test cases executed.

 The numbers of test cases pass.

 The numbers of test cases fail.

 Pass percentage.

 Fail percentage.

 Comments.
Test Summary Report
Defect
 One of the most important information in Test Report is
defect. The report should contain following information
 Total number of bugs.
 Status of bugs (open, closed, responding).
 Number of bugs open, resolved, closed.
 Breakdown by severity and priority.
 Test Report
 Test report is a communication tool between the Test Manager and
the stakeholder. Through the test report, the stakeholder
can understand the project situation, the quality of product and
other things.
 The following scenario shows you why we need a good Test Report
 You co-operate with outsourcing company, its tester after having
performed Performance Testing of the website Guru99 Bank, sends
you a test report like this
Test Report
Test Report
 Test Report
 Detail: You should provide a detailed description of the
testing activity, show which testing you have performed. Do
not put the abstract information into the report, because the
reader will not understand what you said.
 Clear: All information in the test report should
be short and clearly understandable.
 Standard: The Test Report should follow
the standard template. It is easy for stakeholder to review and
ensure the consistency between test reports in many projects.
 Specific: Do not write an essay about the project activity.
Describe and summarize the test result specification and focus
on the main point.