UNIT-05 Marks-12

Testing Tools and Measurements

5.1 Manual Testing and need for Automated Testing
Tools
5.2 Advantages and Disadvantages of Using Tools
5.3 Selecting a Testing Tool
5.4 When to use Automated Test Tools, Testing using
Automated Test Tools
5.5 Metrics and Measurements: Types of Metrics,
Product Metrics and Process Metrics, Object Oriented
Metrics in Testing.
5.1 Manual Testing and need for Automated Testing
Tools
Testing Computer Software manually without using any
Automation Tools
Limitations of Manual Testing:
• Manual Testing requires more time or more resources,
sometimes both
• Performance testing is impractical in manual testing.
• Less Accuracy
• Executing same tests again and again time taking
process as well as Tedious.
• GUI Objects Size difference and Color combinations
etc.. Are not easy to find in Manual Testing.
• Not Suitable for Large scale projects and time bounded
projects.
• Batch Testing is not possible, for each and every test
execution Human user interaction is mandatory.
• Manual Test Case scope is very limited, if it is
Automated test, scope is unlimited.
• Comparing large amount of data is impractical
• Checking relevance of search of operation is difficult
• Processing change requests during software
maintenance takes more time.

Benefits of Automation Testing are:

1. Save Time /Speed: Due to advanced computing
facilities, automation test tools prevail in speed of
processing the tests. Automation saves time as software
can execute test cases faster than human.
2. Reduces the tester’s involvement in executing
tests: It relieves the testers to do some other work.
3. Repeatability/Consistency: The same tests can be
re-run in exactly the same manner eliminating the risk
of human errors such as testers forgetting their exact
actions, intentionally omitting steps from the test
scripts, missing out steps from the test script, all of
which can result in either defects not being identified or
the reporting of invalid bugs (which can again, be time
consuming for both developers and testers to reproduce)
4. Simulated Testing: Automated tools can create many
concurrent virtual users/data and effectively test the
project in the test environment before releasing the
product.
5. Test case design: Automated tools can be used to
design test cases also. Through automation, better
coverage can be guaranteed than if done manually.
6. Reusable: The automated tests can be reused on
different versions of the software, even if the interface
changes.
7. Avoids human mistakes: Manually executing the test
cases may incorporate errors. But this can be avoided in
automation testing.
8. Internal Testing: Testing may require testing for
memory leakage or checking the coverage of testing.
Automation can done this easily.
9. Cost Reduction: If testing time increases cost of the
software also increases. Due to testing tools time and
therefore cost is reduced.
Needs of automation testing:
1. Speed: Think about how long it would take you to
manually try a few thousand test cases for the windows
Calculator. You might average a test case every five
seconds or so. Automation might be able to run 10, 100
even 1000 times that fast.
2. Efficiency
doing anything else. If you have a test tool that reduces
the time it takes for you to run your tests, you have
more time for test planning and thinking up new tests.
3. Accuracy and Precision: After trying a few hundred
cases, your attention may reduce and you will start to
make mistakes .A test tool will perform the same test
and check the result perfectly, each and every time.
4. Resource Reduction: Sometimes it can be physically
impossible to perform a certain test case. The number of
people or the amount of equipment required to create
the test condition could be prohibitive. A test tool can
used to simulate the real world and greatly reduce the
physical resources necessary to perform the testing.
5. Simulation and Emulation: Test tools are used to
replace hardware or software that would normally
interface to your product. This software in ways that you
choose and ways that might otherwise be difficult to
achieve.
6. Relentlessness: Test tool and automation never tire
or give up. It will continuously test the software.
Difference between Manual testing Automated
testing
Automated Testing Manual Testing
• If you have to run a set of • If Test Cases have to be run
tests repeatedly automation is a a small number of times it's
huge gain more likely to perform manual
testing
• Helps performing • It allows the tester to
"compatibility testing" - testing perform more ad-hoc (random
the software on different testing)
configurations
• It gives you the ability to run • Short term costs are reduced
automation scenarios to
perform regressions in a shorter
time
• It gives you the ability to run • The more time tester spends
regressions on a code that is testing a module the grater the
continuously changing odds to find real user bugs
• It's more expensive to • Manual tests can be very
automate. Initial investments time consuming
are bigger than manual testing
• You cannot automate • For every release you must
everything, some tests still have rerun the same set of tests
to be done manually which can be tiresome
5.2 Advantages and Disadvantages of Using Tools

Advantages of using testing tools:

1. Reduce time of testing

2. Improve the bugs finding
3. Deliver the quality software/product
4. Allow to run tests many times with different data
5. Getting more time for test planning
6. Save resources or reduce requirement
7. It is never tired and expert person can work at a
time many tools.
Disadvantages of using testing tools:

1. It's more expensive to automate. Initial investments

are bigger than manual testing
2. Manual tests can be very time consuming.
3. You cannot automate everything; some tests still
have to be done manually.
4. You cannot rely on testing tools always.

5.3 Selecting a Testing Tool

Criteria for Selecting Test Tools:
The Categories for selecting Test Tools are,
1. Meeting requirements;
2. Technology expectations;
3. Training/skills;
4. Management aspects.
1. Meeting requirements-

There are plenty of tools available in the market but

rarely do they meet all the requirements of a given
product or a given organization. Evaluating different
tools for different requirements involve significant effort,
money, and time. Given of the plethora of choice
available, huge delay is involved in selecting and
implementing test tools.

2. Technology expectations-

Test tools in general may not allow test developers to

extends/modify the functionality of the framework. So
extending the functionality requires going back to the
tool vendor and involves additional cost and effort. A
good number of test tools require their libraries to be
linked with product binaries.

3. Training/skills-

While test tools require plenty of training, very few

vendors provide the training to the required level.
Organization level training is needed to deploy the test
tools, as the user of the test suite are not only the test
team but also the development team and other areas like
configuration management.
4. Management aspects-

A test tool increases the system requirement and

requires the hardware and software to be upgraded. This
increases the cost of the already- expensive test tool.

Guidelines for selecting a tool:

1. The tool must match its intended use. Wrong selection

of a tool can lead to problems like lower efficiency and
effectiveness of testing may be lost.
2. Different phases of a life cycle have different quality-
factor requirements. Tools required at each stage may
differ significantly.
3. Matching a tool with the skills of testers is also
essential. If the testers do not have proper training and
skill then they may not be able to work effectively.
4. Select affordable tools. Cost and benefits of various
tools must be compared before making final decision.
5. Backdoor entry of tools must be prevented.
Unauthorized entry results into failure of tool and
creates a negative environment for new tool introduction.
5.4 When to use Automated Test Tools, Testing using
Automated Test Tools
When to use Automated Test Tools
• The application has a very vast area with a high
degree of investing effort in regression.
• Optimization in costs occurred due to manual
errors.
• The software has multiple versions and releases.
• It is cost effective in long run.
• The risk factor is higher for a broader scope of test
execution.
• Cost figures and mathematical calculations are
included in the software functionality.
• There is a greater increase in the execution tempo,
efficiency along with the software quality.
• There is a lesser turnaround time, even for high-risk
software testing.

Testing using Automated Test Tools

Automation Testing these days is a must for most

software projects to ensure automatic verification of
key functionalities. Also help teams efficiently run a
large number of tests in a short period of time.
Listed below are a few tools that help software teams
build and execute automated tests:
Examples:
1. Selenium:
Selenium is a popular testing framework to perform web
application testing across various browsers and
platforms like Windows, Mac, and Linux. With selenium,
you can come up with very powerful, browser-centered
automation testing scripts which are scalable across
different environments. It is compatible with several
programming languages & automation testing
frameworks.
2. Watir:
Watir, pronounced as water, is an open source testing
tool made up of Ruby libraries to automate web
application testing. Loaded with Ruby libraries, it also
supports applications scripted in other languages. You
can link it with databases, export XML files, read files,
spreadsheets, and synchronize code as reusable
libraries. It is a very light-weight open source tool.
3. Ranorex:
Ranorex is flexible, all in one, GUI testing tool using
which you can execute automated tests flawlessly
throughout all environments and devices. When
compared to other GUI testing tools, Ranorex offers
super smart object recognition feature that
automatically detects any change in the user interface
 and keeps the test going. Other features
include reusable code modules, early bug finding, and
integration with other tools.
4. HPE Unified Functional Testing (UFT)
HPE Unified Functional Testing (UFT) software,
formerly known as HP Quick Test Professional (QTP) is
an automated functional GUI testing tool which allows
the automation of user actions on a client based
computer application. It offers features like object
recognition, error handling mechanism, and automated
documentation. It also uses a scripting language to
manipulate the objects and controls of the application
under test.
5. Tricentis Tosca
Tricentis Tosca is a very popular software testing tool
that is used to automate end-to-end testing for
software applications. This tool offers a single
repository for all functional test artifacts, including
requirements, user stories, test data, virtualization
assets. Tosca comes with capabilities like test data
provisioning, service virtualization network, tests
mobile apps, and risk coverage. Tricentis Tosca is a
very popular software testing tool that is used to
automate end-to-end testing for software applications.
This tool offers a single repository for all functional test
artifacts, including requirements, user stories, test
 data, virtualization assets. Tosca comes with
capabilities like test data provisioning, service
virtualization network, tests mobile apps, and risk
coverage.

5.5 Metrics and Measurements: Types of Metrics,

Product Metrics and Process Metrics, Object
Oriented Metrics in Testing.

A Metric is a measurement of the degree that any

attribute belongs to a system, product or process. For
example the number of errors per person hours would be
a metric. Thus, software measurement gives rise to
software metrics. A measurement is an indication of the
size, quantity, amount or dimension of a particular
attribute of a product or process. For example the
number of errors in a system is a measurement.

Software measurement is required to:

• Establish the quality of the current product or

process.
• To predict future qualities of the product or process.
• To improve the quality of a product or process.
• To determine the state of the project in relation to
budget and schedule.
Types of Metrics:
➢ Process quality
➢ Product quality
➢ Objective Metrics
➢ Subjective Metrics

Process quality:
Activities related to the production of software, tasks or
milestones.
1. Process metrics are collected across all projects and
over long periods of time.
2. They are used for making strategic decisions.
3. The intent is to provide a set of process indicators that
lead to long-term software process improvement.
4. The only way to know how/where to improve any
process is to:
• Measure specific attributes of the process.
• Develop a set of meaningful metrics based on these
attributes.
• Use the metrics to provide indicators that will lead to
a strategy for improvement.
Product quality:

Explicit result of the software development activity,

deliverables, products.

1. Product metrics help software engineers to better

understand the attributes of models and assess the
quality of the software.
2. They help software engineers to gain insight into the
design and construction of the software.
3. Focus on specific attributes of software engineering
work products resulting from analysis, design, coding,
and testing.
4. Provide a systematic way to assess quality based on a
set of clearly defined rules.
5. Provide an “on-the-spot” rather than “after-the-fact”
insight into the software development.

Objective Metrics:

1. They are non-negotiable – that is the way they are

defined doesn’t change with respect to the niche or the
type of endeavor they are being applied to.
2. Actual cost or AC is always the total cost actually
incurred in accomplishing a certain activity or a
sequence of activities.