UNIT-5

AUTOMATION AND TESTING TOOLS: The testing techniques, either static or dynamic, can be automated with the
help of software tools which can greatly enhance the software testing process. Testing today is not a manual
operation but is assisted with many efficient tools that help the testers. Testing automation is effective such that any
kind of repetitive and tedious activities can be done by machines, and testers can utilize their time in more creative
and critical work. In the last decade, a lot of testing tools have been developed for use throughout the various SDLC
phases. But the major part is the selection of tools from a pool of various categories of tools. Apart from the high
cost of these tools, a single tool may not cover the whole testing automation. Thus, tools must be selected according
to their application and needs of the organization. Automation is bound to fail if chosen for wrong reasons at the
wrong places in SDLC.

NEED FOR AUTOMATION: -

If an organization needs to choose a testing tool, the following benefits of automation must be
considered
1. Reduction of testing effort: - In verification and validation strategies, numerous test case
design methods have been studied. Test cases for complete software may be hundreds of
thousands or more in number. Executing all of them manually takes a lot of testing effort and
time. Thus, execution of test suits through software tools greatly reduces the amount of time
required.
2. Reduces the testers’ involvement in executing tests: - Sometimes executing the test cases
takes a long time. Automating this process of executing the test suit will relieve the testers to do
some other work, thereby increasing the parallelism in testing efforts.
3. Facilitates regression testing: - As we know, regression testing is the most timeconsuming
process. If we automate the process of regression testing, then testing effort as well as the time
taken will reduce as compared to manual testing.
4. Avoids human mistakes: - Manually executing the test cases may incorporate errors in the
process or sometimes, we may be biased towards limited test cases while checking the
software. Testing tools will not cause these problems which are introduced due to manual
testing
5. Reduces overall cost of the software: - As we have seen that if testing time increases, cost of
the software also increases. But due to testing tools, time and therefore cost can be reduced to a
greater level as testing tools ease the burden of the test case production and execution.
6. Simulated testing: - Load performance testing is an example of testing where the real-life
situation needs to be simulated in the test environment. Sometimes, it may not be possible to
create the load of a number of concurrent users or large amount of data in a project. Automated
tools, on the other hand, can create millions of concurrent virtual users/data and effectively test
the project in the test environment before releasing the product.
7. Internal testing: - Testing may require testing for memory leakage or checking the coverage of
testing. Automation tools can help in these tasks quickly and accurately, whereas doing this
manually would be cumbersome, inaccurate, and time-consuming.
8. Test enablers: - While development is not complete, some modules for testing are not ready. At
that time, stubs or drivers are needed to prepare data, simulate environment, make calls, and
then verify results. Automation reduces the effort required in this case and becomes essential.

9. Test case design: - Automated tools can be used to design test cases also. Through automation,
better coverage can be guaranteed, than if done manually.

CATEGORIZATION OF TESTING TOOLS: -

A single tool may not cover the whole testing process, therefore, a variety of testing tools are
1
 available according to different needs and users. The different categories of testing tools are

STATIC AND DYNAMIC TESTING TOOLS: -

These tools are based on the type of execution of test cases, namely static and dynamic, as
discussed in software testing techniques:

Static testing tools: -

For static testing, there are static program analysers which scan the source program and detect
possible faults and anomalies. These static tools parse the program text, recognize the various
sentences, and detect the following:
 Statements are well-formed. „
 Inferences about the control flow of the program. „
 Compute the set of all possible values for program data.

Static tools perform the following types of static analysis:

1. Control flow analysis: - This analysis detects loops with multiple exits and entry points and
unreachable code.
2. Data use analysis: - It detects all types of data faults.
3. Interface analysis: - It detects all interface faults. It also detects functions which are never
declared and never called or function results that are never used.
4. Path analysis: - It identifies all possible paths through the program and unravels the program’s
control.

2. Dynamic Test Tools: Dynamic testing process is performed by the

dynamic test tools. These tools test the software with existing or current
data. Dynamic test tools comprise the following:
 Test driver: The test driver provides the input data to a module-under-
test (MUT).
 Test Beds: It displays source code along with the program under
execution at the same time.
 Emulators: Emulators provide the response facilities which are used to
imitate parts of the system not yet developed.
 Mutation Analyzers: They are used for testing the fault tolerance of the
system by knowingly providing the errors in the code of the software.

SELECTION OF TESTING TOOLS: -

The big question is how to select a testing tool. It may depend on several factors. What are the
needs of the organization; what is the project environment; what is the current testing
methodology; all these factors should be considered when choosing testing tools. Some guidelines
to be followed by selecting a testing tool are given below.

1. Match the tool to its appropriate use: - Before selecting the tool, it is necessary to know its
use. A tool may not be a general one or may not cover many features. Rather, most of the tools
are meant for specific tasks. Therefore, the tester needs to be familiar with both the tool and its
uses in order to make a proper selection.

2. Select the tool to its appropriate SDLC phase: - Since the methods of testing changes
according to the SDLC phase, the testing tools also change. Therefore, it is necessary to choose
the tool according to the SDLC phase, in which testing is to be done.
2
 3. Select the tool to the skill of the tester: - The individual performing the test must select a tool
that conforms to his skill level. For example, it would be inappropriate for a user to select a tool
that requires programming skills when the user does not possess those skills.

4. Select a tool which is affordable: - Tools are always costly and increase the cost of the project.
Therefore, choose the tool which is within the budget of the project. Increasing the budget of the
project for a costlier tool is not desired. If the tool is underutilization, then added cost will have
no benefits to the project. Thus, once you are sure that a particular tool will really help the
project, then only go for it otherwise it can be managed without a tool also.

5. Determine how many tools are required for testing the system: - A single tool generally
cannot satisfy all test requirements. It may be possible that many test tools are required for the
entire project. Therefore, assess the tool as per the test requirements and determine the
number and type of tools required

6.Select the tool after examining the schedule of testing: - First, get an idea of the entire schedule of
testing activities and then decide whether there is enough time for learning the testing tool and then
performing automation with that tool. If there is not enough time to provide training on the tool, then
there is no use of automation.

COSTS INCURRED IN TESTING TOOLS: -

Automation is not free. Obviously employing the testing tools incur a high cost. Moreover, before
acquiring the tools, significant work is required. Following are some facts pertaining to the cost
incurred in testing tools

1. Automated script development: - Automated test tools do not create test scripts. Therefore, a
significant time is needed to program the tests. Scripts are themselves programming languages.
Thus, automating test execution requires programming exercises.

2. Training is required: - It is not necessary that the tester will be aware of all the tools and can
use them directly. He may require training regarding the tool, otherwise it ends up on the shelf
or implemented inefficiently. Therefore, it becomes necessary that in a new project, cost of
training on the tools should also be included in the project budget and schedule.

3. Configuration management: - Configuration management is necessary to track large number

of fi les and test related artifacts.

4. Learning curve for the tools: - There is a learning curve in using any new tool. For example,
test scripts generated by the tool during recording must be modified manually, requiring tool-
scripting knowledge in order to make the script robust, reusable, and maintainable.

5. Testing tools can be intrusive: - It may be necessary that for automation some tools require
that a special code is inserted in the system to work correctly and to be integrated with the
testing tools. These tools are known as intrusive tools which require addition of a piece of code
in the existing software system. Intrusive tools pose the risk that defects introduced by the code
inserted specifically to facilitate testing could interfere with the normal functioning of the
system.

6. Multiple tools are required: - As discussed earlier, it may be possible that your requirement is
not satisfied with just one tool for automation. In such a case, you have to go for many tools
which incur a lot of cost.

3
 GUIDELINES FOR AUTOMATED TESTING: -
Automation is not a magical answer to the testing problem. Testing tools can never replace the
analytical skills required to conduct testing and manual testing. It incurs some cost as seen above
and it may not provide the desired solution if you are not careful. Therefore, it is necessary that you
carefully plan the automation before adopting it. Decide which tool and how many tools are
required, how much resources are required including the cost of the tool and the time spent on
training.

The guidelines to be followed if you have planned for automation in testing are
1. Consider building a tool instead of buying one, if possible: - It may not be possible every
time. But if the requirement is small and sufficient resources allow, then go for building the tool
instead of buying, after weighing the pros and cons. Whether to buy or build a tool requires
management commitment, including budget and resource approvals.

2. Test the tool on an application prototype: - While purchasing the tool, it is important to
verify that it works properly with the system being developed. However, it is not possible as the

system being developed is often not available. Therefore, it is suggested that if possible, the
development team can build a system prototype for evaluating the testing tool.

1. Not all the tests should be automated: - Automated testing is an enhancement of manual
testing, but it cannot be expected that all test on a project can be automated. It is important to
decide which parts need automation before going for tools. Some tests are impossible to
automate, such as verifying a printout. It has to be done manually.

2. Select the tools according to organizational needs: - Do not buy the tools just for their
popularity or to compete with other organizations. Focus on the needs of the organization and
know the resources (budget, schedule) before choosing the automation tool.

3. Use proven test-script development techniques: - Automation can be effective if proven

techniques are used to produce efficient, maintainable, and reusable test scripts. The following
are some hints:
 Read the data values from either spreadsheets or tool-provided data pools, rather than being
hard-coded into the test-case script because this prevent test cases from being reused. Hard-
coded values should be replaced with variables and whenever possible read data from
external sources.
 Use modular script development. It increases maintainability and readability of the source
code.
 Build library of reusable functions by separating the common actions into shared script
library usable by all test engineers.
 All test scripts should be stored in a version control tool.

4. Automate the regression tests whenever feasible: - Regression testing consumes a lot of
time. If tools are used for this testing, the testing time can be reduced to a greater extent.
Therefore, whenever possible, automate the regression test cases.

OVERVIEW OF SOME COMMERCIAL TESTING TOOLS: -

 Mercury Interactive’s WinRunner: - It is a tool used for performing functional / regression

testing. It automatically creates the test scripts by recording the user interactions on GUI of the
software. These scripts can be run repeatedly whenever needed without any manual
intervention. The test scripts can also be modified if required because there is support of Test
4
 Script language (TSL) with a ‘C’ like syntax. There is also provision for bringing the application
to a known state if any problem has occurred during automated testing. WinRunner executes
the statements by default with an interleaving of one second. But if some activities take more
time to complete, then it synchronizes the next test cases automatically by waiting for the
current operations to be completed.

 Segue Software’s SilkTest: - This tool is also for functional/regression testing. It supports
4Test as a scripting language which is an object-oriented scripting language. SilkTest has a
provision for customized in-built recovery system which helps in continuing the automated
testing even if there is some failure in between.

 IBM Rational SQA Robot: - It is another powerful tool for functional/regression testing.
Synchronization of test cases with a default delay of 20 seconds is also available.

 Mercury Interactive’s LoadRunner: - This tool is used for performance and load testing of a
system. Generally, the tool is helpful for client/server applications of various parameters with
their actual load like response time, the number of users, etc. The major benefit of using this tool
is that it creates virtual users on a single machine and tests the system on various parameters.
Thus, performance and load testing is done with minimum infrastructure.

 Apache’s JMeter: -This is an open-source software tool used for performance and load testing.

 Mercury Interactive’s TestDirector: - TestDirector is a test management tool. It is a web-

based tool with the advantage of managing the testing if two teams are at different locations. It
manages the test process with four phases: specifying requirements, planning tests, running
tests, and tracking defects. Therefore, there is advantage that the tests are planned as per the
requirements evolved. Once the test plan is ready, the test cases are executed. Defect-tracking
can be done in any phase of test process. During defect-tracking, new bug can be reported;
assign responsibility to someone for bug repair; assign priority to the bug; bug repair status can
be analysed, etc. This tool can also be integrated with LoadRunner or WinRunner.

Object Oriented Testing in Software Testing

Software typically undergoes many levels of testing, from unit testing to system or
acceptance testing. Typically, in-unit testing, small “units”, or modules of the
software, are tested separately with a focus on testing the code of that module. In
higher-order testing (e.g, acceptance testing), the entire system (or a subsystem) is
tested with a focus on testing the functionality or external behavior of the system

Testing classes is a fundamentally different issue than testing functions. A function (or
a procedure) has a clearly defined input-output behavior, while a class does not have
an input-output behavior specification. We can test a method of a class using
approaches for testing functions, but we cannot test the class using these
approaches.
 Data dependencies between variables
 Calling dependencies between modules
 Functional dependencies between a module and the variable it computes
 Definitional dependencies between a variable and its types.

5
 But in Object-Oriented systems, there are the following additional dependencies:
 Class-to-class dependencies
 Class to method dependencies
 Class to message dependencies
 Class to variable dependencies
 Method to variable dependencies
 Method to message dependencies
 Method to method dependencies

Web Based Testing – Software Testing

The web application testing involves detection of bugs and to ensure that quality of the
websites is maintained. In order to efficiently plan, and conduct web application testing,
the testers should be aware of its complete architecture, and all the critical features.

The web application testing is an integral part of complete verification of the software. It
includes the validation of all the web features, parameters, browser compatibility etc. It
evaluates the web errors with respect to all the errors of the total software. It is
mandatory to execute the web application test cases on the different browsers, and
operating systems.

Types of Software Web Application Testing

The various types of the software web application testing are listed below −





1. Static Web Based Testing

It involves a static web application where the displayed content is identical to the one
which is saved in the server. This type of application contains a very attractive graphical
user interface, but it only supports the static contents. The static web based testing is
only focussed on verifying the UI which includes the font type and size, color, width,
forms, URLs etc which are normally the part of a normal website.

2. Dynamic Web Based Testing

It involves verification of both the front end and back end components of the websites.
This type of testing consists of checking the dynamic updates on the web as per the
customer requirements. It is mainly focussed on verifying whether the changes made in
the frontend are in sync with the backend and vice-versa.

6
3. E-commerce Web Based Testing
An e-commerce website undergoes complex testing as it is composed of multiple screens.
This type of testing comprises verification of individual modules as well as interaction
among various modules to perform a successful transaction for the customer. For
example, the testing includes validation of individual modules namely login, payment,
product search as well as validation of the complete workflow required to perform a
successful transaction.

4. Mobile Web Based Testing

It involves verification of the website on various mobile devices. It also tests the
responsiveness of web applications on different platforms and devices.

Points to be Considered While Testing a Website:

As the website consists of a frontend, backend, and servers, so things like HTML
pages, internet protocols, firewalls, and other applications running on the servers
should be considered while testing a website. There are various examples of
considerations that need to be checked while testing a web application. Some of them
are:
 Do all pages are having valid internal and external links or URLs?
 Whether the website is working as per the system compatibility?
 As per the user interface-Does the size of displays are the optimal and the best fit
for the website?
 What type of security does the website need (if unsecured)?
 What are the requirements for getting the website analytics, and also controlling
graphics, URLs, etc.?
 The contact us or customer assistance feature should be added or not on the
page, etc.?
Objectives of Web Based Testing:
 Testing for functionality: Make that the web application performs as expected for
all features and functions. Check that user interface elements like form
submissions and navigation work as intended.
 Testing for Compatibility: To make sure it is compatible, test the web application
across a variety of devices, operating systems, and browsers. Verify that the
program operates consistently in a range of settings.
 Evaluation of Performance: Analyze the online application’s overall performance,
speed, and responsiveness. Any performance bottlenecks, such as slow page
loads or delayed server response times, should be located and fixed.
 Testing for load: Examine how well the web application can manage a particular
load or multiple user connections at once. Determine and fix performance
problems when there is a lot of traffic.
 Testing for accessibility: Make sure the online application complies with
applicable accessibility standards (e.g., WCAG) and is usable by people with
disabilities. Make sure the program can communicate with assistive technologies
efficiently.

7
 Testing Across Browsers: Make sure the operation and appearance of the web
application are consistent by testing it in various web browsers. Determine and fix
any problems that might develop with a particular browser.
Steps in Software Testing:
There is a total of 11 steps in software testing. You can read all of them from the
article “General Steps of Software Testing Process” . In web-based testing, various
areas have to be tested for finding the potential errors and bugs, and steps for
testing a web app are given below:
 App Functionality: In web-based testing, we have to check the specified
functionality, features, and operational behavior of a web application to ensure
they correspond to its specifications. For example, Testing all the mandatory fields,
Testing the asterisk sign should display for all the mandatory fields, Testing the
system should not display the error message for optional fields, and also links like
external linking, internal linking, anchor links, and mailing links should be checked
properly and checked if there’s any damaged link, so that should be removed. We
can do testing with the help of Functional Testing in which we test the app’s
functional requirements and specifications.
 Usability: While testing usability, the developers face issues with scalability and
interactivity. As different numbers of users will be using the website, it is the
responsibility of developers to make a group for testing the application across
different browsers by using different hardware. For example, Whenever the user
browses an online shopping website, several questions may come to his/her mind
like, checking the credibility of the website, testing whether the shipping charges
are applicable, etc.
 Browser Compatibility: For checking the compatibility of the website to work the
same in different browsers we test the web application to check whether the
content that is on the website is being displayed correctly across all the browsers
or not.
 Security: Security plays an important role in every website that is available on the
internet. As a part of security, the testers check things like testing the unauthorized
access to secure pages should not be permitted, files that are confined to the
users should not be downloadable without the proper access.
 Load Issues: We perform this testing to check the behavior of the system under a
specific load so that we can measure some important transactions and the load on
the database, the application server, etc. are also monitored.
 Storage and Database: Testing the storage or the database of any web
application is also an important component and we must sure that the database is
properly tested. We test things like finding errors while executing any DB queries,
checking the response time of a\the query, testing whether the data retrieved from
the database is correctly shown on the website or not.

8
 Challenges in testing for web based software
Testing web-based software presents unique challenges due to the dynamic, complex, and interconnected
nature of web applications. Below are the primary challenges encountered during the testing process:

1. Cross-Browser Compatibility

 Challenge: Web applications must function correctly on multiple browsers (e.g., Chrome, Firefox, Safari,
Edge) and browser versions.
 Impact: Variations in rendering engines and JavaScript execution can lead to inconsistent behavior or design
issues.
 Solution: Use tools like BrowserStack or Sauce Labs for testing across browsers and versions.

2. Cross-Platform Compatibility

 Challenge: Applications need to perform well on diverse platforms (desktop, mobile, tablets, and different
operating systems).
 Impact: Differences in screen sizes, resolutions, and OS-specific behaviors make consistent user experiences
difficult to achieve.
 Solution: Perform responsive design testing and use emulators/simulators for various devices.

3. Dynamic and Evolving Content

 Challenge: Web applications often include dynamic content (e.g., APIs fetching live data, personalized feeds)
that changes frequently.
 Impact: Testing dynamic content for correctness and consistency across sessions is difficult.
 Solution: Use tools like Selenium for automated testing of dynamic elements and mock APIs for stable
testing environments.

4. Performance Testing

 Challenge: Web applications must handle varying user loads, especially during peak times.
 Impact: Poor performance, slow response times, or crashes can lead to a negative user experience.
 Solution: Conduct load testing (e.g., using JMeter) and monitor performance under various conditions.

5. Security Testing

 Challenge: Web applications are vulnerable to security threats like SQL injection, cross-site scripting (XSS),
and data breaches.
 Impact: Security vulnerabilities can lead to data loss, reputational damage, and regulatory penalties.
 Solution: Perform penetration testing, use tools like OWASP ZAP, and validate input thoroughly.

9
6. Integration with Third-Party Services

 Challenge: Many web applications rely on third-party APIs or services (e.g., payment gateways, social logins).
 Impact: Failures or changes in third-party APIs can disrupt the application.
 Solution: Use stubs and mocks for testing and set up monitoring for third-party service dependencies.

7. Scalability Testing

 Challenge: Web applications need to scale seamlessly as user traffic grows.

 Impact: Inadequate scalability can lead to slowdowns or crashes under high loads.
 Solution: Simulate high traffic scenarios with tools like LoadRunner or Apache JMeter.

8. Continuous Updates

 Challenge: Frequent updates in agile development cycles may introduce new defects or regressions.
 Impact: Maintaining the quality of frequently changing codebases is time-consuming and complex.
 Solution: Use CI/CD pipelines and automated regression testing tools like Jenkins.

9. Localization and Globalization

 Challenge: Applications targeting global users need to support multiple languages, time zones, and cultural
formats.
 Impact: Errors in date formats, currency conversions, or translated content can affect usability.
 Solution: Conduct localization testing for language and format validation.

10. Network Variability

 Challenge: Users access web applications over networks with varying speeds and reliabilities.
 Impact: Applications may perform poorly on slower networks or fail under unstable conditions.
 Solution: Test under different network conditions using tools like Network Link Conditioner.

11. Accessibility Testing

 Challenge: Ensuring that the application is usable by individuals with disabilities (e.g., vision or hearing
impairments).
 Impact: Failure to meet accessibility standards (e.g., WCAG) can lead to legal penalties and reduced user
base.
 Solution: Use tools like Axe or Wave for accessibility testing and ensure compliance with standards.

12. Browser Extensions and Add-Ons

 Challenge: Some browser extensions or add-ons may interfere with web applications.
 Impact: Unexpected behavior can occur due to conflicts or blocking of essential scripts.

10
  Solution: Test the application with commonly used extensions to identify and resolve compatibility issues.

13. Data Validation and Integrity

 Challenge: Ensuring proper validation and storage of user input data, especially for forms and databases.
 Impact: Incorrect or missing validation can lead to data corruption or vulnerabilities.
 Solution: Implement robust input validation and test edge cases.

11
12
Page 1 of
13