Unit-4

ADVANCED TESTING CONCEPTS

Performance Testing: Load Testing, Stress Testing, Volume Testing,
Fail over Testing, Recovery Testing, Configuration Testing,
Compatibility Testing, Usability Testing, Testing the Documents,
Security Testing, Testing in the Agile Environment,Testing Web
and Mobile Application.
Load Testing:
• It determines the performance of a system, software product, or software
application under real-life based load conditions.
• Basically, load testing determines the behavior of the application when
multiple users use it at the same time. It is the response of the system
measured under varying load conditions. The load testing is carried out for
normal and extreme load conditions.
• The goal of load testing is to identify bottlenecks and determine the
maximum number of users or transactions the system can handle.
The Objective of Load Testing:
• The load testing is used to perform the maximum quantity of software
applications without any breakdown.
• It is used to identify the total count of users that can access the
application simultaneously
• The load testing is used to determine whether the latest infrastructure
can run the software application or not and determine the sustainability
of the application concerning extreme user load.
Load testing techniques includes:

Stress testing: Testing the system’s ability to handle a high load above
normal usage levels
Spike testing: Testing the system’s ability to handle sudden spikes in traffic
Soak testing: Testing the system’s ability to handle a sustained load over a
prolonged period of time
Load Testing Process:
Test Environment Setup: Firstly create a dedicated test environment setup
for performing the load testing. It ensures that testing would be done in a
proper way.
Load Test Scenario: In second step load test scenarios are created. Then
load testing transactions are determined for an application and data is
prepared for each transaction.
Test Scenario Execution: Load test scenarios that were created in previous
step are now executed. Different measurements and metrices are gathered to
collect the information.
Test Result Analysis: Results of the testing performed is analyzed and
various recommendations are made.
Re-test: If the test is failed then the test is performed again in order to get
the result in correct way.
Load Testing Tools:
1. Apache Jmeter
2. WebLoad
3. NeoLoad
4. LoadNinja
5. HP Performance Tester
6. LoadUI Pro
7. LoadView
Advantages:
• Load testing enhances the sustainability of the system or software
application.
• It improves the scalability of the system or software application.
• It helps in the minimization of the risks related to system downtime.
• It reduces the costs of failure of the system.
• It increases customer’s satisfaction.
• Load testing can be complex, requiring specialized knowledge and
expertise to set up and execute effectively.

Stress Testing:
• It is a software testing technique that determines the robustness of
software by testing beyond the limits of normal operation.

• Stress testing is defined as a type of software testing that verifies the

stability and reliability of the system.
• Stress testing is performed to ensure that the system would not crash
under crunch situations.
• Stress testing is also known as Endurance Testing or Torture Testing.

Characteristics of Stress Testing:

• Stress testing analyzes the behavior of the system after a failure.
• Stress testing makes sure that the system recovers after failure.
• It checks whether the system works under abnormal conditions.
• It ensures the display of appropriate error messages when the system is
under stress.
• It verifies that unexpected failures do not cause security issues.
• It verifies whether the system has saved the data before crashing or not.
Process of Stress Testing:
Planning the stress test: This step involves gathering the system data,
analyzing the system, and defining the stress test goals.

Create Automation Scripts: It will create the stress testing automation

scripts and generating the test data for the stress test scenarios.

Script Execution: This step involves running the stress test automation
scripts and storing the stress test results.

Result Analysis: This phase involves analyzing stress test results and
identifying the bottlenecks.

Tweaking and Optimization: This step optimizing the code with the goal
meet the desired benchmarks.
Tools:
1. Jmeter
2. WebLoad
3. NeoLoad
4. LoadNinja
Types:
1. Server-client Stress Testing
2. Product Stress Testing
3. Transactional Stress Testing
4. Systematic Stress Testing
5. Analytical Stress Testing
6. Application Stress Testing
 Advantages:
• Stress testing signifies the system's behavior after failure and makes sure
that the system recovers quickly from the crashes.
• The most important advantage of executing the stress testing will make
the system work in regular and irregular conditions.
• It determines the scalability and enhance the performance of the software.
Disadvantages:
• If we are writing the Stress test script, the person should have enough
scripting knowledge of the language.
• It will require additional resources, which makes this testing bit costlier.
• If we perform the Stress Testing manually, it became complicated task to
complete, and it may also not produce the expected results.
Volume Testing:
• Volume Testing is a type of software testing which is carried out to test a
software application with a certain amount of data.
• It is basically performed to analyze the performance of the system by
increasing the volume of data in the database. Volume testing is
performed to study the impact on response time and behavior of the
system when the volume of data is increased in the database.
Characteristics of Volume Testing:
• Only small amount of data is tested during development phase.
• The test data need to be logically correct.
• The test data is used to assess the performance of the system.
Objectives of Volume Testing:
• To recognize the problems that may be created with large amount of data.
• To check the system’s performance by increasing the volume of data in
the database.
• To find the point at which the stability of the system reduces.
• To identify the capacity of the system or application.
Volume Testing Attributes:
These are the important attributes that are checked during the volume testing:
System’s Response Time: During the volume testing, the response time of
the system or the application is tested. It is also tested whether the system
responses within the finite time or not. If the response time is large then the
system is redesigned.
Data Loss: During the volume testing, it is also tested that there is no data
loss. If there is data loss some key information might be missing.
Data Storage: During the volume testing, it is also tested that the data is
stored correctly or not. If the data is not stored correctly then it is restored
accordingly in proper place.
Data Overwriting: In volume testing, it is tested whether the data is
overwritten without giving prior information to the developer. If it so then
developer is notified.

Advantages:
• Volume testing is helpful in saving maintenance cost that will be spent on
application maintenance.
• It is also helpful in a rapid start for scalability plans.
• Volume testing also helps in early identification of bottlenecks.
• It ensures that the system is capable of real world usage.
Disadvantages:
• More number of skilled resources are needed to carry out this testing.
• It is sometimes difficult to prepare test cases with respect to the number
of volume of data to be tested.
• It is a time consuming technique since it requires lot of time.
• It is a bit costly as compared to another testing technique.
• It is not possible to have the exact break down of memory used in the
real world application.
Fail over testing:
• Failover testing is a technique that validates if a system can allocate extra
resources and backup all the information and operations when a system
fails due to some reason.
• It is preferred that testing should be performed by servers. Active-active
and active-passive are the two most common configurations.
• Both the techniques achieve failover in a very different manner but both
of them are performed to improve the server’s reliability.
• Once the failure in a system is identified, the issue is resolved and the
system gets back to its previous form. The prime purpose of this test is
to ensure to retain data and resources on time when crashes happen.
 Considerable Factors Before Performing Failover Testing:
• The budget has to be the first thing to be taken into consideration
before thinking about performing the Failover test.
• The budget is connected to the frameworks that might get crashed or
break down under pressure/load.
• Always keep in mind that how much time it will take to fix all of the
issues caused by the failure of the system.
• Note down the most likely failures and organize the outcomes according to
how much harm is caused by the failure.

Considerable Factors While Performing Failover Testing:

• Keep a plan of measures to be taken after performing a test.
• Focus on execution of the test plan.
• Set up a benchmark so that performance requirements can be achieved.
Working process of Fail over testing:
1. Consider the factors before performing failover testing like budget, time,
team, technology.
2. Perform analysis on failover reasons and design solutions.
3. Develop test cases to test failover scenarios.
4. Based on the result execute the test plan.
5. Prepare a detailed report on failover.
6. Take necessary actions based on the report.
Examples of Failover Testing:
• Banking and Financial applications
• Telecom applications
• Visa applications
• Trading applications
• Emergency service business applications
• Government applications
• Defense service-related applications
Recovery Testing:
• which aims at testing whether a system can recover from failures or not.
The technique involves failing the system and then verifying that the
system recovery is performed properly.
• To ensure that a system is fault-tolerant and can recover well from
failures.
• A system is expected to recover from faults and resume its work within a
pre-specified time period.
A system or software should be recovery tested for failures like:

• Power supply failure

• The external server is unreachable
• Wireless network signal loss
• Physical conditions
• The external device not responding as expected.

Steps to be performed before executing a Recovery Test:

Recovery Analysis:
• Before implementing the recovery testing, we should make sure that the
proper analysis has to be done to verify the possibility of recovery. The
recovery analysis is necessary for our better understanding of the
recovery-related modification.
Test Plan preparation:
• Designing the test cases keeping in mind the environment and results
obtained in recovery analysis.
Test environment preparation:
• Designing the test environment according to the recovery analysis results.
Maintaining Back-up:
• Information related to the software, like various states of the software and
database should be backed up.
Recovery personnel Allocation:
• For the recovery testing process, it is important to allocate recovery
personnel who is aware and educated enough for the recovery testing
being conducted.
Documentation:
• This step emphasizes on documenting all the steps performed before and
during the recovery testing so that the system can be analyzed for its
performance.
 Test
Recovery Test Plan
Environment
Analysis Preparation
Preparation

Recovery
Maintaning
personnel Documentation
backup
Allocation
Advantages:
• The recovery testing will help us to remove risks.
• The system is more stable, reliable and bug-free once we performed the
recovery testing.
Disadvantages:

• It is an expensive process.
• A skilled person is essential to test the recovery testing. If the untrained
test engineer implements the recovery testing, they should have all the
data for testing.
Configuration Testing:
• It verifies the performance of the system under development against
various combinations of software and hardware to find out the best
configuration under which the system can work without any flaws.
• The different configurations of hardware and software means the multiple
operating system versions, various browsers, various supported drivers,
distinct memory sizes, different hard drive types, various types of CPU etc.
Various Configurations:
Operating System Configuration:
Win XP, Win 7 32/64 bit, Win 8 32/64 bit, Win 10 etc.
Database Configuration:
Oracle, DB2, MySql, MSSQL Server, Sybase etc.
Browser Configuration:
IE 8, IE 9, FF 16.0, Chrome, Microsoft Edge etc.
 Objectives of Configuration Testing:
• To determine whether the software application fulfills the configurability
requirements.
• To determine an optimal configuration of the application under test.
• To do analyse of the performance of software application by changing the
hardware and software resources.
• To do analyse of the system efficiency based on the prioritization.
• To verify how the bugs are reproducible irrespective of the configuration
changes.
Types of Configuration Testing:
Software Configuration Testing:
• Software configuration testing is done over the Application Under Test
with various operating system versions and various browser versions etc.
• It is a time consuming testing as it takes long time to install and uninstall
the various software which are to be used for testing.
• When the build is released, software configuration begins after passing
through the unit test and integration test.
Hardware Configuration Testing:
• Hardware configuration testing is typically performed in labs where
physical machines are used with various hardware connected to them.
• While doing hardware configuration test, the kind of hardware to be
tested and there are several computer hardware which make it next to
impossible to execute all the tests.
Configuration Testing Process:
Firefox

Unit Integrating Test

Chrome
Testing Testing Environment

IE
Client level Testing:
• Client level testing is associated with the usability and functionality testing.
This testing is done from the point of view of its direct interest of the users.
Server level Testing:
• Server level testing is carried out to determine the communication between
the software and the external environment when it is planned to be
integrated after the release.
Compatibility Testing:
• Checking the functionality of an application on different software,
hardware platforms, network &browsers is known as compatibility
testing.
• This compatibility testing is very important in product production and
implementation point of view as it is performed to avoid future issues
regarding compatibility.
Why compatibility testing is important?

• It ensures complete customer satisfaction.

• It provides service across multiple platforms.
• Identifying bugs during development process.
Types of Compatibility Testing:
1. Software
2. Hardware
3. Network
4. Mobile
1.Software Testing:
• Testing the compatibility of an application with an Operating System like
Linux, Mac, Windows.
• Testing compatibility on Database like Oracle SQL server, MongoDB
server.
• Testing compatibility on different devices like in mobile phones,
computers.
2.Hardware Testing: Checking compatibility with a particular size of,
• RAM
• ROM
• Hard Disk
• Memory Cards
• Processor
• Graphics Card
3.Smartphones Testing:
Checking compatibility with different mobile platforms like android, iOS etc.
4.Network Testing:
• Checking compatibility with different,
• Bandwidth
• Operating speed
• Capacity
Types based on Version Testing:

Forward Compatibility Testing: Test the software or application on the new

or latest versions.
For example: Latest Version of the platforms (software)
Win 7 → Win 8 → Win 8.1 → Win 10
Backward Compatibility Testing: Test the software or application on the
old or previous versions.
For example:
Window XP → Vista → Win 7 → Win 8 → Win 8.1
Challenging part to test compatibility testing:
Challenge to test for all platforms
Compatibility testing bug/issue:
These bugs are those that are happening in one platform, but not occur in
another platform.
1. Alignment issue
2. Overlap issue
3. Scattered issue
4. Look and feel issue
Alignment issue:The alignment issue is that in which the element of the page
is not aligned in a proper format as we can see in the below image.
 Overlap issue:
When one attribute is overlapping to another attribute, it may happen when
we are trying to open the application on the different platforms.

Scattered issue:
When the test engineer performing compatibility testing on the application, and
that application is not compatible with all browsers, and platform that's why the
scattered issues may occur.
Look and Feel issue:
When we run the application on different platforms, it may look different
and feel different.
Tools:
LambdaTest
BrowserStack
BrowseEMAll
TestingBot
Usability Testing:
• Checking the user-friendliness, efficiency, and accuracy of the
application is known as Usability Testing.
• Generally, usability testing is performed from an end-user viewpoint to
verify if the system is efficiently working or not.
• The primary purpose of this testing is to check the application should be
easy to use for the end-user, whereas sustaining the client's specified
functional and business requirements.
The user-friendliness can be described as,
• Easy to understand
• Easy to access
• Look and feel
• Faster to Access
• Effective Navigation
• Good Error Handling
Why do we need to perform Usability Testing?
• The flow of an Application should be good
• Navigation steps should be clear
• Content should be simple
• The layout should be clear
• Response time
Parameters of Usability Testing:
Efficienc
y Memora
bility
Error

Paramet
ers

Statisfac Accuran
tion cy
Learnabi
lity
 Usability Testing Process:
• The usability testing process follows a precise set of steps to help the team
get the detailed and helpful response from the end-users.
• In real-time, usability testing tests the application's behavior from the user's
perspective even though it is a time-consuming process, providing the
tester the most precise outcomes from actual testing.

5.Data
1.Planning 3.Team
Analysis
Execution

2.Team 4.Result
Recruiting Documentat 6.Reporting
ion
 Planning:
• The objective of the usability test is covered in the planning step.Where
the team makes the test plan and generates some document samples that
help the testing team complete the usability testing tasks.
Team Recruiting:
• Once the planning phase is completed, we will hire or recruit the test
engineers as per the budget and density of the product. And will validate
the correctness and usability of the product.
• The selection of these test engineers is based on the necessity of testing
along with the number of persons mentioned in the test plan.
Test Execution:
• The test engineers execute the usability testing and implement their
assigned responsibilities. In this situation, the users' needs to test the
product to find irregularity, if any, and also record them correctly.
Test Result Documentation:
• The test result documentation step includes the results based on the test
execution step and then proceeds for further analysis.
Data Analysis:
• The response or the feedback is obtained from usability testing evaluation
in the data analysis phase. And the outcomes are classified.
• In this step, the data from usability tests are evaluated to get implications
and help us provide actionable suggestions to enhance the overall usability
of our product.
Reporting:
• In this, we can report & share the outcomes& suggested modifications with
the development team, designer,& another participant of the particular
project and all the related documents along with audio, databases, screen
recording, etc.
Usability Testing Methods:
1. A/B Testing
2. Hallway Testing
3. Laboratory Usability Testing
4. Expert Review
5. Automated Expert Review
6. Remote Usability Testing
Synchronous Remote Usability Testing
Asynchronous Remote Usability Testing
1.A/B Testing:
• The first usability testing approach is A/B Testing, which includes creating
a similar image of the product without an essential aspect from the original.
2. Hallway Testing:
• In hallway testing, some random people test the application without
having any earlier knowledge of the product instead of skilled
professionals. As a result, we will get more precise outcomes and reliable
responses for further enhancement,
3.Laboratory Usability Testing:
• Generally, it is implemented by the team in an individual lab room and
reporting the results of testing to the related team
4.Expert Review:
• The Expert Review method includes the benefits of a professionals teams
who have in-depth knowledge or experience in the specified field of
performing usability tests.
• The specialist in a specified field is requested to test the product, give the
response, and then submit the outcomes.
• The expert review of usability testing is implemented rapidly and takes less
time than the other type of usability testing because the professionals can
easily identify the loopholes and discover the flaws in the product.
5.Automated Expert Review:
• As the name recommends, automated expert review is executed by writing
automation scripts.
• To execute this usability testing approach, an organization needs to appoint
a resource who is well aware of writing automation scripts and developing
an automation framework.
• The automation test engineers write the test scripts, and when the scripts
are triggered, we can easily implement the test cases. After the
implementation of the test, the results are recorded and submitted.
6.Remote Usability Testing:
• Remote usability testing takes place by people located at remote locations,
which means those situated in various states or sometimes in some other
countries to achieve their testing objectives.
• The remote usability testing is executed remotely and also able to report
the issues if identify any. In this approach, the response can be documented
and submitted by random people, not by the skilled ones.
Advantages:
• With the help of usability testing, we can discover the usability issues
before delivering the final product.
• It helps us to make the software more efficient and applicable.
• The usage of usability testing will help us in receiving related and precise
user responses.
 Disadvantage:
• Usability testing costing also plays an essential role. It requires many
resources to establish a usability test lab, and sometimes hiring or
recruiting a usability test engineer could be costly.
• Usability testing is implemented by the end-users, sometimes it is a bit
harder to identify the volunteers who can work as test engineers.
Security Testing:
• Security testing is an integral part of software testing, which is used to
discover the weaknesses, risks, in the software application and also
help us to stop the nasty attack from the outsiders and make sure the
security of our software applications.
• The primary objective of security testing is to find all the potential
ambiguities and vulnerabilities of the application so that the software does
not stop working.
Principles of Security testing:
1. Availability
2. Integrity
3. Authorization
4. Confidentiality
5. Authentication
6. Non-repudiation
1.Availability:
• The data must be retained by an official person, and they also guarantee
that the data and statement services will be ready to use whenever we need
it.
2.Integrity:
The primary objective of integrity is to permit the receiver to control the data
that is given by the system.
• And also verify that correct data is conveyed from one application to
another.
3.Authorization:
• It is the process of defining that a client is permitted to perform an action
and also receive the services.

4.Confidentiality:
• It is a security process that protracts the leak of the data from the
outsider's .
5.Authentication:
• The authentication process comprises confirming the individuality of a
person, tracing the source of a product that is necessary to allow access to
the private information or the system.

6.Non- repudiation:
• It is used as a reference to the digital security, and it a way of assurance
that the sender of a message cannot disagree with having sent the message
and that the recipient cannot repudiate having received the message.
Key Areas in Security Testing:
1. System Software Security
2. Network Security
3. Server Side Application Security
4. Client Side Application Security
1.System software security:
• In this, we will evaluate the vulnerabilities of the application based on
different software such as Operating system, Database system, etc.
2.Network security:
• We will check the weakness of the network structure.
3.Server-side application security:
• We will do the server-side application security to ensure that the server
encryption and its tools are sufficient to protect the software from any
disturbance.
4.Client-side application security:
• We will make sure that any intruders cannot operate on any browser or
any tool which is used by customers.
Types of Security testing:
1. Security Scanning
2. Risk Assessment
3. Vulnerability Scanning
4. Penetration testing
5. Security Auditing
6. Ethical hacking
7. Posture Assessment
Security Scanning:
• Security scanning can be done for both automation testing and manual
testing. This scanning will be used to find the unwanted file modification
in a web-based application, websites, network, or the file system.

Risk Assessment:
• We will explore the security risk, which can be detected in the association.
The risk can be further divided into three parts, and those are high,
medium, and low. The primary purpose of the risk assessment process is
to assess the vulnerabilities and control the significant threat.
Vulnerability Scanning:
• Vulnerability scanning is performed with the help of automated software
to scan a system to detect the known vulnerability patterns.
 Penetration testing:
• Penetration testing is a security implementation where a cyber-security
professional tries to identify the weakness in the computer system. The
primary objective of this testing is to simulate outbreaks and also finds the
loophole in the system and similarly save from the intruders who can take
the benefits.
Security Auditing:
• Security auditing is an internal inspection of applications and operating
systems for security defects. An audit can also be carried out via line-by-
line checking of code.
Ethical hacking:
• The ethical hacking will help us to increase the security position of the
association because sometimes the ethical hackers use the same tricks,
tools, and techniques that nasty hackers will use, but with the approval of
the official person.
Posture Assessment:
• It is a combination of ethical hacking, risk assessments, and security
scanning, which helps us to display the complete security posture of an
organization.
Advantages:
• Security testing helps improve the overall security of the system by
identifying and fixing vulnerabilities and potential threats.
• Security testing helps organizations to understand the potential risks and
vulnerabilities that they face, enabling them to prepare for and respond to
potential security incidents.
Disadvantages:
• Security testing can be complex, requiring specialized knowledge and
expertise to set up and execute effectively.
• It’s difficult to simulate real-world attacks, and it’s hard to predict how
attackers will interact with the system.
• Security testing can be time-consuming, especially if the system is large
and complex.

Testing The Documentation:

• Testing documentation is the documentation of artifacts that are created
during or before the testing of a software application. Documentation
reflects the importance of processes for the customer, individual and
organization.
• Projects which contain all documents have a high level of
understandability. Careful documentation can save the time, efforts and
wealth of the organization.
• Test document is prepared by every test engineer before starting the test
execution process.
• Once the test document is ready, the entire test execution process depends
on the test document. The primary objective for writing a test document is
to decrease or eliminate the doubts related to the testing activities.
Types of test document:
• Test scenarios
• Test case
• Test plan
• Requirement traceability matrix(RTM)
• Test strategy
• Test data
• Bug report
• Test execution report
 Test Scenarios:
• It is a document that defines the multiple ways or combinations of testing
the application. Generally, it is prepared to understand the flow of an
application. It does not consist of any inputs and navigation steps.
Test case:
• It is an in-details document that describes step by step procedure to test an
application. It consists of the complete navigation steps and inputs and all
the scenarios that need to be tested for the application.
• We will write the test case to maintain the consistency.
Test plan:
It is a document that is prepared by the managers or test lead. It consists of all
information about the testing activities. The test plan consists of multiple
components such as Objectives, Scope, Approach, Test Environments,
Test methodology, Template, Role & Responsibility, etc...
Requirement Traceability Matrix (RTM):
• The Requirement traceability matrix [RTM] is a document which ensures
that all the test case has been covered.
• This document is created before the test execution process to verify that we
did not miss writing any test case for the particular requirement.
Test strategy:
• The test strategy is approved by The Project Manager.
• The test strategy is a high-level document, which is used to verify the test
types (levels) to be executed for the product and also describe that what
kind of technique has to be used and which module is going to be tested.
Test data:
• It is data that occurs before the test is executed. It is mainly used when we
are implementing the test case. Mostly, we will have the test data in the
Excel sheet format and entered manually while performing the test case.
• The test data can be used to check the expected result, which means that
when the test data is entered, the expected outcome will meet the actual
result and also check the application performance by entering the in-correct
input data.
Bug report:
• The bug report is a document where we maintain a summary of all the bugs
which occurred during the testing process.
• This is a crucial document for both the developers and test engineers
because, with the help of bug reports, they can easily track the defects,
report the bug, change the status of bugs which are fixed successfully, and
also avoid their repetition in further process.
Test execution report:
• It is the document prepared by test leads after the entire testing execution
process is completed.
• The test summary report defines the constancy of the product, and it
contains information like the modules, the number of written test cases,
executed, pass, fail, and their percentage.

Benefits of using Documentation:

• Documentation clarifies the quality of methods and objectives.

• It ensures clarity about the stability of tasks and performance.
• It provides feedback on preventive tasks.
• It provides feedback for your planning cycle.
• If we write the test document, we can easily understand the whole process
of the applcation.
Testing in the Agile environment:
• An agile methodology is an iterative approach to software development.
Each iteration of agile methodology takes a short time interval of 1 to 4
weeks. The agile development process is aligned to deliver the changing
business requirement. It distributes the software with faster and fewer
changes.
Principles:
• Constant response
• Continuous testing
• Easy and Clean code
• Less documentation
• Customer satisfaction
• Quick feedback
Life cycle of Agile esting:

5.Test agility
review 1.Assessment

Agile life cycle

4.Daily Scrums
2.Agile planning

3.Release
readiness
Testing web and mobile application:
Mobile application:
• Mobile application testing is a process by which an application software
developed for handheld mobile devices is tested for its functionality,
usability, and consistency.
Web application:
• Web applications are required to be tested on different browsers and
platforms so that we can identifythe areas that need special focus while
testing a web application.