Q5.

You need to develop proper report showing how to apply testing techniques
to the case study mentioned above following the guidance introduced in the
“Introduction to Software Testing” book. You need to demonstrate how to
apply the following techniques on your chosen artifact that can be used.

1 Introduction

1. Activities of a Test Engineer

 Designing test inputs.
 Producing test case values.
 Running test scripts.
 Analyzing results.
 Reporting results to developers and managers.

A test manager is in charge of one or more test engineers.

Test managers set test policies and processes, interact with other managers
on the project, and otherwise help the engineers do their work.

2.Testing Levels Based on Software Activity.

 Acceptance Testing: assess software with respect to requirements.

o Asses OpenCart software based on the requirements report that the
software was built according to.

 System Testing: assess software with respect to architectural design.

o Assess the interface and how easy it is for the user to interact with.

 Integration Testing: assess software with respect to subsystem design.

o Assess the OpenCart interface and the navigation between different
departments.
 Module Testing: assess software with respect to detailed design.
o Assess the software with the other modules such as: Service Registration,
Catalogue Service, Inventory Service, Customer Account Service, Delivery order
Service and so on.

 Unit Testing: assess software with respect to implementation.

o Assess the OpenCart Software regarding the implementation errors or failures
if there is any.

1.1.2 Beizer’s Testing Levels Based on Test Process Maturity.

Level 0 There’s no difference between testing and debugging.

 Compile the OpenCart Software and start giving the software attributes and
see whether there will be any bugs or errors based on the requirements.

Level 1 The purpose of testing is to show that the software works.

 Correctness is virtually impossible to either achieve or demonstrate. Suppose
we run a collection of tests and find no failures. That doesn’t make the
software perfect.

Level 2 The purpose of testing is to show that the software doesn’t work.
 The Tester wants to see the possible errors before the software is launched to
avoid any low feedback from the customers. It may be look a negative goal for
the developers but it’s a valid one.
Level 3 The purpose of testing is not to prove anything specific, but to reduce the
risk of using the software.
 This lets us accept the fact that whenever we use software, we incur some risk.
The risk may be small and the consequences unimportant, or the risk may be
great and the consequences catastrophic, but risk is always there.
 This allows testers and developers to realize that the entire team wants the
same thing which is to reduce the risk of using the software. In level 3 testing,
both testers and developers work together to reduce risk.

Level 4 Testing is a mental discipline that helps all IT professionals develop

higher quality software.
 Once the testers and developers are on the same "team," a company can
advance to true Level 4 testing.
 Testing is a mental discipline that improves quality, according to Level 4
thinking. There are many ways to improve quality, and writing tests that crash
the software is just one of them.
 By adopting this attitude, test engineers can serve as the project's technical
leaders (as is typical in many other engineering disciplines).

1.1.3 Automation of Test Activities.

 Software testing is costly and time-consuming. Software testing can cost up to

50% of the total cost of developing software, and much more for applications
that are safety-critical.
 One objective of software testing is to automate as much as possible in order
to lower costs, minimize human error, and simplify regression testing.
 The test engineer benefits from automating excise duties in numerous ways.
o Eliminating unnecessary duties reduces repetitiveness, making the job of the
test engineer more enjoyable.
o Automation frees up time so that testers may concentrate on the interesting
and difficult jobs related to revenue.
o Automation can assist in removing omission errors, such as forgetting to
update all necessary files with the updated list of expected outcomes.
o Automation reduces some of the test quality variation brought on by individual
differences in potential.
 1.2 Coverage Criteria for Testing.
 Which test inputs are used depends on coverage criteria.
 The community of software testers thinks that making good use of coverage
requirements increases the likelihood that program flaws will be discovered by
test engineers and offers ad hoc assurance that the software is trustworthy
and of high quality.
 Practically speaking, coverage requirements offer helpful guidelines for when
to end testing.

2. Graph Coverage

2.1 Overview

 Several coverage criteria are built on the foundation of directed graphs. The
goal is to obtain a graph abstraction of an artefact that is being tested. For
instance, the most popular source code graph abstraction transforms the code
into a control flow graph.
 Test cases for the artifact are mapped to paths in the graph using the same
abstraction that creates the graph from the artefact. Considering all this, a
graph-based coverage criterion assesses a test set for an artefact in terms of
how the test cases' related pathways "cover" the graph abstraction of the
artifacts.

2.2 Graph Coverage Criteria.

 We divide these criteria into two groups, as is common in the testing

literature.
o Control flow coverage criteria.
o Data flow coverage criteria
2.2.1 Structural Coverage Criteria.

 It starts by defining criteria to visit every node and then every edge in a graph.
 The notation initially appears to make the criteria more difficult to understand,
but in the end, it makes following criteria clearer and more mathematically
accurate, preventing confusion with more complex cases.

2.2.2 Data Flow Criteria.

 based on the idea that we should concentrate on the flows of data values in
order to test a program effectively.
 Defining and using values is how this is accomplished. A definition (def) is a
place in memory where a variable's value is placed (assignment, input, etc.).
An access point for a variable's value is called a usage.

2.2.3 Subsumption Relationships among Graph Coverage Criteria.

 Coverage criteria are often related to one another by subsumption.

 There are many different subsumption relations between the criteria. The
structural coverage relations assume best-effort travelling where appropriate.
The subsumption results hold even if some test requirements are impractical
because best-effort Touring is assumed.
 The subsumption results for data flow criteria are based on three assumptions:
(1) every use is preceded by a def.
(2) every def reaches at least one use.
(3) for every node with multiple outgoing edges, at least one variable is used
on each out edge, and the same variables are used on each out edge.
2.3 Graph Coverage for Source Code.

 Most of the graph coverage criteria were developed for source code.
 We first consider structural coverage criteria and then data flow criteria.

2.3.1 Structural Graph Coverage for Source Code.

 The most popular graph coverage guidelines are based on source code.
 Although specifics differ between programming languages, the general
structure is the same for the majority of them.
 The most common graph is called control flow graph (CFG).
 In control flow graphs, each potential program branch is represented by an
edge, and statements are represented by nodes. A fundamental block is
formally defined as the longest possible sequence of program statements,
each of which must be executed for the entire block to be performed.

2.3.2 Data Flow Graph Coverage for Source Code.

 Some features of programming languages greatly complicate this seemingly

simple definition.
 Not all uses are relevant for data flow analysis.

2.4 Graph Coverage for Design Elements.

 The usage of object-oriented software and data abstraction has resulted in a

greater focus on modularity and reuse.
 This indicates that software testing based on multiple design components
(design elements) is more essential than in the past.
 Integration testing is typically related to these activities. Software components
can be independently tested thanks to modularity, which is often done by
programmers during unit and module testing.
2.4.1 Structural Graph Coverage for Design Elements.

 Creating graphs based on couplings between software components is typically

the first step in providing coverage for design elements.
 By reflecting their connections, coupling calculates the dependency
relationships between two units; errors in one unit may have an impact on the
coupled unit.
 Coupling offers a summary of the software's architecture and design
information. The majority of test criteria for design elements demand that
different linkages between program elements be explored.
 The call graph is the one that is most frequently used for structural design
coverage. The nodes in a call graph stand in for methods (or units), while the
edges signify method calls.

2.4.2 Data Flow Graph Coverage for Design Elements.

 Tests based on control connections between design elements are clear and
relatively simple, and they are probably not very good at identifying flaws. Yet,
data flow connections are frequently very complex and hard to understand.
 The primary issue is where the defs and uses occur. When testing program
units, the defs and uses are in the same unit. During integration testing, defs
and uses are in different units. This section starts with some standard
compiler/program analysis terms.

2.5 Graph Coverage for Specifications.

 Software specifications are another source for graphs that testers can use.
 Many different methods for creating graphs and criteria for covering those
graphs are presented in the literature, but most of them are actually rather
similar.
 We begin by looking at graphs based on sequencing constraints among
methods in classes, then graphs that represent state behavior of software.
2.5.1 Testing Sequencing Constraints.

 Sequencing constraints can be represented in several ways, such as directly,

indirectly, or not at all.
 Sometimes they are encoded as precondition or other specification, but not
directly as a sequencing condition.
 Clearly, formal specifications might help in clarifying the relationships. A
responsible tester must search for formal relationships even when they are
not clearly expressed. A smart tester would certainly apply formal
specifications when they are provided.

2.5.2 Testing State Behavior of Software.

 The software's state behavior can be modelled using graphs based on

specifications by using some type of finite state machine (FSM).
 Several real-world software systems are built on FSM models or may be
modelled as FSMs, which is one benefit of basing testing on FSMs.
 Almost all embedded software falls into this category, including programs used
in remote controls, home appliances, watches, automobiles, cell phones,
traffic signals, train control systems, factory automation, aircraft flight
guidance, and other devices.
 Creating FSMs often has great value. If the test engineer creates an FSM to
describe existing software, he or she will almost certainly find faults in the
software. Some would even argue the converse; if the designers created FSMs,
the testers should not bother creating them because problems will be rare.

2.6 Graph Coverage for Use Cases.

 Software requirements are frequently clarified and expressed using UML use
cases.
 They serve to define the workflow of a computer application by describing the
series of operations that software executes in response to user input.
 Use cases can benefit the tester in starting testing activities early because they
are created early in the software development process.
2.6.1 Use Case Scenarios.
 A use case scenario is a particular instance of or complete journey through a
use case. When use cases are created, a scenario is frequently developed that
should have some semantic sense to the customers.
 It is possible to enumerate every possibility if the use case graph is finite,
which it typically is. The amount of possibilities that are helpful or interesting
from a modelling or test case standpoint can be decreased, though, using
domain expertise.
 Specified path coverage is similar to complete path coverage if the tester or
requirements writer chooses all paths as scenarios. People choose the
scenarios, and they consider their domain knowledge. Thus, it is not a certain
that a defined path's coverage includes its edges or nodes.

2.7 Representing Graphs Algebraically.

 These operations can then be used as a baseline for software testing and to
provide answers to various questions or concerns regarding the graphs.
 Each edge must have unique label or name. The edge names may be added
especially for the algebraic form or they may be derived from labels that are
already connected to the edges.

2.7.1 Reducing Graphs to Path Expressions.

 The procedure described here is not a strict algorithm because it involves
some thought and discretion, but it is sufficient for usage by human testers.
Although this procedure hasn't been automated or implemented in a tool
to our knowledge, it is a unique instance of the widely used method of
creating regular expressions from deterministic FSMs.
2.7.2 Applications of Path Expressions.
 Graphs are represented in an abstract, formal manner using path
expressions. They can therefore be altered to provide us with knowledge
about the graphs they represent.

2.7.3 Deriving Test Inputs.

 Creating covering test cases is the most practical way to use path
expression representations of graphs. Execution should occur for each path,
or path product, defined by the path expression.
 with an appropriate loop restriction. This is a form of specified path
coverage (SPC). If the path expression has an unbounded exponent ('*'), it
can be replaced with one or more carefully selected constant values to
produce the full list of paths. This method will guarantee (that is,
encompass) both node and edge coverage on the graph.

2.7.4 Counting Paths in a Flow Graph and Determining Max Path Length.
 can be used as a basic estimate of the number of tests required to cover
the graph or as a simple complexity metric. The computation of the
maximum number of paths is made possible by the path expressions, which
allow for simple arithmetic.
 It is important to remember that these analyses do not include a feasibility
analysis. Some paths may be infeasible, so this should be interpreted as an
upper, or conservative, bound on the number of paths.

7.5 Minimum Number of Paths to Reach All Edges.

 A related question is how many paths must be traversed to reach all edges.
The process is very similar to counting the maximum number of paths and
uses the same edge weights, but the computation is slightly different.
 A visual inspection of the graph can confirm that all edges can be reached
with two traversals of the graph.
2.7.5 Complementary Operations Analysis.
 The last application of path expressions is not a counting application, but an analysis
that looks for anomalies that may lead to mistakes. It is based on the idea of
“complementary operations.” Two operations are complementary if their behaviors
negate each other, or one must be done before the other.

3 Logic Coverage
3.1 Overview: Logic Predicates and Clauses.

3.2 Logic Expression Coverage Criteria.

3.2.1 Active Clause Coverage.

3.2.2 Inactive Clause Coverage.

3.2.3 Infeasibility and Subsumption.

3.2.4 Making a Clause Determine a Predicate.

3.2.5 Finding Satisfying Values.

3.3 Structural Logic Coverage of Programs.
3.3.1 Predicate Transformation Issues.

3.4 Specification-Based Logic Coverage.

3.5 Logic Coverage of Finite State Machines.

3.6 Disjunctive Normal Form Criteria.

4 Input Space Partitioning.

4.1 Input Domain Modeling.

4.1.1 Interface-Based Input Domain Modeling.

4.1.2 Functionality-Based Input Domain Modeling.

4.1.3 Identifying Characteristics.

4.1.4 Choosing Blocks and Values.

4.1.5 Using More than One Input Domain Model.

4.1.6 Checking the Input Domain Model.

4.2 Combination Strategies Criteria.

4.3 Constraints among Partitions.

5 Syntax-Based Testing.
1 Syntax-Based Coverage Criteria.

5.1.1 BNF Coverage Criteria.

5.1.2 Mutation Testing.
5.2 Program-Based Grammars.

5.2.1 BNF Grammars for Languages.

5.2.2 Program-Based Mutation.

5.3 Integration and Object-Oriented Testing.

5.3.1 BNF Integration Testing.

 As far as we know, BNF testing has not been used at the integration level.

5.3.2 Integration Mutation.

5.4 Specification-Based Grammars.

5.4.1 BNF Grammars.

5.4.2 Specification-Based Mutation.

5.5 Input Space Grammars.

5.5.1 BNF Grammars.

5.5.2 Mutation for Input Grammars.

6 Practical Considerations.

6.1 Regression Testing.

 It is the process of re-testing software that has been adjusted.
 In commercial software development, regression testing makes up the vast
majority of testing work and is crucial to any successful software
development process.
 Large components or systems tend to have large regression test suites.
 Regression testing must be automated, it is important to emphasize. In fact,
it could be said that manual regression testing is the same as performing
nothing at all. There are several different commercially available tools.

6.2 Integration and Testing.

 There are several different-sized parts that make up software. The lowest
level components (classes, modules, and methods) are frequently tested
by a single programmer. Testing must then be done in conjunction with
software integration after that.
 Integration testing is the testing of incompatibilities and interfaces
between otherwise correctly working components. In other words, it is the
testing required to combine smaller functional components into a larger
component. Testing a component that has previously been integrated is
categorically not the same as this.
6.2.1 Stubs and Drivers.
  Developers and testers frequently require additional software components,
also referred to as scaffolding, while evaluating incomplete program pieces.
Test stubs and test drivers are the two most used kinds of scaffolding.
 One of the responsibilities of a test stub is to return values to the calling
component. These values are seldom the same that the actual component
being stubbed would return, or else we would not need the stub, but
sometimes they must satisfy certain constraints.
6.2.2 Class Integration Test Order.

6.3 Test Process.

6.3.1 Requirements Analysis and Specification.
6.3.2 System and Software Design.

6.3.3 Intermediate Design.

6.3.4 Detailed Design.

6.3.5 Implementation.

6.3.6 Integration.
6.3.7 System Deployment.

6.3.8 Operation and Maintenance.

6.3.9 Summary.
6.4 Test Plans.

6.5 Identifying Correct Outputs.

6.5.1 Direct Verification of Outputs.

6.5.2 Redundant Computations.

6.5.3 Consistency Checks.

6.5.4 Data Redundancy.

7.Engineering Criteria for Technologies.

7.1 Testing Object-Oriented Software.

7.1.1 Unique Issues with Testing OO Software.

7.1.2 Types of Object-Oriented Faults.

7.2 Testing Web Applications and Web Services.

7.2.1 Testing Static Hyper Text Web Sites.

7.2.2 Testing Dynamic Web Applications.

7.2.3 Testing Web Services.

7.3 Testing Graphical User Interfaces.

7.3.1 Testing GUI.
8. Building Testing Tools.

8.1 Instrumentation for Graph and Logical Expression Criteria.

8.1.1 Node and Edge Coverage.

8.1.2 Data Flow Coverage.

8.1.3 Logic Coverage.

8.2 Building Mutation Testing Tools.

8.2.1 The Interpretation Approach.
8.2.2 The Separate Compilation Approach.

8.2.3 The Schema-Based Approach.

8.2.4 Using Java Reflection.
8.2.5 Implementing a Modern Mutation System.

Q1. You need to develop proper report showing Software Quality Factors for the
case study mentioned above following the guidance introduced in the “Software
Quality Concepts and Practice” book.

Software Quality Factors:

The structure (categories and factors) of McCall’s classic factor model:

Product operation factors:

 Correctness:
o Related to the outputs of OpenCart’s System such as the
display of the customer’s information or the prices of the
products in the store or the balance of the shop owner and
all the vital procedure that are required from the system.
o A specification is required for each output (system function) which
are:
  The Probability of non-accurate inventory information does
not exceed 1%.
 The probability of any missing data does not exceed 1%.
 Data up to date information of the platform not more than one
day.
 The required response time on any command or procedure on
the OpenCart platform does not exceed 10 seconds.
 Reliability:
o Deals with failure to provide the services such as server
crashing or compromising with any way that stops the
OpenCart platform.
o Its required that the software does not crash by any way at
least on its first 12 months of launch or has any infiltration
by any means regarding the operation of the platform or the
customers or merchants private records or information.
 Efficiency:
o The hardware resources and software requirements to
achieve OpenCart operations in its best state must be
fulfilled.
o OpenCart has certain technical requirements for the
software to operate properly.
 The software must be uploaded to a web server, which
will make the store publicly available on the web.
 When selecting a hosting service, an Apache server is
recommended.
 a database server that supports MySQLi, PDO, or
PostgreSQL. (MySQLi is recommended if possible.)
 Finally, you will need to have the following PHP
libraries installed in your PHP configuration:
  PHP 8.0 or later
 Curl
 GD Library
 Iconv
 Mbstring
 OpenSSL Encrypt
 ZipArchive
 Zlib
 Integrity:
o OpenCart software security must be guaranteed:
 Prevent unauthorized access to the platform.
 Distinguish between users with different authorization
access to the platform (Admins, Users, Customers,
Merchants, Store and Shop owners.)
 Be able to deal with any attempts to destroy any
information (Viruses) and be able to prevent and also
destroy them.
 Usability:
o It is required that there is a way of teaching any new user to
the interface of the platform and facilitate the user
experience as much as possible.
o A 24-hour Customer Service team handles all the
requirements of the users.
o Periodically take feedback from all types of users of the
OpenCart platform to be able to update and change to the
better.
2.Product revision software quality factors:
 Maintainability:
o A software module can contain no more than 30
statements.
o The programming will follow the coding standards
and policies of the firm.
 Flexibility:
o The software should be suitable for users of all types
(Admins, customers, Shop owners and Merchants).
o Non-professionals should be able to use OpenCart to
create new types of reports according to the admin
requirements.
 Testability:
o delivering log files and predefined outcome variables
of the platform.
o Before starting the system, the software system runs
an automatic diagnostic to check that all of its parts
are functioning properly and to create an overview on
any errors that were found.

3. Product transition factors:

 Portability:
o The Platform can work on several types of operating
systems beginning from windows 9.
 o The Platform can work on several types of devices
such as mobile phone, tablet, workstations.
 Reusability:
o The Platform Requires to log in or sign up first to be
able to access the software.
o Billing module.
o Delivery Orders tracking.
 Interoperability:
o creating interfaces to other software systems or
equipment firmware (for example, the firmware of
the Delivery interface and testing equipment
interfaces with the production control software).
